I used to have that feeling until i started playing skyrim with an uncapper. now if I want to switch i just go for it. in fact, with no worries about perk selection the game actually rewards you for trying different things. plus it keeps things interesting.
The only hard part is picking a roleplaying aspect where that makes sense.
Of course I know this feeling, it can result in many unfinished character :D I have a very similar symptome, when I see a very impressive item, I want to build a character using it. Actually these items are: Nhkriin, Vokuun, Otaar and Rahgot :D Recently it was the Archmage robes and Morkei :D