Yes, my dear. It IS you. Here's how I finally got past the horror of caramelizing flan sugar.
It was always a crap shot whether I'd end up with brown caramel or crusty sugar lumps. Now I use a simple idea from Theo's, the candy company out in CA. Since adopting their method, my caramels have been perfect, so when I made flan for a dinner party, I melted the sugar using their method. It was perfect. No graininess. No lumps. No crystallization. No burn spots. And once the pan is empty, a brief soak removes any residual.
For 1 Cup of sugar: Using a non-stick pot, heat the pot and sprinkle 1/4 C of sugar across the bottom. Wait until it starts to melt and then use a silicon spatula to shift the unmelted sugar over the melty stuff. When it's almost liquid, add another 1/4 C, repeating the process with 1/4 of the sugar each time. It will melt quicker after the first entry because now hot liquid sugar is touching the unmelted sugar. It also will get darker as you go along. When all the sugar is melted, there should be steam rising off from it. Pour immediately into flan pan--using pot holders to swirl and coat bottom because the metal will transfer flesh-removing heat to your tender fingertips.
That's it...if you're using more than a cup of sugar, just use 1/4 for each step...that's the key: not adding all of the sugar at one time.
PS: My guess on why some of yours won't melt off the pan is because that bit of sugar reached a higher temperature state (catalyzed) than the rest of the sugar.