There is a reason why some people won’t use glitter in their studios. That stuff gets everywhere. And polymer clay seems to attract it like a magnet.
Pay close attention when you’re working. If you see a speck of glitter or sparkle when it shouldn’t be there, stop right then and take it out. It WILL be much more visible after baking and it will be much more difficult to fix at that point.
Yes, I know, it’s annoying to stop and deal with it. But it’s far more annoying to wish that you had. And it’s not going to disappear on its own.