When the brightness is to low the picture will appear over saturated in color and of coarse to dark, but this is kinda normal for a video overlay when a comfort level has been found within Windows for the setup of your monitor.
There are several choices here, but before I start I strongly recommend resetting your video card to all of the defaults for Color, OpenGL, Direct3D and Overlay and then rebooting to be sure the defaults took.
1. Open WMP and start playing a video and adjust the Overlay settings on your video card. Generally this is a brightness only adjustment. Check several videos to be sure you have it comfortable and real bright scenes are not washed out. Remember you can't judge night scenes because this is most generally a film makers problem and LCDs have no true black only a dark shade of gray.
So white balance becomes the criteria.
2. VLC will pick up the overlay settings you did with WMP and really there shouldn't be a difference here. If there is the first thing I suggest is to reset your preferences or better yet delete the preferences folder. Then try VLC again. VLC offers real filters to do color, brightness etc. in the Extended GUI. When you get it the way you want it go to preferences and press Save. The next time you open the player the settings you made should stick.
What the issue of opening the second instance of VLC is I don't know! But if you do this procedure, I would tend to believe it will disappear.