I have the same problem.
Version 1.1 is not very xp friendly I think.
You may be able to close it properly by right clicking on the task bar button thing where it shows an open window then close.
There is also the vlc keyboard shortcut ctrl+q .