You are not answering the question; I can't help you.
Well then, are you sure you did not change the VLC preferences? LibVLC should play pretty much like VLC with default settings.
Try to comment out the set_hwnd() call and see if it fixes performance problems then. Maybe your window handle is incompatible with some optimizations.
I use "libvlc_media_player_set_hwnd()" to direct the output to "my window".Are you configuring the video output in any way, or do you let LibVLC create its own independent video window?