Spoke too soon. Just got the problem happening. I had the video on pause for quite a longer while than I tested before, and the mac went to sleep for a bit.
When starting back, the video started without the sound, and VLC became irresponsive.
Had to force-quit it.