If in doubt whether you're listening to passthrough or reencoded signal, try increasing volume in VLC. If you hear volume increasing you're in the reencoded mode. If VLC volume slider doesn't affect what you hear, you're in passthrough mode.
HDMI supports 8ch LPCM audio. VLC sends only 6channels. So the issue is that when "HDMI" audio output option is chosen, VLC should output 8ch PCM audio even if the source file only contains 6 channels. It outputs only 6channels, so as a result we hear central channel at the right surround ...