I have come across a possible bug with how VLC handles MP4 metadata when it comes to aspect ratio.
I currently have an iMac computer that is set up for converting VHS tapes to MP4 files. I use an A/V to HDMI converter, which is fed into an inexpensive HDMI capture dongle, and I record the output using QuickTime and then edit and compress the final video using AVIDemux. I render the files at 1280x720, but I force a 4:3 aspect ratio in the MP4 muxer options, which results in a display resolution of 960x720 when played back.
When I play my final encoded files in QuickTime, it displays the video at proper 960x720 display resolution. But if I load the same file in VLC, the video appears at twice it's size at 1280x960. This also affects screenshots as well unless I manually set the screenshot resolution to 960x720 in the Advanced Video Settings.
Is there some sort of scaling option that I'm missing, is it a bug with how VLC reads the MP4 metadata, or is this an issue with how AVIDemux encodes files?
I use VLC at default scaling settings, and only re-assign the screenshot key in options. I tried loading my MP4 files on two VLC installations, one on a Mac with El Capitan, and another with High Sierra. Both Macs run the latest stable VLC release, 3.0.11.1.