> This way also did not work and gave this error now.
that's odd, this option (--qt-compositor) has been here since November 2020, can you check that the your version is recent enough? I assume you downloaded vlc 4.0 from here
https://nightlies.videolan.org/, you may try to download one from last week (the latest has a temporary regression in the player view -_-' )
otherwise you may try without the --qt-compositor option but you'll probably need to disable "integrate video in interface" in the preferences
> but until version 3.0.17 it works normally on this windows
the 4.0 version has a new interface with different requirements. It's supposed to work on windows 7, but maybe there are some edge cases that needs to be covered.