The VideoLAN Forums

I have a relative powerful 4K laptop and can edit 8K files in daVinci etc., but VLC won't play any files over 4K - sound will be played only.
However Virtual Dub (x64) will play 5k, 6k, 8k files.
An example is here:
https://www.dropbox.com/s/f1gbx4pz8ssqp ... t.mov?dl=0
This is file is for test purposes only (no releases from participants).
-----------

HP ZBook 17 G5
Processor Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz 2.21 GHz
Installed RAM 24.0 GB (23.8 GB usable)
NVIDIA QuadroP1000 4GB VRAM
DreamColor 10 bit 4K UHD display

Edition Windows 10 Pro
Version 20H2
Installed on ‎05/‎11/‎2020
OS build 19042.1586
Experience Windows Feature Experience Pack 120.2212.4170.0

VLC 3.0.18 Vetinari

----------

debug:

main debug: processing request item: 8kWatPhnomTest.mov, node: Playlist, skip: 0
main debug: Creating an input for preparsing '8kWatPhnomTest.mov'
main debug: rebuilding array of current - root Playlist
main debug: rebuild done - 2 items, index 1
main debug: starting playback of new item
main debug: resyncing on 8kWatPhnomTest.mov
main debug: 8kWatPhnomTest.mov is at 1
main debug: creating new input thread
main debug: Creating an input for '8kWatPhnomTest.mov'
main debug: requesting art for new input thread
main debug: using timeshift granularity of 50 MiB
main debug: using timeshift path: C:\Users\Mishka\AppData\Local\Temp
main debug: `file:///J:/FujiH2Test/8K/8kWatPhnomTest.mov' gives access `file' demux `any' path `/J:/FujiH2Test/8K/8kWatPhnomTest.mov'
main debug: creating demux: access='file' demux='any' location='/J:/FujiH2Test/8K/8kWatPhnomTest.mov' file='J:\FujiH2Test\8K\8kWatPhnomTest.mov'
main debug: looking for access_demux module matching "file": 15 candidates
main debug: looking for meta fetcher module matching "any": 1 candidates
lua debug: Trying Lua scripts in C:\Users\Mishka\AppData\Roaming\vlc\lua\meta\fetcher
lua debug: Trying Lua scripts in C:\Program Files (x86)\VideoLAN\VLC\lua\meta\fetcher
main debug: no meta fetcher modules matched
main debug: looking for art finder module matching "any": 2 candidates
qt debug: IM: Setting an input
main debug: no access_demux modules matched
main debug: creating access: file:///J:/FujiH2Test/8K/8kWatPhnomTest.mov

etc etc etc

mp4 debug: Apple QuickTime media
mp4 debug: found 3 tracks

etc etc etc

main debug: Stream buffering done (1000 ms in 82 ms)
main debug: resized to 8192x4320
main debug: using vout window module "qt"
main debug: looking for inhibit module matching "any": 1 candidates
main debug: using inhibit module "inhibit"
main debug: Opening vout display wrapper
main debug: looking for vout display module matching "any": 12 candidates
direct3d11 debug: creating Vout EventThread
direct3d11 debug: Win32VoutCreateWindow
main debug: VoutDisplayEvent 'resize' 8192x4320
direct3d11 debug: inclinometer not found. (hr=0x80070490)
direct3d11 debug: created video sub-window
direct3d11 debug: Vout EventThread running
main debug: resized to 3840x1970
main debug: VoutDisplayEvent 'resize' 3840x1970
direct3d11 debug: Created the D3D11 device type 1 level b100.
direct3d11 debug: NVIDIA WDDM driver 27.21.14.6172
direct3d11 debug: supports colorspace RGB Rec.709 gamma:22 range:FULL
direct3d11 debug: supports colorspace RGB Rec.709 gamma:22 range:STUDIO
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:22 range:STUDIO
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:22 range:FULL
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:2084 range:FULL
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:2084 range:STUDIO
direct3d11 debug: Output max luminance: 270.0, colorspace RGB Rec.709 gamma:22 range:FULL, bits per pixel 10
direct3d11 debug: using colorspace RGB Rec.709 gamma:22 range:FULL
direct3d11 debug: Using pixel format VA_NV12 for chroma DX11
main debug: VoutDisplayEvent 'resize' 8218x4391
direct3d11 debug: using GPU render fence
direct3d11 debug: Direct3D11 resources created
direct3d11 debug: Direct3D11 device adapter successfully initialized
direct3d11 debug: Direct3D11 Open Succeeded
main debug: using vout display module "direct3d11"
direct3d11 debug: D3D11 pool succeed with 40 surfaces (8192x4352) context 0x1f037f30
main debug: original format sz 8192x4320, of (0,0), vsz 8192x4320, 4cc DX11, sar 1:1, msk r0x0 g0x0 b0x0
main debug: removing module "freetype"
main debug: looking for text renderer module matching "any": 3 candidates
main debug: VoutDisplayEvent 'resize' 3840x1970
direct3d11 debug: Detected size change 3735x1970
freetype debug: Using DWrite backend
freetype debug: DWrite_GetFamily(): family name: Arial
freetype debug: DWrite_ParseFamily(): using font at index 0 with weight 400 for bold: 0, italic: 0
freetype debug: DWrite_ParseFamily(): using font at index 1 with weight 700 for bold: 1, italic: 0
freetype debug: DWrite_ParseFamily(): using font at index 2 with weight 400 for bold: 0, italic: 1
freetype debug: DWrite_ParseFamily(): using font at index 3 with weight 700 for bold: 1, italic: 1
main debug: using text renderer module "freetype"
main debug: looking for hw decoder module matching "none": 2 candidates
main debug: no hw decoder modules matched
avcodec debug: trying format dxva2_vld
main debug: removing module "direct3d11"
direct3d11 debug: Direct3D11 resources destroyed
direct3d11 debug: Direct3D11 device adapter closed
direct3d11 error: SetThumbNailClip failed: 0x800706f4
direct3d11 debug: Win32 Vout EventThread terminating
direct3d11 debug: Win32VoutCloseWindow
direct3d11 debug: WinProc WM_DESTROY

etc etc etc

main debug: removing module "freetype"
main debug: removing module "yuvp"
main debug: removing module "swscale"
main error: failed to create video output

Yes, I can reproduce the issue, the VLC window is constantly resized while it's playing fine with other players including FFplay.

You should create a new report ticket on the VLC Trac with your above report, the link to your sample file and the link to this thread:
https://code.videolan.org/videolan/vlc/-/issues