Hi,
you might also already have seen: https://developer.apple.com/videos/play/fall2017/503/
So with tvOS 11.2 with an AppleTV 4K connected to an UHD/HDR TV-Set, it is possible to change the display settings to provide best quality playback, like using HDR when the source offers it.
I think, it would be best to do this inside TVVLCKit or maybe better libVLC.
Any plans for this?
I would try to help, but I do not own a AppleTV 4K.