h264 display resolution 2 pixels taller = green band

[fourpointsix]

VLC 2.2.4 (Windows) is interpreting an h264 video as being 2 pixels taller than it is, resulting in a green line a the bottom of the video. I've searched google and this board and while I've found posts that touch on similar problems, I cannot find a clear explanation or a solution.

Context
I transcoded a DVD MPEG-2 704x480 interlaced (top field first) 16:9 video to h264 in an MKV container using ffmpeg N-83882-g58f0bbc (yesterday's build) with no scaling or resizing or cropping filters so the output resolution should equal the input resolution.

Transcoding command:

Code: Select all

ffmpeg -i Original.vob -t 10 -an -c:v h264 -b:v 2500k -flags +ildct+ilme -pix_fmt yuv420p h264.mkv

ffprobe MPEG-2 input video:

Code: Select all

Stream #0:1[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, smpte170m, top first), 704x480 [SAR 40:33 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

ffprobe transcoded h264 video:

Code: Select all

Stream #0:0: Video: h264 (High), yuv420p(top coded first (swapped)), 704x480 [SAR 40:33 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)

Problem
The original VOB plays in VLC in the correct resolution with no artifacts but the transcoded file has a green line at the bottom of the video and VLC reports its resolution as this:

Resolution: 704x482
Display resolution: 704x480

The resolution is two pixels too tall. I've been reading about "resolution" vs. "display resolution" and the consensus seems to be that the "resolution" is really an internal buffer resolution. Since this buffer is 2 pixels greater than the input resolution, I assume the remaining pixels are left as zeroes. In YUV zero equals a shade of green which would explain the green band artifact.

So my question is, why is my file being decoded using an incorrectly-sized buffer and is there a way to fix it? I don't know if this is a problem with the input file or a bug in VLC's h264 decoder or its renderer.

Troubleshooting

• I tried an MP4 container to rule out MKV as the cause. It exhibited the same problem.
• I transcoded to MPEG-2 and it reported the correct "resolution" and "display resolution" (704x480)
• I transcoded to h265 and it reported the correct "resolution" and "display resolution" (704x480)

VLC debug messages
This is just a small sampling of the debug messages. They're the ones that seemed most relevant to the problem. Note that the only mention of the incorrect resolution is a core debug line showing "original format sz 704x482".

Code: Select all

mkv debug: | | | + Track Video mkv debug: | | | | + width=704 mkv debug: | | | | + height=480 mkv debug: | | | | + Track Video Interlaced=1 mkv debug: | | | | + Unknown (N7libebml9EbmlDummyE) mkv debug: | | | | + display width=16 mkv debug: | | | | + display height=9 . swscale debug: 32x32 (32x32) chroma: YUVA -> 16x16 (16x16) chroma: RGBA with scaling using Bicubic (good quality) core debug: using video filter2 module "swscale" core debug: looking for video filter2 module matching "any": 58 candidates yuvp debug: YUVP to YUVA converter core debug: using video filter2 module "yuvp" core debug: Deinterlacing available core debug: deinterlace -1, mode linear, is_needed 0 . direct3d debug: Direct3d Device: AMD Radeon HD 5670 4098 26840 0 core debug: VoutDisplayEvent 'resize' 869x518 window direct3d warning: trying surface pixel format: YV12 direct3d debug: selected surface pixel format is YV12 . core debug: using vout display module "direct3d" core debug: original format sz 704x482, of (0,0), vsz 704x480, 4cc I420, sar 40:33, msk r0x0 g0x0 b0x0

If more information is needed to track down the cause or a solution I'll be glad to provide it. Thank you!

[Jean-Baptiste Kempf]

core debug: original format sz 704x482, of (0,0), vsz 704x480, 4cc I420, sar 40:33, msk r0x0 g0x0 b0x0

This seems correct. Try VLC 3.0.0 from nightly builds, please.

[fourpointsix]

The green band is no longer present in the latest nightly release (3.0.0-20170313-0506). Thank you!

It still shows "Buffer dimensions: 704x482" though. Obviously it's an internal thing and doesn't affect the output anymore but I'm curious why it's being processed at those dimensions. It seems only some h264 videos exhibit this difference between display and buffer dimensions. It's especially odd to me because 482 is not divisible by 4.