amd64 VLC fails to play mencoder-produced h264 movies

[liontooth]

I would like to file a bug report; the bug is reasonably well confirmed and tested. I'm posting it here rather than in the bug database because so instructed.

Summary:

In brief, the problem is that h264 files produced with mencoder don't display properly on Linux amd64. The picture is full of blocks to the point of looking milky. The sound is perfect.

Delimiting the problem:

xine and mplayer will play the same files perfectly on amd64. In addition, vlc on i686 plays the files perfectly -- the very same files that don't display correctly on amd64.

Finally, h264 files created with ffmpeg play perfectly fine with VLC on Linux amd64.

This is true of several (all I've tried) versions of VLC from Debian's 0.8.4 to the nightly build 0.8.5-svn20060304-0.

The problem appears to be triggered by mencoder's way of creating h264 files, and it is triggered only for VLC on Linux amd64.

Details:

When I encode a file with mencoder, using a script with these commands:

mencoder $1 -o /dev/null -oac copy -ovc x264 \
-x264encopts pass=1:bitrate=$2:bframes=3:subq=2:threads=2

echo "Starting second pass"
mencoder $1 -o $STEM-h264-$2.avi -oac mp3lame -lameopts cbr:br=64 -ovc x264 \
-x264encopts pass=2:bitrate=$2:bframes=3:subq=6:threads=2:frameref=4:8x8dct

I get a file that plays without problems in mplayer/kplayer, xine (1.1.1), and ffplay. In vlc, the sound is fine, but the video is all fuzzy, lots of small blocks, and unwatchable (8-second sample available on request -- I'm happy to send a strace).

If instead of mencoder I use this command in ffmpeg, vlc has no problems playing the resulting file:

ffmpeg -i $1 -f avi -vcodec h264 -b $2 -s 640x480 -acodec mp3 $OUTFILE

Does someone know what's going on? This is with vlc for amd64, nightly build wxvlc 0.8.5-svn20060304-0 on Debian sid.

I'd love to have this fixed, as I'd like to be able to stream the files produced by mencoder. BTW I asked on the mencoder list and they suggested it was a VLC problem -- which is plausible, since ffplay shows the mencoder file fine.

Please someone respond to this -- I had this posted on the Linux list for three weeks with no response. There's got to be a way to report a bug!

Dave

[The DJ]

Do all versions of the various players use the same ffmpeg version? Because it sounds like a 64bit problem in a specific part of the ffmpeg decoder that is only triggered in case you encode in this specific way.

[liontooth]

Hi DJ,

Let me see if I understand what you suggest may be happening. Is it the case that VLC uses ffmpeg to decode (play) h264 videos? In my Debian installation, VLC is not shown to depend on ffmpeg -- and that's using videolan's own nightly builds, not the official Debian version.

If this is the case, your hypothesis is plausible, but I'm not sure how to test it -- which different versions of ffmpeg may be worth testing? What can I do to narrow down the conditions that trigger this bug?

Dave