Artifacts/pixelation in playback

[DaveQ]

Hello all,

I've done both a Google and forum search but was unable to find this particular complaint. I'm surprised by this as I would have thought it would be something that's been covered before, but maybe my searches didn't use the right terms or something. In any case, I apologize if this has been covered before and would be happy with a redirect to the previous topic if that's the case.

Anyway, I have noticed that playback quality of H.264 encoded videos with VLC is often inferior to playback with other tools, due to video artifacts/pixelation in VLC. The same exact files played back with other programs (QuickTime Player, PLEX, etc) do not exhibit this behavior. I notice it most often when I'm playing back movies I ripped from my own collection and converted with HandBrake. For example, this is a screenshot of what I get with VLC:



This is the exact same file played back in QuickTime player:



And this is in the old QuickTime 7 player:



I noticed this in VLC 1.X, and so I tried updating to the latest (2.0.5). Same behavior. I'm currently running Mountain Lion (10.8.2), but noticed this originally when I was on Snow Leopard (10.6.X).

Any suggestions?

Thanks,
Dave

[fkuehne]

Is this limited to a certain set of files?

Chances are high that the other playback software you mentioned are using hardware acceleration for full-HD H264 contents. This is something, VLC does not do at this point. However, we are going to add in our next major release 2.1, which will be out late this spring (let's say April - without warranties). In fact, this should solve your issue.

[DaveQ]

I have noticed this on certain files, and thinking back on it, they're all HD content. One example is when I ripped/converted my Prometheus blu-ray. There's an early scene in a dark cave; one of the characters is waving a flashlight around. As the beam passes in front of the camera there's a moment of severe pixelation similar to what's shown in the Pixar screen capture above. I was using VLC to spot check the encode to make sure it looked good before moving it to my Plex server. When I saw the pixelation I just thought I had used too low of a quality setting in HandBrake, and re-encoded at a higher setting. This produced a larger file but the pixelation was still there when I checked it in VLC. So I adjusted the quality even higher. I kept doing this until the pixelation in that scene was minimized. It was hardly noticeable, but it was still there, and these settings resulted in an absurdly large file with a much higher average/peak bitrate than I had used on any other movie.

So I moved the biggest file to my Plex server, and then checked it on my television (using Plex for playback). The pixelation that was minor but still visible when played back in VLC was completely gone. That seemed weird, so I went back and checked the next smaller file, viewing it with Plex instead of VLC. No pixelation. I went back to the original file -- the first one, made with the original settings, and which showed severe pixelation in that one scene with VLC --and it also had no pixelation when played back with Plex.

After this I stopped relying on VLC to spot-check my encodes, using QuickTime player or Plex instead. Digging through my files, I found a couple other examples like this, where VLC would show artifacts and the others didn't. But in each of those examples, the scene presented complex action or (as in that one scene in Prometheus) a dark scene with something bright moving through it.

This Pixar example is not a complex scene... the portion of the frame with the artifact is just a blue field with a minor gradient. So I thought I would post the samples here and ask about it. What puzzles me is that if it's due to a lack of hardware acceleration, why did increasing the bit rate for Prometheus improve the situation rather than make it worse?

I don't mean to be critical. I like VLC and would prefer to use it over QuickTime player and for spot checks before moving my movies to my Plex server. Hopefully the 2.1 update will fix this issue.

Thanks,
Dave