RC3 H264 encoding quality worse than 0.86i with low bitrates

[globaltrasher]

I think I got it FIXED!!!

Instead of above default values, set:

How AQ attributes bits = 1
Strength of AQ = 0.50

Does that fix the issue for anyone else?

[kdh]

i'll test your settings with what i'm doing over here: viewtopic.php?f=4&t=60947&start=15

and I'll report back. thanks for working on this!

[kdh]

FYI, its not How AQ attributes bits = 1

its How AQ Distrubutes Bits = 1.

But yea! I was able to change my frame rates from 15 to 30, and the over all quality looks really good. even tho I'm using such low VB rates I'm very happy with the quality now!

I will be doing a live test tomorrow(monday) August 10th from 9pm till Midnight Cen time. I'll post the link.

[ILEoo]

Ah, nice if you have been able to pinpoint something to affect on your issue. Could you try to either use aq-strenght or aq-mode (only one of them at the time) to see if only changing one of them helps.

[ILEoo]

Vlc did use old default on x264 aq-mode, but it should be fixed on next release (and next nightly build),
thou it doesn't really explain why it showed up only on windows side, and quality issue ain't on linux side.

Also because of that, I'm not at all sure if that was really the problem.

[kdh]

Ah, nice if you have been able to pinpoint something to affect on your issue. Could you try to either use aq-strenght or aq-mode (only one of them at the time) to see if only changing one of them helps.

When I get home from work I will test this.

[kdh]

I can't seem to find an aq-mode setting. Are you really talking about Direct MV prediction mode?

[ILEoo]

I can't seem to find an aq-mode setting. Are you really talking about Direct MV prediction mode?

No, AQ-mode,

Code: Select all

--sout-x264-aq-mode

[kdh]

ah-ha! We've got 3 live shows tonight, and none of those guys do video streaming. I'll have to test tomorrow.

[kdh]

ive been using these settings windows side for my project

:sout-x264-aq-mode=0 :sout=#transcode{vcodec=h264,vb=160,fps=15,scale=0.5,acodec=mp3,ab=96,channels=2,samplerate=44100,audio-sync,venc=x264{keyinit=15}}:std{access=udp,mux=ts,dst=everydayjunglist.org:8777}

and it works well. the edges of my frame get easily distorted but the center looks good. you can see what im talking about on my home page

http://www.everydayjunglist.org look on the right corner of our home page to see the edj tv sneak peak beta to see the video i'm talking about.

thats me playing by the way.. that was a live show i did last wednesday.

[dvdeurse]

I don't think that the right aq-mode setting solves this problem. Even after disabling aq-mode, I still get the problem mentioned in this thread. The configuration I've used is as follows:
* VLC 1.0.1 and VLC 0.9.8i (both Windows versions)
* capture stream from a direct show filter
* transcode this stream into H.264/AVC, with a bit rate of 500 kbit/s

This is the command I used:

Code: Select all

vlc.exe -vvv -Idummy --reset-config dshow:// :dshow-vdev="Osprey-5X0 Video Device 2" :dshow-adev="none" :dshow-size="768x432" :dshow-fps=25 :dshow-caching=500 :sout=#transcode{vcodec=x264,vb=500,threads=4,venc=x264{keyint=32,min-keyint=32,bframes=0,ref=1,aq-mode=0}}:std{access=file,mux=es,dst=out.264}

A picture with a comparison and analyze of both encoded files can be seen at http://multimedialab.elis.ugent.be/user ... arison.jpg. As you can see, the visual quality of version 1.0.1 is unacceptable. Version 1.0.1 seems to have a problem with the rate controller: when you push the encoder a little bit (i.e., encode a complex scene with a given bit budget that is lower than the bits the encoder would spend in VBR setting), it starts allocating very little bits to I frames (as can be seen from the analyze, where the red lines are the sizes of the I frames, the blue lines the size of the P frames). The latter results in a very bad visual quality. Note that I tested the same settings with an x264 stand-alone encoder and the visual quality was perfect. So, there seems to be a problem with the steering of the rate controller of the x264 encoder within VLC ...

Any suggestions?

[kdh]

I've spent a lot of time reading the wiki about command line options. Add this below to your stream:

--sout-ffmpeg-qmin=10 --sout-ffmpeg-rc-buffer-size=20000 --sout-ffmpeg-qmax=30 --sout-ffmpeg-qscale=1 --sout-ffmpeg-luma-elim-threshold=-4 --sout-ffmpeg-chroma-elim-
threshold=7

I've using it in my project and I've gotten really good gains.

hope that helps.

[ILEoo]

I don't think that the right aq-mode setting solves this problem. Even after disabling aq-mode, I still get the problem mentioned in this thread. The configuration I've used is as follows:
* VLC 1.0.1 and VLC 0.9.8i (both Windows versions)
* capture stream from a direct show filter
* transcode this stream into H.264/AVC, with a bit rate of 500 kbit/s

. Note that I tested the same settings with an x264 stand-alone encoder and the visual quality was perfect. So, there seems to be a problem with the steering of the rate controller of the x264 encoder within VLC ...

Any suggestions?

Did you test that x264 stand-alone encode dated same as vlc 1.0.1 release? as x264 has fixed some bugs/got improvements meanwhile. So could you try nightly build trunk and compare it to current x264 stand-alone encoder instead.

[ILEoo]

I've spent a lot of time reading the wiki about command line options. Add this below to your stream:

Code: Select all

--sout-ffmpeg-qmin=10 --sout-ffmpeg-rc-buffer-size=20000 --sout-ffmpeg-qmax=30 --sout-ffmpeg-qscale=1 --sout-ffmpeg-luma-elim-threshold=-4 --sout-ffmpeg-chroma-elim- threshold=7

I've using it in my project and I've gotten really good gains.
hope that helps.

Can you paste some 'without' and 'with' sample pictures of those settings, and if they really do clear difference (on nightlies) I don't see why we couldn't change ffmpeg-encoder module defaults. Those don't really affect at all on h264 encoding, so in that part it's kinda offtopic.

[dvdeurse]

Did you test that x264 stand-alone encode dated same as vlc 1.0.1 release? as x264 has fixed some bugs/got improvements meanwhile. So could you try nightly build trunk and compare it to current x264 stand-alone encoder instead.

Here is a new comparison: http://multimedialab.elis.ugent.be/user ... arison.jpg.
The versions that I used:
* x264 rev 1259 (http://mirror01.x264.nl/x264/revision1259/x264.exe)
* VLC branch-20090914-0005 (http://nightlies.videolan.org/build/win ... -win32.exe

For the record: the x264 command was

Code: Select all

x264.exe --keyint 32 --min-keyint 32 --bframes 0 --ref 1 --bitrate 500 --aq-mode 0 -o out.264 input.yuv 720x432

As you can see, the problem is still there ...

[kdh]

I've spent a lot of time reading the wiki about command line options. Add this below to your stream:

Code: Select all

--sout-ffmpeg-qmin=10 --sout-ffmpeg-rc-buffer-size=20000 --sout-ffmpeg-qmax=30 --sout-ffmpeg-qscale=1 --sout-ffmpeg-luma-elim-threshold=-4 --sout-ffmpeg-chroma-elim- threshold=7

I've using it in my project and I've gotten really good gains.
hope that helps.

Can you paste some 'without' and 'with' sample pictures of those settings, and if they really do clear difference (on nightlies) I don't see why we couldn't change ffmpeg-encoder module defaults. Those don't really affect at all on h264 encoding, so in that part it's kinda offtopic.

Hmm.. Its a little hard as I'm not saving my older streams. I can tell that using --sout-ffmpeg-luma-elim-threshold=-4 --sout-ffmpeg-chroma-elim-
threshold=7 made the biggest improvements with my settings.

[ILEoo]

[
Here is a new comparison: http://multimedialab.elis.ugent.be/user ... arison.jpg.
The versions that I used:
* x264 rev 1259 (http://mirror01.x264.nl/x264/revision1259/x264.exe)
* VLC branch-20090914-0005 (http://nightlies.videolan.org/build/win ... -win32.exe

For the record: the x264 command was

Code: Select all

x264.exe --keyint 32 --min-keyint 32 --bframes 0 --ref 1 --bitrate 500 --aq-mode 0 -o out.264 input.yuv 720x432

As you can see, the problem is still there ...

Did you use same parameters with vlc also eg

Code: Select all

#transcode{venc=x264{keyint=32,bframes=0,ref=1,aq-mode=0},vb=500}

?

[dvdeurse]

Did you use same parameters with vlc also eg

Code: Select all

#transcode{venc=x264{keyint=32,bframes=0,ref=1,aq-mode=0},vb=500}

?

Yes I did:

Code: Select all

vlc.exe -vvv -Idummy --reset-config dshow:// :dshow-vdev="Osprey-5X0 Video Device 2" :dshow-adev="none" :dshow-size="768x432" :dshow-fps=25 :dshow-caching=500 :sout=#transcode{vcodec=x264,vb=500,venc=x264{keyint=32,min-keyint=32,bframes=0,ref=1,aq-mode=0}}:std{access=file,mux=es,dst=out.264}

[ILEoo]

Could you also try same with nightly trunk build?

[dvdeurse]

Same result (i.e., bad quality) with the nightly trunk build (trunk-20090916-2205): http://multimedialab.elis.ugent.be/user ... _trunk.jpg

[dvdeurse]

I did also the test (i.e., same VLC parameters as described above) with the linux version of VLC (1.0.1). Surprisingly, the linux version does not have this problem: http://multimedialab.elis.ugent.be/user ... _linux.jpg. Maybe something goes wrong during the cross-compilation to Windows?

[Jean-Baptiste Kempf]

I did also the test (i.e., same VLC parameters as described above) with the linux version of VLC (1.0.1). Surprisingly, the linux version does not have this problem: http://multimedialab.elis.ugent.be/user ... _linux.jpg. Maybe something goes wrong during the cross-compilation to Windows?

O_o

[dvdeurse]

Is there anyone that can confirm these problems with the windows version of VLC 1.01?

[ILEoo]

There have been people complaining about windows side quality on x264 before, and we're checking that possible crosscompile bug. But as VLC doesn't have that many windows developers, it takes littlebit time (and as 1.0.2 is coming out soon (Tm), it takes time too).

[DGMurdockIII]

i can on win 7