RC3 H264 encoding quality worse than 0.86i with low bitrates

[thomasferre]

Hi

I have already posted in the VLC 1.0 section but with no solution for the moment ...

I am happily using VLC for several years to do live H264 multicast streaming.

I used to use 0.86i with no pb with my personnal H264 encoding parameters.

I upgraded to new RC3.

The new X264 library with RC3 is using a lot less CPU than the 0.86i (which is a good thing) but produces poorer quality with low bitrates (<=512 kb/s).

This quality pb never happen in RC3 with higher bitrates.

After some tests, it seems that this pb appears in RC3 ONLY with bitrates <512 kb/s.

Does anyone met the same pb ?

Increasing the subme to 7 or 8 or 9 with 1.0 RC3 improves encoding quality with bit rates between 512 kb/s and 1000 kb/s.

But the pb remains with VERY low bitrates.

For example, with a 384 kb/s H264 stream, VLC 0.86i produces a much better quality image with the same parameter (except the subme at 7 for 0.86i and at 9 for 1.0 RC3).

Is there any other parameter that I can change in 1.0 RC3 H264 to improve it ?

You can do the test :

try to encode with H264 at 384 kb/s with 0.86i and 1.0 RC3 and you will see a BIG quality difference.

Thanks for your help.

Regards

[ILEoo]

could you post your whole transcode-block?

[thomasferre]

Hi

Sorry but I don't understand what you call my "whole transcode-block"

Is it my X264 configuration ?

I am using the same X264 configuration with VLC 1.0 RC3 and VLC 0.86i.

As you can see, with a 384 kb/s bit-rate, there is a BIG quality difference ...

VLC 0.86 i :


VLC 1.0 RC3 :


Once again, the quality difference is only happening with very LOW bitrates (<512 kb/s).

With high bitrates, X264 encoding with 0.86i and 1.0 RC3 are the same quality.

Thanks for your help.

[ILEoo]

Yes, I mean your X264 configuration. Also if you haven't compiled those versions yourself (eg encoding in windows or mac) x264 version has changed too.

[thomasferre]

I am using the Windows version of VLC downloaded from videolan.org.

Here is my X264 configuration file in VLC :

All default except

sout-x264-partitions and sout-x264-direct set to none to save cpu

I am using the same X264 configuration with VLC 0.86i and 1.0 RC3

--

[x264] # Encodeur vidéo H264/MPEG4 utilisant la bibliothèque x264

# integer (Taille maximale du GOP)
#sout-x264-keyint=250

# integer (Taille minimale du GOP)
#sout-x264-min-keyint=25

# integer (Agressivité supplémentaire des images I)
#sout-x264-scenecut=40

# integer (Images B entre images I et P.)
#sout-x264-bframes=0

# integer (Utilisation adaptative d’images B)
#sout-x264-b-adapt=1

# integer (Biaiser l'utilisation des images B)
#sout-x264-b-bias=0

# boolean (Garder des images B en références)
#sout-x264-bpyramid=0

# boolean (CABAC)
#sout-x264-cabac=1

# integer (Nombre d’images de référence)
#sout-x264-ref=1

# boolean (Pas de filtre de boucle)
#sout-x264-nf=0

# string (Paramètres AlphaC0 et Beta du filtre de boucle)
#sout-x264-deblock=0:0

# string (Niveau H.264)
#sout-x264-level=5.1

# boolean (Mode de désentrelacement)
#sout-x264-interlaced=0

# integer (QP)
#sout-x264-qp=26

# integer (VBR par qalité)
#sout-x264-crf=0

# integer (QP minimum)
#sout-x264-qpmin=10

# integer (QP max)
#sout-x264-qpmax=51

# integer (Saut de QP maximum)
#sout-x264-qpstep=4

# float (Tolérance moyenne sur le débit)
#sout-x264-ratetol=1.000000

# integer (Débit maximum local)
#sout-x264-vbv-maxrate=0

# integer (Buffer VBV)
#sout-x264-vbv-bufsize=0

# float (Remplissage initial du buffer VBV)
#sout-x264-vbv-init=0.900000

# float (Facteur de QP entre I et P)
#sout-x264-ipratio=1.400000

# float (Facteur de QP entre P et B)
#sout-x264-pbratio=1.300000

# integer (Différence de QP entre chrominance et luminance)
#sout-x264-chroma-qp-offset=0

# integer (Controle de vitesse multi-passes)
#sout-x264-pass=0

# float (Compression dynamique de QP)
#sout-x264-qcomp=0.600000

# float (Réduire les fluctuations de QP)
#sout-x264-cplxblur=20.000000

# float (Réduire les fluctuations de QP)
#sout-x264-qblur=0.500000

# integer (Distributions des bits par AQ)
#sout-x264-aq-mode=2

# float (Force pour AQ)
#sout-x264-aq-strength=1.000000

# string (Partitions à considérer)
sout-x264-partitions=none

# string (Mode de prédiction des vecteurs de mouvement directs)
sout-x264-direct=none

# integer (Taille de prédiction des vecteurs de mouvement directs)
#sout-x264-direct-8x8=-1

# boolean (Prédiction pondérée pour les images B)
#sout-x264-weightb=0

# string (Méthode d’estimation des mouvements)
#sout-x264-me=hex

# integer (Distance maximale de recherche)
#sout-x264-merange=16

# integer (Distance maximale de recherche)
#sout-x264-mvrange=-1

# integer (Minimiser la taille des tampons entre les threads)
#sout-x264-mvrange-thread=-1

# integer (Qualité de l’estimation de mouvement sous-pixel et de la décision de partition)
sout-x264-subme=7

# boolean (Décider des références pour chaque partition)
#sout-x264-mixed-refs=0

# boolean (Chroma dans l’estimation de mouvement)
#sout-x264-chroma-me=1

# boolean (Taille de la transformation spatiale adaptative)
#sout-x264-8x8dct=0

# integer (Quantisation treillis RD)
#sout-x264-trellis=0

# boolean (Détection SKIP précoce sur images P)
#sout-x264-fast-pskip=1

# boolean (Coéfficient de seuil sur les images P)
#sout-x264-dct-decimate=1

# integer (Résolution de bruit)
#sout-x264-nr=0

# integer (Inter luma quantization deadzone)
#sout-x264-deadzone-inter=21

# integer (Intra luma quantization deadzone)
#sout-x264-deadzone-intra=11

# boolean (Non-deterministic optimizations when threaded)
#sout-x264-non-deterministic=0

# boolean (Optimisations CPU)
#sout-x264-asm=1

# boolean (Calcul PSNR)
#sout-x264-psnr=0

# boolean (Calcul SSIM)
#sout-x264-ssim=0

# boolean (Mode « calme »)
#sout-x264-quiet=0

# integer (SPS and PPS id numbers)
#sout-x264-sps-id=0

# boolean (Access unit delimiters)
#sout-x264-aud=0

# boolean (Statistiques)
#sout-x264-verbose=0

# string (Nom du fichier de stats en mode 2 passes)
#sout-x264-stats=x264_2pass.log
--

[ILEoo]

could you also include your transcode-settings, no need to include defaults.

[thomasferre]

All default except :

# integer (Nombre de threads)
sout-transcode-threads=4

# boolean (Priorité élevée)
sout-transcode-high-priority=1

I am using a dual core.

I tried with sout-transcode-high-priority=0 and sout-transcode-threads=0 but the pb remains ...

[ILEoo]

you don't define bitrate there at all, so default is 800k, how do you encode those <512kbps streams then?

[thomasferre]

In fact, I am using a batch file :

start vlc.exe dshow:// :dshow-vdev="Osprey-100 Video Device 1" :dshow-fps=25.000 :dshow-adev="SigmaTel Audio" :sout=#transcode{vcodec=h264,vb=384,scale=1,acodec=mp4a,ab=64,channels=1,samplerate=44100}

[ILEoo]

did you actually check that bitrate keeps that 384? as default for x264 bitrate is only average-bitrate, not cbr-style top.

[thomasferre]

Yes, when I stream, I can see that this is really a 384 kb/s stream

[thomasferre]

Nobody is interessed by this subject ?

[elpechos]

I just upgraded from a version less than 1.0.0 to the latest goldeneye release

I noticed the h264 quality was worse at low bitrates and was googling for a solution.

You can notice it in particular if you encode one video with theora at 400kbits or so and one with h264 using the latest VLC

The H264 version is markedly worse than the theora encode -- which isn't the case on previous VLC versions.

[Rémi Denis-Courmont]

It would help more if people were actually sending patches to fix the default settings of the x264 plugin instead of whining at length here.

[elpechos]

I'm not particularly certain it is a matter to do with the default settings
I've tried doing encodes with very aggressive/cpu intensive x264 settings and the performance was still surprisingly poor

Perhaps some settings aren't being passed to x264 and the encode has poor rate control or such?

[thomasferre]

Hello

Like elpechos, I tried A LOT of different X264 encoding parameters with VLC 1.0 but none are producing a decent quality video.

I went to the VLC IRC channel to tell the pb.

Some people there told me : "go to X264 IRC channel"

I did.

Some people at the X264 IRC channel told me that they are "not responsible for VLC to --please stay polite-- thing up with X264" and advise me to go back to VLC IRC channel ...

Like I said, I am happily using VLC for about 5 years.

I really like this piece of software.

But denying X264 enconding algo pb is not very constructive ...

So I get back to 0.86i which is producing a very good H264 encoded video (but with a heavy cpu usage for live stream)

I notice a strange behavior with X264 encoding with VLC 1.0 :

the quality is ok for a few minutes then it gets progressively worse and worse and it never come back to good.

To see that, you need to encode live for a few minutes and displaying live the encoded video.

Did you met the same pb ?

[aka]

I got the same problem with you. I don't think x264 settings should be blamed.
I have compiled 0.8 and 1.0 with the same version of x264, 0.8 doesn't lose quality while 1.0 behave the same as official release.
I checked the settings in x264.c and found no important changes.
After v0.9,Transcoding with x264 always crashes(maybe just under windows?) until recent releases. I can't figure out what happens between those minor versions.

[ILEoo]

I have tested h264 encoding on current master, and I'm really unable to reproduce this quality-drop you seem to suffer.

Could you tell me your whole sout-config (eg do you stream it, or file it etc). And if you use linux or os X os Windows.

[aka]

Windows Version.

But when I build vlc git version with x264 which 0.8 used,it gets the same quality as 0.8 version.

I found that qp is the cause of the quality penalty .

With the same x264 config options in vlc, new version transcode with the average qp of 40 while old version is 25. That make new version produce much worse quality with larger bitrate.

So can someone finger out which config options of x264 make the difference?

[ILEoo]

Could you try nightly build master (trunk) builds on windows too. Currently vlc doesn't touch the qp if user hasn't requested qp-mode.

[aka]

With slightly improvement,windows nightly build version still has the problem.

I just tried 1.0.2-git version on linux, it doesn't have such problem.

Here is my command line and results

Code: Select all

vlc.exe --intf rc -vvv d:\temp\output.avi --sout=#transcode{venc=x264{me=hex,weightb=0,partitions=normal,ratetol=1.0,merange=16,subme=5,bframes=0,b-bias=0,ref=1,verbose},vcodec=h264,vb=600,scale=1,thread=4,fps=29.97}:duplicate{dst=std{access=file,mux=mp4,dst=d:\test.mp4}}

results:

source: 1 min part of beijing olympic mtv
download here http://218.240.151.140/output.7z
windows version
0.8 release
x264 [info]: slice I:8 Avg QP:24.25 size: 12399
x264 [info]: slice P:1790 Avg QP:29.02 size: 2481
x264 [info]: mb I I16..4: 69.4% 0.0% 30.6%
x264 [info]: mb P I16..4: 5.6% 0.0% 1.5% P16..4: 18.6% 8.2% 1.5% 0.0% 0
.0% skip:64.5%
x264 [info]: kb/s:605.4

1.0 release
x264 [info]: slice I:17 Avg QP:43.10 size: 3005
x264 [info]: slice P:1779 Avg QP:47.65 size: 2503
x264 [info]: mb I I16..4: 92.8% 0.0% 7.2%
x264 [info]: mb P I16..4: 22.6% 0.0% 0.1% P16..4: 69.9% 5.9% 1.3% 0.0% 0
.0% skip: 0.3%
x264 [info]: coded y,uvDC,uvAC intra:1.1% 17.6% 0.5% inter:0.3% 3.8% 0.0%
x264 [info]: kb/s:601.3

nightly build
x264 [info]: slice I:17 Avg QP:41.61 size: 3629
x264 [info]: slice P:1779 Avg QP:46.69 size: 2521
x264 [info]: mb I I16..4: 91.1% 0.0% 8.9%
x264 [info]: mb P I16..4: 22.6% 0.0% 0.1% P16..4: 69.6% 6.1% 1.3% 0.0% 0
.0% skip: 0.3%
x264 [info]: coded y,uvDC,uvAC intra:1.5% 20.8% 0.5% inter:0.8% 4.7% 0.0%
x264 [info]: kb/s:606.9

1.0.2-git on linux
x264 [info]: slice I:17 Avg QP:21.68 size: 17148
x264 [info]: slice P:1781 Avg QP:26.86 size: 2337
x264 [info]: mb I I16..4: 56.4% 0.0% 43.6%
x264 [info]: mb P I16..4: 5.6% 0.0% 1.6% P16..4: 20.7% 8.7% 1.4% 0.0% 0.0% skip:62.0%
x264 [info]: coded y,uvDC,uvAC intra:26.7% 43.6% 8.6% inter:7.5% 6.2% 0.2%
x264 [info]: kb/s:593.8

[ILEoo]

Did you do compile in linux against contrib-tree and it's x264, or did you have some external x264 (and if so, what version).

As on vlc <-> x264 it doesn't check if encoder is windows or linux, so in that sens there shouldn't be difference because of that.

[aka]

The versions which I list in the results are from official binary(linux version comes from ubuntu ppa),not my personal builds .

[ILEoo]

You didn't get any different log-messages on windows? Also could you try with threads=1 in windows to see if it has any effect on quality (no thread, threads) and with psnr option on x264.

[globaltrasher]

Same problem here. My 350k H.264+AAC live TV stream looks much worse when created with 1.0.1 with the same x264 settings as with 0.86i. On several occasions I've also seen 1000k streams to be OK for a a minute or so, and then suddenly drop significantly in quality.

However I noticed that under Preferences -> Input/Codecs -> Video codecs -> X264 there are several new configuration settings in 1.0.1, compared to 0.86i. Hopefully this may help us figure this problem out:

Multipass ratecontrol = 0
Multipass ratecontrol:
- 1: First pass, creates stats file
- 2: last pass, does not overwrite stats file
- 3: Nth pass, overwrites stats file

How AQ attributes bits = 2
Defines bitdistribution mode for AQ, default 2
- 0: Disabled
- 1: Avoid moving bits between frames
- 2: Move bits between frames

Strength of AQ = 1.00
Strength to reduce blocking and blurring in flat and textured areas, default 1.0, recommended to be between:
- 0.2-0.5: weak AQ
- 1.5: strong AQ

Maximum motion vector length = -1
Maximum motion vector length in pixels. -1 is automatic based on level

Minimum buffer space between threads = -1
Minimum buffer space between threads. -1 is automatic based on number of threads

Non-deterministic optimizations when threaded = 0
Slightly improve quality of SMP, at the cost of repeatability

Filename for 2 pass stats file
Filename for 2 pass stats filefor multipass encoding

I'm thinking, something about AQ...? Playing with this setting now...