Huh, it is not cool, that there is no answer for this problem. Maybe I can give You some extra information...
1. I noticed, that peak problem is connected to dynamic screen changes. I think that encoder tries to keep fps and quality, that why there is a peak.
Peak are not very often, I can notice them at decoder as disorder on screen (squares on picture, wheeze sound). Also, peaks are very short, but annoying.
2. Cpu usage is always at the same level - 30-40%. It's P4 2.8Ghz with 512MB RAM, and it is only doing encoding. System is Debian Unstable, vlc 0.9.0-svn (deb build), but the same problem was on 0.85 series.).
3. Regarding documentation there are two possibilities:
venc=ffmpeg allows to set the encoder to use, where:
*keyint=80 is the maximal amount of frames between two key frames
*hurry-up allows the encoder to decrease the quality of the stream if the CPU can't keep up with the encoding rate
*vt=800000 is the tolerance in kbit/s for the bitrate of the outputted video
--> I tried it, it does not help.
or
bitrate=3000000 is the average bitrate of the stream ,
maxbitrate=4000000 is the maximum bitrate of the stream
-->but I think this works only for hardware encoding cards...
Please help. This is simple video IN stream, from analog source, and I just can't keep it stable, I fought, that vb parameter is enough... I have only 4Mbit upload, so when the peak is happens there is disaster at decoding site.