transcode video process question regarding b_need_duplicate

[haydenm315]

I've been tracking down an issue with regards to maintaining near constant bitrate transcoding from mpeg4 to x264. No matter what options I set for x264, I see large spikes in bitrate. X264 logging indicates that the output bitrate is correct. P and I frame placement looks ok from X264's perspective. Running ffprobe on the stream being output shows periods where I'm receiving I frames at a very high rate. This indicates that vlc is doing something in the middle.

I found a code block in /modules/stream_out/transcode/video.c that I'm trying to understand. The code exists in the transcode_video_process function. My understanding of the below code is that in the unlikely event that video has drifted 50ms, don't send the picture, or in the unlikely case that the video is 50ms too early, send a duplicate. My observation is that the conditional which causes duplicates to be sent, causes a number of problems. When this occurs, a spurt of duplicates are sent for a number of seconds or more. This causes the bitrate to increase significantly. We also packetize for HLS, and this seems to break streaming to the latest generation of apple devices. If I comment out the conditional which sets the duplicate flag, bitrate is correct, and streaming works on the latest generation of apple devices. Can someone better explain the conditional below? What is the purpose of the duplicate flag?

Code: Select all

if( unlikely( i_video_drift < (i_master_drift - 50000) ) ) { #if 0 msg_Dbg( p_stream, "dropping frame (%i)", (int)(i_video_drift - i_master_drift) ); #endif picture_Release( p_pic ); continue; } else if( unlikely( i_video_drift > (i_master_drift + 50000) ) ) { #if 0 msg_Dbg( p_stream, "adding frame (%i)", (int)(i_video_drift - i_master_drift) ); #endif b_need_duplicate = true; }

[haydenm315]

Anybody? This code wrecks havoc with x264 constant bitrate. It also screws up HLS streaming badly. These few lines of code have caused some major problems over here. Respond here, or deal with me in GIT when I blow these lines away.

[Rémi Denis-Courmont]

The purpose is to keep the frame rate within certain boundaries in case of drift.

[haydenm315]

Thanks for the response which has helped me understand what Vlc's doing here better than before. I've been studying the decode, transcode, and encode sections of code and have noticed some significant issues with regards to transcoding out at a much lower framerate than the input. If input is 30fps and output is 15fps, things seem to work well. The following code block drops every other decoded picture that have arrived too soon, and we proceed to the next picture. At 30fps, each frame is displayed for 1/30 a second. I'm seeing drifts between -26000 and -60000. Every other frame is dropped, and all is mostly well.

Code: Select all

if( unlikely( i_video_drift < (i_master_drift - 50000) ) ) { #if 0 msg_Dbg( p_stream, "dropping frame (%i)", (int)(i_video_drift - i_master_drift) ); #endif picture_Release( p_pic ); continue; }

If I try to reduce the transcode output fps further down to a value of 5, things get really messed up. The following code block has issues in this scenario. It's extremely likely that the p_pic->date - i_pts < -MASTER_SYNC_MAX_DRIFT, so sync is always being reset. 25 pictures a second should be dropped, but instead they are causing sync resets because they satisfy this conditional. When this occurs, the function no longer drops pictures it doesn't need, but rather adds duplicate frames due to the i_pts update from the picture. This prevents the correct output fps and results in a massive bitrate increase.

Code: Select all

if ( unlikely( p_pic->date - i_pts > MASTER_SYNC_MAX_DRIFT || p_pic->date - i_pts < -MASTER_SYNC_MAX_DRIFT ) ) { msg_Dbg( p_stream, "drift is too high, resetting master sync" ); date_Set( &id->interpolated_pts, p_pic->date ); i_pts = p_pic->date + 1;

I know that increasing the MASTER_SYNC_MAX_DRIFT to prevent sync resets to account for difference in 30fps input and 5fps output seems to work, but I'm assuming the MASTER_SYNC_MAX_DRIFT is set to a value which represents the maximum sync difference that can't be noticed by a human. Maybe the sync reset code could be disabled if the streaming scenario doesn't involve an audio track?

[haydenm315]

I think this code has changed significantly on the trunk compared to the release we're using. I'll test with the latest and see what that brings.