multi-instance raw image callback, error with memory leakage

[Donggyu Lee]

I'm using libvlc.dll(2.1.3 windows install version binary) for getting raw image buffer from rtsp video source.
It's seems good when I use only one instance for one rtsp video source.

But I need to get 20 video data simultaneously (with one application).
When I try do this, there is memory leakage with libavcodec_plugin.dll.

I do those things.

1. make new instances for 20 video sources (registering pre-render, post-render callback functions)
I considered use only one instance for 20 video sources, but I think there's no way to get 'channel number information' (1~20)
with the render callback functions.

char smem_options[1000];
sprintf(smem_options
, "#transcode{vcodec=RV24}:smem{"
"video-prerender-callback=%lld,"
"video-postrender-callback=%lld,"
"video-data=%lld,"
"no-time-sync},"
, cbVideoPrerender
, cbVideoPostrender //This would normally be useful data, 100 is just test data
, (long long int)200 //Test data
);
const char * const vlc_args_smem[] = {
"-I", "dummy", // Don't use any interface
"--ignore-config", // Don't use VLC's config
"--verbose=1", // Be verbose
"--sout", smem_options, // Stream to memory
"--no-audio" // Audio Off
};
m_pInstance[nCh] = libvlc_new(sizeof(vlc_args_smem) / sizeof(vlc_args_smem[0]), vlc_args_smem); // nCh = 0 ~19

2. make new media from rtsp video address using 'libvlc_media_new_location'
libvlc_media_t* pMedia = libvlc_media_new_location (m_pInstance[nCh], siteAddress);

3. make new player from the media using 'libvlc_media_player_new_from_media'
and release the media
m_pPlayer[nCh] = libvlc_media_player_new_from_media (pMedia);
libvlc_media_release (pMedia); /* No need to keep the media now */

4. attach event callbacks
libvlc_event_attach(m_pEventManager[nCh], libvlc_MediaPlayerOpening, handleEvent, (void*)nCh);
libvlc_event_attach(m_pEventManager[nCh], libvlc_MediaPlayerBuffering, handleEvent, (void*)nCh);
libvlc_event_attach(m_pEventManager[nCh], libvlc_MediaPlayerPlaying, handleEvent, (void*)nCh);
libvlc_event_attach(m_pEventManager[nCh], libvlc_MediaPlayerPaused, handleEvent, (void*)nCh);
libvlc_event_attach(m_pEventManager[nCh], libvlc_MediaPlayerStopped, handleEvent, (void*)nCh);
libvlc_event_attach(m_pEventManager[nCh], libvlc_MediaPlayerEndReached, handleEvent, (void*)nCh);
libvlc_event_attach(m_pEventManager[nCh], libvlc_MediaPlayerEncounteredError, handleEvent, (void*)nCh);

5. play video
int nRet = libvlc_media_player_play(m_pPlayer[nCh]);


I defined base callback functions for capturing raw image

void cvVideoPreRender(int nCh, void* pVideoData, uint8_t** pPixelBuffer, int nSize)
{
if( nCh < 0 || RTSP_MAX_CHANNEL <= nCh ) // RTSP_MAX_CHANNEL =20
return;

if ( g_nVideoBufferSize[nCh] < nSize || g_pVideoBuffer[nCh] == NULL )
{
if( g_pVideoBuffer[nCh] )
free(g_pVideoBuffer[nCh]);
g_pVideoBuffer[nCh] = (uint8_t *)malloc(nSize);
g_nVideoBufferSize[nCh] = nSize;
}
*pPixelBuffer = g_pVideoBuffer[nCh];
}

void cbVideoPostrender(int nCh, void *p_video_data, uint8_t *p_pixel_buffer,
int width, int height, int pixel_pitch, int size, int64_t pts)
{
if( nCh < 0 || RTSP_MAX_CHANNEL <= nCh )
return;

if( g_pImgVideo[nCh] == NULL ) // IplImage in OpenCV
return;

BufferToIplImageResize(p_pixel_buffer, width, height, g_pImgVideo[nCh]); // convert buffer to IplImage resizing

if( g_pMainWnd && g_pMainWnd->m_bCreated )
g_pMainWnd->DisplayImage(nCh, g_pImgVideo[nCh]);
}

and define 20 functions for pre-render, post-render callbacks each.

void cbVideoPrerender0(void *p_video_data, uint8_t **pp_pixel_buffer, int size)
{ cvVideoPreRender(0, p_video_data, pp_pixel_buffer, size); }
void cbVideoPostrender0(void *p_video_data, uint8_t *p_pixel_buffer, int width, int height, int pixel_pitch, int size, int64_t pts)
{ cbVideoPostrender(0, p_video_data, p_pixel_buffer, width, height, pixel_pitch, size, pts); }
void cbVideoPrerender1(void *p_video_data, uint8_t **pp_pixel_buffer, int size)
{ cvVideoPreRender(1, p_video_data, pp_pixel_buffer, size); }
void cbVideoPostrender1(void *p_video_data, uint8_t *p_pixel_buffer, int width, int height, int pixel_pitch, int size, int64_t pts)
{ cbVideoPostrender(1, p_video_data, p_pixel_buffer, width, height, pixel_pitch, size, pts); }
...
void cbVideoPrerender19(void *p_video_data, uint8_t **pp_pixel_buffer, int size)
{ cvVideoPreRender(19, p_video_data, pp_pixel_buffer, size); }
void cbVideoPostrender19(void *p_video_data, uint8_t *p_pixel_buffer, int width, int height, int pixel_pitch, int size, int64_t pts)
{ cbVideoPostrender(19, p_video_data, p_pixel_buffer, width, height, pixel_pitch, size, pts); }

using cbVideoPrerender0~cbVideoPrerender19 for cbVideoPrerender param in smem_options,
using cbVideoPostrender0~cbVideoPostrender19 for cbVideoPostrender param in smem_options.

I do not write 'release codes' for players and instances here.


this applicatino seems good for the first time,
but a few seconds later, the memory begins bigger and bigger resulting in 'no response' error of the application.

When I see the messages from libvlc module with 'DebugView' utility,
I can see those messages while the memory getting bigger.
"avcodec encoder warning: almost fed libavcodec with two frames with the same PTS (427799291736) "
"avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 427799245010, last: 427799411676)"

when the application is about to be killed, I can see
"swscale filter warning: can't get output picture"
"main generic warning: can't get output picture"

Does anybody know solutions about this?
Can I use multiple instances for 20 rtsp video sources with no leakage or error?
Or can you recommend other library getting rtsp video source, which I can get raw images for multiple rtsp video?
Please help me!

[Rémi Denis-Courmont]

Please use the official API for render callbacks. Unfortunately, this is a bit too vague to help. I can only guess your callbacks are too slow to keep up.

[Donggyu Lee]

Please use the official API for render callbacks. Unfortunately, this is a bit too vague to help. I can only guess your callbacks are too slow to keep up.

When I remove copy and resize function and display function from post-render callback,
the same warnings and memory leakage occur as ever.
And I used the sample source from github "https://gist.github.com/TimSC/4121862"
modified a little to add video channel number information.
Thank you.