libvlc - imem and raw video input

[tulkas85]

hi, I'm trying to play raw frame from memory with vlc.

I have a GigE camera that puts in memory raw frame with mono 8bit format (gray scale).

I use imem to put in vlc these buffers, I followed this topic https://forum.videolan.org/viewtopic.php?f=32&t=93842 to do this, and it seems to work properly (callbacks are invoked).

But I have errors on stdout after each callback, it seems that codec setting is wrong (I don't know what could be for a raw video).

Is possible to set something like : --imem-codec=none ?

My first step is to play this raw video in vlc , next step is to stream it instead of playing, but I don't know if it's possible.

[Rémi Denis-Courmont]

"GREY" is the VLC FOURCC for 8-bits gray-scale. You can pass "undf" if you just want to test that feeding data works but don´t want it decoded.

[tulkas85]

Hi have same error using "grey" or "undf" , my vlc version is 2.1.3 for Windows

Code: Select all

main decoder error: no suitable decoder module for fourcc `undf'. VLC probably does not support this sound or video format. VLC does not support the audio or video format "undf". Unfortunately there is no way for you to fix this

There is a way to show this raw stream on screen ?

[Rémi Denis-Courmont]

GREY, not grey.

[tulkas85]

I tried so "GREY" nothing changed.

main decoder error: no suitable decoder module for fourcc `GREY'. VLC probably does not support this sound or video format.
main decoder error: No suitable decoder module
main decoder error: VLC does not support the audio or video format "GREY". Unfortunately there is no way for you to fix this.

[Rémi Denis-Courmont]

Then you're not setting other parameters correctly (or the rawvid decoder is missing).

[tulkas85]

this is my piece of code for vlc connection, it's only a first approach obviously

Code: Select all

const uint8_t *pImageBufferGlobal; int globaldts=0; int globalpts=0; int globalsize=0; int myImemGetCallback (void *data, const char *cookie, int64_t *dts, int64_t *pts, unsigned *flags, size_t * bufferSize, void ** buffer){ int ret=0; int size=globalsize; *buffer=malloc(size); memcpy ( *buffer, data, size ); *dts=globaldts; globaldts=globaldts+33333; *pts=globalpts; globalpts=globalpts+33333; *bufferSize=size; return ret; } int myImemReleaseCallback (void *data, const char *cookie, size_t bufferSize, void * buffer){ int ret=0; return ret; } DWORD WINAPI connect_vlc(LPVOID lpParameter){ // VLC options char imem_options[256]; sprintf(imem_options, "--imem-get=%ld", myImemGetCallback); char imem_options2[256]; sprintf(imem_options2, "--imem-release=%ld", myImemReleaseCallback); char imem_options3[256]; sprintf(imem_options3, "--imem-data=%ld", pImageBufferGlobal); //vlc --demux rawvideo --rawvid-fps 25 --rawvid-width 1920 --rawvid-height 1080 --rawvid-chroma I420 input.yuv const char *vlc_args[] = { //"--extraintf=logger", // Log anything //"--verbose=2", // Be much more verbose then normal for debugging purpose imem_options,imem_options2,imem_options3, "--imem-codec=GREY", "--imem-cat=2", "--imem-cookie=test" //"--sout='#transcode{vcodec=h264}'" //"--imem-caching=3000" }; // load the vlc engine libvlc_instance_t *inst; libvlc_media_player_t *mp; libvlc_media_t *m; string URL; inst = libvlc_new(sizeof(vlc_args) / sizeof(vlc_args[0]), vlc_args); m = libvlc_media_new_location(inst, "imem://"); mp = libvlc_media_player_new_from_media(m); libvlc_media_release(m); libvlc_media_player_play(mp); return 0; }

and verbose log

Code: Select all

main libvlc debug: VLC media player - 2.1.3 Rincewind main libvlc debug: Copyright © 1996-2014 the VideoLAN team main libvlc debug: revision 2.1.3-0-ge6a71cc main libvlc debug: configured with ../extras/package/win32/../../../configure '--enable-update-check' '--enable-lua' '--enable-faad' '--enable-flac' '--enable-theora' '--enable-twolame' '--enable-quicktime' '--enable-avcodec' '--enable-merge-ffmpeg' '--enable-dca' '--enable-mpc' '--enable-libass' '--enable-x264' '--enable-schroedinger' '--enable-realrtsp' '--enable-live555' '--enable-dvdread' '--enable-shout' '--enable-goom' '--enable-caca' '--disable-sdl' '--enable-qt' '--enable-skins2' '--enable-sse' '--enable-mmx' '--enable-libcddb' '--enable-zvbi' '--disable-telx' '--enable-nls' '--disable-dirac' '--host=i686-w64-mingw32' 'host_alias=i686-w64-mingw32' main libvlc debug: using multimedia timers as clock source main libvlc debug: min period: 1 ms, max period: 1000000 ms main libvlc debug: searching plug-in modules main libvlc debug: loading plugins cache file c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\plugins\plugins.dat main libvlc debug: recursively browsing `c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\plugins' main libvlc debug: saving plugins cache c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\plugins\plugins.dat main libvlc debug: plug-ins loaded: 392 modules main libvlc debug: translation test: code is "C" main libvlc debug: CPU has capabilities MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 FPU main input debug: Creating an input for 'imem://' The thread 'Win32 Thread' (0x1acc) has exited with code 0 (0x0). main input debug: using timeshift granularity of 50 MiB, in path 'C:\Users\Fabio\AppData\Local\Temp' main input debug: `imem://' gives access `imem' demux `' path `' main input debug: creating demux: access='imem' demux='' location='' file='(null)' main demux debug: looking for access_demux module matching "imem": 12 candidates 'AcquireContinuous.exe': Loaded 'C:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\plugins\access\libaccess_imem_plugin.dll', Binary was not built with debug information. access_imem demux debug: Using get(0047706b), release(004757b6), data(05590040), cookie(baslertest) access_imem demux debug: Video GREY 0x0 SAR 0:0 frame rate 0/0 main input debug: selecting program id=0 main demux debug: using access_demux module "access_imem" main decoder debug: looking for decoder module matching "any": 39 candidates rawvideo decoder error: invalid display size 0x0 main decoder debug: no decoder modules matched main decoder error: no suitable decoder module for fourcc `GREY'. VLC probably does not support this sound or video format. main decoder error: No suitable decoder module main decoder error: VLC does not support the audio or video format "GREY". Unfortunately there is no way for you to fix this. main decoder debug: killing decoder fourcc `GREY', 0 PES in FIFO main demux meta debug: looking for meta reader module matching "any": 2 candidates lua demux meta debug: Trying Lua scripts in C:\Users\Fabio\AppData\Roaming\vlc\lua\meta\reader lua demux meta debug: Trying Lua scripts in c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\lua\meta\reader main demux meta debug: no meta reader modules matched main input debug: `imem://' successfully opened main input debug: Buffering 0% main input debug: Buffering 11% main input debug: Buffering 22% main input debug: Buffering 33% main input debug: Buffering 44% main input debug: Buffering 55% main input debug: Buffering 66% main input debug: Buffering 77% main input debug: Buffering 88% main input debug: Buffering 99% main input debug: Stream buffering done (333 ms in 30 ms) main input debug: Decoder buffering done in 0 ms

[Rémi Denis-Courmont]

rawvideo decoder error: invalid display size 0x0
...

[tulkas85]

ok, thank you.

Now I can see video output, but video is broken


I set these options :

Code: Select all

const char *vlc_args[] = { //"--extraintf=logger", // Log anything "--verbose=2", // Be much more verbose then normal for debugging purpose imem_options,imem_options2,imem_options3, "--imem-codec=GREY", "--imem-cat=2", "--imem-cookie=baslertest", "--imem-width=1280", "--imem-height=960", "--imem-fps=30", "--sout=#stream_out_duplicate{dst=display,dst=std{access=file,dst=c:\\go.avi}}" };

maybe fps doesn't read correctly becouse I have :
rawvideo decoder warning: invalid frame rate 0/0, using 25 fps instead


log:

Code: Select all

main libvlc debug: VLC media player - 2.1.3 Rincewind main libvlc debug: Copyright © 1996-2014 the VideoLAN team main libvlc debug: revision 2.1.3-0-ge6a71cc main libvlc debug: configured with ../extras/package/win32/../../../configure '--enable-update-check' '--enable-lua' '--enable-faad' '--enable-flac' '--enable-theora' '--enable-twolame' '--enable-quicktime' '--enable-avcodec' '--enable-merge-ffmpeg' '--enable-dca' '--enable-mpc' '--enable-libass' '--enable-x264' '--enable-schroedinger' '--enable-realrtsp' '--enable-live555' '--enable-dvdread' '--enable-shout' '--enable-goom' '--enable-caca' '--disable-sdl' '--enable-qt' '--enable-skins2' '--enable-sse' '--enable-mmx' '--enable-libcddb' '--enable-zvbi' '--disable-telx' '--enable-nls' '--disable-dirac' '--host=i686-w64-mingw32' 'host_alias=i686-w64-mingw32' main libvlc debug: using multimedia timers as clock source main libvlc debug: min period: 1 ms, max period: 1000000 ms main libvlc debug: searching plug-in modules main libvlc debug: loading plugins cache file c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\plugins\plugins.dat main libvlc debug: recursively browsing `c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\plugins' main libvlc debug: saving plugins cache c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\plugins\plugins.dat main libvlc debug: plug-ins loaded: 392 modules main libvlc debug: translation test: code is "C" main libvlc debug: CPU has capabilities MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 FPU main input debug: Creating an input for 'imem://' The thread 'Win32 Thread' (0x1708) has exited with code 0 (0x0). main stream output debug: using sout chain=`stream_out_duplicate{dst=display,dst=std{access=file,dst=c:\go.avi}}' main stream output debug: stream=`stream_out_duplicate' main stream out debug: looking for sout stream module matching "stream_out_duplicate": 21 candidates stream_out_duplicate stream out debug: creating 'duplicate' stream_out_duplicate stream out debug: * adding `display' main stream output debug: stream=`display' main stream out debug: looking for sout stream module matching "display": 21 candidates main stream out debug: using sout stream module "stream_out_display" stream_out_duplicate stream out debug: * adding `std{access=file,dst=c:\go.avi}' main stream output debug: stream=`std' main stream out debug: looking for sout stream module matching "std": 21 candidates main stream out debug: set config option: sout-standard-access to file main stream out debug: set config option: sout-standard-dst to c:\go.avi main access out debug: looking for sout access module matching "file": 8 candidates access_output_file access out debug: file access output opened (c:\go.avi) main access out debug: using sout access module "access_output_file" main mux debug: looking for sout mux module matching "avi": 10 candidates mux_avi mux debug: AVI muxer opened main mux debug: using sout mux module "mux_avi" stream_out_standard stream out debug: using `file/avi://c:\go.avi' main stream out debug: using sout stream module "stream_out_standard" main stream out debug: using sout stream module "stream_out_duplicate" main input debug: using timeshift granularity of 50 MiB, in path 'C:\Users\Fabio\AppData\Local\Temp' main input debug: `imem://' gives access `imem' demux `' path `' main input debug: creating demux: access='imem' demux='' location='' file='(null)' main demux debug: looking for access_demux module matching "imem": 12 candidates access_imem demux debug: Using get(00477061), release(004757b6), data(053f0040), cookie(baslertest) access_imem demux debug: Video GREY 1280x960 SAR 0:0 frame rate 0/0 main input debug: selecting program id=0 main demux debug: using access_demux module "access_imem" main decoder debug: looking for packetizer module matching "any": 21 candidates rawvideo decoder warning: invalid frame rate 0/0, using 25 fps instead main decoder debug: using packetizer module "rawvideo" main input debug: starting in async mode main demux meta debug: looking for meta reader module matching "any": 2 candidates lua demux meta debug: Trying Lua scripts in C:\Users\Fabio\AppData\Roaming\vlc\lua\meta\reader lua demux meta debug: Trying Lua scripts in c:\Program Files\Basler\pylon 2.3\Samples\C++2010\Debug_1394\lua\meta\reader main demux meta debug: no meta reader modules matched main input debug: `imem://' successfully opened main input debug: Buffering 0% main input debug: switching to sync mode main stream output debug: adding a new sout input (sout_input:09297220) main input debug: Buffering 9% stream_out_duplicate stream out debug: duplicated a new stream codec=GREY (es=0 group=0) main input debug: Buffering 18% main decoder debug: looking for decoder module matching "any": 39 candidates main input debug: Buffering 27% main input debug: Buffering 37% main input debug: Buffering 46% main input debug: Buffering 55% main input debug: Buffering 64% rawvideo decoder warning: invalid frame rate 0/0, using 25 fps instead main input debug: Buffering 74% main decoder debug: using decoder module "rawvideo" stream_out_duplicate stream out debug: - added for output 0 main input debug: Buffering 83% main mux debug: adding a new input mux_avi mux debug: adding input main input debug: Buffering 92% stream_out_duplicate stream out debug: - added for output 1 main input debug: Stream buffering done (305 ms in 157 ms) main input debug: Decoder buffering done in 0 ms main spu text debug: looking for text renderer module matching "any": 2 candidates freetype spu text debug: Using Arial as font from file C:\Windows\Fonts\arial.ttf freetype spu text debug: using fontsize: 2 main spu text debug: using text renderer module "freetype" main scale debug: looking for video filter2 module matching "any": 55 candidates swscale scale debug: 32x32 chroma: YUVA -> 16x16 chroma: RGBA with scaling using Bicubic (good quality) main scale debug: using video filter2 module "swscale" main scale debug: looking for video filter2 module matching "any": 55 candidates yuvp scale debug: YUVP to YUVA converter main scale debug: using video filter2 module "yuvp" main video output debug: Deinterlacing available main video output debug: deinterlace 0, mode blend, is_needed 0 main video output debug: Opening vout display wrapper main vout display debug: looking for vout display module matching "any": 10 candidates direct3d vout display debug: creating Vout EventThread direct3d vout display debug: DirectXCreateWindow main window debug: looking for vout window hwnd module matching "any": 3 candidates main window debug: no vout window hwnd modules matched direct3d vout display debug: created video sub-window direct3d vout display debug: Vout EventThread running direct3d vout display debug: disabling screen saver direct3d vout display debug: Direct3d Device: Intel(R) HD Graphics 32902 338 9 direct3d vout display warning: trying surface pixel format: X8R8G8B8 direct3d vout display debug: selected surface pixel format is X8R8G8B8 direct3d vout display debug: Using D3DTEXF_LINEAR for minification direct3d vout display debug: Using D3DTEXF_LINEAR for magnification direct3d vout display debug: Direct3D scene created successfully direct3d vout display debug: Direct3D device adapter successfully initialized main vout display debug: using vout display module "direct3d" main vout display debug: A filter to adapt decoder to display is needed main filter debug: looking for video filter2 module matching "any": 55 candidates swscale filter debug: 1280x960 chroma: GREY -> 1280x960 chroma: RV32 with scaling using Bicubic (good quality) main filter debug: using video filter2 module "swscale" main vout display debug: Filter 'Swscale' (092c3624) appended to chain main video output debug: original format sz 1280x960, of (0,0), vsz 1280x960, 4cc GREY, sar 1:1, msk r0x0 g0x0 b0x0 main video output warning: picture is too late to be displayed (missing 214 ms) main video output warning: picture is too late to be displayed (missing 189 ms) main video output warning: picture is too late to be displayed (missing 164 ms) main video output warning: picture is too late to be displayed (missing 142 ms) main video output warning: picture is too late to be displayed (missing 116 ms) main video output warning: picture is too late to be displayed (missing 100 ms) main video output warning: picture is too late to be displayed (missing 74 ms) main video output warning: picture is too late to be displayed (missing 48 ms) main video output warning: picture is too late to be displayed (missing 21 ms) main video output warning: picture is too late to be displayed (missing 35 ms) main video output debug: picture might be displayed late (missing 9 ms) main video output warning: picture is too late to be displayed (missing 36 ms) main video output debug: picture might be displayed late (missing 11 ms) mux_avi mux debug: writing header main vout display debug: auto hiding mouse cursor main vout display debug: auto hiding mouse cursor main video output debug: picture might be displayed late (missing 14 ms) main vout display debug: auto hiding mouse cursor main vout display debug: auto hiding mouse cursor main vout display debug: auto hiding mouse cursor

[Rémi Denis-Courmont]

That's somewhat obviously a line pitch problem. I'm not sure imem can do what you want.

[tulkas85]

maybe I can try others decoder ? How can I obtain a complete list of installed decoder ?

I can change the type of pixel if it's necessary. I have these options:

Code: Select all

enum PixelFormatEnums { PixelFormat_Mono8, //!<Sets the pixel format to Mono 8 PixelFormat_Mono1Packed, //!<Sets the pixel format to Mono 1 Packed PixelFormat_Mono10, //!<Sets the pixel format to Mono 10 PixelFormat_Mono12, //!<Sets the pixel format to Mono 12 PixelFormat_Mono12Packed, //!<Sets the pixel format to Mono 12 Packed PixelFormat_Mono16, //!<Sets the pixel format to Mono 16 PixelFormat_Mono2Packed, //!<Sets the pixel format to Mono 2 Packed PixelFormat_Mono4Packed, //!<Sets the pixel format to Mono 4 Packed PixelFormat_BayerGR8, //!<Sets the pixel format to Bayer GR 8 PixelFormat_BayerRG8, //!<Sets the pixel format to Bayer RG 8 PixelFormat_BayerGB8, //!<Sets the pixel format to Bayer GB 8 PixelFormat_BayerBG8, //!<Sets the pixel format to Bayer BG 8 PixelFormat_BayerGR10, //!<Sets the pixel format to Bayer GR 10 PixelFormat_BayerRG10, //!<Sets the pixel format to Bayer RG 10 PixelFormat_BayerGB10, //!<Sets the pixel format to Bayer GB 10 PixelFormat_BayerBG10, //!<Sets the pixel format to Bayer BG 10 PixelFormat_BayerGR12, //!<Sets the pixel format to Bayer GR 12 PixelFormat_BayerRG12, //!<Sets the pixel format to Bayer RG 12 PixelFormat_BayerGB12, //!<Sets the pixel format to Bayer GB 12 PixelFormat_BayerBG12, //!<Sets the pixel format to Bayer BG 12 PixelFormat_YUV422Packed, //!<Sets the pixel format to YUV 422 Packed PixelFormat_YUV422_YUYV_Packed, //!<Sets the pixel format to YUV 422 (YUYV) Packed PixelFormat_BayerGB12Packed, //!<Sets the pixel format to Bayer GB 12 Packed PixelFormat_BayerGR12Packed, //!<Sets the pixel format to Bayer GR 12 Packed PixelFormat_BayerRG12Packed, //!<Sets the pixel format to Bayer RG 12 Packed PixelFormat_BayerBG12Packed, //!<Sets the pixel format to Bayer BG 12 Packed PixelFormat_BayerGR16, //!<Sets the pixel format to Bayer GR 16 PixelFormat_BayerRG16, //!<Sets the pixel format to Bayer RG 16 PixelFormat_BayerGB16, //!<Sets the pixel format to Bayer GB 16 PixelFormat_BayerBG16 //!<Sets the pixel format to Bayer BG 16 };

my first test was : PixelFormat_Mono8 using decoder GREY
second test : PixelFormat_YUV422_YUYV_Packed using decoder YUY2 (https://wiki.videolan.org/YUV/)

both has same output