display vmem output in a pictureBox

[alb84]

Hello everybody, I finally managed to link unmanaged libVLC with managed Winforms and setup a little project where the vmem unlock callback calls a marshalled delegate which invoke an event whose handler display the frame in a picturebox. I am doing this because I need to display the video with text and polilynes overlaying on the frames and syncrhonization between polylines and video is quite important.

The problem is that at the moment I am displaying the image on the form calling the SetPixel method for each pixel:

Code: Select all

Bitmap bmp(VIDEO_WIDTH, VIDEO_HEIGHT, Imaging::PixelFormat::Format32bppRgb); for(int x=0; x<VIDEO_WIDTH; x++) for(int y=0; y<VIDEO_HEIGHT; y++) { int a, r, g, b; b=context->pixels[(y*VIDEO_WIDTH+x)*4+0]; g=context->pixels[(y*VIDEO_WIDTH+x)*4+1]; r=context->pixels[(y*VIDEO_WIDTH+x)*4+2]; a=context->pixels[(y*VIDEO_WIDTH+x)*4+3]; bmp.SetPixel(x, y, System::Drawing::Color::FromArgb(a, r, g, b)); } this->pictureBox1->Image=bmp.GetThumbnailImage(VIDEO_WIDTH, VIDEO_HEIGHT, nullptr, System::IntPtr::Zero); System::Drawing::Graphics ^g = System::Drawing::Graphics::FromImage(this->pictureBox1->Image); g->DrawImage(bmp.GetThumbnailImage(VIDEO_WIDTH, VIDEO_HEIGHT, nullptr, System::IntPtr::Zero), 0, 0); System::Drawing::Pen ^p = gcnew System::Drawing::Pen(System::Drawing::Color::Green, 4); g->DrawLine . . . bla bla bla

but as you can imagine this is very slow...

I'd like to find a way to access the Image directly... someone told me to use GCHandle to pin the pixel of the Image, but I don't know what to do!
I have also found this link which seems quite useful http://www.bobpowell.net/lockingbits.htm

It would be great to do something like BitmapData->Scan0 = context->pixels so that there is no overhead for rearranging the pixel values.

Yet I don't know if this is possible.

There is also another problem, I've looked for documentation for vmem-chroma but I found nothing... Anyway RV32 gives me BGRA data, but .NET Image data is arranged in ARGB, is there a way to tell vmem to arrange the data as I want so that I can save the time or the rearrangement? I think it is the main cause of slowness and my project should work realtime.

[ninjamint]

found this @ dreamincode.net, credits goto microchip... use his code as a reference.
direct URL: http://www.dreamincode.net/forums/showtopic14788.htm

pointers are always going to be the fastest way of manipulating data, because its direct access to the bitmaps pixel memory;

Code: Select all

using System; using System.Collections.Generic; using System.Text; using System.Drawing; using System.Drawing.Imaging; namespace SampleGrabberNET { public unsafe class UnsafeBitmap { Bitmap bitmap; // three elements used for MakeGreyUnsafe int width; BitmapData bitmapData = null; Byte* pBase = null; public UnsafeBitmap(Bitmap bitmap) { this.bitmap = new Bitmap(bitmap); } public UnsafeBitmap(int width, int height) { this.bitmap = new Bitmap(width, height, PixelFormat.Format24bppRgb); } public void Dispose() { bitmap.Dispose(); } public Bitmap Bitmap { get { return(bitmap); } } private Point PixelSize { get { GraphicsUnit unit = GraphicsUnit.Pixel; RectangleF bounds = bitmap.GetBounds(ref unit); return new Point((int) bounds.Width, (int) bounds.Height); } } public void LockBitmap() { GraphicsUnit unit = GraphicsUnit.Pixel; RectangleF boundsF = bitmap.GetBounds(ref unit); Rectangle bounds = new Rectangle((int) boundsF.X, (int) boundsF.Y, (int) boundsF.Width, (int) boundsF.Height); // Figure out the number of bytes in a row // This is rounded up to be a multiple of 4 // bytes, since a scan line in an image must always be a multiple of 4 bytes // in length. width = (int) boundsF.Width * sizeof(PixelData); if (width % 4 != 0) { width = 4 * (width / 4 + 1); } bitmapData = bitmap.LockBits(bounds, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb); pBase = (Byte*) bitmapData.Scan0.ToPointer(); } public PixelData GetPixel(int x, int y) { PixelData returnValue = *PixelAt(x, y); return returnValue; } public void SetPixel(int x, int y, PixelData colour) { PixelData* pixel = PixelAt(x, y); *pixel = colour; } public void UnlockBitmap() { bitmap.UnlockBits(bitmapData); bitmapData = null; pBase = null; } public PixelData* PixelAt(int x, int y) { return (PixelData*)(pBase + y * width + x * sizeof(PixelData)); } } public struct PixelData { public byte blue; public byte green; public byte red; } }

I wouldn't suggest using that code exactly, but take from it the important bits, and use that!

[alb84]

Thanks I see, I had already found a similar project (FastPixel on codeproject) where an extension of the class Bitmap was made in order to override the SetPixel method using the direct access with BitmapData. Of course this is an improvement.

But what I was looking for was a solution in order no tot cycle over all the pixel to rearrange the byte disposition from RGBA to ABGR.
Is there a way to force the byte disposition of the vmem output of libVLC to be ABGR and nor RGBA so that I can just copy the memory or maybe even better copy the pointer to the data buffer returned by libVLC to BitmapData->Scan0 in order to save all the overhead time of rearranging the pixels?

Thanks

[Rémi Denis-Courmont]

If you really don't want to cycle through the pixels, you should use I420 and preserve the resolution. Otherwise VLC will have to change the chroma anyway.

[alb84]

So you are saying that if I use I420 instead of RV32 and I do no scale the resolution it gives me ARGB? This is great!

[alb84]

I've got a problem, changing from RV32 to I420 I get an "Access violation writing location", the code where it occurs is not debuggable. I think this is because it is libVLC code. The disassembly is
0522185B rep movs byte ptr es:[edi],byte ptr [esi]
but I imagine it can be no help...

Anyway could you please tell me where I can find a documentation of the possible vmem-chroma values? Maybe I420 was not included in the libVLC.lib I have extracted from the libvlc.dll or in the other plugins dll which come with a standard VLC Windows installation...

[alb84]

Sorry everything worked fine simply like this:

Code: Select all

Bitmap bmp(VIDEO_WIDTH, VIDEO_HEIGHT, VIDEO_WIDTH*4, Imaging::PixelFormat::Format32bppRgb, IntPtr(context->pixels)); this->pictureBox1->Image=bmp.GetThumbnailImage(VIDEO_WIDTH, VIDEO_HEIGHT, nullptr, System::IntPtr::Zero);

Don't know if creating a Bitmap for each frame and then an Image from a Bitmap and assigning the Image to the PictureBox is the more clean and speady solution but it works fine and the performances are also acceptable... of course buch better than when I used SetPixel.