The VideoLAN Forums

Posted: **05 Sep 2023 18:24**

Hello,
I made Delphi form application and I have implemented usage of libvlc.dll and sdl2.dll to get video frame.
Sdl is using lock, unlock and display function where you work with PSDL_Texture. According to my test showing video in SDL window use alot of CPU and very small amount of GPU (setting '--avcodec-hw=dxva2' didn't help at all).
If I removed the SDL window and tryed to display video in Panel by calling

Code: Select all

libvlc_media_player_set_hwnd(AMediaPlayer, Pointer(Panel1.Handle));

the CPU usage was low and the GPU was used for decoding/rendering the video. This is great but in this case I am not able to get vide frame since in this case lock, unlock and display functions were never triggered.

Code: Select all

libvlc_video_set_callbacks(AMediaPlayer, @lock, @unlock, @display, @FrameContext[iId]);

So my questions/desire:
- I want to get each video frame so I can display it with IDirect3DTexture9, IDirect3DSurface9.
- As I can see I can't get dx9 texture out of vlc directly?
- so which aproach would be the best to do this on GPU and not on CPU site?

Did I set the hardwer decoding wrong?

Code: Select all

  //Before opening an instance:
  AVlcArgs[0] := ''; //PAnsiChar(libvlc_dynamic_dll_path);
  AVlcArgs[1] := '--no-video-title-show';
  AVlcArgs[2] := '--no-xlib'; 
  AVlcArgs[3] := '--no-audio'; 
  AVlcArgs[4] := '--network-caching=200'; 
  AVlcArgs[5] := '--rtsp-caching=100'; 
  AVlcArgs[6] := '--sout-rtp-caching=200'; 
  AVlcArgs[7] := '--avcodec-hw=any';
  AVlcArgs[8] := '--vout=directx'; 
  AVlcArgs[9] := '--ffmpeg-hw';
  AVlcArgs[10] := '--extraintf=logger'; 
  AVlcArgs[11] := '--verbose=2'; // 
  AVlcArgs[12] := '--file-logging';
  AVlcArgs[13] := '--logfile=vlc.log'; 
  
  var p_li := libvlc_new(Length(AVlcArgs)-1, @AVlcArgs[0]);
  
  //And before play:
  libvlc_media_add_option(AMedia, PAnsiChar('--network-caching=200'));
  libvlc_media_add_option(AMedia, PAnsiChar('--rtsp-caching=100'));
  libvlc_media_add_option(AMedia, PAnsiChar('--sout-rtp-caching=200'));
  libvlc_media_add_option(AMedia, PAnsiChar('--avcodec-hw=any'));
  libvlc_media_add_option(AMedia, PAnsiChar('--vout=directx'));
  libvlc_media_add_option(AMedia, PAnsiChar('--ffmpeg-hw'));

Posted: **06 Sep 2023 08:29**

You can't have the cake and eat it. If you use hardware decoding, the frames are in GPU memory and thus not efficiently accessible from CPU.

Posted: **07 Sep 2023 09:17**

You can't have the cake and eat it. If you use hardware decoding, the frames are in GPU memory and thus not efficiently accessible from CPU.

Is it possible to call
libvlc_video_set_callbacks(AMediaPlayer, @lock, @unlock, @display, @FrameContext[iId]);
without using SDL window?
And use libvlc_video_set_callbacks to create dxTexture (IDirect3DTexture9, IDirect3DSurface9)?
To minimize CPU usage?

Currently I don't know how to get vlc video frame in
function lock(opaque: Pointer; planes: PVCBPlanes): Pointer; cdecl;

My current code

Code: Select all

function lock(opaque: Pointer; planes: PVCBPlanes): Pointer; cdecl;
var
  AContext: PFrameContext;
  lockedRect: D3DLOCKED_RECT;
  pitch: Integer;
  vlcPixelData: Pointer;
  vlcPitch: Integer;
begin
  Result := nil;

  if Assigned(opaque) then
  begin
    AContext := PFrameContext(opaque);

    if Assigned(AContext) then
    begin
      AContext.dx9Texture.GetSurfaceLevel(0, AContext.dx9Surface);
      AContext.dx9Surface.LockRect(lockedRect, nil, 0);
      pitch := lockedRect.Pitch;
	  
      vlcPixelData := planes[0];
      vlcPitch := AContext.VideoWidth;

      AContext.dx9Surface.UnlockRect;

      Result := planes[0];
    end;
  end;
end;

Posted: **07 Sep 2023 15:29**

Currently my last code looks like this
Before play

Code: Select all

    //libvlc_media_player_set_hwnd(AMediaPlayer, Pointer(Panel1.Handle)); [b]This is not set so the lock function is triggered?[/b]
    libvlc_video_set_callbacks(AMediaPlayer, @lock, @unlock, @display, @FrameContext[iId]);

    libvlc_video_set_format(AMediaPlayer, 'BGRA', FrameContext[iId].VideoWidth, FrameContext[iId].VideoHeight, FrameContext[iId].VideoWidth * 4);
    libvlc_video_set_format_callbacks(AMediaPlayer, @setup, @cleanup);
  end;

  if (libvlc_media_player_is_playing(FMediaPlayerList[iId]) = 0) then
    libvlc_media_player_play(FMediaPlayerList[iId]);

And my lock function

Code: Select all

function lock(opaque: Pointer; planes: PVCBPlanes): Pointer; cdecl;
var
  AContext: PFrameContext;
  lockedRect: D3DLOCKED_RECT;
  pitch: Integer;
  vlcPixelData: Pointer;
  x, y: Integer;
begin
  Result := nil;

  if Assigned(opaque) then
  begin
    AContext := PFrameContext(opaque);

    if Assigned(AContext) then
    begin
      AContext.dx9Texture.GetSurfaceLevel(0, AContext.dx9Surface);
      AContext.dx9Surface.LockRect(lockedRect, nil, 0);
      pitch := lockedRect.Pitch;

      vlcPixelData := planes[0]; //[b]I don't know if this is right[/b]
      if not Assigned(vlcPixelData) then
        Exit;

      for y := 0 to AContext.VideoHeight - 1 do
      begin
        for x := 0 to AContext.VideoWidth - 1 do
        begin
          PByte(Integer(lockedRect.pBits) + y * pitch + x * 4)^ := PByte(Integer(vlcPixelData) + (y * AContext.VideoWidth + x) * 4)^;         // B  [b]I am getting exception here. x and y value is random when exception occures[/b]
          PByte(Integer(lockedRect.pBits) + y * pitch + x * 4 + 1)^ := PByte(Integer(vlcPixelData) + (y * AContext.VideoWidth + x) * 4 + 1)^; // G
          PByte(Integer(lockedRect.pBits) + y * pitch + x * 4 + 2)^ := PByte(Integer(vlcPixelData) + (y * AContext.VideoWidth + x) * 4 + 2)^; // R
          PByte(Integer(lockedRect.pBits) + y * pitch + x * 4 + 3)^ := PByte(Integer(vlcPixelData) + (y * AContext.VideoWidth + x) * 4 + 3)^; // A
        end;
      end;

      AContext.dx9Surface.UnlockRect;
      Result := planes[0];
    end;
  end;
end;

I have bolded my question/note in the code.
It looks like I am not getting correct data in vlcPixelData := planes[0];
How to know if the lock function was triggered with good data, if the video frame is ok or not. How to know if I can copy the pixels in to dx9 texture?

Posted: **07 Sep 2023 19:39**

Again, you can't have it both ways. If you use GPU acceleration, you can't access the bitmaps from CPU.

Posted: **07 Sep 2023 20:07**

Again, you can't have it both ways. If you use GPU acceleration, you can't access the bitmaps from CPU.

As you can see from my post I am trying to get dx9 texture (currently its not important if this will be on GPU or CPU).
Why I get exception in for loop when trying to copy vlc video frame to dx9 texture?

Posted: **13 Sep 2023 07:20**

What you are looking for is the GPU version of libvlc_video_set_callbacks, which is called libvlc_video_set_output_callbacks. You will need a nightly libvlc 4.x build. See https://videolan.videolan.me/vlc/group_ ... 775926b808

Posted: **13 Sep 2023 18:41**

What you are looking for is the GPU version of libvlc_video_set_callbacks, which is called libvlc_video_set_output_callbacks. You will need a nightly libvlc 4.x build. See https://videolan.videolan.me/vlc/group_ ... 775926b808

thanks, that looks promising

Posted: **14 Sep 2023 18:25**

What you are looking for is the GPU version of libvlc_video_set_callbacks, which is called libvlc_video_set_output_callbacks. You will need a nightly libvlc 4.x build. See https://videolan.videolan.me/vlc/group_ ... 775926b808

I tryed to play video with this 4.0 libvlc but I get nil in my line (p_li is nil)
var p_li := libvlc_new(Length(AVlcArgs)-1, @AVlcArgs[0]);

Are there some examples how to use this 4.0? How to initialize and set everything before playing the video?

Posted: **15 Sep 2023 15:31**

Currently with version 4.0 I am not able to play the video. in log it looks like something is corrupted
AMedia := libvlc_media_new_path(libvlcInstance, PAnsiChar(AnsiString('./videoLobby.mkv')));

main debug: using audio output module "mmdevice"
main debug: keeping audio output
main debug: Creating an input for '??'
main debug: using timeshift granularity of 50 MiB
main debug: `file:///I:/VLC/VLC_4/Win32/Debug/%10%A0%A1%01%01' gives access `file' demux `any' path `/I:/VLC/VLC_4/Win32/Debug/%10%A0%A1%01%01'
main debug: creating access: file:///I:/VLC/VLC_4/Win32/Debug/%10%A0%A1%01%01
main debug: (path: I:\VLC\VLC_4\Win32\Debug\ ¡)
main debug: looking for access module matching "file": 5 candidates
dvdnav error: Could not open I:\VLC\VLC_4\Win32\Debug\ ¡ with libdvdcss.
dvdnav error: Can't open I:\VLC\VLC_4\Win32\Debug\ ¡ for reading
dvdnav error: vm: failed to open/read the DVD
dvdnav warning: cannot open DVD (I:\VLC\VLC_4\Win32\Debug\ ¡)
filesystem error: cannot open file I:\VLC\VLC_4\Win32\Debug

if I use libvlc_media_new_location its the same but nothing in log file only in console
Debug Output:
main input error: Could not open MRL »??«.

The VideoLAN Forums

Delphi (pascal), video frame, sdl2, directX

Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX

Re: Delphi (pascal), video frame, sdl2, directX