Find the best option to minimize the lag of libVLC Android app java

[bobyguy]

Hello,

Reposting my post from stackoverflow (I hope it is an acceptable pratice)

https://stackoverflow.com/questions/771 ... -app-java
Viewing a rtsp stream on my libVLC, I get 1s of delay. I tried to use all the possible options to be set in libVLC I could find to reduce the delay.

I have a rtsp server on my host machine that streams in RTSP with the following code

Code: Select all

/v4l2rtspserver/v4l2rtspserver -W 320 -H 240 -F 30 -P 8554 $mycam that outputs that:

Code: Select all

[NOTICE] /v4l2rtspserver/main.cpp:269 Version: 0.3.6-3-g233b631 live555 version:2022.10.01 [NOTICE] /v4l2rtspserver/src/V4l2RTSPServer.cpp:37 Create V4L2 Source.../dev/video0 [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2Device.cpp:133 driver:uvcvideo capabilities:84200001 mandatory:4000001 [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2Device.cpp:136 /dev/video0 support capture [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2Device.cpp:139 /dev/video0 support streaming [ERROR] /v4l2rtspserver/libv4l2cpp/src/V4l2Device.cpp:212 /dev/video0: Cannot set pixelformat to:H264 format is:YUYV [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2Device.cpp:225 /dev/video0:MJPG size:320x240 bufferSize:153600 [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2Device.cpp:246 fps:1/30 [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2Device.cpp:247 nbBuffer:0 [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2MmapDevice.cpp:49 Device /dev/video0 [NOTICE] /v4l2rtspserver/libv4l2cpp/src/V4l2MmapDevice.cpp:73 Device /dev/video0 nb buffer:10 [NOTICE] /v4l2rtspserver/src/V4l2RTSPServer.cpp:62 Create Source .../dev/video0 [NOTICE] /v4l2rtspserver/inc/BaseServerMediaSubsession.h:49 format:video/JPEG [NOTICE] /v4l2rtspserver/inc/V4l2RTSPServer.h:234 Play this stream using the URL "rtsp://XXXXXXX:8554/unicast" [NOTICE] /v4l2rtspserver/src/V4L2DeviceSource.cpp:96 handleCmd_SETUP:SETUP rtsp://192.168.5.1:8554/unicast/track1 RTSP/1.0 CSeq: 4 User-Agent: LibVLC/4.0.0-dev (LIVE555 Streaming Media v2022.07.14) Transport: RTP/AVP;unicast;client_port=57666-57667

Code: Select all

String rtspStreamUrl; Resources res = getResources(); rtspStreamUrl = preferences.getString("rtspStreamUrl", getResources().getString(R.string.Default_RtspStreamUrl)) ; /// ArrayList<String> options = new ArrayList<>(); options.add("-vvv"); options.add("--low-delay"); options.add("--network-caching=100"); //options.add("--file-caching=100"); //options.add("--sub-track=0"); //options.add("--rtsp-tcp"); LibVLC libVLC = new LibVLC(this, options); // VLC's MediaPlayer org.videolan.libvlc.MediaPlayer vlcMediaPlayer = new org.videolan.libvlc.MediaPlayer(libVLC); SurfaceView surfaceView = findViewById(R.id.surface_view); FrameLayout.LayoutParams params = new FrameLayout.LayoutParams(FrameLayout.LayoutParams.MATCH_PARENT, FrameLayout.LayoutParams.MATCH_PARENT); surfaceView.setLayoutParams(params); vlcMediaPlayer.getVLCVout().setVideoView(surfaceView); vlcMediaPlayer.getVLCVout().attachViews(); Media media = new Media(libVLC, Uri.parse(rtspStreamUrl)); /// media.setHWDecoderEnabled(true, true); media.addOption(":network-caching=250");//golden was 250 media.addOption(":clock-jitter=0");//golden was 0 media.addOption(":clock-synchro=0"); media.addOption(":no-dr"); media.addOption(":drop-late-frames"); media.addOption(":skip-frames"); media.addOption(":live-caching=50"); // for example, to set it to 50ms /// vlcMediaPlayer.setMedia(media); media.release(); DisplayMetrics metrics = new DisplayMetrics(); getWindowManager().getDefaultDisplay().getMetrics(metrics); int screenHeight = metrics.heightPixels; int screenWidth = metrics.widthPixels; vlcMediaPlayer.getVLCVout().setWindowSize(screenWidth, screenHeight); // This sets the dimensions VLC will use vlcMediaPlayer.setAspectRatio("16:9"); // or whatever aspect ratio you want, e.g., "4:3" vlcMediaPlayer.play();

I have this version of the lib:

implementation 'org.videolan.android:libvlc-all:4.0.0-eap11'

Any brilliant idea on how to debug this, or what to change to reduce the delay?
Thanks a lot!


I tried to visualize the stream on a separate client on the same devide (Qgroundcontrol) and I have no visible delay.

EDIT: It turns out that remove the -F frame rate options from the server decreases the delay but it is still pretty big, around 600ms