I have tried to use "libvlc_video_get_height/width" but it always returns 0 saying "No active video output". However the RTP stream is playing and I can see it.
How can I retrieve video dimensions to make the canvas with the correct size?
This is a timing issue.
What I have observed is this...
Inside libvlc the listener fires the "playing" event.
If you immediately try and invoke video_get_width you get the error you quoted.
You need to wait a bit longer then invoke video_get_width.
As a curiosity, since I don't really use libvlc anymore, I changed my vlcj project as an experiment thus:
1. In the MediaPlayer class instance creation, kick off a new thread.
2. The new thread has a for loop that loops forever.
3. The thread sleeps for one second.
4. The thread invokes has_vout.
5. If has_vout returns zero, go back to #3.
6. If has_vout returns non-zero, invoke video_get_width, video_get_height, invoke listeners with a new "meta data available" event and exit the thread
This approach does indeed work.
In short, if after you receive a "playing" event you create a thread loop that sleeps every Xms and invokes has_vout, eventually (very quickly actually) it will return non-zero and then you can invoke get_width/get_height.
It would be a lot easier if the "playing" event was generated _after_ the video output was guaranteed to be available.
What I have learned is that the threading and synchronisation code required on the Java side to integrate properly with libvlc can be very intricate and hard to get a full grasp of, and even though I made best efforts with my own vlcj project, I can not guarantee that there are no thread dead-locks or subtle races. I certainly don't recommend trying to re-use the MediaPlayer instance for example.
Hope that helps you solve your problem.