Streaming UVC camera and USB Mic - Linux

[gusarg81]

Hi,

I am developing a project for a Smart Doorbell, by using a SBC Rock64 (v2.0) with a Arducam UVC USB camera (Arducam B0205) and a USB MIC (this one just a testing, I will use a simple USB Mic) ReSpeaker USB Mic Array.
The Linux distro is Debian Buster.

Now, I managed to stream video only by:

cvlc -vvv v4l2:///dev/video0:width=1920:height=1080:fps=30:chroma=MJPG --sout '#transcode{acodec=none}:rtp{sdp=rtsp://:8554/}'

How can add also the mic capture to this streaming?
My Mic hardware:

arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: rki2ssound [rk-i2s-sound], device 0: ff010000.i2s-rk3328-hifi rk3328-hifi-0 []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: rki2ssound [rk-i2s-sound], device 1: ff010000.i2s-snd-soc-dummy-dai snd-soc-dummy-dai-1 []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 3: ArrayUAC10 [ReSpeaker 4 Mic Array (UAC1.0)], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0

Which is the card 3.

Also as a plus (but no so important at this stage, but would be at the end, how can add hwaccel, since right now it use the 4 cores at 100%).

Thanks in advance.

[Rémi Denis-Courmont]

Add a slave input for the audio device.

[gusarg81]

Hi, thanks for reply. Can you elaborate please?

I used like this but there is no sound in the stream:

Server:

cvlc -vvv v4l2:///dev/video0:width=1920:height=1080:fps=30:chroma=MJPG:live-caching=300:input-slave=alsa://plughw:3,0 --sout '#transcode{acodec=mp3,ab=192,channels=2,samplerate=44100}:rtp{sdp=rtsp://:8554/}'

Client:

vlc -vvv --network-caching 200 rtsp://10.0.1.4:8554/

Mic does work when I test with on server:

arecord -D plughw:3,0 out.wav

Any ideas? Thanks in advance.

[gusarg81]

Hi,

I made some progress by installing pulseaudio in the SBC board (and in PC, since it has a desktop environment, pulseaudio was already installed):

cvlc -vvv v4l2:///dev/video0:width=1920:height=1080:fps=30:chroma=MJPG:live-caching=300 --input-slave=pulse://alsa_input.usb-SEEED_ReSpeaker_4_Mic_Array__UAC1.0_-00.multichannel-input --sout '#transcode{acodec=mp4a,ab=192,channels=6,samplerate=44100}:rtp{sdp=rtsp://:8554/}'

On the PC, works just fine. On the SBC the Mic is not being captured and shows this error (without stop):

...
avcodec encoder warning: cannot send one frame to encoder -22
avcodec encoder warning: cannot send one frame to encoder -22
avcodec encoder warning: cannot send one frame to encoder -22
avcodec encoder warning: cannot send one frame to encoder -22
...

Any ideas? Thanks.

[wout]

I get the same error code for sending mp3's to the chromecast audio. Ever got around to fixing the problem?

[Rémi Denis-Courmont]

If it works on PC but not on the SBC, there's probably a difference in the installed software. Are you sure you're using the same VLC and avcodec versions?

Also, well, realistically, if it requires specific hardware to reproduce, you're on your own to debug.