VLCKit tvOS: prevent Audio mix-down to Stereo

ssbmaccom · Postby **ssbmaccom** » 09 Dec 2016 21:04

In my App I am currently experimenting to allow playback of TS streams (live TV from Sat Dish receivers) with Multi-Channel Audio.
Well - Audio PassThrough (using spdiff out) is colliding with the Audio effects of tvOS (navigation sounds).

I activated debugLogging and watched the results.
I can see, that VLC is recognizing the right formats and preparing a conversion pipeline - and it always ends with

Code: Select all

2016-12-09 19:19:04.097436 vuplusTV[530:567245] DEMUX_SET_ES 1283
2016-12-09 19:19:04.097484 vuplusTV[530:567245] enabling pid 32 from program 131
2016-12-09 19:19:04.097526 vuplusTV[530:567245] enabling pid 1279 from program 131
2016-12-09 19:19:04.097837 vuplusTV[530:567245] enabling pid 1283 from program 131
2016-12-09 19:19:04.097890 vuplusTV[530:567245] enabling pid 1284 from program 131
2016-12-09 19:19:04.097973 vuplusTV[530:567245] enabling pcr pid 1279 from program 131
2016-12-09 19:19:04.103911 vuplusTV[530:567457] A/52 channels:6 samplerate:48000 bitrate:384000
2016-12-09 19:19:04.104075 vuplusTV[530:567457] restarting module due to input format change
2016-12-09 19:19:04.106350 vuplusTV[530:567457] removing module "avcodec"
2016-12-09 19:19:04.107248 vuplusTV[530:567457] ffmpeg codec (ac3_at) stopped
2016-12-09 19:19:04.107438 vuplusTV[530:567457] looking for decoder module matching "any": 32 candidates
2016-12-09 19:19:04.107513 vuplusTV[530:567457] reusing audio output
2016-12-09 19:19:04.107601 vuplusTV[530:567457] VLC is looking for: 'a52 ' 48000 Hz 3F2R/LFE frame=1536 samples/1536 bytes
2016-12-09 19:19:04.110180 vuplusTV[530:567457] we set the AU format: [48000.000000][mcpl][9][8][1][8][2][32]
2016-12-09 19:19:04.110293 vuplusTV[530:567457] the actual set AU format is [48000.000000][mcpl][9][8][1][8][2][32]
2016-12-09 19:19:04.391617 vuplusTV[530:567457] audio output unit started: 0
2016-12-09 19:19:04.391749 vuplusTV[530:567457] analog AudioUnit output successfully opened
2016-12-09 19:19:04.391816 vuplusTV[530:567457] output 'f32l' 48000 Hz Stereo frame=1 samples/8 bytes
2016-12-09 19:19:04.391997 vuplusTV[530:567457] looking for audio volume module matching "any": 2 candidates
2016-12-09 19:19:04.392093 vuplusTV[530:567457] using audio volume module "float_mixer"
2016-12-09 19:19:04.392143 vuplusTV[530:567457] input 'a52 ' 48000 Hz 3F2R/LFE frame=1536 samples/1536 bytes
2016-12-09 19:19:04.392262 vuplusTV[530:567457] looking for audio filter module matching "scaletempo": 11 candidates
2016-12-09 19:19:04.392319 vuplusTV[530:567457] format: 48000 rate, 6 nch, 4 bps, fl32
2016-12-09 19:19:04.392392 vuplusTV[530:567457] params: 30 stride, 0.200 overlap, 14 search
2016-12-09 19:19:04.392460 vuplusTV[530:567457] 1.000 scale, 1440.000 stride_in, 1440 stride_out, 1152 standing, 288 overlap, 672 search, 2400 queue, fl32 mode
2016-12-09 19:19:04.395198 vuplusTV[530:567457] using audio filter module "scaletempo"
2016-12-09 19:19:04.395549 vuplusTV[530:567457] conversion: 'a52 '->'f32l' 48000 Hz->48000 Hz 3F2R/LFE->3F2R/LFE
2016-12-09 19:19:04.395691 vuplusTV[530:567457] looking for audio converter module matching "any": 6 candidates
2016-12-09 19:19:04.395747 vuplusTV[530:567457] no audio converter modules matched
2016-12-09 19:19:04.395817 vuplusTV[530:567457] looking for audio converter module matching "any": 6 candidates
2016-12-09 19:19:04.395862 vuplusTV[530:567457] no audio converter modules matched
2016-12-09 19:19:04.395907 vuplusTV[530:567457] cannot find decoder for conversion pipeline
2016-12-09 19:19:04.395987 vuplusTV[530:567457] cannot add user audio filter "scaletempo" (skipped)
2016-12-09 19:19:04.396034 vuplusTV[530:567457] removing module "scaletempo"
2016-12-09 19:19:04.396148 vuplusTV[530:567457] conversion: 'a52 '->'f32l' 48000 Hz->48000 Hz 3F2R/LFE->Stereo
2016-12-09 19:19:04.396231 vuplusTV[530:567457] looking for audio converter module matching "any": 6 candidates
2016-12-09 19:19:04.396280 vuplusTV[530:567457] no audio converter modules matched
2016-12-09 19:19:04.397893 vuplusTV[530:567457] looking for audio converter module matching "any": 6 candidates
2016-12-09 19:19:04.398031 vuplusTV[530:567457] no audio converter modules matched
2016-12-09 19:19:04.398107 vuplusTV[530:567457] cannot find decoder for conversion pipeline
2016-12-09 19:19:04.398245 vuplusTV[530:567457] cannot setup filtering pipeline
2016-12-09 19:19:04.560673 vuplusTV[530:567457] removing module "float_mixer"
2016-12-09 19:19:04.560793 vuplusTV[530:567457] keeping audio output
2016-12-09 19:19:04.568906 vuplusTV[530:567457] failed to create audio output
2016-12-09 19:19:04.569057 vuplusTV[530:567457] option quiet does not exist
2016-12-09 19:19:04.569301 vuplusTV[530:567457] CPU flags: 0x00000068
2016-12-09 19:19:04.570854 vuplusTV[530:567457] codec (ac3_at) started
2016-12-09 19:19:04.570930 vuplusTV[530:567457] using decoder module "avcodec"
2016-12-09 19:19:04.571506 vuplusTV[530:567457] reusing audio output
2016-12-09 19:19:04.571607 vuplusTV[530:567457] VLC is looking for: 's16l' 48000 Hz 3F2M/LFE frame=1 samples/12 bytes
2016-12-09 19:19:04.574073 vuplusTV[530:567457] we set the AU format: [48000.000000][mcpl][9][8][1][8][2][32]
2016-12-09 19:19:04.574152 vuplusTV[530:567457] the actual set AU format is [48000.000000][mcpl][9][8][1][8][2][32]
2016-12-09 19:19:04.859856 vuplusTV[530:567457] audio output unit started: 0
2016-12-09 19:19:04.859952 vuplusTV[530:567457] analog AudioUnit output successfully opened
2016-12-09 19:19:04.860021 vuplusTV[530:567457] output 'f32l' 48000 Hz Stereo frame=1 samples/8 bytes
2016-12-09 19:19:04.860122 vuplusTV[530:567457] looking for audio volume module matching "any": 2 candidates
2016-12-09 19:19:04.860171 vuplusTV[530:567457] using audio volume module "float_mixer"
2016-12-09 19:19:04.860216 vuplusTV[530:567457] input 's16l' 48000 Hz 3F2M/LFE frame=1 samples/12 bytes
2016-12-09 19:19:04.860297 vuplusTV[530:567457] looking for audio filter module matching "scaletempo": 11 candidates
2016-12-09 19:19:04.860347 vuplusTV[530:567457] format: 48000 rate, 6 nch, 4 bps, fl32
2016-12-09 19:19:04.860409 vuplusTV[530:567457] params: 30 stride, 0.200 overlap, 14 search
2016-12-09 19:19:04.860471 vuplusTV[530:567457] 1.000 scale, 1440.000 stride_in, 1440 stride_out, 1152 standing, 288 overlap, 672 search, 2400 queue, fl32 mode
2016-12-09 19:19:04.861799 vuplusTV[530:567457] using audio filter module "scaletempo"
2016-12-09 19:19:04.861861 vuplusTV[530:567457] conversion: 's16l'->'f32l' 48000 Hz->48000 Hz 3F2M/LFE->3F2M/LFE
2016-12-09 19:19:04.861949 vuplusTV[530:567457] looking for audio converter module matching "any": 6 candidates
2016-12-09 19:19:04.862025 vuplusTV[530:567457] s16l->f32l, bits per sample: 16->32
2016-12-09 19:19:04.862094 vuplusTV[530:567457] using audio converter module "audio_format"
2016-12-09 19:19:04.862231 vuplusTV[530:567457] conversion pipeline complete
2016-12-09 19:19:04.862302 vuplusTV[530:567457] conversion: 'f32l'->'f32l' 48000 Hz->48000 Hz 3F2M/LFE->Stereo
2016-12-09 19:19:04.862385 vuplusTV[530:567457] looking for audio converter module matching "any": 6 candidates
2016-12-09 19:19:04.862431 vuplusTV[530:567457] using audio converter module "simple_channel_mixer"
2016-12-09 19:19:04.862473 vuplusTV[530:567457] conversion pipeline complete
2016-12-09 19:19:04.862541 vuplusTV[530:567457] looking for audio resampler module matching "any": 1 candidates
2016-12-09 19:19:04.863320 vuplusTV[530:567457] using audio resampler module "ugly"
2016-12-09 19:19:04.864262 vuplusTV[530:567457] playback way too early (-143292): playing silence
2016-12-09 19:19:04.864336 vuplusTV[530:567457] inserting 6878 zeroes

I already change the [AVAudioSession sharedSession] to use the max available number of channels - which is 6 when setting Dolby 5.1 in Audio System Preferences. Usually Apps run with only 2 channels enabled.

Anyway... it always comes with mix-down to stereo. It would not even be necessary to decode a52, but simply to pass it to audio out without colliding with tvOS sound effects - or simply drop the "conversion: 'f32l'->'f32l' 48000 Hz->48000 Hz 3F2M/LFE->Stereo" module in the pipeline. Until this module all looks promising.

Is there a chance to tell VLCKit NOT to use this stereo mixdown module?

Postby **fkuehne** » 14 Dec 2016 23:21

No, the audio output module does only support 2.0 PCM at the moment. It is scheduled to be changed quite soon for iOS and chances are high that the new patch will also work for tvOS. Further down the road, we will also add a public API for audio pass-through of AC-3 and E-AC-3 content as already supported by VLC's official tvOS app.

ssbmaccom · Postby **ssbmaccom** » 15 Dec 2016 12:06

Thanks for your reply.
What I figured out is, that for tvOS the same code is used as for iOS. Especially VLCKit/libvlc/vlc/modules/audio_output/audiounit_ios.m.
There the code is forcing the AudioUnit (and the required filters) to mix-down to 32-bit float, LPCM, Stereo (line 225 to 232) for non-spdif.
For iOS this will usually make sense, as there are no 5.1 headphones - only for Airplay something better would be useful.

When using spdif the AudioUnit is setup to LPCM, 16-bit integer, Stereo. But in this case libVLC will render the audio stream as is and in case of 5.1 tvOS runs into issues, when it tries to mix system sounds (Navigation sounds) with the audio stream from libVLC.

I may do some experiments on my local sandbox but for a final solution we/you should consider to add a new file audiounit_tvos.m, fixup the Makefiles accordingly. Then we can prepare dedicated audio handling for tvOS.

BTW: build of the framework fails due to an issue in gas when building libvpx contrib for tvOS Simulator. Therefore there had been no nightly build (iOS, tvOS and MacVLC) since Dec. 3rd

The VideoLAN Forums

VLCKit tvOS: prevent Audio mix-down to Stereo

VLCKit tvOS: prevent Audio mix-down to Stereo

Re: VLCKit tvOS: prevent Audio mix-down to Stereo

Re: VLCKit tvOS: prevent Audio mix-down to Stereo

Who is online