Chromecast subtitles support

[zwaa]

I hope this thread stays active so everyone can follow the status of Chromecast subtitles without having to create a new topic for each new version of VLC.

Now on VLC 3.4.8 (in my case, iPhone), what are the prospects for casting subtitles to Chromecast devices?
Is there a concrete plan yet?

[fkuehne]

VLC version 4 is supposed to have that feature and as soon as the iOS port switches to that version, the feature will be available. This is not imminent.

[fkuehne]

You can find the bug report on our tracker here: https://code.videolan.org/videolan/vlc/-/issues/19310

[zwaa]

You can find the bug report on our tracker here: https://code.videolan.org/videolan/vlc/-/issues/19310

Thanks for the link to the tracker page.
I've studied it meticulously, and have a suggestion: see point 2.

For the benefit of everyone who, like me, are interested in the problems involved with Chromecasting subtitles, I've tried to interpret and explain roughly what the tracker page says, with some comments.

1. Obviously, you must use a multiplexer (mux), a data selector to sync (switch between at high speed) audio and video into one file stream, and in order to also sync the subtitles into that stream, the WebVTT format is apparently the only format that can be used because VLC uses the WebM media file format to embed it within mkv (Matroska tracks container). But according to the tracker page, Chromecast doesn’t notice the WebVTT track. The author speculates that some "dedicated command" (not further explained) might be needed.

2. The above is at the heart of the issue: how to package the subtitles file so Chromecast devices recognizes it?

• I think that maybe, just maybe I've found an article that can shed some light on it: "Use Media Tracks (Google Cast SDK)". However, I don't have the time or the tools to look closer into it:
https://developers.google.com/cast/docs ... dia_tracks

• If the article above helps, obviously most of the text below is superfluous.

3. The author remarks that "it seems all subtitles are split file" and bases that on two statements on this page:
https://developers.google.com/cast/docs/media

• The first statement says "With adaptive bitrate streaming protocols, you must implement CORS":
https://developers.google.com/cast/docs ... _protocols
CORS is a security policy for web applications, and adaptive bitrates are associated with data streams that are split into 10-second segments (see note: *HLS).

• And the second statement says "Your subtitle resources must implement CORS":
https://developers.google.com/cast/docs ... d_captions

4. Apart from WebVTT files, Chromecast also supports the TTML and CEA-608/708 subtitle formats, but it’s hinted that either the WebM file format or the mkv container - or both - doesn’t support them. So if only WebVTT works, and the subtitles are in another format, they’d have to be extracted/transcoded (probably in their entirety) first.

5. "Side loaded subtitles": this is not further explained, but could perhaps open up the possibility of applying the TTML or CEA-608/708 formats in stead?

• The author argues that in order for "side loading" the subtitles to work, trigging (using a "command") the Chromecast device to recognize the subtitles is still necessary, although I think in theory it may not actually be so when "side loading" (whatever that is), and it’s also possible that TTML or CEA-608/708, if either can be used, is recognized without a trigger. But all this is just speculations: I actually don't believe that "side loading" the subtitles would work, regardless how you'd implement it.

• The author also establishes that "side loading" would require extracting/transcoding the subtitles in their entirety before streaming them and concludes that "The only option in that case seems to be to provide the subtitles in segments (*HLS, ...), which probably would need the whole media to be adaptive" (the reasoning is unclear)

6. Lastly, there’s the option of "Burning in subtitles", which simply means to add the subtitles onto the video frames. That would require VLC to extract the subtitles file, and it would also make redundant any subtitles characteristics (fonts/effects/colors/sizes/etc) set in the Chromecast device, so VLC would need to choose and apply those characteristics in its stead.

Note: *HLS, or HTTP Live Streaming, is an online streaming protocol developed by Apple. Support for the protocol is widespread in media players (including Chromecast), web browsers, mobile devices, and streaming media servers, and it’s the most popular video streaming protocol today:
https://castr.com/blog/what-is-hls-stre ... w-it-works

• From the article above:

HLS uses a system called Adaptive Bitrate Streaming. It splits your video up into 10-second segments. It then creates multiple versions of each segment in different quality levels. All of this happens behind the scenes, in real-time.

Next, it looks at your viewer’s device, connection speed, and preferences. The HLS protocol determines the ideal video for the situation and sends it across. The idea is to give a smooth viewing experience with no lags or stutters.

Because the video is broken up into 10-second segments, HLS can adjust quality on the fly. This means that as a user’s internet connection changes, the HLS powered stream will switch back and forth between video qualities by changing which segments of video are delivered.

[daimubate]

Can't wait to be able to cast movies+subtitles from VLC to my Chromecast.