VLC, Matroska, and ReplayGain

[AliceWonderMisc]

I have some confusion regarding Matroska and :

A) How to tag for ReplayGain in general

B) How to tag for ReplayGain when different audio streams are present

C) How to tag for ReplayGain with Opus specifically

D) How VLC interprets the specification, assuming it does at all.

-=-

A - How to tag for ReplayGain in general

According to the Matroska Tag Specification there are two "official" tags for ReplayGain

https://matroska.org/technical/specs/tagging/index.html

These are REPLAYGAIN_GAIN and REPLAYGAIN_PEAK

I'm assuming that for album gain/peak they would be put in either TargetType ALBUM (ID of 50) or COLLECTION (ID of 70) and for track within a TargetType TRACK (ID of 30) so the user could specify when playing which gain they prefer applied (usually former when listening to album and latter in non-album playlists)

However where the specification confuses me is the type, that page says they are Binary and not String.

At https://github.com/mpv-player/mpv/issues/2128 it looks like there's also confusion there as to whether binary is really appropriate or whether string is what should be used, and it looks like MPV is actually using string. If it is binary, are we suppose to include the white space and dB in the binary represention of the REPLAYGAIN_GAIN tag? What does VLC expect the tag to be?

B - How to tag for ReplayGain when different audio streams are present

I'm creating Matroska audios with both AAC and Opus streams. It's spoken content, MP3 really sucks for spoken content, Opus is the best for that and vast majority of Matroska capable media players can handle Opus but I'm hoping that by having the AAC there too, people who want to add the audios to an iTunes based library can convert to .m4a without needing to transcode from Opus - the converters can just extract the AAC stream. FFMpeg can do it that way for example with the right switches. I have no clue if the converters that exist do that but I'm hoping some do.

Anyway proper ReplayGain to apply actually depends upon the lossy encoding itself. Usually they are fairly close but with Opus I suspect they are the most different because Opus applies R128Gain (similar but different) internally.

You can see the difference by taking a lossless WAVE file, use wavegain to calculate the gain, then encode it into several different lossy formats. Convert those lossy formats back to wave and run wavegain in the results - and you will see different values from the original wave and from each other. Usually not a lot but there.

So what I'm doing is creating individual .xml files for the Matroska tags DTD for each stream that has only the stream specific tags (right now just ENCODER, REPLAYGAIN_GAIN, and REPLAYGAIN_PEAK)

Then I use the --tags n:foo.xml switch to mkvmerge to specify which stream the metadata applies to.

Using mkvinfo it looks like I am doing it right, but using the mediainfo program it always displays the stream specific tags for the last one given as if they were global tags, and when I view the metadata in VLC it appears to do the same thing.

Thus --tags 0:aac.xml --tags 1:opus.xml results in the opus specific tags looking like global tags to both mediainfo and to VLC but since AAC is the first stream, it's the stream VLC plays by default - meaning if VLC is applying the REPLAYGAIN_GAIN tag at all, it is probably applying the Opus specific stream tag to the AAC stream.

If I changed the order of the streams so Opus stream was first then it would be reversed.

C) How to tag for ReplayGain with Opus specifically

Part of the Opus specification is a different gain adjustment scheme called R128 Gain, which I believe is an EU standard, meaning that to comply with EU laws certain types of media (TV advertisements for example) they have to conform to that standard.

My understanding and this might be wrong is that the R128Gain information is in a header for the Opus stream itself and does not rely upon tag based metadata (I think like how lame applies ReplayGain) and that an Opus complian decoder will apply the gain when decoding the audio.

If every audio in a playlist was Opus we wouldn't neet to worry about ReplayGain but since playlists often don't exclusively have Opus, for consistent volume we need to apply a ReplayGain that adjust the gain further after R128Gain has been applied.

When I decode opus to wave using FFmpeg and use wavegain, the suggested adjustment (small sample size) has both been different and smaller than with other lossy formats indicating to me (and I do need to do more experiments) that FFmpeg applies R128Gain to the Opus stream when decoding to WAVE so the resulting small adjustment brings the audio to the same perceived volume as other formats that don't use R128Gain but only use ReplayGain.

If my understanding of this incorrect please tell me but if my understanding is correct, it highlights the important of having VLC apply the ReplayGain associated with the specific stream.

D) How VLC interprets the specification, assuming it does at all.

I like to try to do things to specified standard but at the end of the day if the majority of media players that do anything with ReplayGain don't follow the standard but use some other logic, it is better to use what players actually support.

What does VLC do with ReplayGain and Matroska containers?

AFAIK there isn't a program like mp3gain, aacgain, or vorbisgain that will apply the tags to a Matroska file so what I'm doing is decoding the lossy format to wave and using wavegain to get the values and creating the metadata manually (which I have to do anyway to have an accurate ENCODER metadata for what encoded the stream) so I have some flexibility in how to create the tags, I want to do it in a way that VLC understands. Well preferably in a way that both VLC and foobar2000 understands but VLC is more important to me.

-=-

Thank you for your time.

[AliceWonderMisc]

Okay B is solved - I was lacking a Target in the track specific XML files, I had assumed that mkvmerge --track N:foo.xml would do the right thing but it didn't, for some reason it was assigning it a target of movie (50) and while I'm not sure why that caused them to look like global tags to mediainfo and vlc, that is what happened.

Specifying a TargetType of "TRACK" with value of 30 in the stream specific xml fixed that issue.

Still don't know if I should keep using string type for the ReplayGain data or use binary like the matroska spec suggests (Issue A).

I think I am doing C correctly, and D of course I can't answer myself.