Bug: RMIDI (.rmi) Files ignore embedded soundfonts

[spessasus]

I think I've found a bug in the VLC MIDI player, but trying to report a bug to GitLab returns "Your account is pending approval from your GitLab administrator and hence blocked. Please contact your GitLab administrator if you think this is an error." so I'm posting it here.

The issue
The RIFF MIDI (.rmi) files can have embedded sf2 soundfonts in them, and VLC just ignores them.
Also I know that fluidsynth (which VLC uses for MIDI playback) doesn't support .rmi files natively. (see https://github.com/FluidSynth/fluidsynth/issues/1356)
This means that VLC has explicitly added support for these, which means that the format is supported, although incorrectly.
Some files use different channels for drums and drum maps specific to the soundfont, which breaks with every other soundfont except the embedded one.

Working software
I know 2 programs that read the embedded sf2 in the .rmi files correctly:

• Falcosoft's Midi Player 6 (Windows): https://falcosoft.hu/softwares.html

• SpessaSynth Midi Synthesizer: https://spessasus.github.io/SpessaSynth

Test files
I have several .rmi files for testing but am unsure how to attach them here. Please let me know how I can provide these files or if there is another method for sharing them with the development team.
Though the Falcosoft's Midi Player 6 comes with one of these, named "GRABBAG_EmbeddedSF2.rmi"

How to reproduce

• Get VLC

• Get an .rmi file with embeddedSF2. (For example the one bundled with Midi Player 6)

• Play it with SpessaSynth or MP6

• Try to play it with VLC and hear incorrect sounds

Tested version
The bug isn't OS dependent, hence I'm posting this here. But I've tested it on Arch Linux and Windows 10.
Vlc version (arch)

Code: Select all

$ vlc --version VLC media player 3.0.21 Vetinari (revision 3.0.21-0-gdd8bfdbabe8) VLC version 3.0.21 Vetinari (3.0.21-0-gdd8bfdbabe8) Compiled by builduser on (Jun 16 2024 20:52:10) Compiler: gcc version 14.1.1 20240522 (GCC) This program comes with NO WARRANTY, to the extent permitted by law. You may redistribute it under the terms of the GNU General Public License; see the file named COPYING for details. Written by the VideoLAN team; see the AUTHORS file.

[Rémi Denis-Courmont]

RMI is just a stupid RIFF header in front of the real SMF (.mid) file. VLC knows how to skip it. It was originally just Microsoft hijacking MIDI files.

If what you claim is true (and I've no reasons to doubt it), then somebody thought it was a good idea to redefine RIFF MIDI to embed fonts. Well that's really dumb IMO. I know karaoke embeds lyrics with MIDI. But a sound font is going to be orders of magnitude than the MIDI, so that's really really dumb.

And so, no VLC doesn't support that but it's a stretch to call it a bug. Also it is likely not fixable because FluidSynth only supports local files for fonts, not abstract bitstreams.

[spessasus]

RMI is just a stupid RIFF header in front of the real SMF (.mid) file. VLC knows how to skip it. It was originally just Microsoft hijacking MIDI files.

If what you claim is true (and I've no reasons to doubt it), then somebody thought it was a good idea to redefine RIFF MIDI to embed fonts. Well that's really dumb IMO. I know karaoke embeds lyrics with MIDI. But a sound font is going to be orders of magnitude than the MIDI, so that's really really dumb.

And so, no VLC doesn't support that but it's a stretch to call it a bug. Also it is likely not fixable because FluidSynth only supports local files for fonts, not abstract bitstreams.

Hi Rémi,
Thanks for your response.
I would like to adress some of your points:
Reading the soundfont
Looking at the file with a hex editor, it looks like the soundfont is just another chunk in the file, like the SMF data chunk.
If VLC knows how to skip the RIFF header to get to the SMF data, why can't it do the same for the embedded soundfont?
The embedded RMI file looks like this:
"RIFF" chunk: "RMID" type

• "data" chunk: (the SMF data)

• "LIST" chunk: "INFO" type (the file metadata)

• "RIFF" chunk: "sfbk" type (the sf2 data)

Wouldn't it be simple to check for the sfbk chunk in the .rmi file?
Here's some JS pseudo-code with the basic logic:

Code: Select all

function loadRMID(binary) { // note: header is the fourcc and type is the first 4 letters in the data chunk const mainChunk = loadRIFF(binary); if(mainChunk.header != "RIFF" || mainChunk.type != "RMID") { throw new Error("not an RMI file"); } while(mainChunk.readIndex < mainChunk.data.length) { const chunk = loadRIFF(mainChunk.data) switch(chunk.header) { case "data": loadMIDI(chunk.data); break; case "RIFF": loadSoundFont(chunk.data) } } }

About fluidsynth accepting files
A simple solution would be to put the sf file in a temporary folder then tell fluidsynth to load that file.

About the whole idea
Also, I think the embedded soundfont is a great idea. This solves the biggest MIDI problem: different soundfonts across different OSes and devices.
Having an embedded soundfont ensures that the MIDI sounds the same everywhere.
The files I have are really small, (like 5MB) because the soundfonts appear to be trimmed specifically for the SMF sequence.

I have uploaded the test RMI file to filetransfer.io: https://filetransfer.io/data-package/RaocztZ8#link
Feel free to test it out!