Slow loading of audio when playing some mkv files

[Doctor ex Machina]

When I start playing or fastforward certain mkv files, it takes about half a second for the audio to start playing. Meanwhile the video starts instantly. This happens both on my laptop and the vlc app in my phone. I suspect it happens only with some x265 files but I'm not entirely sure. It's not a big problem, under normal circumstance you don't even notice, but sometimes when speech starts right at the very beginning of the video, there is no way for me to hear the first word.

Today I was editing some mkv files which didn't have this problem. I was trying to change default audio and subitles track using mkvtoolnix. For some reason the new files did have the sound issue. Then I ran those original files through mkvtoolnix without changing literally anything and the issue still appeared. I tried playing the files in other video players and none of them exhibited the same problem with sound, only the vlc in my laptop and phone.

Is it some setting in vlc that I should change? Or maybe some setting in mkvtoolnix that I could change when editing those files?

[Lotesdelere]

https://forum.videolan.org/viewtopic.php?f=14&t=156242