I observed the same problem on mac os 10.13.1 over both smb and afp. I have not tried other players.
I wonder if there are any debugging tools available for vlc, like "displaying current buffer size" and so on. My guess is prefetching is not fired up at the right moment, hence the delay.