This delays may be caused by mux ts encoding. Try to see into sout-ts- options. In particular, sout-ts-shaping=10 and sout-ts-dts-delay=20 may help. But note, that some minimum caching' delay are necessary, you may combine sout-ts-shaping, sout-ts-dts-delay and sout-udp-caching on transmit side, and...