Problems streaming with capture card

[JHT]

I have one box set up as a streamer using an MPG file as source sending it with RTP with MPG Transport system to another box set up as player and it will receive the stream and show it.
If I change the source to a Datapath E2s capture card using Direct Show, I can see the capture fine locally on the streamer box but the player wont show anything.
I have tried changing to RTP Video and audio with same results.
I am not doing any transcoding and I have tried with and with streaming all elementary streams.
What am I doing wrong?

[Rémi Denis-Courmont]

It's hard to say without logs. You may need to transcode.