Page 1 of 1

invmem RV24 wrong-endian channel masks?

Posted: 12 Apr 2011 02:43
by plalonde
I'm using the invmem in-memory codec, and the red/blue channels are coming out inverted. Looking at modules/codec/invmem.c:183-186 in the case for VLC_CODEC_RGB24, I see the code that sets up the channel masks. Apart from being endian-dependent, I think the constants are incorrect: in-memory RGB ordering for little-endian architectures (which most of us are on) requires rmask = 0x0000ff and bmask = 0xff0000 in order to stride memory as R0G0B0R1G1B1...

The case for VLC_CODEC_RGB32 needs a similar fix.

Does anyone think different before I figure out how to submit a patch?

Paul