We're talking about cropping with --sout "#transcode{....,croptop=50,cropbottom=50,...}" , right?
This is broken for some time now, I think since 0.8.1 or earlier. A fix is postponed until some filter rework is done for 0.8.5.
You can download my patch for cropping and some more format magic here:
http://urichter.cjb.net/vdr/?h=vlc-croppadd&l=en
Cheers,
Udo