Having had the same problem, i have been using a work around that seems to be working for me, in VLC prefs on my mac, i set the OSD Font Renderer to be Freetype 2 - now, my text effects work again, i can set my own font size in pixels, everything is as it should be.
I suspect leaving the font renderer as default means it uses the Quartz one, which is lovely, but no effects
Ok just to explain a little more clearly, im using 1.0.0 RC3 Goldeneye for Intel. Also noticed in the past that effects stopped working, but didnt investigate why. Today, knowing the GF is going to want to watch films, i thought id try and make it work.
In the Advanced preferences, in the main Subtitle / OSD tab - i played around with selecting different renderers and settings -eventually, selecting Freetype2 got my effects working again. My ideal settings and effects?:
Font - Arial
Size - 18px
Colour- White
Effect - Background
Works great - don't know what id do without the background effect - ps. are there any settings for it somewhere?