I struggled with this today, much to the detriment of my piles of laundry and messy room. For me the problem was that the unmodified keys (I wanted plain ol' up & down to control the volume) were already assigned to keys, BUT they were keys that were not available through the GUI hotkey change interface (even with "advanced options" checked).
The solution was obvious when I opened the config file in a text editor (back it up 1st) and saw that unmodified up and down are already assigned for dvd menu navigation (not something I found I could change in the regular ol' hotkeys interface)
~Original VLCrc config:
(the # is commenting out; which I took as "this is the default", confirmed by the fact that other successfully assigned hotkeys were not preceded by the hash and had the values I expected)
# Navigate up (key) ********* Probably doesn't work cuz it's assigned twice, but I don't use VLC for DVD's so I wouldn't know
#key-nav-up=Up
# Navigate down (key) ********* See above
#key-nav-down=Down
********* blah blah blah other hotkeys
# Volume up (key) ********* Doesn't work
key-vol-up=Up
# Volume down (key) ********* See above
key-vol-down=Down
So I changed it to
~Modified VLCrc config:
# Navigate up (key) ********* I chose the Alt-Ctrl modifiers b/c AFAIK they were unused, you could also use the value "Unset" if you didn't want to use 'em at all
key-nav-up=Alt-Ctrl-Up
# Navigate down (key) ********* See above
key-nav-down=Alt-Ctrl-Down
********* blah blah blah other hotkeys
# Volume up (key) ********* Hooray, it works!!!
key-vol-up=Up
# Volume down (key) ********* See above
key-vol-down=Down
To find your config file:
http://www.videolan.org/doc/faq/en/index.html#id269879
Dsastray