Thanks for the reply.
What I am experiencing is that the player sends out undecoded output only when it sees that s/pdif is "available". In other words, if I select from control panel my inboard sound card, which supports s/pdif, I will get the digital output. Unfortunately, this card re-samples the 44.1 khz dts to 48.0khz, as most sound cards do when they aren't receiving signal from a hardware player. thus I can play ac3 files from hard drive (dvd rips) but not dts-cd files.
That is why I have the outboard usb sound card, which is designed not to resample like the SoundBlaster line of products. Even though it is selected from control panel as the default audio device, VLC evidently cannot see that it supports s/pdif, and turns the 5.1 channel dts-cd data stream into stereo, even when I tell VLC to "use s/pdif output when available".
I have another software player, JRiver Media Jukebox, that is not so "smart" and does output dts-cd s/pdif to my outboard sound card, so I know it is possible, and daily see that it works in 5.1 surround. But Media Jukebox doesn't play ac3. Consequently I have to toggle between 2 sound cards and 2 media players to go between ac3 and dts, and I can't do this reliably after several beers in the middle of a party
I'm hoping VLC can be the player that will play both ac3 48000 and dts 44100 with s/pdif output, into my usb-to-toslink outboard passthrough sound card, but I would have to override the "if available" check, or disable the dts-to-stereo decoding that is going on now.
I appreciate the help. At worst, I may have to keep toggling and go easy on the beers.
donEsteban