How to fetch urls, which is served with 206 - Partial Content

[jakobdo]

Hello,
i have been using this script for a while:
https://bbs.archlinux.org/viewtopic.php?id=183762
Worked like a charm. But the last days it did not work.
I found that 3 out of 4 returned status code 206, and if i "printed" the response, is was not fulfilled.
Can vlc.stream somehow tell server not to serve 206 or can vlc.stream download the entire file, even if it get the status 206?

I just started using lua, so if this is a noob question, please bear with me.

[Rémi Denis-Courmont]

206 is perfectly fine and expected.

[jakobdo]

206 is maybe ok. But vlc.stream(url) only reads the "first" part.

local stream = vlc.stream(url)
repeat
line = stream:readline()
string = string..line
until line ~= nil
return json.decode(string)

Will return something like:
[{"id":351,"key":"indiedance","name":"Indie Dance","playlist":"http:\/\/listen.di.fm\/public2\/indiedance.pls"},{"id":349,"key":"jazzhouse","name":"Jazz House","playlist":"http:\/\/listen.di.fm\/public2\/jazzhouse.pls"},{"id":352,"key":"liquidtrap","name":"Liquid Trap","playlist":"http:\/\/listen.di.fm\/public2\/liquidtrap.pls"},{"id":350,"key":"idm",
Which is not the full response.
The "code" expect the entire json-response like this:
http://listen.di.fm/public2

[roland1]

The code itself has flaws

Code: Select all

repeat line = stream:readline() string = string..line -- error if line == nil until line ~= nil -- breaks if line is _not_ nil

In effect it reads only the first line if existent, else an error occours.
So, I tested...

Code: Select all

local sink = {} repeat local line = stream:readline() table.insert(sink, line) -- no error, no string pooling. until line == nil local string = table.concat(sink, "\n") vlc.msg.info(string)

... and voila, it doesn't work any better;)
Instead, there is a overwhelming chance to get only fragments with vlc.stream.
Maybe by coincidence, I got an error message when the json was _complete_:
[] access_http access error: error: HTTP/1.1 416 Requested Range Not Satisfiable
Btw, my browser has no difficulties to load that page.

[jakobdo]

The download also works fine directly in my browser.
But when i try "your" new code, i still just get the "first" part of the url / json.
So i don't think your code is better then the first one provided.

[roland1]

Short story (at least short enough to repeat it),

... and voila, it doesn't work any better;)
[...]

Long story, Your code fails (ever tried ""..nil in Lua?) whenever there is not exactly one line in the stream, although, this seems to be _no_ problem as apparently the jsons are packed _without_ newlines. But, if this'd be expected (I'd not do it, I don't need to) then why not simply

Code: Select all

local string = stream:readline()

So I think my code is, as always, extra-galactic better than the first one provided, though I'd have prefered stream:read(chunksize) but this'd have made even more noise in the source.

[jakobdo]

I have tested the code:

local string = stream:readline()
Same problem.
It is like the vlc.stream does not support 206.
I don't know if I somehow can tell the vlc.stream not to support partial content?
Or if i can read "response" code in vlc.stream and keep reading while i get data or something.
Or maybe i just need to learn LUA.

[roland1]

You could repeat (some 10 times) open+read stream until json.decode succeed, ugly but could work as there is some chance to get the whole json. At the cost of having wget on Your (linux) machine this is nicer:

Code: Select all

-- -- di_vlc.lua - VLC Service Discovery Plugin -- Adds Digital Imported based Radiostations to the VLC Playlist Window -- -- App (*nix): /usr/lib/vlc/ -- (win): C:\Program Files\VideoLAN\VLC\ -- -- Base (*nix): ~/.local/share/vlc/ -- (win): %appdata%\vlc\ -- -- Folder Structure: -- $base/lua/sd/di_vlc.lua # This Script -- $base/lua/sd/modules/common.luac # Found in $app/lua/intf/modules -- function descriptor() return {title = "DI Radio Stations"} end local json local function stream2table(url) -- requires wget. local pr = io.popen("wget -q -O - "..url) local s = pr:read"*a" pr:close() return json.decode(s) end function main() json = require "dkjson" local stations = {["DI.FM"] = {"di.fm", "2"}, -- 2 = aac ["JazzRadio"] = {"jazzradio.com", "3"}, -- 3 = mp3 (aac pls = 404) ["RockRadio"] = {"rockradio.com", "2"}, -- 2 = aac ["Sky.FM"] = {"sky.fm", "3"}} -- 3 = mp3 (aac pls = 404) for key, data in pairs(stations) do local site = "http://www."..data[1] local url = "http://listen."..data[1].."/public"..data[2] local ok, result = pcall(stream2table, url) local node = vlc.sd.add_node({title = key}) if ok then for index, item in ipairs(result) do node:add_subitem({title = item["name"], path = item["playlist"], description = item["description"], genre = item["name"], url = site, publisher = "Digital Imported"}) end else vlc.msg.info("stream2table error @ "..url) end end end

[jakobdo]

I dont like the "wget" solution. )
I have just downloaded the json files and i am loading them offline.
Not perfect, but it works.
Too bad lua cant handle these partial content.