Setting custom HTTP headers inside SD scripts

[ale5000]

Previously I was appending the custom headers to the User-Agent but it was blocked in VLC.
My script strictly depend from custom HTTP headers, is there a new way to set custom headers?

[Jean-Baptiste Kempf]

Previously I was appending the custom headers to the User-Agent but it was blocked in VLC.
My script strictly depend from custom HTTP headers, is there a new way to set custom headers?

Ale5000, you're back???

Useragent, yes; custom headers, no.

[ale5000]

Useragent, yes; custom headers, no.

The Rai streaming server don't work at all without setting authentication in the custom HTTP header.
Is there any alternative to fix my script?

A lot of work was needed for creating the Rai TV script and it is frustrating that it cannot work for a minimal thing.

Edit: It would be nice to be able to set custom headers in a way like this: http://w3.impa.br/~diego/software/luasocket/http.html

[ale5000]

Ale5000, you're back???

I haven't too much free time but yes
How are you?

[Jean-Baptiste Kempf]

Ale5000, you're back???

I haven't too much free time but yes
How are you?

Great! We've made sooo much progress on subtitles

[mederi]

Edit: It would be nice to be able to set custom headers in a way like this: http://w3.impa.br/~diego/software/luasocket/http.html

Check following topic: https://forum.videolan.org/viewtopic.php?f=29&t=109033

[Rémi Denis-Courmont]

Previously I was appending the custom headers to the User-Agent but it was blocked in VLC.
My script strictly depend from custom HTTP headers, is there a new way to set custom headers?

Yes, send your custom HTTP request directly over sockets. The low-level HTTP request API has been not exposed to Lua.

[ale5000]

Previously I was appending the custom headers to the User-Agent but it was blocked in VLC.
My script strictly depend from custom HTTP headers, is there a new way to set custom headers?

Yes, send your custom HTTP request directly over sockets. The low-level HTTP request API has been not exposed to Lua.

Well, with some efforts maybe I can do this but then I will receive the streaming data directly from the live streaming; how can I make VLC play data directly?

Is there a way to pipe them directly to VLC?
I cannot use a temporary file since it is a live streaming and grow indefinitely and also it is ugly to use it.

[Rémi Denis-Courmont]

Lua scripts can only generate playlist items, not streams. You may need to write a custom native plugin for that specific use case - but on the bright side, a native plugin might be able to use the low-level HTTP request API.

[ale5000]

Lua scripts can only generate playlist items, not streams. You may need to write a custom native plugin for that specific use case - but on the bright side, a native plugin might be able to use the low-level HTTP request API.

Isn't there any plan to extend the functionality of LUA Addons?
They are nice but there are a lot of limitations.

Workaround this limitation will require a lot of hacks

[Rémi Denis-Courmont]

I don't think Lua should be allowed to generate actual streams. Performance would most certainly suck.

[ale5000]

I don't think Lua should be allowed to generate actual streams. Performance would most certainly suck.

Well, this is just the last resort until someone add the feature to pass custom headers (that shouldn't be hard, I don't need to set cookies or things where security can be a problem but just custom non-standard headers "viaurl" and "ttAuth", maybe also others to simulate a real browser but they aren't really important).

Obviously I wouldn't pass data directly if it was possible, but this option is useful for edge cases, one can also implement a full demuxer in Lua in a short time (until a proper one in C is implemented).
In my "IIS Smooth Streaming parser" Lua script that I have written a lot of time ago I actually create a full container data from scratch in a temp file then appended the video data from the streaming (endlessly) and passed the link of the temp file to another VLC instance; it worked fine but the downside is that the temp file grow indefinitely until you kill the first VLC instance since the live stream never end.

For now I'm thinking of maybe having 2 instances of VLC; one with a Lua SD that pass a link like http://127.0.0.1 that point to a local proxy in Lua in another instance of VLC (where its only function is to create the request with custom headers and pass the data as is).

[Rémi Denis-Courmont]

"Custom headers" will never happen. The high-level HTTP API cannot know how to handle unknown headers, by definition. You have to use the low-level HTTP API if you want to insert custom headers.

[ale5000]

Custom headers shouldn't be handled at all (they should just be passed), it should simply reject them if they contain special characters like ' " : \r \n \0 (or to be safer allow only alphanumeric characters) under header field name and under header field value (name and value handled separately) and nothing bad can happen.
And also it could skip the ones already handled like: Connection, Cookie, User-Agent, etc.

What can happen in the worst case?
I don't see how it could cause problems.

[Rémi Denis-Courmont]

Custom headers shouldn't be handled at all (they should just be passed), it should simply reject them if they contain special characters like ' " : \r \n \0 (or to be safer allow only alphanumeric characters) under header field name and under header field value (name and value handled separately) and nothing bad can happen.
And also it could skip the ones already handled like: Connection, Cookie, User-Agent, etc.

What can happen in the worst case?

1) A new version VLC supports the header line, and suddenly everything breaks.
2) The header line changes the server behaviour in a way that the high-level state machine does not handle.
3) It is unknown whether the header line is a connection or an end-to-end header line.
...
You cannot simply bypass a state machine and expects things to keep working.

You have to remove the state machine - use the low-level HTTP message API.

[ale5000]

1) They are custom headers so they will never be supported;
2) The problem is that the missing of the header now cause changes in the server behaviour; it is always possible to have changes in the server behavior also without custom headers;
3) I don't really understand this point, I know only request headers and response headers; beside that it can be added in the C code an array (or another variable type) that tag the origin of every header, the ones set by VLC and the ones set by Lua so that in every step is known where the header originate.

So the worst thing that can happen is that the Lua script stop working which is what it do now, so it isn't really worse.

[Rémi Denis-Courmont]

1) Begging the question.
2) Red herring.
3) If you don't even know, then how will code know?

[ale5000]

Ale5000, you're back???

I haven't too much free time but yes
How are you?

Great! We've made sooo much progress on subtitles

I haven't yet had time to look at them directly but from the VLC changelog I see many new subtitles format supported and many interesting changes, great work

[ale5000]

3) If you don't even know, then how will code know?

Quote from: http://freesoft.org/CIE/RFC/2068/143.htm

13.5.1 End-to-end and Hop-by-hop Headers
For the purpose of defining the behavior of caches and non-caching proxies, we divide HTTP headers into two categories:

End-to-end headers, which must be transmitted to the ultimate recipient of a request or response. End-to-end headers in responses must be stored as part of a cache entry and transmitted in any response formed from a cache entry.
Hop-by-hop headers, which are meaningful only for a single transport-level connection, and are not stored by caches or forwarded by proxies.
The following HTTP/1.1 headers are hop-by-hop headers:

Connection
Keep-Alive
Public
Proxy-Authenticate
Transfer-Encoding
Upgrade
All other headers defined by HTTP/1.1 are end-to-end headers.

Hop-by-hop headers introduced in future versions of HTTP MUST be listed in a Connection header, as described in section 14.10.

So, all custom headers are end-to-end headers, from the quote it doesn't seems hard to detect it.

Edit:
A possible implementation idea:

Code: Select all

my_item.set_custom_request_header(string name, string val, bool end_to_end_header=true)

Example:

Code: Select all

local element = { title=current_title, path=current_path } local my_item = vlc.sd.add_item(element) my_item.set_custom_request_header("MyCustomHeader1", "123", true) my_item.set_custom_request_header("MyCustomHeader2", "345", true)

[ale5000]

Edit: It would be nice to be able to set custom headers in a way like this: http://w3.impa.br/~diego/software/luasocket/http.html

Check following topic: https://forum.videolan.org/viewtopic.php?f=29&t=109033

It may be a solution but it is too much problematic to load compiled code since it is OS dependent and the script will never be universal.

[Rémi Denis-Courmont]

So, all custom headers are end-to-end headers, from the quote it doesn't seems hard to detect it.

Wrong. The specification only says that Connection, Keep-Alive, Public, Proxy-Authenticate, Transfer-Encoding, Upgrade are hop-by-hop, and other header lines from the specification are end-to-end.

That says nothing of custom headers.