Request for package: tinyproxy

[mk1]

Is it possible to get https://tinyproxy.github.io/ as a package to install on the custom firmware please?

Reason:
When trying to download a .ts using webif "OPT+ -> Download" using Safari (or indeed Chrome, Firefox or Opera) on iOS 13.3.1 the download always fails.
Issuing the same request as Safari but with curl gives: curl: (18) transfer closed with outstanding read data remaining
After some trial and error I've found that removing the Connection: keep-alive HTTP Request header resolves the issue.
I wrote a simple Proxy server in Java which forwards requests to the original DLNA download URL after removing that header. Pointing the browser at my Proxy server results in a successful download.
It seems, unsurprisingly, that there are some operability issues with the Humax DLNA server so my plan was to:
1) Install tinyproxy on the Humax and configure it to forward requests to the DLNA port after removing the header
2) Modify the webif "OPT+ -> Download" to use the tinyproxy port instead of the DLNA port

 

[Black Hole]

When trying to download a .ts using webif "OPT+ -> Download" using Safari (or indeed Chrome, Firefox or Opera) on iOS 13.3.1 the download always fails.

Downloads work fine for me, but my recordings are decrypted therefore the WebIF download button does not use DLNA.

Once you click "download", pre iOS13 nothing seemed to be happening but it downloaded in the background. The problem was to know when it had finished and where to find the file. With iOS13 there is a download icon at top right in Safari (next to the share button) with a tiny progress bar under it, and clicking it brings up a progress status panel and a listing of all existing downloads. These are accessible through the Files app.

However, what I prefer to do is use the FileManager (from the App Store) and link it to SMB on the HDR-FOX (samba package). This will only work with decrypted files.

 

[mk1]

Downloads work fine for me, but my recordings are decrypted therefore the WebIF download button does not use DLNA.

Yes it seems to be the Humax DLNA that's causing the issue. Hence my idea for proxying it.

Once you click "download", pre iOS13 nothing seemed to be happening but it downloaded in the background. The problem was to know when it had finished and where to find the file. With iOS13 there is a download icon at top right in Safari (next to the share button) with a tiny progress bar under it, and clicking it brings up a progress status panel and a listing of all existing downloads. These are accessible through the Files app.

Correct.
In my case, the progress bar completes but next to it is the reload circle and the magnifying glass. Clicking the magnifying glass shows the file in the Files app but at 0%. Retrying the download gives the same result.

However, what I prefer to do is use the FileManager (from the App Store) and link it to SMB on the HDR-FOX (samba package). This will only work with decrypted files.

As you deduced, my files are encrypted. Also, we're always low on disk space so I'm not sure auto-decryption would be feasible.

I'm assuming it's quite a bit of work to add a new package then?

 

[Ezra Pound]

If you have code that runs on the HDR-Fox T2 then it is possible to create a 'package' in the correct format by following the notes on the Wiki here :-
https://wiki.hummy.tv/wiki/Create_Packages
All packages are held on a server run by af123 the creator the Custom FirmWare, you would have to submit your package to him in order to get his approval

See also :-
https://wiki.hummy.tv/wiki/Compile_Software_for_the_Humax

 

[Black Hole]

I'm assuming it's quite a bit of work to add a new package then?

Not necessarily, but it has a small audience because your particular problem is not common. If you want to run up a package yourself (and make it available generally) go ahead, otherwise expect it to be added at the end of a long list of "nice to haves".

But, you're not just suggesting a new package, you would also have to modify WebIF so that if the package is present it does something different.

As you deduced, my files are encrypted. Also, we're always low on disk space so I'm not sure auto-decryption would be feasible.

How low? If you are running so low you can't auto-decrypt new recordings as they happen, you are also in danger of losing recordings because of disk space issues or file system retries. Setting auto-decryption might take a long time, but it would eat through the backlog eventually (and, I think, prioritise new recordings). To reduce the disk space overhead, turn off sending original recordings to the recycle bin (decryption doesn't fail, so the backup isn't necessary).

But, it's easy enough to click OPT+ >> Decrypt before you click OPT+ >> Download. That solves your problem without a new package.

And there are other ways to skin the cat: Instead of downloading via Safari and WebIF, try downloading via VLC and DLNA.

 

[af123]

I haven't checked but the web server that is already shipped with the custom firmware (lighttpd) is fairly feature-rich and can definitely be configured as a reverse proxy (for example, it fulfils this role in order to present the webshell interface). You might be able to easily make it do what you need here.

 

[/df]

Anyhow the problem must be down to some bug or feature of the iOS 13 web client code that AIUI other browser suppliers are forced to use on iPhones and presumably it's not going to go away, so a solution could be useful. Although if it's this bug it's been there since iOS 8, so maybe not so much.

 

[mk1]

This is what I did to resolve my issue:

1. Upgraded lighttpd to version 1.4.53 (not sure this was actually necessary but it looked like a pre-req for some of the proxy functionality)
2. Created /mod/etc/lighttpd/extra.d/proxy.conf containing the following:

server.modules += ( "mod_proxy" )
$SERVER["socket"] == ":81" {
  # Proxy requests on port 81 to port 9000
  proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "9000" ) ) )
}

3. Restarted lighttpd
4. Took a backup of /mod/webif/html/m/browse.jim and made the following diff (note, based on webif 1.4.2-6):

humax# diff browse.jim.bak browse.jim
--- browse.jim.bak
+++ browse.jim
@@ -94,11 +94,13 @@
                set synopsis [cgi_quote_html [$ts get synopsis]]
        }
+       # Replace port 9000 in the DLNA url with port 81 to route download requests to the proxy
+       set dlnaurl [string map {:9000 :81} [system dlnaurl [file normalize $file]]]
+ 
+       # Add a download link pointing to port 81
        puts "<li>
                <img class=va border=0 width=80 src=/img/$img.png>
                <h3>$bfile</h3>
                <p class=ui-li-count>
-                       <span class=filesize> $sz </span>
+                       <span class=filesize> <a href=\"$dlnaurl $file\" download><img class=\"va\" src=\"/lib/jquery.plugin/contextMenu/images/download.png\" alt=\"Download\">Download</a>  $sz </span>
                </p>
                <p class=icons>
        "

This works because mod_proxy issues an HTTP/1.0 Connection: close request to the Humax DLNA server thus overriding the HTTP/1.1 Connection: keep-alive request from the browser.

Thanks to @af123 for the pointer on lighttpd's proxy capabilities and for everyone else's suggestions . My next project will be to get DLNA downloads from VLC working!