[qtube] Webif front end for youtube-dl

[4ndy]

I checked 20 minutes after the recording finished. Auto log shows scans and the decrypt and shrink processes were added to the queue as expected.

Although the queue page refreshes itself every minute or so nothing was processed.

The box then went to sleep (power saving).

30 minutes later auto processing still not started working through the queue.

Any ideas?


Sent from my SM-G950F using Tapatalk

 

[4ndy]

I think I have identified the problem.

In the middle of my queue was a youtube film which had a very long url string. Once I deleted this from the queue, it started running.

I am sorry, I can't recall what it was, so that we can recreate, but there seems to a restriction in the url field length and presumably a validation process before processing the queue.

Sent from my SM-G950F using Tapatalk

 

[MymsMan]

Is there anything in auto.log for the period?

 

[4ndy]

This is the section showing the scan before I deleted the problem download. The list of Qtube items has been truncated before the very long URL that appears to be the cause the issue.

27/10/2018 18:46:02 - Registered newk with priority 1000 (::newk::run)
27/10/2018 18:46:03 - Registered sweeper with priority 700 (::sweeper::run)
27/10/2018 18:46:03 - Registered flag 'sweeper' for sweeper
27/10/2018 18:46:03 - Scanning media for flags...
27/10/2018 18:46:03 -  C: 32657   900  decrypt - /mnt/hd2/My Video/! Films/Weekend With Lulu.ts
27/10/2018 18:46:03 -  C: 32658   400   shrink - /mnt/hd2/My Video/! Films/Weekend With Lulu.ts
27/10/2018 18:46:03 -  C: 32656   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00pcm3h/life-10-primates
27/10/2018 18:46:03 -  C: 32655   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00p90d6/life-9-plants
27/10/2018 18:46:03 -  C: 32654   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00p4rl4/life-8-creatures-of-the-deep
27/10/2018 18:46:03 -  C: 32653   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00p1n00/life-7-hunters-and-hunted
27/10/2018 18:46:03 -  C: 32652   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00nxks3/life-6-insects
27/10/2018 18:46:03 -  C: 32651   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00ntlyx/life-5-birds
27/10/2018 18:46:03 -  C: 32650   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00nqbkb/life-4-fish
27/10/2018 18:46:03 -  C: 32649   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00nkpcc/life-3-mammals
27/10/2018 18:46:03 -  C: 32648   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00nj6dr/life-2-reptiles-and-amphibians
27/10/2018 18:46:03 -  C: 32647   200    qtube - https://www.bbc.co.uk/iplayer/episode/b00ncr13/life-1-challenges-of-life
27/10/2018 18:46:03 -  C: 32646   200    qtube - https://www.bbc.co.uk/iplayer/episode/p00bf6pz/blackadder-blackadder-goes-forth-2-plan-b-corporal-punishment
27/10/2018 18:46:03 -  C: 32645   200    qtube - https://www.bbc.co.uk/iplayer/episode/p00bf6md/blackadder-blackadder-goes-forth-1-plan-a-captain-cook
27/10/2018 18:46:03 -  C: 32644   200    qtube - https://www.bbc.co.uk/iplayer/episode/b0bp9kx0/royal-opera-house-lessons-in-love-and-violence
27/10/2018 18:46:03 - Scan completed (0.91 seconds)
27/10/2018 18:46:03 - Active flags: shrinkR decryptR shrink decrypt dedup sweeper
27/10/2018 18:46:03 - /media/My Video                                                                  - shrinkR decryptR shrink decrypt
27/10/2018 18:46:03 - /media/My Video/Sally Ann                                                        - dedup shrink decrypt
27/10/2018 18:46:03 - /media/My Video/Line of Duty                                                     - shrink decrypt
27/10/2018 18:46:03 - /media/My Video/Red Cap                                                          - shrink decrypt
27/10/2018 18:46:03 - /media/My Video/Danny Baker Takes Over                                           - shrink decrypt
27/10/2018 18:46:03 - /media/My Video/Midsomer Murders

 

[/df]

Is there a way to get it to download to a specific directory ?

The download directory is set in the youtube wrapper script from the youtube-dl CF package. However if you specify a different output parameter in the options field I'd expect it to override that setting. Try something like (for HDR) this to use dirName under the normal Video directory:

-o "/mnt/hd2/My Video/dirName/%(title)s.%(ext)s"

 

[MymsMan]

The download directory is set in the youtube wrapper script from the youtube-dl CF package. However if you specify a different output parameter in the options field I'd expect it to override that setting. Try something like (for HDR) this to use dirName under the normal Video directory:

-o "/mnt/hd2/My Video/dirName/%(title)s.%(ext)s"

Unfortunately that gives

ERROR: fixed output name but more than one file to download

Yotube-dl doesn't like having duplicate -o specifications in the options

I could probably fix in qtube by avoiding using the wrapper code.

 

[Black Hole]

One of those things it would be nice to have a text entry box for (or even better a navigator), defaulting to My Video.

 

[/df]

...
Youtube-dl doesn't like having duplicate -o specifications in the options
...

With apologies to prpr, the wrapper could be modified to deal with it.

#! /bin/sh
# youtube-dl wrapper

# check for output parameters - only 1 allowed
outdir=
for arg in "$@"; do
    case $arg in
    -o|--output)
        outdir=$arg
        break
        ;;
    esac
done

# add model-specific default output parameter if needed
if [ -n "$outdir" ]; then
    # caller overrides output
    outdir=
  else
    if [ `cat /etc/model` = HDR ]; then
        outdir='/mnt/hd2/My Video'
    else
        outdir='/media/drive1/Video'
    fi                              
    outdir="$outdir/%(title)s.%(ext)s"
fi                                           

# shell syntax: ${var:-stuff}->null if var null or unset, else expansion of stuff
python /mod/bin/youtube-dl --config-location /mod/etc/youtube-dl.conf ${outdir:+ -o "$outdir"} "$@"

How about setting up a bind mount like /media/[Video] in the next CF to simplify scripts like this? Or is it too late?

 

[prpr]

With apologies to @prpr, the wrapper could be modified to deal with it.

No need for apologies. My bash scripting skills are fairly limited, so the thanks are due to you.
I'll package that up as well.

 

[/df]

No need for apologies. My bash scripting skills are fairly limited, so the thanks are due to you.
I'll package that up as well.

Great. Though I do keep a pedant hat nearby ready for use and would point out that a script starting with #!/bin/sh should be just a plain POSIX standard Bourne shell script, as that one is. Bash is a luxury in the Humax CF world.

Shellcheck is helpful, even if it also has a pedant hat.

 

[Black Hole]

 

[af123]

Great. Though I do keep a pedant hat nearby ready for use and would point out that a script starting with #!/bin/sh should be just a plain POSIX standard Bourne shell script, as that one is. Bash is a luxury in the Humax CF world.

Absolutely! Too many people do not know what is POSIX, what is ksh88/93 and what is bash. The Humax default shell is busybox ash anyway, and this script needs to use syntax compatible with that. I would not be tempted to change the shebang line to invoke bash, it's much bigger than ash and it's doubtful that you need the extensions anyway.

 

[prpr]

Too many people do not know what is POSIX, what is ksh88/93 and what is bash.

Too right. Me being one of them. Are these our faults or those of the people who write them?
There are too many damned shells and I'm damned if I can be bothered to learn all the differences, or even all the names, any more.
Whatever, I can never remember the flamin' syntax of things that ought to be simple but aren't, or don't seem to follow natural flow. Then I get irritated with it (and me).

 

[/df]

...
There are too many damned shells and I'm damned if I can be bothered to learn all the differences, or even all the names, any more.
...

Anyone who has written Unix shell scripts will sympathise. But we can let the computer can take the strain, instead of fiddling about with the script until it works. As I said,

...
Shellcheck is helpful, even if it also has a pedant hat.

For instance in my youtube-dl wrapper above, it tells me that instead of `cat /etc/model`, we should have had "$(cat /etc/model)" (whether in [[d]a]sh or bash). There is a good rationale for this but it's an example of what I meant by its 'pedant hat' . Much more to the point, the good news is that there were no other diagnostics.

 

[Black Hole]

Multiple shell languages seems to be a case of "we can, so we will". Best to standardise on one - but if machine capacity comes into it so that a full implementation won't fit, the compromise means it is not possible to stamdardise!

 

[af123]

For instance in my youtube-dl wrapper above, it tells me that instead of [icode]cat /etc/model[/ICODE], we should have had "$(cat /etc/model)" (whether in [[d]a]sh or bash). There is a good rationale for this but it's an example of what I meant by its 'pedant hat' . Much more to the point, the good news is that there were no other diagnostics.

Exactly why I always have SHELLCHECK_OPTS="-e SC2006" amongst others : )

 

[4ndy]

Seems to have stopped working in both real time and queue, I get the message shown.

Tried two boxes (both auto-updated to latest version of QTube when I opened webif), and a dozen or so iplayer links.

For Example

 

[MymsMan]

Seems to have stopped working in both real time and queue, I get the message shown.

Oh dear, not sure what has happened following the lastest youtube-dl
Must be something I'm doing wrong since youtube-dl itself is working

Try putting something in the options field (-v) because it is handling null options strings differently

 

[4ndy]

-v seems to do it.

Sent from my SM-G950F using Tapatalk

 

[MymsMan]

Hopefully fixed qtube Version 0.1.0-1