Black Hole
May contain traces of nut
Got a problem now where the queue process never terminates even though the task has completed, and now I'm getting Python errors, so I think my system is broken and all bets are off.
[youtube-dl] Download files from youtube.com or other video platforms
- Thread starter prpr
- Start date
Black Hole
May contain traces of nut
Right, well I switched to a different HDR-FOX and there is definitely a problem with queued rather than immediate Qtube downloads. It's hard to say what the problem is, because the logging for the queued download is very sparse, all I get is the first line (unless that's really all there should have been):
...and then nothing for ages, except at some point it reported (no timestamp):
...but the queue task kept running until I gave up after a couple of hours and killed it:
Run exactly the same process except immediate rather than queued, and it works fine:
The download completed in less than a minute, and I expect the FFMPEG conversion from .m4a to .mp3 to take about 90 minutes. Not being able to queue the episodes is painful!
Code:
06/05/2026 11:34:02 - ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
06/05/2026 11:34:02 - Starting queued download URL https://www.bbc.co.uk/programmes/m002v88g Options -x --audio-format mp3 QID 487925...and then nothing for ages, except at some point it reported (no timestamp):
Code:
ERROR: unable to download video...but the queue task kept running until I gave up after a couple of hours and killed it:
Code:
06/05/2026 13:51:09 - Caught error: 06/05/2026 11:34:34 - [bbc.co.uk] m002v88g: Downloading video page
06/05/2026 13:51:09 - -code 1 -level 0 -errorinfo {::qtube::dequeue /mod/webif/plugin/qtube/queue.hook 32 {{queue exec} youtube --newline -x --audio-format mp3 https://www.bbc.co.uk/programmes/m002v88g 2>@::aio.handle16 | awk {{print strftime("%d/%m/%Y %H:%M:%S -"), $0; fflush(); }} >@::aio.handle16} ::auto::runplugin {} 1 {::qtube::dequeue ::<reference.<queue__>.00000000000000000006> https://www.bbc.co.uk/programmes/m002v88g} {} /mod/webif/lib/auto/deq 205 {::auto::runplugin qtube dequeue ::<reference.<queue__>.00000000000000000006> https://www.bbc.co.uk/programmes/m002v88g}} -errorcode {CHILDKILLED 8237 SIGTERM}
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderrRun exactly the same process except immediate rather than queued, and it works fine:
Code:
13:52:51 - --------------------------------------------------------------
13:52:51 - Starting immediate download of https://www.bbc.co.uk/programmes/m002v88g Options -x --audio-format mp3
13:52:51 - Be VERY patient - it can take a couple of minutes for download to start!
13:52:51 - youtube --newline -x --audio-format mp3 https://www.bbc.co.uk/programmes/m002v88g
13:52:51 - {*}youtube --newline -x --audio-format mp3 https://www.bbc.co.uk/programmes/m002v88g
13:53:24 - [bbc.co.uk] m002v88g: Downloading video page
13:53:28 - [bbc.co.uk] m002v88g: Downloading playlist JSON
13:53:30 - [bbc.co.uk] m002v88d: Downloading media selection JSON
13:53:30 - [bbc.co.uk] m002v88d: Downloading MPD manifest
13:53:31 - [bbc.co.uk] m002v88d: Downloading MPD manifest
13:53:33 - [bbc.co.uk] m002v88d: Downloading m3u8 information
13:53:35 - [bbc.co.uk] m002v88d: Downloading m3u8 information
13:53:36 - [bbc.co.uk] m002v88d: Downloading MPD manifest
13:53:37 - [bbc.co.uk] m002v88d: Downloading MPD manifest
13:53:37 - [bbc.co.uk] m002v88d: Downloading m3u8 information
13:53:37 - [bbc.co.uk] m002v88d: Downloading m3u8 information
13:53:44 - [dashsegments] Total fragments: 273
13:53:44 - [download] Destination: /mnt/hd2/My Video/John_le_Carre_The_Spy_Who_Came_in_From_the_Cold_2._Outside.m4a
13:53:44 - [download] 81.0% of ~66.69MiB at 2.85KiB/s ETA 01:15:58
13:53:44 - [download] 81.0% of ~66.69MiB at 8.33KiB/s ETA 25:58
13:53:44 - [download] 81.0% of ~66.69MiB at 18.88KiB/s ETA 11:27
13:53:44 - [download] 81.0% of ~66.69MiB at 39.63KiB/s ETA 05:27
13:53:44 - [download] 81.0% of ~66.69MiB at 78.42KiB/s ETA 02:45
13:53:44 - [download] 81.1% of ~66.69MiB at 150.25KiB/s ETA 01:25
(etc)The download completed in less than a minute, and I expect the FFMPEG conversion from .m4a to .mp3 to take about 90 minutes. Not being able to queue the episodes is painful!
Black Hole
May contain traces of nut
It's a shame we can't queue anything other than a .ts for conversion – that way I would be able to download the files immediately and then perform all the conversions in the background. I could just take this off the Humax, but I think we should get to the bottom of the problem.
EEPhil
Number 28
That looks like a very long time for a 30 minute audio file.
Black Hole
May contain traces of nut
But that's what it is!
Code:
13:54:22 - [download] 100% of 66.72MiB in 00:38
13:54:23 - [ffmpeg] Correcting container in "/mnt/hd2/My Video/John_le_Carre_The_Spy_Who_Came_in_From_the_Cold_2._Outside.m4a"
13:54:51 - [ffmpeg] Destination: /mnt/hd2/My Video/John_le_Carre_The_Spy_Who_Came_in_From_the_Cold_2._Outside.mp3
15:22:29 - Deleting original file /mnt/hd2/My Video/John_le_Carre_The_Spy_Who_Came_in_From_the_Cold_2._Outside.m4a (pass -k to keep)
OP
prpr
Well-Known Member
Yes, it's broken for me too, as I said a few weeks ago.
You could use the -v option, but it doesn't help a whole lot.
I'm not sure what generates those. Maybe it's Python. Anyway, I guess they are related to killing the task.
I've no idea what's causing it to hang nor how to debug it. Do you know when it last worked? I can't see any package updates, that are possibly related to causing this, in the last 6 months. Any ideas @/df ?
Black Hole
May contain traces of nut
Memory fails me.
Isn't it an adequate clue that it works direct from Qtube but fails when queued?
Last edited:
EEPhil
Number 28
The processor in the Humax must be under powered or ffmpeg is timesharing with very low priority. My 22 year old laptop (XP) converts the file in 2.5 minutes.
Black Hole
May contain traces of nut
^ This. Didn't you realise? It's not actually underpowered for what it is supposed to do, it was never supposed to be software-transcoding audio! We can't leverage the hardware pipelines it has, and I'm not sure it has any pipelines for encoding anyway.
This is tolerable because I can just leave it to take as long as it likes – but that works better when I can set a series of tasks on the queue and not need to keep feeding it manually! Extraction of audio from .ts is much quicker (no transcoding required), if you're willing to accept the ,mp3 is actually MP2.
Black Hole
May contain traces of nut
Something to do with it not getting sufficient processing priority and missing a response deadline perhaps? That could put the change at the other end of the line and not to do with any package update this end.
TimK2015
Forum Supporter
I have tried to download, with Qtube on my Fox T2, the John le Carre that BH mentioned earlier and it fails every time from the 'all episodes' on the 'programme website' page. I have downloaded the individual episodes ok. I've also downloaded other complete books from the Books & Stories page again going to 'all episodes'.
John le Carre - https://www.bbc.co.uk/programmes/m002b5vb/episodes/player - this fails every time
Dark Frontier from Books & Stories - https://www.bbc.co.uk/programmes/m002vx3b/episodes/player - downloads ok
There's also a couple of episodes of Lucy Worsley's Lady Swindlers that refuse to download.
John le Carre - https://www.bbc.co.uk/programmes/m002b5vb/episodes/player - this fails every time
Dark Frontier from Books & Stories - https://www.bbc.co.uk/programmes/m002vx3b/episodes/player - downloads ok
There's also a couple of episodes of Lucy Worsley's Lady Swindlers that refuse to download.
EEPhil
Number 28
I had heard that some Humax processors weren't well powered. I suppose I thought the processing required for adequate recording of two items and playing back of another might require a bit more oomph than it really does.
Black Hole
May contain traces of nut
It hardly needs any oomph at all. The SoC is designed to implement a PVR essentially in hardware, the processor only does the light management stuff – configuring the hardware in the main, plus networking stacks. The data path for the TS stream includes hardware encryption and codecs, and DMA data transfer.
EEPhil
Number 28
I'm probably getting confused with later models (5000T, Aura) with all the posible apps. VLC (Aura) can get a bit temperamental.
OP
prpr
Well-Known Member
Clearly not.
I had another look today (using the link that TimK2015 said works for him) and this is how it ended up:
Code:
UID PID PPID C STIME TTY STAT TIME CMD
root 1408 1 0 May04 ? SNs 0:24 /mod/sbin/crond
root 9534 1408 0 09:46 ? SN 0:00 /bin/sh -c /mod/webif/lib/auto/deq >/dev/null 2>>/mod/tmp/auto.log
root 9538 9534 0 09:46 ? SN 0:00 /mod/bin/jimsh /mod/webif/lib/auto/deq
root 9546 9538 0 09:46 ? SN 0:00 /bin/sh /mod/bin/youtube --newline https://www.bbc.co.uk/programmes/m002vx3b/episodes/player
root 9547 9538 0 09:46 ? SN 0:00 awk {print strftime("%d/%m/%Y %H:%M:%S -"), $0; fflush(); }
root 9549 9546 1 09:46 ? SN 1:53 python /mod/lib/python2.7/dist-packages/youtube-dl --external-downloader wget --hls-prefer-ffmpeg --cache-dir /mod/.cache/youtube-dl --newline https://www.bbc.co.uk/programmes/m002vx3b/episodes/playerThe other interesting thing is that it consistently fails around the 1:53/1:54 mark on CPU time. This is possibly a clue, but I'm not sure to what.
You're in the realms of fantasy again Jones.
Black Hole
May contain traces of nut
So... what are you saying? Why should there be an ffmpeg process until the download has completed and it's moved on to post-processing?
A few weeks ago, I was able to queue a QTube job and get the video out, somewhat to my surprise. I think it was Paul Carrack's RAH gig from YT, a ~300MB combined A-V file in YT's only accessible format 18. Perhaps the queued job fails if it has to run a fancy conversion (not just copying streams), or perhaps even when merging. The queue log together with
qtube.log have helped in the past. When I'm in the same place (or even country) as the apparently working machine I'll check the details. A work-around could be to extract the video and audio streams separately (use , to separate the video and audio format selectors in the -F parameter instead of + for merge) and then merge them manually.
Black Hole
May contain traces of nut
So far as I can see, that has no bearing on the current problem. These are audio downloads and I'm having no difficulty downloading them and converting them directly on the qtube page, but they fail when queued for background processing. So far as I can see the download does not even start, and the process is shown as running in the queue, until terminated with kill.
Perhaps my observation doesn't help but I've had problems since last year using the qtube approach but have found doing the client terminal window method works better as it often times out but using the client window method I can simply repeat the command and it will then pick up where it left off. Most downloads these days take about 5-6 times repeating the command to get the video and audio to finish downloading.