Problems with a Qtube Process on HD-FOX...

[Black Hole]

This is a HD-FOX, and (possibly foolishly) I decided to torture it by downloading the Strictly opening show. I presume the download will be several gigabytes, so it was never a good idea in the first place (not on my broadband), although I did download the Victor Borge programme successfully (about 1GB) taking 2 hours.

After a day, the qtube log is stuck on downloading:



I'm not sure why (not investigated, perhaps it's the debug level), but this machine doesn't log download progress – it only reports when the download is complete.

The process queue looks as if something is happening:



There is a .part file, 1.5GB in size, but it seems to have stopped being written at about 11am yesterday (that's its timestamp in the ls -l anyway).

What I want to do is reboot, which (I presume) would end up with the qtube task being resubmitted and the download being resumed, but I am reluctant because there seems to be another problem which might require information gathering:

We really need an easier way of getting rid of long running queued tasks,

Storing the PID of the currently running queued task in the queue.db entry would be one option but it is fairly complicated to add a new field to an existing database definition but then I realised that is not actually needed.
By design only one queued task can be running at a time so rather than a pid field in each queue entry it only needs a single pid to be stored which could be in the settings database or in a simple file.

So,

1. When starting a queued task save the pid of the started task in file/db
2. (optionally) When task ends delete saved pid
3. Add 'Kill current task' button to queue display button, button action issues kill pid command
4. (optionally) Disable kill button when no task active, Display pid on Queue panel

This is now implemented in 1.5.2-13 in the Beta area.
Any other packages with plugins (qtube) need modifying to use system qexec rather than exec in queue.hook

Note I have an (apparently) running task, yet the Kill Running button is greyed out (see screenshot). What debug info does the team need?

(All this started because I used RS to schedule recording on two machines while I was away, and neither succeeded! HD3 was nominally "working" but somehow screwed up and RTS not working until it had a reboot, HDR1 WebIF was running but the humaxtv process had crashed.)

 

[SemiRamis]

(All this started because I used RS to schedule recording on two machines while I was away, and neither succeeded! HD3 was nominally "working" but somehow screwed up and RTS not working until it had a reboot, HDR1 WebIF was running but the humaxtv process had crashed.)

Is it not on the iplayer? Why this gigantic effort?

 

[Black Hole]

Is it not on the iplayer? Why this gigantic effort?

Have you tried using the TV Portal on the Humax lately? No, I don't have smart TVs... and then there's the stress-testing of the various CF functions to be considered. It didn't get where it is today without people stress-testing!

If you have nothing useful to say, STFU.

 

[prpr]

Note I have an (apparently) running task, yet the Kill Running button is greyed out (see screenshot). What debug info does the team need?

None. The qtube package needs updating (or releasing, I forget where it got to).
Having said that, the whole system qexec thing has generated some logging unpleasantness (that I'm kinda surprised nobody has mentioned), and I just haven't had the cycles to look at it again (like with many others).

How big is your swapfile by the way?

 

[Black Hole]

None. The qtube package needs updating (or releasing, I forget where it got to).

So I can reboot and/or delete queue.db with gay abandon then.

How big is your swapfile by the way?

I guess that might be relevant to the patch-up, but it hasn't got that far. 134MB by the looks.

 

[prpr]

So I can reboot and/or delete queue.db

You can do the former and it should restart from where it is.
If you do the latter then it won't, and you'll need to re-queue it, but it should then carry on as above.

 

[/df]

I think we've previously concluded that 512MB swap may be needed for the fixup to run with large videos (or downgrade the selected resolution).

 

[Black Hole]

Meanwhile, I've done it on another machine (HDR-FOX). I think I'll kill queue.db.

Curious: why 134MB? Even allowing for M = 10^6 rather than 2^20, that seems a strange figure for me to have chosen.

 

[bottletop]

I think we've previously concluded that 512MB swap may be needed for the fixup to run with large videos (or downgrade the selected resolution).

It hasn't finish downloading, so it did not reach the fixup stage.
This is indicated on the first post - the qtube.log does not have '[ffmpeg] Fixing malformed AAC bitstream' message for the last download. Eg the sequence should be similar to the log for the previous download.

It appears to have stalled during the download. Is the filesystem ok? Is the drive SMART statistics ok?

 

[prpr]

why 134MB?

What's the actual size in bytes?

that seems a strange figure for me to have chosen.

We will never know what weird things go on in inside your head.

 

[bottletop]

..
There is a .part file, 1.5GB in size, but it seems to have stopped being written at about 11am yesterday (that's its timestamp in the ls -l anyway).
..

Forgot to ask, is there only one .part file?
Usually the download may have

file.mp4.part​

file.mp4.part-Fragnnnn.part​

during the download stage.

Assuming similar download options and the files available,

Victor Borges (29 minutes duration) took 1:42 for 1020 MiB​

So extrapolating this

Strictly Come Dancing 22 Launch (99 minutes duration)​

should take 3.33 times longer, so take roughly 6:00 to download 3400MiB

But the .part file is only 1.5GB, so that tracks with the stall at roughly 11:00.
Maybe look at what else happened during the time it stopped writing to the drive. Eg Not sure if loss of broadband will cause this issue.
Did (telnet) top or dfshow anything useful?

Edit1: only just noticed the first line of your post #8. If it suits, after successful download on HDR, maybe try again on the HD box to see if you can recreate the problem? (It may be a repeatable issue on that unit.)

 

[Black Hole]

Forgot to ask, is there only one .part file?
Usually the download may have
file.mp4.partfile.mp4.part-Fragnnnn.partduring the download stage.

Only the one.

Did (telnet) top or dfshow anything useful?

I imagine that's too late now?

Maybe look at what else happened during the time it stopped writing to the drive. Eg Not sure if loss of broadband will cause this issue.

It would be no surprise if the networking had baulked it, I don't recall whether the powerlines crashed during that time, but I thought using qtube would make it robust against external interruptions? Perhaps it expects a restart.

Whatever, I'm satisfied the problems have been explained.

What's the actual size in bytes?

134217728

So it's 128 GiB MiB then.

 

[prpr]

MiB. You have to be so careful with these units.