Unresponsive after Fixing malformed AAC bitstream with qtube

[Black Hole]

Right click

Yeah, I can paste in, but not copy out! Pain in the arse.

Ctl-V doesn't seem to work

The problem is that in the command line context, Ctrl+V will be received by the command interpreter as literally a Ctrl+V (ditto Ctrl+C etc). What you have to do is engage some higher-level component (ie the GUI part of webshell) to offer a paste option through the browser. I see you've ended up with a screenshot instead of copying text into a code box.

 

[Jeddy]

Well that's annoying! After 3 hours or so it looks like the box turned itself off (no doubt due to lack of activity on my part)




and this time it thinks it's completed...



and the only 'relevant' file I can see is the one at 2.19GB which I downloaded through iplayer.

So, I take it I need to try yet again and make sure I play with the box every hour or so to keep it active?

Also annoying is previously when I selected View qtube.log it used to take me to that



but now I get the choice of which to select



the most recent being the one selected for the above.

 

[Ezra Pound]

Well that's annoying! After 3 hours or so it looks like the box turned itself off (no doubt due to lack of activity on my part)

Settings >> System >> Power Management >> Automatic Power Down >> Off ?

 

[Black Hole]

but now I get the choice of which to select

That's strange. I was going to say "weird", because (as you say) I expect the button to go straight to the qtube.log file... but it looks like qtube.log does not exist.

Don't quote me, but I think the logic goes something like this:

• qtube.log is not persistent, it gets deleted at the end of the run (I have thought that unnecessary and inconvenient).
  
  
• The logging system overall splits log files when they exceed a size threshold (I believe mainly to prevent huge loading times in the viewer app), according to a setting in WebIF >> Settings >> Advanced Settings.

The conclusion is that what you are seeing is the retained splits from the previous failed run which generated a huge log file. The current qtube.log got deleted, so the log viewer couldn't find it and offered a list of existing .log files to choose from.

 

[Jeddy]

First thanks for the Auto Power down reminder - I'll give it another go later with that turned off.

I don't think that's quite right. You'll notice in post 22 above that the last time in the log is 17:18 when it had downloaded 80.2%. That's obviously not a complete download and it wouldn't have been able to redo the entire 6Gb before I posted an hour later (the time estimate was about 3:45 hours). I'll agree there are 2 retained splits from the failed run (when it turned itself off). When I turned back on it didn't redo anything so there is no new log? There is also no file again suggesting it didn't redo the download.
But I think your point is essentially valid - they are old logs from the failed download but there is no current log to view as it didn't attempt to redo it (fortunately otherwise it would probably never finish if it keeps turning itself off until I change the setting).

 

[MymsMan]

qtube.log is not persistent, it gets deleted at the end of the run (I have thought that unnecessary and inconvenient).

Not true, it is persistent and can include the results of many downloads

The logging system overall splits log files when they exceed a size threshold (I believe mainly to prevent huge loading times in the viewer app), according to a setting in WebIF >> Settings >> Advanced Settings.

Truish, it just renames the current log adding a timestamp and a new log will be created on first activity. To view those archived logs you need to go to the Diag page where they should be listed in the right hand columns . The "view Qtube.log" button is simple minded and doesn't know when a log has been archived.

The size of log file and number/age of old logs to retain can be controlled via Settings->Advanced Settings

AFAIK youtube.dl can restart an incomplete download but if it fails in post-processing, including Fixing malformed AAC bitstream, there is little progress logging and no easy way to restart it

 

[Black Hole]

Not true, it is persistent and can include the results of many downloads

You're right, I see mine is still there, but what's happened to the prototype qtube.log file in this instance?

 

[Jeddy]

So how come my incomplete download hasn't restarted?

 

[MymsMan]

So how come my incomplete download hasn't restarted?

I am surprised there are no entries in your queue, how long do you keep completed items according to settings?
To restart the download you will need to reenter the URL on the qtube page, if the entry still existed on the queue you could have chosen the Re-submit option

but what's happened to the prototype qtube.log file in this instance

After the old qtube.log is renamed there is no (prototype) qtube.log until the next time qtube is run

 

[Jeddy]

Sorry, where is that in the settings? This one?

If so, we're well with the 7 days.
Previously I was able to resubmit etc., I'm not going to say it is 'to blame' but it is since I changed the storage to 256 things have started behaving differently - before that, the download continued even past 3 hours to 100% without the box powering down even though I wasn't playing with the remote (just monitoring via the web interface as I was doing above).

 

[Jeddy]

Out of curiosity I thought I'd try a short download




which after selecting View Queue



although this time selecting View Qtube.log does show



It appears that when the box turned itself off, it just forgot everything.

 

[MymsMan]

If you run the download immediately then it doesn't go anywhere near the queue and just runs it immediately but it is possibly subject to browser timeouts

Running in the background is much better for anything than the smallest files because you can set up a list of files to download without waiting for each to finish and the download will be automatically restarted should the humax be powered off or crash. Downloads can also be deferred to a quiet time.

 

[Black Hole]

before that, the download continued even past 3 hours to 100% without the box powering down even though I wasn't playing with the remote

The auto power-down setting isn't going to change just because we changed our swap space size!

 

[Jeddy]

Thanks, I'll use background for large in future.

Sure, I wasn't suggesting it had, as I said, I'm not going to blame it, the timing just seemed to coincide for some reason.

 

[Jeddy]

Well that's useful, I resubmitted the download request and it started where it left off, i.e. at 80.2%
Quickly downloaded the rest and all seems to work now!
Thanks!!

 

[Black Hole]

Maybe the size of swapper's swap file needs a tweak in general; your task seems to have only just exceeded the available space, so doubling ought to eliminate further problems entirely (for any reasonable qtube task, anyway), for everyone.

I resubmitted the download request and it started where it left off, i.e. at 80.2%

Yes, it's a feature of youtube-dl to resume (rather than restart) broken downloads – something I'm particularly grateful for with my broadband!

 

[Jeddy]

Maybe the size of swapper's swap file needs a tweak in general; your task seems to have only just exceeded the available space, so doubling ought to eliminate further problems entirely (for any reasonable qtube task, anyway), for everyone.


Yes, it's a feature of youtube-dl to resume (rather than restart) broken downloads – something I'm particularly grateful for with my broadband!

It was an unusually large download at 3 hours, although I guess given the increasing length of some films etc. maybe an increase in the default size wouldn't be a bad idea, I'm assuming the extra 128MB is on disk so you're not robbing Peter to pay Paul.

A useful feature - I think it surprised me most because I couldn't find any evidence of the previous download either as a 4GB file under media or in the logs. Although it does raise a minor concern. I may be wrong, but if I couldn't see the part file and had just given up I'd have 'lost' 4GB of disc space to a partial download wouldn't I? Or do such things get tidied up after e.g. 7days?

 

[Black Hole]

I'm assuming the extra 128MB is on disk so you're not robbing Peter to pay Paul.

/dev/sd*3 (/mnt/hd3) is a reserved 10GB partition which the standard firmware uses for the streaming buffer and the cookie cache. Nicking a quarter of a gig out of that isn't going to make much difference.

if I couldn't see the part file and had just given up I'd have 'lost' 4GB of disc space to a partial download wouldn't I?

Seems plausible, but maybe youtube-dl tidies up next run. You won't have been able to find buffer files through the WebIF GUI tools (which have a limited view dedicated to their purpose), so you would have to use the command line (or maybe FTP/SMB/NFS).

Incidentally, while checking into hd3 I found logs from fixdisk runs hiding there I wasn't previously aware of.

 

[bottletop]

..
Incidentally, while checking into hd3 I found logs from fixdisk runs hiding there I wasn't previously aware of.

Aren't they the ones that are viewable via WebIF/Diag/ - right hand side - View Log Files/fix-disk.X.log etc? Where X starts from zero for the latest

 

[Black Hole]

Indeed, lack of observation on my part, but how come they are there when (presumably) hd3 is also off-line for fixing? I guess they have to be copied there from RAMdisk afterwards.

Returning to the subject of partial qtube / youtube-dl downloads, I find the download process uses three intermediate files *.part, *.part-Frag#.part, and *.ytdl, where "*" is the target output filename (including extension, eg .mp4) in the target folder. Thus, if the config is set to put output in My Video, these files will be found in My Video but with those appended extra extensions thus rendering them invisible using the WebIF media browser (which filters for recognised media files). So far as I can see, the only clean-up is when the process completes; any aborted downloads will hang around indefinitely (unless resumed).

NB: Edited because friggin' markdown interpreted asterisks as commands to italicise. Bloody stupid if you ask me.