• The forum software that supports hummy.tv has been upgraded to XenForo 2.3!

    Please bear with us as we continue to tweak things, and feel free to post any questions, issues or suggestions in the upgrade thread.

Unencrypt vs Webif with Decrypt

sceptic said:
Ah I see. I thought perhaps that if the decrypted file size was the same as the encrypted file size then it would be safe to delete it...
Click to expand...

That's one of the checks that the process does but it still moves the original file to the bin if you have undelete installed. It costs nothing and will be automatically removed after a period of time or sooner if disk space is running low.

Any ideas what is the cause of those connection refused errors in my log?
Click to expand...
Yes, it means that the media server isn't running. Most likely the box is half awake recording or ready to record. It would be better for the log to say that rather than connection refused however.
 
OK that makes sense, thanks for clarifying. Regarding the log I think it would be easier to read if it just contained errors and actual decrypt actions. Its quite long at present as the entire directory tree is listed every time it runs. A timestamp might be a useful addition too for debugging purposes.
 
af123 said:
should the recursion break for any special directory (i.e. one that has a name starting with a [ character)? I don't think it should.
Click to expand...
I think the recursion should NOT apply to any directory starting with a '['.
 
That's how it will be in the next version, although it isn't a clear-cut decision and I have not prevented anyone from specifically setting a [ directory for recursive operations.

Quite a few people use flatten and have created top level folders called things like [Films] that they don't want flattening. They could reasonably expect decrypt to recurse into those. Under the new system they'll need to set recursive decrypt on the top-level My Video directory and then separately on any special directories in the rest of the hierarchy. It will now never go into the bin though, even if it is explicitly flagged.
 
af123 said:
They could reasonably expect decrypt to recurse into those.
Click to expand...

Not really, but if they did these folders could be specifically marked as you say. We just need to make sure everyone knows what "[" means.

Most likely anything flattened will have been part of a series, flattened into the top level (and recursive decrypted), and then moved by the user into a "[" folder. The only problem is if somebody modifies the record folder to target a "[" (or moves the recording before it got decrypted).
 
Another new version uploaded which uses a cleaner persistent log file which auto-rotates at 2MiB.
The log file includes timestamps too and is much less verbose but also provides more information when things are processed.
I've also improved the behaviour when the DLNA server isn't present (half awake or portal) or goes away during decrypt operations.

Code: 
28/01/2013 00:39 - -------------------------------------------------------
28/01/2013 00:39 - Media scan starting, DLNA server status: 1
28/01/2013 00:40 - dedup scan completed in 43.895 seconds.
28/01/2013 00:40 -   DECRYPT: /media/My Video/Stargazing LIVE/1_6_Stargazing_LIVE
28/01/2013 00:40 -   DLNA: http://172.29.0.252:9000/web/media/4449.TS
28/01/2013 00:47 - Done... 2.90 GiB in 448.245 seconds - 6.63 MiB/s

Special directories are now treated differently too. If they are encountered during recursion then they, and all their descendants, are skipped. If they are themselves flagged for a recursive operation then that will be honoured.
The dustbin is special - it will never be processed.
 
Would you be prepared to update unencrypt to the same standard, or perhaps cause it to invoke the same routines as recursive auto-decrypt? I am considering the situation where a non-networked HDR-FOX is running a limited set of packages loaded by USB, and doesn't necessarily have the full WebIF loaded (perhaps auto-decrypt should call an updated unencrypt, rather than the other way around).
 
Thanks for the super fast turnaround on this. Since picking up the new version I've made one recording which has been decrypted but I can't find a log file at all now to view, either in webif diagnostics or via telnet... Unfortunately I don't know the right command to search for it so my telnet checking is probably missing it somewhere!
 
The log is now called auto.log and should show up in the diagnostics screen, otherwise it is in /mod/tmp/ on the disk.
 
Out of interest, what decryption rate did you see? I get around 6MiB/s and shrink operations are a tad faster at 9MiB/s.
 
af123 said:
Out of interest, what decryption rate did you see? I get around 6MiB/s and shrink operations are a tad faster at 9MiB/s.
Click to expand...

I get similar values to you for Decrypt. I've not enabled shrink as yet. Media scanning times vary a lot depending upon what the box was doing at the 10min slot when it runs. I was thinking about suggesting that it could perhaps be enhanced to only scan 'new' files (without requiring a full scan of the filesystem each time). But I guess this would require a hook into the Auto Update mechanism to flag the recording or are recordings stored in a DB with a timestamp that can just be queried to the last scan time? Is the full media scan likely to slow down the box at all when the box is busy or am I worrying about nothing?
 
Can you provide some log extracts showing the scan times? I'm trying to get a feel for how well it's performing. There are a number of ways that I could look at improving this area.
 
With no recordings underway it takes about 5-6 secs to scan. With 1 recording going this increases to around 95 secs or so. I have one timing for two recordings running (1HD and 1SD) of around 101 secs.
 
Here are some timing taken with no recording taking place
Code: 
29/01/2013 19:00 - Media scan starting, DLNA server status: 1
29/01/2013 19:00 - dedup scan completed in 0.629 seconds.
29/01/2013 19:00 -  DECRYPT: /media/My Video/Africa/Africa_20130123_2101
29/01/2013 19:00 -  DLNA: http://10.0.0.200:9000/web/media/1803.TS
29/01/2013 19:10 - Done... 3.55 GiB in 612.503 seconds - 5.94 MiB/s
29/01/2013 19:10 - decrypt scan completed in 616.258 seconds.
29/01/2013 19:10 -  SHRINK: /media/My Video/Africa/Africa_20130123_2101
29/01/2013 19:10 -          Estimate 6% saving.
29/01/2013 19:10 -          Shrinking...
29/01/2013 19:15 -
Saved: 943309/19856683 packets, 172.73/3635.87 MiB (4.75%)
29/01/2013 19:15 - Done... 3.55 GiB in 301.421 seconds - 12.06 MiB/s
29/01/2013 19:15 - shrink scan completed in 304.851 seconds.
29/01/2013 19:15 - mpg scan completed in 1.781 seconds.
29/01/2013 19:15 - Media scan completed in 923.645 seconds.

With a single Standard Def. Recording taking place
Code: 
 Media scan starting, DLNA server status: 1
29/01/2013 20:10 - dedup scan completed in 1.224 seconds.
29/01/2013 20:10 -  DECRYPT: /media/My Video/Rory and Will - Champions Of___/Rory and Will - Champions Of____20130128_2100
29/01/2013 20:10 -  DLNA: http://10.0.0.200:9000/web/media/1863.TS
29/01/2013 20:15 - Done... 1.67 GiB in 320.69 seconds - 5.32 MiB/s
29/01/2013 20:15 - decrypt scan completed in 322.728 seconds.
29/01/2013 20:15 -  SHRINK: /media/My Video/Rory and Will - Champions Of___/Rory and Will - Champions Of____20130128_2100
29/01/2013 20:15 -          Estimate 10% saving.
29/01/2013 20:15 -          Shrinking...
29/01/2013 20:18 -
Saved: 955544/9319039 packets, 174.97/1706.55 MiB (10.25%)
29/01/2013 20:18 - Done... 1.67 GiB in 162.367 seconds - 10.51 MiB/s
29/01/2013 20:18 - shrink scan completed in 167.34 seconds.
29/01/2013 20:18 - mpg scan completed in 3.231 seconds.
29/01/2013 20:18 - Media scan completed in 494.644 seconds.

This time recording two Hi-Def programs, chase playing a third Hi-Def. and decrypting / Shrinking another Hi-Def. file
Code: 
29/01/2013 21:10 - Media scan starting, DLNA server status: 1
29/01/2013 21:10 - dedup scan completed in 1.134 seconds.
29/01/2013 21:10 -  DECRYPT: /media/My Video/Lost Kingdoms of South America/Lost Kingdoms of South America_20130128_2104
29/01/2013 21:10 -  DLNA: http://10.0.0.200:9000/web/media/1865.TS
29/01/2013 21:16 - Done... 1.50 GiB in 368.461 seconds - 4.17 MiB/s
29/01/2013 21:16 - decrypt scan completed in 372.715 seconds.
29/01/2013 21:16 -  SHRINK: /media/My Video/Lost Kingdoms of South America/Lost Kingdoms of South America_20130128_2104
29/01/2013 21:16 -          Estimate 22% saving.
29/01/2013 21:16 -          Shrinking...
29/01/2013 21:19 -
Saved: 1867733/8390468 packets, 341.99/1536.64 MiB (22.26%)
29/01/2013 21:19 - Done... 1.50 GiB in 178.984 seconds - 8.59 MiB/s
29/01/2013 21:19 - shrink scan completed in 186.791 seconds.
29/01/2013 21:19 - mpg scan completed in 2.308 seconds.
29/01/2013 21:19 - Media scan completed in 563.094 seconds.
 
Now the webif recursive functionality is available I was wondering how feasible it would be to add Archive and recursive Archive options to the menu along the same lines as decrypt / shrink, but where an archive destination (network share or external USB) would need to be entered to define the destination. Anything 'archived' would then be moved to the archive location by the webif background job. Users with NAS centralised storage could use this to archive some or all of their recordings automatically.
 
My Hummy is running quick tonight, it managed to Decrypt at 7.56MiB/s ! :) It seems faster since I switched to 2.15...
 
Currently the webif auto script has a bug - it still descends into protected folders (those with names starting with '['). You need to add something like
Code: 
     if {!$force && ![file exists "$dir/.auto$attr"]} {
        log "returning - not processing special directory $dir"
        return
    }
after handling recursion ~ line 340
 
You must log in or register to reply here.
Back
Top