Force DLNA Index, Auto Decrypt, Auto Shrink

MymsMan said:
Getting back to the real subject of discussion the recording of Countdown finished today at 14:59 and was immediately Flattened, similarly for 32 Brinkburn Street at 15:00 :)
The 15:05 scheduled scan started but failed for some reason just after the start of the decrypt scan line 210 :(
The 15:25 scheduled scan ran normally decrypting two recordings :)

I don't like the decrease in frequency of scheduled to 20 minutes but do like the offsetting from the hour to reduce the likelihood of end of recording scans colliding with scheduled scans, I still depend on the scheduled runs to actually do the actual decryption and scheduling of ad detection so still prefer a more frequent scan to reduce chance of long waits. I would prefer 5-59/10 or 5-59/15.

On a related theme would it be possible to initiate a run of Auto (and Detectads) at system boot to handle any processing that had not been completed because of the system shutting itself down after a scheduled recording.
Click to expand...

I can't immediately see why that 15:05 scheduled scan failed.. is that with the debugging level set to max?
I have always wanted to reduce the frequency once the triggering mechanism is working properly but I can see that it wouldn't help your situation since flatten is (currently) implemented as a post-decryption-scan plugin.
A scan following boot is definitely a good idea. Since detectads is a plugin it would run too.
 
af123 said:
Try this:

Code: 
ndex: webif/plugin/flatten/auto.hook
===================================================================
--- webif/plugin/flatten/auto.hook   (revision 2404)
+++ webif/plugin/flatten/auto.hook   (working copy)
@@ -2,7 +2,7 @@
proc ::flatten::processdir {dir} {
   global root

-   if {[string match {\[*} $dir]} {
+   if {[string match {\[*} [file tail $dir]]} {
     log "  Special" 2
     return
   }
Click to expand...

That does the trick :)
 
af123 said:
I can't immediately see why that 15:05 scheduled scan failed.. is that with the debugging level set to max?
I have always wanted to reduce the frequency once the triggering mechanism is working properly but I can see that it wouldn't help your situation since flatten is (currently) implemented as a post-decryption-scan plugin.
A scan following boot is definitely a good idea. Since detectads is a plugin it would run too.
Click to expand...
With logging at max it becomes more difficult to see the wood for the trees - over 500 lines per scan but I will turn it on when I know recordings are due to end

Relevant lines for processing the News
Code: 
904 20/04/2015 22:35:04 -   /media/My Video/BBC News at Ten_20150420_2159.ts - Can't use helper.
903 20/04/2015 22:35:04 -   /media/My Video/BBC News at Ten_20150420_2159.ts - database is locked
902 20/04/2015 22:35:04 -   /media/My Video/BBC News at Ten_20150420_2159.ts - Not yet indexed, trying helper.
After that I started watching the recording so subsequent scans found recording in use

Detectads is a plugin but it also has a scheduled task which should be run at startup
 
Today after Countdown the Scan found the DLNA server to be inactive and although the scan navigated the entire directory tree it didn't report which files were/were not decrypted
Code: 
2889 21/04/2015 15:05:01 -       [/media/My Video/ABC/Father Brown]
2888 21/04/2015 15:05:01 -     [/media/My Video/ABC]
2887 21/04/2015 15:05:01 - DECRYPT: [/media/My Video]
2886 21/04/2015 15:05:01 -   [/media/My Video]
2885 21/04/2015 15:05:01 - decrypt scan starting.
2884 21/04/2015 15:05:01 - Root device: 2050
2883 21/04/2015 15:05:01 - Dustbin: /media/My Video/[Deleted Items]
2882 21/04/2015 15:05:01 - Media scan starting, DLNA server status: 0
2881 21/04/2015 15:05:01 - DLNA Server is NOT running.
The DLNA server is not shown in the log as active at any time since the system woke up to start recording and when it went back into standby at 15:45
The Server was active for the period following waking up for an earlier scheduled reminder event and following a subsequent manual start.

So why didn't Humax start the server when waking up to perform recordings which would need indexing?
 
Much more interesting is what happens after the system was rebooted and DLNA server started running
The 16:05 scan successfully decrypts one program then dies as it is about to process the entry for /media/My Video/Countdown_20150421_1410.ts
The 16:15 scan dies at exactly the same point
but the 16:25 successfully finds that "/media/My Video/Countdown_20150421_1410.ts - has been indexed.", decrypts it and goes on to complete the scan successfuly.
Code: 
21/04/2015 16:05:02 - Registered ::detectads::chancheck for postdecrypt hook with priority 30.
21/04/2015 16:05:02 - Registered ::flatten::scan for postdecryptscan hook with priority 50.
21/04/2015 16:05:02 - Registered ::flatten::scandir for postdecryptsingledir hook with priority 50.
21/04/2015 16:05:02 - Registered ::sweeper::scan for postdecryptscan hook with priority 50.
21/04/2015 16:05:02 - Registered ::sweeper::scansingledir for postdecryptsingledir hook with priority 50.
21/04/2015 16:05:02 - -------------------------------------------------------
21/04/2015 16:05:02 - DLNA Server is running.
21/04/2015 16:05:02 - Media scan starting, DLNA server status: 1
21/04/2015 16:05:02 - Dustbin: /media/My Video/[Deleted Items]
21/04/2015 16:05:02 - Root device: 2050
21/04/2015 16:05:02 - decrypt scan starting.
21/04/2015 16:05:02 -   [/media/My Video]
21/04/2015 16:05:02 - DECRYPT: [/media/My Video]
21/04/2015 16:05:02 -   /media/My Video/Banished_20150416_2100.ts - Already decrypted.
21/04/2015 16:05:02 -   /media/My Video/Eggheads_20150420_1759.ts - Already decrypted.
21/04/2015 16:05:02 -   /media/My Video/New_ The Gadget Show_20150420_1859.ts - Already decrypted.
21/04/2015 16:05:02 -   /media/My Video/Decimate_20150421_1500.ts - has been indexed.
21/04/2015 16:05:02 -   DECRYPT: /media/My Video/Decimate_20150421_1500
21/04/2015 16:05:02 -   DLNA: http://127.0.0.1:9000/web/media/24758.TS
21/04/2015 16:08:37 -   Removing/binning old copy.
21/04/2015 16:08:38 - Done... 1.43 GiB in 215.468 seconds - 6.8 MiB/s
21/04/2015 16:08:38 - Running postdecrypt plugin ::detectads::chancheck (priority 30) [<reference.<ts_____>.00000000000000000008>]
21/04/2015 16:08:38 -   DETECTADS: Checking /media/My Video/Decimate_20150421_1500.ts (BBC ONE South) for channel exclusion
21/04/2015 16:08:38 - Checking exclusion entry: BBC *
21/04/2015 16:08:38 - Matched exclusion entry BBC * - skipping ad detection
21/04/2015 16:08:38 - Completed postdecrypt plugin ::detectads::chancheck
21/04/2015 16:08:38 -   /media/My Video/32 Brinkburn Street_20150420_1415.ts - Already decrypted.
21/04/2015 16:08:38 -   /media/My Video/Decimate_20150420_1500.ts - Already decrypted.
21/04/2015 16:08:38 -   /media/My Video/Brain of Britain_20150420_1500.ts - Already decrypted.
21/04/2015 16:08:38 -   /media/My Video/Banished_20150409_2102.ts - Already decrypted.
21/04/2015 16:15:02 - Registered ::detectads::chancheck for postdecrypt hook with priority 30.
21/04/2015 16:15:02 - Registered ::flatten::scan for postdecryptscan hook with priority 50.
21/04/2015 16:15:02 - Registered ::flatten::scandir for postdecryptsingledir hook with priority 50.
21/04/2015 16:15:02 - Registered ::sweeper::scan for postdecryptscan hook with priority 50.
21/04/2015 16:15:02 - Registered ::sweeper::scansingledir for postdecryptsingledir hook with priority 50.
21/04/2015 16:15:02 - -------------------------------------------------------
21/04/2015 16:15:03 - DLNA Server is running.
21/04/2015 16:15:03 - Media scan starting, DLNA server status: 1
21/04/2015 16:15:03 - Dustbin: /media/My Video/[Deleted Items]
21/04/2015 16:15:03 - Root device: 2050
21/04/2015 16:15:03 - decrypt scan starting.
21/04/2015 16:15:03 -   [/media/My Video]
21/04/2015 16:15:03 - DECRYPT: [/media/My Video]
21/04/2015 16:15:03 -   /media/My Video/Banished_20150416_2100.ts - Already decrypted.
21/04/2015 16:15:03 -   /media/My Video/Eggheads_20150420_1759.ts - Already decrypted.
21/04/2015 16:15:03 -   /media/My Video/New_ The Gadget Show_20150420_1859.ts - Already decrypted.
21/04/2015 16:15:03 -   /media/My Video/Decimate_20150421_1500.ts - Already decrypted.
21/04/2015 16:15:03 -   /media/My Video/32 Brinkburn Street_20150420_1415.ts - Already decrypted.
21/04/2015 16:15:03 -   /media/My Video/Decimate_20150420_1500.ts - Already decrypted.
21/04/2015 16:15:03 -   /media/My Video/Brain of Britain_20150420_1500.ts - Already decrypted.
21/04/2015 16:15:03 -   /media/My Video/Banished_20150409_2102.ts - Already decrypted.
21/04/2015 16:25:02 - Registered ::detectads::chancheck for postdecrypt hook with priority 30.
21/04/2015 16:25:02 - Registered ::flatten::scan for postdecryptscan hook with priority 50.
21/04/2015 16:25:02 - Registered ::flatten::scandir for postdecryptsingledir hook with priority 50.
21/04/2015 16:25:02 - Registered ::sweeper::scan for postdecryptscan hook with priority 50.
21/04/2015 16:25:02 - Registered ::sweeper::scansingledir for postdecryptsingledir hook with priority 50.
21/04/2015 16:25:02 - -------------------------------------------------------
21/04/2015 16:25:02 - DLNA Server is running.
21/04/2015 16:25:02 - Media scan starting, DLNA server status: 1
21/04/2015 16:25:02 - Dustbin: /media/My Video/[Deleted Items]
21/04/2015 16:25:02 - Root device: 2050
21/04/2015 16:25:02 - decrypt scan starting.
21/04/2015 16:25:02 -   [/media/My Video]
21/04/2015 16:25:02 - DECRYPT: [/media/My Video]
21/04/2015 16:25:02 -   /media/My Video/Banished_20150416_2100.ts - Already decrypted.
21/04/2015 16:25:02 -   /media/My Video/Eggheads_20150420_1759.ts - Already decrypted.
21/04/2015 16:25:02 -   /media/My Video/New_ The Gadget Show_20150420_1859.ts - Already decrypted.
21/04/2015 16:25:02 -   /media/My Video/Decimate_20150421_1500.ts - Already decrypted.
21/04/2015 16:25:02 -   /media/My Video/32 Brinkburn Street_20150420_1415.ts - Already decrypted.
21/04/2015 16:25:02 -   /media/My Video/Decimate_20150420_1500.ts - Already decrypted.
21/04/2015 16:25:02 -   /media/My Video/Brain of Britain_20150420_1500.ts - Already decrypted.
21/04/2015 16:25:02 -   /media/My Video/Banished_20150409_2102.ts - Already decrypted.
21/04/2015 16:25:03 -   /media/My Video/Countdown_20150421_1410.ts - has been indexed.
21/04/2015 16:25:03 -   DECRYPT: /media/My Video/Countdown_20150421_1410
21/04/2015 16:25:03 -   DLNA: http://127.0.0.1:9000/web/media/24764.TS
21/04/2015 16:27:13 -   Removing/binning old copy.
21/04/2015 16:27:13 - Done... 1.05 GiB in 130.837 seconds - 8.22 MiB/s
21/04/2015 16:27:13 - Running postdecrypt plugin ::detectads::chancheck (priority 30) [<reference.<ts_____>.00000000000000000018>]
21/04/2015 16:27:13 -   DETECTADS: Checking /media/My Video/Countdown_20150421_1410.ts (Channel 4) for channel exclusion
21/04/2015 16:27:13 - Checking exclusion entry: BBC *
21/04/2015 16:27:13 - Checking exclusion entry: * Radio
21/04/2015 16:27:14 -   DETECTADS: Queued /media/My Video/Countdown_20150421_1410.ts for advert detection
21/04/2015 16:27:14 - Completed postdecrypt plugin ::detectads::chancheck
21/04/2015 16:27:14 -   /media/My Video/Have I Got a Bit More News for You_20150420_2246.ts - Already decrypted.
21/04/2015 16:27:15 -   /media/My Video/32 Brinkburn Street_20150421_1415.ts - has been indexed.
21/04/2015 16:27:15 -   DECRYPT: /media/My Video/32 Brinkburn Street_20150421_1415
21/04/2015 16:27:15 -   DLNA: http://127.0.0.1:9000/web/media/24765.TS
21/04/2015 16:30:20 -   Removing/binning old copy.
21/04/2015 16:30:22 - Done... 1.3 GiB in 186.827 seconds - 7.1 MiB/s
21/04/2015 16:30:22 - Running postdecrypt plugin ::detectads::chancheck (priority 30) [<reference.<ts_____>.00000000000000000022>]
21/04/2015 16:30:22 -   DETECTADS: Checking /media/My Video/32 Brinkburn Street_20150421_1415.ts (BBC ONE South) for channel exclusion
21/04/2015 16:30:22 - Checking exclusion entry: BBC *
21/04/2015 16:30:22 - Matched exclusion entry BBC * - skipping ad detection
21/04/2015 16:30:22 - Completed postdecrypt plugin ::detectads::chancheck
21/04/2015 16:30:22 -   /media/My Video/Homes Under the Hammer_20150421_0601.ts - Already decrypted.
21/04/2015 16:30:22 -   /media/My Video/New_ The Gadget Show_20150413_1901.ts - Already decrypted.
21/04/2015 16:30:23 -     [/media/My Video/ABC]
21/04/2015 16:30:24 -       [/media/My Video/ABC/Father Brown]
.... Rest of scan completes normally
 
MymsMan said:
So why didn't Humax start the server when waking up to perform recordings which would need indexing?
Click to expand...
That's normal. The DLNA server doesn't get started when the Humax is in half-awake mode (such as when it has woken up to make a recording).
 
MymsMan said:
Much more interesting is what happens after the system was rebooted and DLNA server started running
Click to expand...
It looks, then, like it is failing at 'lassign [$ts dlnaloc "127.0.0.1"] url'
I wonder if that is because the database is locked? That seems likely - I'll add a catch around that and a retry.
 
af123 said:
That's normal. The DLNA server doesn't get started when the Humax is in half-awake mode (such as when it has woken up to make a recording).
Click to expand...
So even in the brave new new world I will need to schedule dummy reminders after recordings to ensure that they can be decrypted promptly! :eek:
 
MymsMan said:
So even in the brave new new world I will need to schedule dummy reminders after recordings to ensure that they can be decrypted promptly! :eek:
Click to expand...
Or inject a power command via the ir package (easy enough as [system lastbootreason 0] will let you find out if it booted for a recording).
It should be possible to put the box back into standby following decryption too - might take a little while to get the logic right though.
 
af123 said:
It should be possible to put the box back into standby following decryption too - might take a little while to get the logic right though.
Click to expand...

I wouldn't want it to go back to sleep until DetectAds had finished and you couldn't detect if someone had actually started to watch the TV
 
MymsMan said:
I wouldn't want it to go back to sleep until DetectAds had finished and you couldn't detect if someone had actually started to watch the TV
Click to expand...
Not in the general case, no, but my boxes are set to always start up on channel 200 so I could fairly safely put the box back to sleep if it was woken up for a scheduled recording, the automatic processing has completed and it was still on channel 200 with no recordings being played.
I can see from your logs that you've been playing a bit with detectads : )
 
it seems to me that all this talk is more about quickly decrypting AFTER the recording happened. Is any of this relevant to being able to stream almost live recording say 15 seconds behind live?
 
af123 said:
I can see from your logs that you've been playing a bit with detectads : )
Click to expand...
Unfortunately njm hasn't been able to work on the package since last October so I started to teach myself jim/tcl
So far I have managed to implement:
  • Channel exclusion list to avoid wasting time on BBC and radio recordings
  • Option to run nicesplice to crop ads after detection complete
  • Start detectads in separate process after decryption (chrontabs entry only really needed for restarts and insurance)
I have sent changes to njm but he hasn't commented yet
 
mihaid said:
it seems to me that all this talk is more about quickly decrypting AFTER the recording happened. Is any of this relevant to being able to stream almost live recording say 15 seconds behind live?
Click to expand...
It is not directly relevant to live streaming but getting the bugs out of the dlna helper function is an important precursor to any progress on live streaming.

My, limited, experiments so far show that it is possible to access the current recording stream using the technique outlined by oatcakes but not the entire recording. If you start accessing the recording in VLC fifteen minutes after the start you will happily see the first 15 minutes but not the 15 minutes of additional recording whilst you were watching the start of the show - to see that you would need to restart and skip the first 15 minutes.

My interest is not streaming for network viewing per se but being able to access the decrypted stream so that ad breaks can be detected whilst the recording is in progress. I have ideas of how to handle the problem for detectads but not for network streaming in general.
 
af123 said:
I have always wanted to reduce the frequency once the triggering mechanism is working properly but I can see that it wouldn't help your situation since flatten is (currently) implemented as a post-decryption-scan plugin.
Click to expand...
The reason Flatten and Sweeper were originally moved to be post-decryption scans was because moving/renaming recordings invalidated the dlna index and thus made the eventual running of decryption slow and unpredictable. Now with the new index helper function the index invalidation is no longer such an issue and there is no real reason that Flatten and Sweeper couldn't be run pre-decryption scan.

This would actually be quite a trivial change to make in the .hook files though it might be best to make it a Settings option since some users might not have the autodecrypt flag set for the target folders of the move/copy.

The fly in the ointment is that the autotrigger scans only look in the original folder and wouldn't find the moved files ;)
 
Last edited:
MymsMan said:
My, limited, experiments so far show that it is possible to access the current recording stream using the technique outlined by oatcakes but not the entire recording. If you start accessing the recording in VLC fifteen minutes after the start you will happily see the first 15 minutes but not the 15 minutes of additional recording whilst you were watching the start of the show - to see that you would need to restart and skip the first 15 minutes.
Click to expand...
There might be a way around this. When I was viewing the timeshift buffer from one HDR-FOX on another by DLNA I noticed that, if I first deleted the contents of the 'Tsr' folder so that the files were recreated, I could only watch from the start of the buffer to the time when the file was loaded: playback would the stop when that point was reached. I could then start from the beginning again, but now more of the programme was loaded so I could skip to where I left off and watch the next section. However, if you view the timeshift buffer without deleting the 'Tsr' contents, '0.TS' is overwritten from the start rather than completely deleted. The upshot is that when you load the buffer by DLNA, the apparent length of the programme is much longer than the section being viewed so you can watch the whole programme, for example, without having to stop and reload the stream. I don't know where the apparent length of the buffer comes from; is it the NTS file or is it read directly from the TS file at the time of loading? If you could somehow trick the system into thinking that an ongoing recording is already full length you may be able to decrypt in real time.
 
How does one cd into a dot folder (Telnet)? From /media/My Video I can ls -a and see .autodecrytr, but I can't list what's in it or cd into it.
 
It's a file, not a directory.
Use 'ls -al' to see the flags and note the absence of a d in the first column.
It's a flag file used to indicate that recursive auto decrypt is enabled so it will be empty.
 
Ah, OK. I was looking for the helper files, but I guess they get removed again when finished. I mistakenly saw the . file and assumed it was the hidden directory (oops).
 
You must log in or register to reply here.
Back
Top