[DetectAds] Announcing DetectAds version 2

MymsMan said:
After difficulty trying to remember all the ins and outs of cross compiling I have uploaded a version, 0.2.1-6, that appears to work with the new libsndfile package and hopefully resolves problems with large files.
Click to expand...
Can't comment on the large files, but this seems to be doing OK

Only niggles I noticed were all in Rugby WCup recordings
- missed a few breaks on a first pass - re-ran it manually and they were added
- hasn't happened again since I switched to the 'chase-processing' method early last week

so, all in all looking good. I like the functionality, and figure +ve feedback is always welcome :)
 
OwenMc said:
Can't comment on the large files, but this seems to be doing OK

Only niggles I noticed were all in Rugby WCup recordings
- missed a few breaks on a first pass - re-ran it manually and they were added
- hasn't happened again since I switched to the 'chase-processing' method early last week

so, all in all looking good. I like the functionality, and figure +ve feedback is always welcome :)
Click to expand...

Thanks for the compliments :D

I am surprised that you got different results on rerunning - the data would be identical so the results should be the same, chase processing shouldn't make a difference
 
MymsMan said:
I am surprised that you got different results on rerunning - the data would be identical so the results should be the same, chase processing shouldn't make a difference
Click to expand...
Yeah, I can't come up with a good reason for that at all - I didn't notice any reboots had happened around those scans, which is the only theory I came up with.
Maybe the silences were right on the threshold for acceptance, and tipped the other side on the 2nd run? There end my guesses :)
 
A couple of points. I have been using the package for a while now and it seems to be working OK and what's more, HWMBO is quite impressed (good result there then), but Despite having 'Dustbin/delete original recording after processing?' set to Yes, I still get two recordings in the Media List and File List, one flagged -Crop Done and the other one no so flagged and no associated recording in the Deleted Items folder. Should this be happening (I do have Undelete running), and if so, why? As it's a bit off putting having two entries for each programme in the Media list and 8 in the file list.
On a slightly different tack, would it be possible to alter the settings so that the user can enter their own equivalent of '-Crop Done' and 'Decrypt Done' flags to something else, say 'Cropped' into the title and filename? I know I could do this in Sweeper, but feel that it would be better done 'at source'. I've also got an SD recording from 'Yesterday' recorded on 16-09-15 that is flagged 'Decrypt Done'. Where did that come from?
 
Within the next couple of days I should be posting the next version of DetectAds which will restore the original file name and title when processing is complete, handle length errors better, hide -dec files and also return the system to standby if awakened by chaseget.

Sorry for delays but I managed to find the circular tuit today.

Not sure why deleting original isn't working for you - hopefully it will be fixed in the update.
 
MymsMan said:
Not sure why deleting original isn't working for you - hopefully it will be fixed in the update.
Click to expand...
I assume from this that it should do then. Further than that, I tried a test recording this afternoon on More4. Four In A bed and am left with three in the CF Media list. One named Four in a bed {date}.ts @609.64MiB, another one named Four in a bed -Decrypt {date}.ts @609.64MiB and Four in a bed -Crop {date}.ts @353.93MiB and of course the same thing in [FlatView]. And a total of 9 associated files in Win File explorer. No sign of a -Crop Done.ts anywhere though. So it's leaving two superflous copies laying about in Media List ans where's the -Crop Done file? Not in the [Deleted Items]
 
OwenMc said:
Only niggles I noticed were all in Rugby WCup recordings
- missed a few breaks on a first pass - re-ran it manually and they were added
- hasn't happened again since I switched to the 'chase-processing' method early last week
Click to expand...

It happened yesterday with the chase-processing setting on
- file name on-screen had 'len-error' at the end
- 6 breaks on opening the file for play
- re-processed, and a 7th was added

Again, just rugby - I've yet to see it for anything else. Maybe something to do with how they set up the live event ads?
2nd recording yesterday stopped recording at the kickoff (most annoying), so there may be a few hiccups around event info?

Not too bothered, as it's all nearly over anyway, but just to keep the info stream ticking over :)
O
 
Getting different results following a length error is quite plausible since the length error is unlikely to recur once recording has completed - they seem to occur if the Humax DLNA indexer kicks in before chaseget has managed to retrieve the end of program. The new version will automatically queue the recording to retry ad detection if the error amounts to more than 30 seconds of recording.
 
I have just uploaded detectads 0.2.2-0 and chaseget 0.1.1.-1 to the package repository which contain a number of changes requested on this forum.

Output files are renamed to the original file name & title on completion of processing.
File renaming will not take place if any of the following is true:
  • The "Dustbin/delete original recording after processing?" option is set to NO
  • The Input or Output files are Inuse (e.g. being played or processed by another task)
  • There is a length error on the -dec output file
If a length error is significant, more than 30 seconds long, the input file will be automatically queued to retry ad detection since it is unlikely to recur again.

If you do not use auto crop the Bookmarks set by DetectAds will now be automatically 'inverted' to meet nicesplice's requirements removing the need to perform that operation on the Crop menu page before performing the Crop operation manually

ChaseGet will return the system to standby if it appears safe to do so.
To reduce the risk of upsetting important family members if the box shuts down without warning (The Customised Firmware has no ability to put a 2 minute standby warning on screen like the Humax does) you can specify a time range in the ChaseGet Settings during which automatic return to standby won't operate.
 
I'll monitor the new release for the week and report back anything noteworthy.

OwenMc said:
2nd recording yesterday stopped recording at the kickoff (most annoying), so there may be a few hiccups around event info?
O
Click to expand...
Off topic, but following on from above....

Last night box was powered on when I got home, RC was being 'ignored' - not a reaction at all, so I hit the old-reliable power-button on the back to regain a semblance of control.
Further checking showed ALL Sunday's subsequent recordings were knackered, starting at kick-off and all the way through the evening
- Humax recording list showed full-length recording times, but wouldn't play anything
- web i/f showed these events with 0 bytes for both orig event + attempted decrypt event

so I'll not blame that on the event info - methinks I had other problems :) :)

O
 
Had another fail this evening. I was recording two progs at the same time (both with ads) and one failed to crop properly and left a -crop file, the other was OK. I still have them and the log.
 
Please post or attach to a private message the relevant log file section and I'll take a look but some length errors seem to be unavoiadable
 
I also had an'All Stars Mr. & Mrs' from 20:00 last night leave the original and a -Crop (not -Crop Done) file I haven't tried playing the -crop file yet, but The Celebrity Chase certainly hacked off HWMBO by missing bits. I had to revert to my EE Box to see the end. Here you go with the log.
Code: 
797 24/10/2015 22:05 - FV: --> Skipping (in-use): Psycho Pussies When Cats Attack 24-10-15 1908
796 24/10/2015 22:04 - FV: --> Removing Formula 1_ US Grand Prix - Qualifying____20151024_2145 (inode mismatch)
795 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 119
794 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 317
793 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
792 /mod/webif/plugin/flatview/auto.hook:119: Error: could not read "/media/My Video/ [FlatView]/Formula 1_ US Grand Prix - Qualifying____20151024_2145.ts": No such file or directory
791 24/10/2015 22:02 - FV: +++ Adding Formula 1_ US Grand Prix - Qualifying____20151024_2145
790 epgpatch: No patch required.
789 epgpatch: Processing '/media/My Video/Formula 1_ Hungarian Grand Prix -___/Formula 1_ US Grand Prix - Qualifying____20151024_2145'
788 24/10/2015 21:26 - FV: --> Skipping (in-use): Psycho Pussies When Cats Attack 24-10-15 1908
787 24/10/2015 20:10 - FV: +++ Adding Psycho Pussies When Cats Attack 24-10-15 1908
786 24/10/2015 20:10 - FV: +++ Adding The Chase Celebrity Special 24-10-15 1858
785 24/10/2015 20:10 - FV: +++ Adding The Chase Celebrity Special -Crop 24-10-15 1858
784 24/10/2015 20:10 - FV: --> Removing Psycho Pussies_ When Cats Attack_20151024_1908 (orphaned)
783 24/10/2015 20:08 - FV: +++ Adding Psycho Pussies_ When Cats Attack_20151024_1908
782 24/10/2015 20:08 - FV: --> Removing Psycho Pussies_ When Cats Attack_20151024_1908 (inode mismatch)
781 24/10/2015 20:07 - FV: --> Removing Psycho Pussies_ When Cats Attack_20151024_1908-crop (orphaned)
780 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 119
779 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 317
778 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
777 /mod/webif/plugin/flatview/auto.hook:119: Error: could not read "/media/My Video/ [FlatView]/Psycho Pussies_ When Cats Attack_20151024_1908-crop.ts": No such file or directory
776 24/10/2015 20:06 - FV: +++ Adding Psycho Pussies_ When Cats Attack_20151024_1908
775 epgpatch: No patch required.
774 epgpatch: Processing '/media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908'
773 24/10/2015 19:58 - FV: +++ Adding New_ The Chase_ Celebrity Special_20151024_1858
772 epgpatch: No patch required.
771 epgpatch: Processing '/media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858'
Psycho Pussies was OK.
What is the entry for the F1 all about, that was on BBC2.
There wer no len errors reported in these recordings, but I have had one recently
 
Trev said:
Code: 
24/10/2015 22:04 - FV: --> Removing Formula 1_ US Grand Prix - Qualifying____20151024_2145 (inode mismatch)
795 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 119
794 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 317
793 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
792 /mod/webif/plugin/flatview/auto.hook:119: Error: could not read "/media/My Video/ [FlatView]/Formula 1_ US Grand Prix - Qualifying____20151024_2145.ts": No such file or directory
791 24/10/2015 22:02 - FV: +++ Adding Formula 1_ US Grand Prix - Qualifying____20151024_2145
What is the entry for the F1 all about, that was on BBC2.
Click to expand...
This must have been created in the Flat View area previously. The message "inode mismatch" means that the file in the FlatView area and its source in the hierarchical tree no longer match. This could be because another process (possibly detactads) has renamed the files to something else and replaced the original with a modified copy.

In this situation flatview deletes the entry in the FlatView area and creates a new link to the (now modified) source.
 
Would any of this be caused by sweeper changing the date format in the filename? I notice that 'Psycho Pussies' has a load of entries with both date formats. Psycho Pussies played OK with the ads removed and nothing missing. It was The Chase that screwed up.
 
When I said "the log" I should have been specific and said detectads.log, sorry for misleading

Detectads should not have been processing the Grand Prix qualifying since it it is a BBC program, but it could be a Sweeper entry renaming the file if you have been attempting to remove underscores from the name. It is pity that FV doesn't include seconds in its message time stamps since a minute is a very long time in computing terms

The reason DetectAds keeps both Original and -crop files when it detects length errors is so so that you can quickly go back to the original if needed however it is not always DetectAds that causes ends of files to go missing - it could be an error in the Accurate Recording markers from the broadcaster.
DetectAds is Never responsible for any missing data in the original.
 
Stupid me:oops:. Here you go.
Code: 
458 24/10/2015 21:45:46 RM(9443)- Matched exclusion entry BBC * - skipping ad detection
457 24/10/2015 21:45:46 RM(9443)- DETECTADS: Checking /media/My Video/Formula 1_ Hungarian Grand Prix -___/Formula 1_ US Grand Prix - Qualifying____20151024_2145.ts (BBC TWO HD) for channel exclusion
456 24/10/2015 20:06:47 DA(2462)- done...processed /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908.ts in 3457.631s 00:57:38 - 5 ad breaks detected
455 24/10/2015 20:06:47 DA(2462)- /mod/tmp/Psycho Pussies_ When Cats Attack_20151024_1908-dec.ts deleted
454 24/10/2015 20:06:34 DA(2462)- /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908.ts deleted
453 24/10/2015 20:06:32 NS(2462)- New Program Length = 2622s
452 24/10/2015 20:06:32 NS(2462)- Wrote 65592 entries to /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908-crop. Stripped 772826 packets (144904k) of EPG data
451 24/10/2015 20:06:32 NS(2462)- -+++++++------+++++++++++++-----++++++++++++++-----+++++-
450 24/10/2015 20:06:22 DA(2462)- ad break found 56:37-57:47 (3397 - 3467) Frames: 84937 - 86668
449 24/10/2015 20:00:14 DA(2462)- ad break found 46:12-50:03 (2772 - 3003) Frames: 69302 - 75079
448 24/10/2015 19:59:26 DA(1880)- done...processed /media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858.ts in 3574.791s 00:59:35 - 5 ad breaks detected
447 24/10/2015 19:59:26 DA(1880)- /mod/tmp/New_ The Chase_ Celebrity Special_20151024_1858-dec.ts deleted
446 24/10/2015 19:59:11 DA(1880)- Incomplete data retrieval -14 bytes missing (00:00:00)
445 24/10/2015 19:59:11 NS(1880)- New Program Length = 2489s
444 24/10/2015 19:59:11 NS(1880)- Wrote 71156 entries to /media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858-crop. Stripped 555789 packets (104210k) of EPG data
443 24/10/2015 19:59:11 NS(1880)- -+++++++++++++++++++----+++++++---+++++++++---+++++++++----
442 24/10/2015 19:59:09 DA(1880)- ad break found 55:30-59:19 (3330 - 3559) Frames: 83251 - 88977
441 24/10/2015 19:46:06 DA(1880)- ad break found 43:05-45:26 (2585 - 2726) Frames: 64617 - 68154
440 24/10/2015 19:41:48 DA(2462)- ad break found 27:33-31:24 (1653 - 1884) Frames: 41335 - 47104
439 24/10/2015 19:34:45 DA(1880)- ad break found 31:38-34:00 (1898 - 2040) Frames: 47461 - 51004
438 24/10/2015 19:24:08 DA(1880)- ad break found 20:11-23:03 (1211 - 1383) Frames: 30285 - 34579
437 24/10/2015 19:23:42 DA(2462)- ad break found 8:33-13:27 (513 - 807) Frames: 12818 - 20179
436 24/10/2015 19:10:54 DA(2462)- ad break found 0:00-0:19 (0 - 19) Frames: 1 - 474
435 24/10/2015 19:09:11 RM(2512)- DETECTADS: Ad-detection flag already set /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908-crop.ts
434 24/10/2015 19:09:11 RM(2512)- DETECTADS: Checking /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908-crop.ts (Channel 5) for channel exclusion
433 24/10/2015 19:09:10 NS(2462)- progLen = 0s, 0 bookmarks, HD = 0
432 24/10/2015 19:08:12 DA(2462)- ==DETECTADS Chase Run: /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908.ts
431 24/10/2015 19:08:11 RM(2459)- DETECTADS: Started /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908.ts for chaserun advert detection
430 24/10/2015 19:08:11 RM(2459)- DETECTADS: Checking /media/My Video/Psycho Pussies_ When Cats Attack_20151024_1908.ts (Channel 5) for channel exclusion
429 24/10/2015 19:01:22 DA(1880)- ad break found 0:00-0:29 (0 - 29) Frames: 1 - 729
428 24/10/2015 18:59:53 RM(1928)- DETECTADS: Ad-detection flag already set /media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858-crop.ts
427 24/10/2015 18:59:53 RM(1928)- DETECTADS: Checking /media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858-crop.ts (ITV HD) for channel exclusion
426 24/10/2015 18:59:51 NS(1880)- progLen = 0s, 0 bookmarks, HD = 1
425 24/10/2015 18:58:52 DA(1880)- ==DETECTADS Chase Run: /media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858.ts
424 24/10/2015 18:58:52 RM(1878)- DETECTADS: Started /media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858.ts for chaserun advert detection
423 24/10/2015 18:58:52 RM(1878)- DETECTADS: Checking /media/My Video/The Chase_ Celebrity Special/New_ The Chase_ Celebrity Special_20151024_1858.ts (ITV HD) for channel exclusion
I'll recheck the and of The Chase again. I was being submitted to a whole bunch of tongue pie by HWMBO at the time and rescuing the situation was considerably higher on my priorities list than investigating what was wrong. I'll go and have a look now.
Right then. The broken one (cut off the end) had the -crop suffix and did not have New: in the info panel and was flagged as 41 mins
The other one was flagged as 59 mins, It was all there and had been bookmarked.
There was another copy in /Media/My Videos/[Deleted Items]/WIF_autodecrypt. This one was 59 mins, had New_ in the filename and New: in the WIf info panel and had the AD icon flag.
 
Last edited:
You must log in or register to reply here.
Back
Top