Detect Ads

Umpa · Oct 3, 2018

Just to say, that since last night its started to work. Before when I tried it spewed some error or another, nothing ever showed up in the log. So - hey ho that's life.
Today I am welcomed telling me its been updated, so I think what the hell I will give it another go, and boom loads of text and after a short while, all the adds have gone.

Very nice !

Trev · Oct 3, 2018

What do you mean by "Loads of text"? When? Where?

Umpa · Oct 4, 2018

I should have perhaps taken a screen shot, just loads of output text scrolled up the screen. Did not understand any of it, but was a welcome site compared to the 3 lines of error text it gave me before the box took an update.

Point being that it magically worked for me, and all I needed to do was wait until the next day - lol. Kinda like when the computer fairy fixes your pc overnight lol.

Umpa · Oct 4, 2018

So this program is good, I wonder if I can use it to retrospectively remove adds from recordings that have been moved over to my NAS and subsequently deleted from the Humax.

Ideally It would do this overnight and work directly from the NAS - as I have lots of programmes, that said if its going to work, I am happy to move them back to the NAS for processing.

On another note, and I am sorry if this is a stretch to far (I'm only a user) - could the program save the adverts it removes, and put them in a directory for safe keeping. Old adverts are sometimes useful to look back on. It would be really cool if that could be done.

Black Hole · Oct 4, 2018

Set bookmarks instead of cropping (in the settings), then you can do what you want afterwards.

MymsMan · Oct 4, 2018

Umpa said:
I should have perhaps taken a screen shot, just loads of output text scrolled up the screen. Did not understand any of it, but was a welcome site compared to the 3 lines of error text it gave me before the box took an update.

I don't remember you ever reporting your previous errors - we may have been able to solve the problem months ago.

Umpa said:
So this program is good, I wonder if I can use it to retrospectively remove adds from recordings that have been moved over to my NAS and subsequently deleted from the Humax.

Ideally It would do this overnight and work directly from the NAS - as I have lots of programmes, that said if its going to work, I am happy to move them back to the NAS for processing.

Do you still have the hmt and nts files associated with the recordings? If you do then you can probably run detectads if not you would need to use sidecar to recreate them.
There are also PC based ad-detection options

Umpa said:
On another note, and I am sorry if this is a stretch to far (I'm only a user) - could the program save the adverts it removes, and put them in a directory for safe keeping. Old adverts are sometimes useful to look back on. It would be really cool if that could be done.

There's no accounting for taste.
Use the DetectAds to bookmark the ads you can then use Crop (nicesplice) with the invert option to keep what would normally be thrown away

Umpa · Oct 4, 2018

Thanks for the info. Not used it before so would not have reported errors. This is a first for me and it worked after the update.

Umpa · Oct 6, 2018

Sorry for the delay in replying properly - I have been sick for the last few days and not really been up to it.

MymsMan said:
I don't remember you ever reporting your previous errors - we may have been able to solve the problem months ago.

I could not report errors as I tried it one day and it failed, tried it the next day and it worked, turns out the package was updated that night. At first I thought it was something I was doing hence I re-tried the following day. It worked a treat and was very impressed with it [probable because It seemed like magic to me] I had no idea how it worked under the hood.

MymsMan said:
Do you still have the hmt and nts files associated with the recordings? If you do then you can probably run detectads if not you would need to use sidecar to recreate them.
There are also PC based ad-detection options

No they get stripped away during the backup process, I did not know they were needed for this process, I always thought they were subtitles and stuff like that, its good to know they can be rebuilt though. I realise you probably think I'm a bit of an idiot - but understand that when I installed the CF back in the beginning it was only so I could decrypt and back up the Humax, I never looked any further than that. That feature alone was what I was looking for. I upgraded the firmware every now and again - just because - never really understanding what else it did. Just seemed a good idea - newer is always better right ?

MymsMan said:
There's no accounting for taste.
Use the DetectAds to bookmark the ads you can then use Crop (nicesplice) with the invert option to keep what would normally be thrown away

Nostalgia is a very powerful tool.

Trev · Oct 7, 2018

Had a couple of error messages referring to Length Errors from Detectads along the lines of "Queing for retry due to significant length error" on last night's X Factor (I know), and today's GP live build up. Had a few of these lately. Here's what I recon is the significant extract from detectads log

Code:

2
06/10/2018 06:00:51 RM(3787)-   DETECTADS: Checking /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Live_20181006_0600.ts (Channel 4 HD) for channel exclusion
06/10/2018 06:00:52 RM(3787)-   DETECTADS: Started /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Live_20181006_0600.ts for chaserun advert detection
06/10/2018 06:00:52 DA(3789)- ==DETECTADS Chase Run: /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Live_20181006_0600.ts
06/10/2018 06:03:51 NS(3789)- progLen = 0s, 0 bookmarks, HD = 1
06/10/2018 06:06:02 DA(3789)-   ad break found 0:00-0:14 (0 - 14) Frames: 1 - 358
06/10/2018 06:25:42 NS(3789)- -++++++++++++++++++++++++++++++++
06/10/2018 06:25:42 NS(3789)- Read failure/EOF within frame 49525, readPosition 1240455793 Fsize 1240465683, chunk 0, frameSize 80832, towrite 80689, skipping rest of file
06/10/2018 06:52:29 NS(3789)- ++++++++++++++++++++++++++++++++++++++++++
06/10/2018 06:52:29 NS(3789)- Read failure/EOF within frame 112717, readPosition 2486182705 Fsize 2486185984, chunk 0, frameSize 5439, towrite 5296, skipping rest of file
06/10/2018 07:17:05 NS(3789)- +++++++++++++++++++++++++++++++++++++++++
06/10/2018 07:17:05 NS(3789)- Read failure/EOF within frame 174577, readPosition 3850235873 Fsize 3850244096, chunk 0, frameSize 152905, towrite 152874, skipping rest of file
06/10/2018 07:30:05 DA(3789)-   ad break found 84:52-87:51 (5092 - 5271) Frames: 127302 - 131785
06/10/2018 07:30:14 NS(3789)- ++++++++++++++++----+
06/10/2018 07:30:14 NS(3789)- Read failure/EOF within frame 206262, readPosition 4569533809 Fsize 4569542656, chunk 0, frameSize 24192, towrite 24049, skipping rest of file
06/10/2018 07:38:34 NS(3789)- ++++++++++++++
06/10/2018 07:38:34 NS(3789)- Read failure/EOF within frame 227674, readPosition 5031182833 Fsize 5031186432, chunk 0, frameSize 43584, towrite 43441, skipping rest of file
06/10/2018 07:41:17 NS(3789)- +++++
06/10/2018 07:41:17 NS(3789)- Read failure/EOF within frame 235240, readPosition 5192804017 Fsize 5192851456, chunk 0, frameSize 73181, towrite 73038, skipping rest of file
06/10/2018 07:47:19 NS(3789)- ++++++++++++
06/10/2018 07:47:19 NS(3789)- Read failure/EOF within frame 252266, readPosition 5542118947 Fsize 5542125568, chunk 0, frameSize 130254, towrite 130225, skipping rest of file
06/10/2018 07:52:08 DA(3789)-   ad break found 106:00-109:34 (6360 - 6574) Frames: 159002 - 164354
06/10/2018 08:13:28 DA(3789)-   ad break found 127:10-130:49 (7630 - 7849) Frames: 190751 - 196233
06/10/2018 08:27:42 DA(3789)-   ad break found 141:07-145:06 (8467 - 8706) Frames: 211666 - 217647
06/10/2018 08:36:46 NS(3789)- ----++++++++++++++++++++++++++++++-----++++++++++++++++-----+++++++++++++++
06/10/2018 08:36:46 NS(3789)- Read failure/EOF within frame 364667, readPosition 7870851313 Fsize 7870861312, chunk 0, frameSize 38976, towrite 38833, skipping rest of file
06/10/2018 08:38:10 DA(3789)-   ad break found 155:44-157:03 (9344 - 9423) Frames: 233611 - 235581
06/10/2018 08:38:11 NS(3789)- -
06/10/2018 08:38:11 NS(3789)- Wrote 342834 entries to /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Live_20181006_0600-crop. Stripped 2234343 packets (418939k) of EPG data
06/10/2018 08:38:11 NS(3789)- New Program Length = 8479s
06/10/2018 08:38:11 DA(3789)- Final bookmarks: 14 5092 5271 6360 6574 7630 7849 8467 8706 9344 9423
06/10/2018 08:38:11 DA(3789)- /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Live_20181006_0600.ts deleted
06/10/2018 08:38:38 DA(3789)- /mod/tmp/F1 Japanese GP Qualifying Live_20181006_0600-dec.ts deleted
06/10/2018 08:38:38 DA(3789)- done...processed /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Live_20181006_0600.ts in 9286.746s 02:34:47 - 6 ad breaks detected
Removed /mod/tmp/F1 Japanese GP Qualifying Live_20181006_0600-inp.ts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Live_20181006_0600-inp.nts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Live_20181006_0600-inp.hmt<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Live_20181006_0600-dec.ts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Live_20181006_0600-dec.nts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Live_20181006_0600-dec.hmt<br>
06/10/2018 10:28:50 RM(3436)-   DETECTADS: Checking /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Replay_20181006_1028.ts (Channel 4 HD) for channel exclusion
06/10/2018 10:28:50 RM(3436)-   DETECTADS: Started /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Replay_20181006_1028.ts for chaserun advert detection
06/10/2018 10:28:51 DA(3440)- ==DETECTADS Chase Run: /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Replay_20181006_1028.ts
06/10/2018 10:31:49 NS(3440)- progLen = 0s, 0 bookmarks, HD = 1
06/10/2018 10:33:59 DA(3440)-   ad break found 0:00-0:17 (0 - 17) Frames: 1 - 418
06/10/2018 10:47:28 DA(3440)-   ad break found 12:45-16:46 (765 - 1006) Frames: 19119 - 25149
06/10/2018 10:59:21 DA(3440)-   ad break found 24:46-28:52 (1486 - 1732) Frames: 37144 - 43292
06/10/2018 11:26:36 DA(3440)-   ad break found 52:10-56:16 (3130 - 3376) Frames: 78244 - 84389
06/10/2018 11:46:36 DA(3440)-   ad break found 72:20-76:25 (4340 - 4585) Frames: 108491 - 114617
06/10/2018 12:08:27 DA(3440)-   ad break found 94:00-98:06 (5640 - 5886) Frames: 140994 - 147144
06/10/2018 12:22:33 DA(3440)-   ad break found 108:18-112:23 (6498 - 6743) Frames: 162441 - 168577
06/10/2018 12:34:36 DA(3440)-   ad break found 124:43-125:32 (7483 - 7532) Frames: 187067 - 188308
06/10/2018 12:34:37 NS(3440)- -++++++++++++++++-----+++++++++++-----++++++++++++++++++++++++++++++++++++++++-----++++++++++++++++++++++++++++-----++++++++++++++++++++++++++++++-----++++++++++++++++-----+++++++++++++++++-
06/10/2018 12:34:37 NS(3440)- Wrote 246729 entries to /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Replay_20181006_1028-crop. Stripped 1631838 packets (305969k) of EPG data
06/10/2018 12:34:37 NS(3440)- New Program Length = 5997s
06/10/2018 12:34:37 DA(3440)- Final bookmarks: 17 765 1006 1486 1732 3130 3376 4340 4585 5640 5886 6498 6743 7483 7532
06/10/2018 12:34:37 DA(3440)- /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Replay_20181006_1028.ts deleted
06/10/2018 12:34:58 DA(3440)- /mod/tmp/F1 Japanese GP Qualifying Replay_20181006_1028-dec.ts deleted
06/10/2018 12:34:58 DA(3440)- done...processed /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Qualifying Replay_20181006_1028.ts in 7389.423s 02:03:09 - 8 ad breaks detected
Removed /mod/tmp/F1 Japanese GP Qualifying Replay_20181006_1028-inp.ts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Replay_20181006_1028-inp.nts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Replay_20181006_1028-inp.hmt<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Replay_20181006_1028-dec.ts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Replay_20181006_1028-dec.nts<br>
Removed /mod/tmp/F1 Japanese GP Qualifying Replay_20181006_1028-dec.hmt<br>
06/10/2018 20:32:45 RM(3426)-   DETECTADS: Checking /media/My Video/The X Factor/New_ The X Factor_20181006_2032.ts (ITV HD) for channel exclusion
06/10/2018 20:32:45 RM(3426)-   DETECTADS: Started /media/My Video/The X Factor/New_ The X Factor_20181006_2032.ts for chaserun advert detection
06/10/2018 20:32:45 DA(3429)- ==DETECTADS Chase Run: /media/My Video/The X Factor/New_ The X Factor_20181006_2032.ts
06/10/2018 20:35:45 NS(3429)- progLen = 0s, 0 bookmarks, HD = 1
06/10/2018 20:37:36 DA(3429)-   ad break found 0:00-0:17 (0 - 17) Frames: 1 - 423
06/10/2018 20:47:16 DA(3429)-   ad break found 10:39-12:51 (639 - 771) Frames: 15977 - 19285
/mod/webif/lib/settings.class:109: Error: database is locked
in procedure 'getfile' called at file "/mod/bin/chaseget", line 435
in procedure '<reference.<setting>.00000000000000000078>' called at file "/mod/bin/chaseget", line 116
in procedure 'settings _nval_setting' called at file "/mod/lib/jim/oo.tcl", line 52
at file "/mod/lib/jim/oo.tcl", line 73
at file "/mod/webif/lib/settings.class", line 109
06/10/2018 20:55:08 NS(3429)- -+++++++++---++++++++
06/10/2018 20:55:09 NS(3429)- Read failure/EOF within frame 32007, readPosition 1047395875 Fsize 1047445504, chunk 0, frameSize 216846, towrite 216817, skipping rest of file
06/10/2018 20:55:09 NS(3429)-
06/10/2018 20:55:09 NS(3429)- Wrote 28288 entries to /media/My Video/The X Factor/New_ The X Factor_20181006_2032-crop. Stripped 310263 packets (58174k) of EPG data
06/10/2018 20:55:09 NS(3429)- New Program Length = 1131s
06/10/2018 20:55:09 DA(3429)- Final bookmarks: 17 639 771
06/10/2018 20:55:09 DA(3429)- /media/My Video/The X Factor/New_ The X Factor_20181006_2032.ts Incomplete data retrieval 52371456 bytes missing (00:03:40)
06/10/2018 20:55:09 DA(3429)- Queing /media/My Video/The X Factor/New_ The X Factor_20181006_2032.ts for retry of detectads due to significant file length error
06/10/2018 20:55:13 DA(3429)- /mod/tmp/New_ The X Factor_20181006_2032-dec.ts deleted
06/10/2018 20:55:13 DA(3429)- done...processed /media/My Video/The X Factor/New_ The X Factor_20181006_2032.ts in 1168.223s 00:19:28 - 2 ad breaks detected
Removed /mod/tmp/New_ The X Factor_20181006_2032-inp.ts<br>
Removed /mod/tmp/New_ The X Factor_20181006_2032-inp.nts<br>
Removed /mod/tmp/New_ The X Factor_20181006_2032-inp.hmt<br>
Removed /mod/tmp/New_ The X Factor_20181006_2032-dec.ts<br>
Removed /mod/tmp/New_ The X Factor_20181006_2032-dec.nts<br>
Removed /mod/tmp/New_ The X Factor_20181006_2032-dec.hmt<br>
07/10/2018 04:58:53 RM(6051)-   DETECTADS: Checking /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Build Up_20181007_0458.ts (Channel 4 HD) for channel exclusion
07/10/2018 04:58:53 RM(6051)-   DETECTADS: Started /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Build Up_20181007_0458.ts for chaserun advert detection
07/10/2018 04:58:53 DA(6057)- ==DETECTADS Chase Run: /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Build Up_20181007_0458.ts
07/10/2018 05:01:47 NS(6057)- progLen = 0s, 0 bookmarks, HD = 1
07/10/2018 05:02:50 DA(6057)-   ad break found 0:00-0:03 (0 - 3) Frames: 1 - 81
/mod/webif/lib/settings.class:109: Error: database is locked
in procedure 'getfile' called at file "/mod/bin/chaseget", line 435
in procedure '<reference.<setting>.00000000000000000076>' called at file "/mod/bin/chaseget", line 116
in procedure 'settings _nval_setting' called at file "/mod/lib/jim/oo.tcl", line 52
at file "/mod/lib/jim/oo.tcl", line 73
at file "/mod/webif/lib/settings.class", line 109
07/10/2018 05:19:04 DA(6057)-   ad break found 15:06-19:22 (906 - 1162) Frames: 22661 - 29057
07/10/2018 05:19:05 NS(6057)- -+++++++++++++++++++++++----
07/10/2018 05:19:05 NS(6057)- Wrote 36487 entries to /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Build Up_20181007_0458-crop. Stripped 233249 packets (43734k) of EPG data
07/10/2018 05:19:05 NS(6057)- New Program Length = 902s
07/10/2018 05:19:05 DA(6057)- Final bookmarks: 3 906 1162
07/10/2018 05:19:05 DA(6057)- /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Build Up_20181007_0458.ts Incomplete data retrieval 42299392 bytes missing (00:02:47)
07/10/2018 05:19:05 DA(6057)- Queing /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Build Up_20181007_0458.ts for retry of detectads due to significant file length error
07/10/2018 05:19:08 DA(6057)- /mod/tmp/F1 Japanese GP Live Build Up_20181007_0458-dec.ts deleted
07/10/2018 05:19:08 DA(6057)- done...processed /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Build Up_20181007_0458.ts in 1041.03s 00:17:21 - 2 ad breaks detected
Removed /mod/tmp/F1 Japanese GP Live Build Up_20181007_0458-inp.ts<br>
Removed /mod/tmp/F1 Japanese GP Live Build Up_20181007_0458-inp.nts<br>
Removed /mod/tmp/F1 Japanese GP Live Build Up_20181007_0458-inp.hmt<br>
Removed /mod/tmp/F1 Japanese GP Live Build Up_20181007_0458-dec.ts<br>
Removed /mod/tmp/F1 Japanese GP Live Build Up_20181007_0458-dec.nts<br>
Removed /mod/tmp/F1 Japanese GP Live Build Up_20181007_0458-dec.hmt<br>
07/10/2018 05:58:46 RM(11233)-   DETECTADS: Checking /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Live_20181007_0558.ts (Channel 4 HD) for channel exclusion
07/10/2018 05:58:46 RM(11233)-   DETECTADS: Started /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Live_20181007_0558.ts for chaserun advert detection
07/10/2018 05:58:47 DA(11239)- ==DETECTADS Chase Run: /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Live_20181007_0558.ts
07/10/2018 06:01:46 NS(11239)- progLen = 0s, 0 bookmarks, HD = 1
07/10/2018 06:04:45 DA(11239)-   ad break found 0:00-0:12 (0 - 12) Frames: 1 - 292
07/10/2018 06:53:37 NS(11239)- -++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
07/10/2018 06:53:37 NS(11239)- Read failure/EOF within frame 146482, readPosition 3321146353 Fsize 3321151488, chunk 0, frameSize 24760, towrite 24617, skipping rest of file
07/10/2018 07:01:35 NS(11239)- ++++++++++++++++
07/10/2018 07:01:35 NS(11239)- Read failure/EOF within frame 169514, readPosition 3811671598 Fsize 3811672064, chunk 0, frameSize 6531, towrite 6385, skipping rest of file
07/10/2018 07:54:40 DA(11239)-   ad break found 111:35-114:15 (6695 - 6855) Frames: 167381 - 171365
07/10/2018 08:11:06 DA(11239)-   ad break found 129:31-131:28 (7771 - 7888) Frames: 194271 - 197211
07/10/2018 08:11:07 NS(11239)- +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----++++++++++++++++++++++++--
07/10/2018 08:11:07 NS(11239)- Wrote 344682 entries to /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Live_20181007_0558-crop. Stripped 2005561 packets (376042k) of EPG data
07/10/2018 08:11:07 NS(11239)- New Program Length = 7599s
07/10/2018 08:11:07 DA(11239)- Final bookmarks: 12 6695 6855 7771 7888
07/10/2018 08:11:07 DA(11239)- /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Live_20181007_0558.ts deleted
07/10/2018 08:11:33 DA(11239)- /mod/tmp/F1 Japanese Grand Prix Live_20181007_0558-dec.ts deleted
07/10/2018 08:11:33 DA(11239)- done...processed /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Live_20181007_0558.ts in 7787.212s 02:09:47 - 3 ad breaks detected
Removed /mod/tmp/F1 Japanese Grand Prix Live_20181007_0558-inp.ts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Live_20181007_0558-inp.nts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Live_20181007_0558-inp.hmt<br>
Removed /mod/tmp/F1 Japanese Grand Prix Live_20181007_0558-dec.ts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Live_20181007_0558-dec.nts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Live_20181007_0558-dec.hmt<br>
07/10/2018 08:30:18 RM(24867)-   DETECTADS: Checking /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Reaction_20181007_0830.ts (Channel 4 HD) for channel exclusion
07/10/2018 08:30:19 RM(24867)-   DETECTADS: Started /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Reaction_20181007_0830.ts for chaserun advert detection
07/10/2018 08:30:19 DA(24869)- ==DETECTADS Chase Run: /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese GP Live Reaction_20181007_0830.ts
07/10/2018 08:33:13 NS(24869)- progLen = 0s, 0 bookmarks, HD = 1
07/10/2018 12:28:50 RM(23745)-   DETECTADS: Checking /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Replay_20181007_1228.ts (Channel 4 HD) for channel exclusion
07/10/2018 12:28:50 RM(23745)-   DETECTADS: Started /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Replay_20181007_1228.ts for chaserun advert detection
07/10/2018 12:28:50 DA(23751)- ==DETECTADS Chase Run: /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Replay_20181007_1228.ts
07/10/2018 12:31:49 NS(23751)- progLen = 0s, 0 bookmarks, HD = 1
07/10/2018 12:32:51 DA(23751)-   ad break found 0:00-0:16 (0 - 16) Frames: 1 - 401
07/10/2018 12:49:35 DA(23751)-   ad break found 15:04-19:10 (904 - 1150) Frames: 22600 - 28738
07/10/2018 13:20:51 DA(23751)-   ad break found 46:19-50:24 (2779 - 3024) Frames: 69478 - 75598
07/10/2018 13:27:31 NS(23751)- -+++++++++++++++++++++++-----+++++++++++++++++++++++++++++++++++++++++++++++++-----++++++++++++
07/10/2018 13:27:31 NS(23751)- Wrote 130418 entries to /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Replay_20181007_1228-crop. Stripped 799341 packets (149876k) of EPG data
07/10/2018 13:27:31 NS(23751)- New Program Length = 2939s
07/10/2018 13:27:31 DA(23751)- Final bookmarks: 16 904 1150 2779 3024
07/10/2018 13:27:32 DA(23751)- /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Replay_20181007_1228.ts deleted
07/10/2018 13:27:47 DA(23751)- /mod/tmp/F1 Japanese Grand Prix Replay_20181007_1228-dec.ts deleted
07/10/2018 13:27:47 DA(23751)- done...processed /media/My Video/F1 Japanese GP Qualifying Live/F1 Japanese Grand Prix Replay_20181007_1228.ts in 3358.676s 00:55:59 - 3 ad breaks detected
Removed /mod/tmp/F1 Japanese Grand Prix Replay_20181007_1228-inp.ts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Replay_20181007_1228-inp.nts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Replay_20181007_1228-inp.hmt<br>
Removed /mod/tmp/F1 Japanese Grand Prix Replay_20181007_1228-dec.ts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Replay_20181007_1228-dec.nts<br>
Removed /mod/tmp/F1 Japanese Grand Prix Replay_20181007_1228-dec.hmt<br>

Any thoughts on the possible cause please?

~~OOH, thats crap. I'll have another go. It would appear that copy, paste doesn't work too well.~~
Ah, that's better. Errors were for 06/10/2018

MymsMan · Oct 7, 2018

Very odd, it is failing retrieving one of the settings - something that is normally very reliable, some other process must have been setting a value at the time.

I will change the code to check the setting less frequently but I wonder if @af123 could check for and retry locked database conditions within the settings object code since retrieving settings is spread throughout the webif.

MymsMan · Dec 22, 2018

MymsMan said:
Very odd, it is failing retrieving one of the settings - something that is normally very reliable, some other process must have been setting a value at the time.

I will change the code to check the setting less frequently but I wonder if @af123 could check for and retry locked database conditions within the settings object code since retrieving settings is spread throughout the webif.

I realised I hadn't actually made this change when I suffered the same problem myself recently

So a new version of chaseget (0.1.2-0) has now been uploaded

Matthew · Dec 22, 2018

Is there any way to adjust the settings Ad Detector uses, as I nearly always manually crop the detected file but wish to avoid the sound cutoff at the edit I have to alter the ad 'out points' by 3-4 secs, if I could ajust settings to avoid that would save me much time.

MymsMan · Dec 23, 2018

Matthew said:
Is there any way to adjust the settings Ad Detector uses, as I nearly always manually crop the detected file but wish to avoid the sound cutoff at the edit I have to alter the ad 'out points' by 3-4 secs, if I could ajust settings to avoid that would save me much time.

Add -padSec 4to the Other options on the settings page

Trev · Dec 23, 2018

What exactly is the problem and what does -padSec 4 actually do?
I haven't noticed any sound cutoff. But perhaps that's because I haven't been paying attention.

Black Hole · Dec 23, 2018

A general problem with nicesplice-implemented cuts is the audio stream can take a few seconds to sort itself out, resulting in broken or no audio temporarily. My understanding is that this setting adds a little padding to the detect-ads cut point, to give the audio stream time to settle before the "meat" starts again.

Not a problem for us bookmarkers of course.

Trev · Dec 23, 2018

Thanks BH.
I actually had a good stiff RTFM in the interim. That also gave me a better idea of what's what. But your explanation above expands on that nicely.
It's probably that I don't remember it happening rather than it not happening for me.
I assume (or is it presume?) that the addition of a positive value to padSec makes the 'ad end' cut earlier than it normally would have done.

MymsMan · Dec 23, 2018

Trev said:
I assume (or is it presume?) that the addition of a positive value to padSec makes the 'ad end' cut earlier than it normally would have done.

Yes - it shortens the ad break by the specified amount at both ends of the break
You can use a negative value if you want to try and cut out the sponsors bumper messages

Matthew · Dec 28, 2018

MymsMan said:
Add -padSec 4to the Other options on the settings page

Thanks, is there any way to just modify the 'out point' I don't have a problem with the 'in point' I can see there is no setting but if you could say where that setting is applied in the code that would help.

MymsMan · Dec 28, 2018

padsec is actually one of the settings that is handled in the Silence program a C++ routine that has to be compiled using a cross compiler

It would be possible to add separate in/out padding in the Jim code layer but since padding already existed in Silence I just exposed the existing settings rather than reinventing the wheel.

Matthew · Dec 30, 2018

MymsMan said:
padsec is actually one of the settings that is handled in the Silence program a C++ routine that has to be compiled using a cross compiler

It would be possible to add separate in/out padding in the Jim code layer but since padding already existed in Silence I just exposed the existing settings rather than reinventing the wheel.

Thanks for the pointers, I've done the following mod, creating a new option (padOut) so my friends can use it. Please let me know what you think.

padOut sets number of seconds to pad, however start of file bookmark is always padded 2 secs (as that's all it needs there). Setting padOut means the last bookmark is ignored as that cleans the crop at the end.

padSec overrides padOut

I`ve also added a button to the detect page to jump straight to the crop page once ad-detect done.

Code:

--- /mod/webif/plugin/detectads/detectads_orig.jim
+++ mod/webif/plugin/detectads/detectads.jim
@@ -152,7 +152,8 @@
       admaxsec  65
       padsec     0
       makethm    0
-      thmoffset  0}
+      thmoffset  0
+      padout     0}
 
     # Override default from settings DB
     foreach {key defvalue} [array get optarray] {
@@ -467,6 +468,7 @@
     $queue_db close
 }
 
+# Manual
 proc ::detectads::run {ts} {
     set file [$ts get file]
 
@@ -479,7 +481,11 @@
            log "Cannot acquire exclusive lock $file, terminating." 0
            return {"DEFER" "Cannot acquire exclusive lock"}
     }
-
+    set stime [$ts get start]
+    set etime [$ts get end]
+    set durn $($etime-$stime)
+    set theend 0
+    set padding 0
     set size [$ts size]
     set numAdBreaks 0
     set bname [file rootname [file tail $file]]
@@ -495,7 +501,14 @@
     foreach {key value} [array get ::opts] {
         set $key $value
     }
-
+       
+    if {$padsec} {
+    set padout 0
+    }
+    if {$padout} {
+    log "Padding out points by $padout secs and ignoring program end bookmark" 0
+    }
+   
     set tailPid 0; # unused (for killing head of pipe on timeout - useful with tail -f recording.ts | ffmpeg ... | silence...)
 
     set silenceArgs "\
@@ -545,9 +558,22 @@
                     startSeconds \
                     endMinutes \
                     endSeconds]} {
-
+   
             set totalStartSeconds [expr {$startMinutes * 60 + $startSeconds}]
-            set totalEndSeconds [expr {$endMinutes * 60 + $endSeconds}]
+            if {$cropcmd eq " " && $totalStartSeconds < 10 && $padout} {
+            set padding 2
+            } elseif {$padout} {
+            set padding $padout
+            }
+            set totalEndSeconds [expr {$endMinutes * 60 + $endSeconds - $padding}]
+            set endMinutes [expr int($totalEndSeconds / 60)]
+            set endSeconds [expr ($totalEndSeconds % 60)]
+            if { [expr {$totalEndSeconds + 10}] > $durn} {
+               set totalEndSeconds [expr {$totalEndSeconds + $padding}]
+               if {$padout} {
+                  set theend 1
+               }
+            }
 
             log "  ad break found $startMinutes:$startSeconds-$endMinutes:$endSeconds ($totalStartSeconds - $totalEndSeconds)" 0
 
@@ -565,7 +591,11 @@
                         ladd bookmarks 0 $totalStartSeconds $totalEndSeconds
                     }
                 } else {
-                    ladd bookmarks $totalStartSeconds $totalEndSeconds
+            if {$theend == 0} {
+            ladd bookmarks $totalStartSeconds $totalEndSeconds
+                     } else {
+            ladd bookmarks $totalStartSeconds
+                     }
                 }
             }
             # Update bookmarks and build up crop cmd line as each break is detected.- mymsman  150408
@@ -746,6 +776,7 @@
     return $retcode
 }
 
+# Chaserun
 proc ::detectads::chaserun {ts {Qid 0} {retrynumb 0}} {
     # run ad detection against currently recording program using dlna helper - mymsman 150505
     # creates decrypted, cropped and shrunk copy of file in video root directory.
@@ -768,12 +799,23 @@
     }
     set statustok [system startop -multiple detectads $file]
     set size [$ts size]
+    set stime [$ts get start]
+    set etime [$ts get end]
+    set durn $($etime-$stime)
+    set theend 0
+    set padding 0
     set numAdBreaks 0
 
     # set each option from settings/overrides
     foreach {key value} [array get ::opts] {
         set $key $value
-    }
+    }       
+    if {$padsec} {
+    set padout 0
+    }
+    if {$padout} {
+    log "Padding out points by $padout secs and ignoring program end bookmark" 0
+    }   
     set tailPid 0; # unused (for killing head of pipe on timeout - useful with tail -f recording.ts | ffmpeg ... | silence...)
 
     set silenceArgs "\
@@ -955,7 +997,20 @@
                     endSeconds]}  {
 
             set totalStartSeconds [expr {$startMinutes * 60 + $startSeconds}]
-            set totalEndSeconds [expr {$endMinutes * 60 + $endSeconds}]
+            if {$cropcmd eq " " && $totalStartSeconds < 10 && $padout} {
+            set padding 2
+            } elseif {$padout} {
+            set padding $padout
+            }
+            set totalEndSeconds [expr {$endMinutes * 60 + $endSeconds - $padding}]
+            set endMinutes [expr int($totalEndSeconds / 60)]
+            set endSeconds [expr ($totalEndSeconds % 60)]
+            if { [expr {$totalEndSeconds + 10}] > $durn} {
+               set totalEndSeconds [expr {$totalEndSeconds + $padding}]
+               if {$padout} {
+                  set theend 1
+               }
+            }
 
             log "  ad break found $startMinutes:$startSeconds-$endMinutes:$endSeconds ($totalStartSeconds - $totalEndSeconds) Frames: $startFrame - $endFrame" 0
 
@@ -975,7 +1030,11 @@
                         ladd bookmarks 0 $totalStartSeconds $totalEndSeconds
                     }
                 } else {
-                    ladd bookmarks $totalStartSeconds $totalEndSeconds
+            if {$theend == 0} {
+            ladd bookmarks $totalStartSeconds $totalEndSeconds
+                     } else {
+            ladd bookmarks $totalStartSeconds
+                     }
                 }
             }
             # Update bookmarks and build up crop cmd line as each break is detected.- mymsman  150408

Code:

--- /mod/webif/plugin/detectads/detectads-orig
+++ mod/webif/plugin/detectads/detectads
@@ -133,6 +133,7 @@
                 puts "-brkMinSec  60   = Minimum length of an ad break in seconds"
                 puts "-adMaxSec   65   = Maximum length of a single ad in seconds"
                 puts "-padSec     0    = Adjust ends of ad breaks by n seconds"
+                puts "-padOut     0    = Adjust out points of ad breaks by n seconds"
                 puts "-makeThm    n    = Make thumbnail if it doesn't already exist"
                 puts "-thmOffset  0    = Thumbnail offset from programme start"
         }

Code:

--- /mod/webif/plugin/detectads/web/index_orig.jim
+++/mod/webif/plugin/detectads/web/index.jim
@@ -34,6 +34,10 @@
 
     <p>See <b><a href=\"http://wiki.hummy.tv/wiki/DetectAds\" target=\"_blank\">DetectAds user guide</a></b> for help</p>
     <p>Use <b><a href=\"/settings/settings.jim\" target=\"_blank\">Settings page</a></b> to alter default DetectAds options or override them below</p>
+<span class=hidden id=params
+    file=\"[cgi_quote_url $rfile]\"
+    dir=\"[cgi_quote_url $dir]\"
+></span>
 <table class=keyval cellpadding=5>
 <tr><th>File:</th><td>$non_escaped_file</td></tr>
 "
@@ -85,6 +89,7 @@
 }
 puts "
 <button id=back>Back to Media Browser</button>
+<button id=crop>Crop File</button>
 </div>
 <div id=results class=\"hidden\">
 <div class=\"running blood\">

Code:

--- /mod/webif/plugin/detectads/web/script_orig.js
+++ /mod/webif/plugin/detectads/web/script.js
@@ -1,5 +1,6 @@
 var curval = 0;
 var $slider;
+var madebm = false;
 
 function draw_slider() {
    $slider = $('#slider');
@@ -62,6 +63,8 @@
             $('.running').slideUp('slow');
             loaddata(data, true);
             loaddata('\n\n=== Refresh page to see bookmarks ===');
+        $('#crop').button('option', 'disabled', false);
+        madebm = true;
          },
          error : function(_, _, e) {
             if (window.console)
@@ -143,6 +146,15 @@
 
    });
 
+   $('#crop').button().click(function() {
+    values = $.trim($('#bookmarks').text()).split(/ +/);
+    if ((values.length && values[0] != '') || madebm)
+    {
+    window.location = '/browse/crop/crop.jim?file=' + $('#params').attr('file');
+    }
+    else alert('No bookmarks, cannot crop.');
+});
+   
    $('#reset').button({
    }).on('click', function() {
       $(this).button('option', 'disabled', true);
@@ -176,3 +188,11 @@
    });
 
 });
+
+$(document).ready(function() { 
+
+   $('#crop').button('option', 'disabled', true);
+   var values = $.trim($('#bookmarks').text()).split(/ +/);
+   if (values.length && values[0] != '') $('#crop').button('option', 'disabled', false);
+ 
+});

Detect Ads

Member

The Dumb One

Member

Member

May contain traces of nut

Ad detector

Member

Member

The Dumb One

Ad detector

Ad detector

Active Member

Ad detector

The Dumb One

May contain traces of nut

The Dumb One

Ad detector

Active Member

Ad detector

Active Member