Auto Detect Ad Breaks

[Deleted member 473]

Update: processing the [130 minute] film took 37 minutes.

I reckon I can FTP a film onto my laptop, even over WiFi, and then strip out its ads and save it to mpg using VideoRedo, in far less time than that, and also get it right!

Have you noticed that on most films the ad breaks get less frequent the further you are into the film? Do the advertisers assume people have given up half way?

 

[af123]

Where is the bottle-neck in this process? Would it be more efficient to strip out the audio stream into a separate file first? If it is ffmpeg taking time to report on audio signal power, would it be straightforward to compile an executable which is optimised for that one task?

ffmpeg is extracting the audio stream (along with any required conversions for HD) and feeding it into the silence detection and analysis program. Overall there probably isn't any benefit in separating the two parts of the pipeline although I did when testing because I wanted to repeatedly run the analysis.

Newer versions of ffmpeg have a built-in silence detection filter so, in principal, the job could be done in one pass. The silence cluster analysis would then be done in something like Jim - and that's what njm has proposed as a possibility. I built a few versions of ffmpeg but couldn't get the filter to work properly in the limited time I had.

 

[Black Hole]

I reckon I can FTP a film onto my laptop, even over WiFi, and then strip out its ads and save it to mpg using VideoRedo, in far less time than that, and also get it right!

But that's not an unattended process. The goal for this is an automatic background process like decryption is now.

njm has sent me the files for packaging up as a beta!

njm definitely qualifies for induction to the ranks of BYTs!

 

[mihaid]

Is the recording definitely decrypted?

You were right it was not decrypted. confusion between the HD thing and decryption. I have now decrypted it and did the detectad but no bookmarks, could the process be interrupted by an auto switch off? I have used detect ad on a 30 min ch4 prog and it seems to have worked, it found one ad which is probably right.

 

[af123]

But that's not an unattended process. The goal for this is an automatic background process like decryption is now.

That's already done in njm's code but I left it disabled for this first beta. It doesn't yet handle being powered off in progress though.

More thoughts...how about a central database for ad breaks? Once one box has done this detection process, it could record the bookmark positions to that database, taking into account padding and any local time offset, and then other boxes could automatically apply those same bookmark positions. That would help boxes which record from standby and then power off. When they power on again and decrypt the recording, the bookmarks may already be available, saving the overhead of this process.

 

[ChrisDaniels]

Awesome idea

 

[njm]

Thanks for testing, everybody. As af123 said, it's early days yet.

I'm glad it's working at least some of the time. My intention is to have this running automatically in the background after a recording is decrypted. I've been running it like this on the auto decrypt hook, but it doesn't work in all cases. E.g., if the hdr goes into standby part way through the ad detection process then ads won't be flagged again. I've had some ideas about how to solve this but I haven't had time to implement them yet. I'll get some time on Sunday.

I think a central database is a good idea.

Sent from my HTC One using Tapatalk

 

[ChrisDaniels]

Solved my earlier problem. I had content share turned off, so nothing was decrypted.


Sent from my iPhone using Tapatalk

 

[Black Hole]

More thoughts...how about a central database for ad breaks? Once one box has done this detection process, it could record the bookmark positions to that database, taking into account padding and any local time offset, and then other boxes could automatically apply those same bookmark positions. That would help boxes which record from standby and then power off. When they power on again and decrypt the recording, the bookmarks may already be available, saving the overhead of this process.

I don't twig how you can sync up one machine's recording with the database ad positions. Where is your time reference coming from?

 

[af123]

The RS server will do. It already has the facility to provide a current timestamp.

However, it's probably simpler than that. We know the scheduled start time and the time the recording started. In an individual box, the detected bookmark positions can be converted to offsets from the scheduled start time. On any other box, the process can be reversed.

 

[Black Hole]

I think that will be susceptible to errors of up to a minute.

How can I get detectads running on HDR1? I've tried removing it and reinstalling, but "Run Analysis" just stares at me like I'm mad or something. All I get is the spinners "Analysing..." and "Still working...", not even the intro debug lines.

 

[mihaid]

I've tried it twice on the decrypted hunger games, no bookmarks done. The last message said one ad found refresh page to see bookmarks but nothing was there.

 

[Black Hole]

I had a similar result to that, but it worked when retried.

 

[Deleted member 473]

But that's not an unattended process. The goal for this is an automatic background process like decryption is now.

It would be attended for the two minutes it takes to find and cut adverts. That's possibly less than searching all those bookmarks, which, incidentally, I have never ever used.

 

[Black Hole]

Oh, you reckon you can do the whole process with only two minutes of your attention - in that case, carry on (if you have no use for detectads, butt out and leave this topic to those who do).

 

[mihaid]

I tried to do a different recording of 2 plus hours, started decrypt The T2 was dual recording and after a minute of decrypting it crashed. It's probably too taxing.

 

[prpr]

I tried to do a different recording of 2 plus hours, started decrypt The T2 was dual recording and after a minute of decrypting it crashed. It's probably too taxing.

It's possibly out of memory. Do you have swapper installed?

 

[af123]

I think that will be susceptible to errors of up to a minute.

The RS server can act as a time source to reduce that to a second or two (depending on network latency).
Try rs time from the CLI.

How can I get detectads running on HDR1? I've tried removing it and reinstalling, but "Run Analysis" just stares at me like I'm mad or something. All I get is the spinners "Analysing..." and "Still working...", not even the intro debug lines.

From telnet:

humax# cd /mod/webif/plugin/detectads
humax# ./test "/media/My Video/A Recording File.ts"

 

[af123]

I tried to do a different recording of 2 plus hours, started decrypt The T2 was dual recording and after a minute of decrypting it crashed. It's probably too taxing.

That's just dual recording + decrypting one file? Should be well within the capabilities of the box.
Crashed as in said Crash on the front panel?

 

[Deleted member 473]

Oh, you reckon you can do the whole process with only two minutes of your attention - in that case, carry on (if you have no use for detectads, butt out and leave this topic to those who do).

Now! Now!

It doesn't take long to manually detect the ad breaks, start and end, of a recording on a fast computer. A film will have maybe 5 or 6 ad breaks, fairly evenly spaced to start, but thinning towards the end. Often the breaks are about 4 minutes long. I haven't ever missed one yet. Perhaps two minutes was ambitious, but I don't see there being much saving in time from sticking bookmarks in if they have to be verified.

Now, if the whole process of detecting adverts and removing them were automated on the Humax, that would be something. I just question the reliability of it. I have bought video editors that purport to do this and they make a total balls up of it, so I can't see how it could be easy.

Most of my recordings are on Freesat, anyway, so it won't affect me either way!