[sweeper] Custom rules to manage recordings

[af123]

I haven't been able to make successful sweeper rules using the 'scheduled duration' criteria.

'Recording duration' seems to work as a criteria but not 'scheduled duration'.

Sounds like a bug. I'll look later.

 

[af123]

I've released sweeper 2.0.6 - changes below.

• Fix bug with the scheduled duration condition (it was using seconds instead of minutes);
• New condition series for folder rules which detects if the folder was created automatically due to a series recording;
• Folder rules now run against all sub-folders of the folder being processed. Previously they were only run against folders which were created automatically due to a series recording (which is what sweeper was originally created to handle). All of your existing folder rules will automatically be updated to include the new series condition so that the behaviour of your rules is unchanged by this update;
• The GUI shows more explanatory text in folder rules - trying to make the current behaviour more obvious;
• Conditions which previously checked for a substring can now also be set up to use pattern matching by including a * in them. For example, to check for medialist titles which start with New: you can use New:* as the condition argument. Sweeper web interface has online help for this when editing a condition.

With the change to folder processing, sweeper possibly becomes slightly more useful. I'm still working out how to extend it for some of the other things that have been requested.

Folder rules still work in the same way in that a single sample recording from the folder is used for matching and then the actions refer to the folder itself. This is because the original purpose was for automatically filing folders created for series recordings.

 

[af123]

After adding the folder rule I waited for about 2 hours.

Shrink & Decrypt worked on 1 file but neither file was renamed.

Try increasing the debug level for automatic processing. That might show what's going wrong.

Is there any way to automatically delete the empty folders once the files have been moved to the main Radio Recordings folder?

Sweeper should do that automatically for you when it moves the recordings across, assuming the folder is then empty.

 

[rowanmoor]

I'm not sure if this could be possible with sweeper or not. One thing I miss on the Toppy is the ability to see how long everything is at a glance. So when we are deciding what to watch a quick glance down the list shows us how long everything is so we can see what will fit in the time we have. Because everything is filed in series folders on the Humax, I have to go into the folder to see how long and what channel (advert breaks or not) the programs are.

I'm reluctant to use flatten to put everything in the root because you only see a few items per page (nowhere near as many as the toppy) so we would end up with 10s of pages to scroll through.

I have wondered if this can be solved by renaming the series folders to include the channel and recording length. If you rename a series folder, then do new recordings of that series go into the same folder or is the original name re-created? I'm guessing the latter and newk does something really clever to get round that. Could sweeper rename a folder? Can it delete an empty folder (posts above suggest not)?

 

[ian_j]

Tried again with updated Sweeper and the process very nearly works 100%.

I created 2 dummy recordings that finished at around 11:20 this morning.

The files moved from their respective folders to Radio Recordings\ (folder created using Sweeper) but the original folders have not been deleted.

The files then renamed fine (Sweeper rule added to Radio Recordings\ folder using right click menu after it was created)

I then added decrypt & shrink to the folder both of which have now run, decrypt and shrink did take nearly an hour to execute (they have only just run while I was typing this and after I took the following screen grab of my auto log.

I'm pasting the log here as there are some errors and I wondered if these relate to the original radio recording folders not being removed.

 

[ian_j]

Try increasing the debug level for automatic processing. That might show what's going wrong.


Sweeper should do that automatically for you when it moves the recordings across, assuming the folder is then empty.

How do I increase the debug level, a recording that took place this morning at 11:30 is still not shrunk.

The empty folders are not being deleted.

rpb424 how do I setup/automate running your Tidy script?

 

[cdmackay]

How do I increase the debug level

I think it's:

Settings - General Settings - Auto-processing log level

 

[rpb424]

rpb424 how do I setup/automate running your Tidy script?

May be best not to try it for the time being, since auto deleting a folder using it will also lose any sweeper configuration stored on that folder, which would need manually re-creating if and when sweeper makes that folder again in the future.

I'm working on a more sophisticated version that stores off sweeper configs before deleting a folder, and will reapply them the next time it detects that folder has appeared again.

Advancements to sweeper itself may make what I'm doing redundant anyway, but it is good bash scripting practice for me whilst I'm doing it and an interesting challenge.

 

[ian_j]

Hmmm yeah I forgot it would also delete the Radio Recordings folder which I leave after moving my recordings over to PC.

 

[sceptic]

Regarding sweeping to an external source (i.e a NAS) for archiving/additional capacity etc. I would be nice to be able to constrain the sweeping to certain time periods (between X and Y) of my choosing when I know my HDR is not going to be recording anything else. Previously when I've tried copying recordings over and the box has been dual recording I have found it can become very laggy on occasions. Perhaps a condition for the current time/hour?

 

[af123]

Perhaps a condition for the current time/hour?

Nice idea, I'll add one. I'll probably do current time as HHMM so you can do or {now {< 0800} now {> 2200}} or similar.

 

[Andy Edgeworth]

I must admit, I just set it not to do anything for 6 hours after a recording. Has the advantage that if I record anything, I can watch the same evening. If I don't watch, it will be archived off.

 

[af123]

New now condition added to version 2.0.7. Thanks @sceptic.

 

[ian_j]

af123 any reason the empty folders on my system were not being deleted after the files were moved and they were empty?

 

[af123]

af123 any reason the empty folders on my system were not being deleted after the files were moved and they were empty?

Can't think offhand, but the auto.log should say why it can't delete a folder (it reports which files are 'blocking' the deletion).

 

[ian_j]

Can't think offhand, but the auto.log should say why it can't delete a folder (it reports which files are 'blocking' the deletion).

Reply #85 has my Auto Log in it I can't make any sense of it though regarding folders.

 

[Black Hole]

Look at the lines which say "blocking file" - that means an action is being blocked by the presence of a file.

 

[ian_j]

Look at the lines which say "blocking file" - that means an action is being blocked by the presence of a file.

I checked on the WebIF and each folder was 4k. Using ftp though each folder was empty so no idea what was stopping their deletion.

 

[sceptic]

New now condition added to version 2.0.7. Thanks @sceptic.

Thanks for the fast turnaround @af123. I think this is your best package yet!

 

[cdmackay]

I'm also seeing top-level series folders not being deleted, after sweeper has moved their contents elsewhere.

I see "Failed to remove directory" in auto.log, which then lists dozens of blocking files. but the dirs in question are empty, at least when viewed via WebIF, so I don't quite understand what that's telling me?

Web interface version: 1.0.15-5 (Lighttpd)
Custom firmware version: 2.23 (build 2035)
Humax Version: 1.03.12 (kernel HDR_1.02.32)​