[sweeper] Custom rules to manage recordings

[vibez]

Thanks again, It's working perfectly

 

[hairy_mutley]

I seem to have some missing icons on the sweeper page (Firefox shows the broken image symbol). I presume that the problem is fairly recent as Duplicate Rule and Edit Condition were show broken, so I did a page refresh to see if that would help and the Delete Rule icon has now gone as well!

 

[af123]

I seem to have some missing icons on the sweeper page (Firefox shows the broken image symbol). I presume that the problem is fairly recent as Duplicate Rule and Edit Condition were show broken, so I did a page refresh to see if that would help and the Delete Rule icon has now gone as well!

That's due to the web interface update. I have a new sweeper package almost ready which will fix that.

 

[af123]

I will be releasing a new sweeper package shortly which changes how a global rule works. I think that the new behaviour is actually what most people already expect it to do so hopefully there will be no surprises.

To recap the current behaviour, there are three rule types:

• normal - runs against all recordings within the folder;
• folder - runs against all sub-folders within the folder. In order to determine if a folder matches, sweeper looks inside the sub-folder and selects the newest recording which is not in-use. If that recording matches the criteria then the folder is deemed to have matched and the actions are performed. Folder actions apply to the whole folder rather than individual items within. There are extra criteria and actions for folder rules;
  
• global - does both of the above but only standard criteria and actions are available.

The original idea of global was to save having to duplicate rules after many people reported that they ended up putting the same rule in twice, once with and once without the folder prefix.

The new sweeper package will deprecate the global rule type and add a new one, recurse:

• recurse <N> - runs against all recordings within the folder and all recordings <N> levels deep within sub-folders and their descendants.

global will continue to be accepted as an alias for recurse 1; that means that global rules will now be tested and applied individually to all recordings within the folder and all recordings within direct sub-folders. As I wrote above, I think a lot of people already think that they work like this.

(You may have spotted that recurse 0 has the same behaviour as a normal rule, and in fact that's how normal rules are now implemented internally, but the web interface continues to make a distinction between the two types).

 

[Black Hole]

So now every recording is tested individually before being processed, even in recursed folders (as opposed to if the first one tested passes they all pass)?

 

[af123]

So now every recording is tested individually before being processed, even in recursed folders (as opposed to if the first one tested passes they all pass)?

Yes, and the rule is applied to recordings one by one rather than to the folder as a whole.

 

[peterworks]

It is probably my lack of understanding but...
A couple of days ago I entered a new sweeper rule whereby A Place in the Sun is moved to a folder of the same name. I saved it and all looked okay.
However looking in the auto.log I noticed that the file was trying to be moved on each run but, as it had already been moved, it failed.


Looking at the text file in sweeper I saw that it had put 'Recurse 1' in which explained why I was getting the output in auto.log.
I subsequently used the text editor to remove 'Recurse 1' and created a new test rule - 'Recurse 1' was automatically added.

I removed the test rule and the auto.log shows that all is now working correctly.
Looking at the post above it states there are 3 rules - normal, folder and global. How do I know which is which ?
Also, from reading the post above, I presume when I set up a new rule it is global - how can I change it to normal ?

Thanks and sorry if I have misunderstood the workings of sweeper...

 

[af123]

That seems to be a bug. When you create a rule you get a choice of three options - the first one is adding recurse 1 when it should not be.

 

[af123]

Fixed in version 2.1.1, now available.

 

[MontysEvilTwin]

I want a rule to delete the timestamp from the filenames of films. I set up the following:

# Remove Film Timestamp
genre Film filename *_********_**** action {renamefile {%orig%regsub,_********_****\s,,}}

This succesfully matches films with a timestamp in the filename but it fails to rename them.

28/06/2016 18:52 - + Sweeper processing /media/My Video/Departures_20160621_0207.ts
28/06/2016 18:52 - Processing [genre Film filename *_********_**** action {renamefile {%orig%regsub,_********_****\s,,}}]
28/06/2016 18:52 -    genre(Film)
28/06/2016 18:52 -      MATCH
28/06/2016 18:52 -    filename(*_********_****)
28/06/2016 18:52 -      MATCH
28/06/2016 18:52 -    action(renamefile {%orig%regsub,_********_****\s,,})
28/06/2016 18:52 - ACTION: renamefile(%orig%regsub,_********_****\s,,) [0]
28/06/2016 18:52 -   Expanded [%orig%regsub,_********_****\s,,] -> [Departures_20160621_0207]
28/06/2016 18:52 -    - Calling expand_regsub({_********_****\s} {})
28/06/2016 18:52 - Error. %regsub - couldn't compile regular expression pattern: nested count
28/06/2016 18:52 -      Result: (Departures_20160621_0207)
28/06/2016 18:52 - Renaming /media/My Video/Departures_20160621_0207.ts to Departures_20160621_0207
28/06/2016 18:52 - ... ERROR Target already exists

What am I doing wrong? Also, sometimes films are recorded in two parts (e.g. when there is a news bulletin in the middle). To cope with this, it would be useful to append a number (e.g. '_2') to the end of the target filename if a file of the same name already exists.

 

[af123]

I want a rule to delete the timestamp from the filenames of films. I set up the following:

# Remove Film Timestamp
genre Film filename *_********_**** action {renamefile {%orig%regsub,_********_****\s,,}}

...
What am I doing wrong?

The regsub function expects a regular expression rather than a wildcard pattern. In a regular expression a . represents any character so you could replace the *s with .s, however this is a bit neater - \d{8} means 8 digits.

genre Film filename *_????????_???? action {renamefile {%orig%regsub,_\d{8}_\d{4},,}}

Also, sometimes films are recorded in two parts (e.g. when there is a news bulletin in the middle). To cope with this, it would be useful to append a number (e.g. '_2') to the end of the target filename if a file of the same name already exists.

For handling up to four files with the same prefix, this will work. There isn't currently a way of handling an arbitrary number...

genre Film filename *_????????_???? action {set {newname=%basename%regsub,_\d{8}_\d{4},,}}
varset newname fileexists %%newname.ts action {set {newname=%%newname_2}}
varset newname fileexists %%newname.ts action {set {newname=%%newname%regsub,_\d$,_3,}}
varset newname fileexists %%newname.ts action {set {newname=%%newname%regsub,_\d$,_4,}}
varset newname action {renamefile %%newname}

 

[MontysEvilTwin]

Thanks very much. I can't get the second part to work though. I pasted in the section to sweeper.conf and it presents each line with 'varset' as a discrete unnamed rule. If I remove the carriage returns and leave one space between each line, then sweeper ignores them.

 

[af123]

Thanks very much. I can't get the second part to work though. I pasted in the section to sweeper.conf and it presents each line with 'varset' as a discrete unnamed rule. If I remove the carriage returns and leave one space between each line, then sweeper ignores them.

Strange - works for me. A slightly more optimised version is this:

genre Film filename *_????????_???? action {set {newname=%basename%regsub,_\d{8}_\d{4},,}}
lastrule "" fileexists %%newname.ts action {set {newname=%%newname_2}}
lastrule "" fileexists %%newname.ts action {set {newname=%%newname%regsub,_\d$,_3,}}
lastrule "" fileexists %%newname.ts action {set {newname=%%newname%regsub,_\d$,_4,}}
varset newname action {renamefile %%newname}

That's four separate rules.

I'll update sweeper tonight with a couple of new features and then you will be able to use the following single rule to achieve this. The new in-line regsub makes for them more readable too (IMHO).

genre Film filename ~_\\d{8}_\\d{4}$ action {renamefile %asuniqfilename/%regsub:%orig:_\\d{8}_\\d{4}::/}}

Example:

humax# ls -1
The Boat that Rocked_20160601_1234.hmt
The Boat that Rocked_20160601_1234.nts
The Boat that Rocked_20160601_1234.ts
The Boat that Rocked_20160601_2345.hmt
The Boat that Rocked_20160601_2345.nts
The Boat that Rocked_20160601_2345.ts
The Boat that Rocked_20160601_3456.hmt
The Boat that Rocked_20160601_3456.nts
The Boat that Rocked_20160601_3456.ts

humax# cat .sweeper
genre Film filename ~_\\d{8}_\\d{4}$ action {renamefile %asuniqfilename/%regsub:%orig:_\\d{8}_\\d{4}::/}}

humax# /mod/webif/plugin/sweeper/test `pwd`

humax# ls -1
The Boat that Rocked.hmt
The Boat that Rocked.nts
The Boat that Rocked.ts
The Boat that Rocked_2.hmt
The Boat that Rocked_2.nts
The Boat that Rocked_2.ts
The Boat that Rocked_3.hmt
The Boat that Rocked_3.nts
The Boat that Rocked_3.ts

 

[MontysEvilTwin]

Strange - works for me. A slightly more optimised version is this:

genre Film filename *_????????_???? action {set {newname=%basename%regsub,_\d{8}_\d{4},,}}
lastrule "" fileexists %%newname.ts action {set {newname=%%newname_2}}
lastrule "" fileexists %%newname.ts action {set {newname=%%newname%regsub,_\d$,_3,}}
lastrule "" fileexists %%newname.ts action {set {newname=%%newname%regsub,_\d$,_4,}}
varset newname action {renamefile %%newname}

That's four separate rules.

I'll update sweeper tonight with a couple of new features and then you will be able to use the following single rule to achieve this. The new in-line regsub makes for them more readable too (IMHO).

genre Film filename ~_\\d{8}_\\d{4}$ action {renamefile %asuniqfilename/%regsub:%orig:_\\d{8}_\\d{4}::/}}

Example:

humax# ls -1
The Boat that Rocked_20160601_1234.hmt
The Boat that Rocked_20160601_1234.nts
The Boat that Rocked_20160601_1234.ts
The Boat that Rocked_20160601_2345.hmt
The Boat that Rocked_20160601_2345.nts
The Boat that Rocked_20160601_2345.ts
The Boat that Rocked_20160601_3456.hmt
The Boat that Rocked_20160601_3456.nts
The Boat that Rocked_20160601_3456.ts

humax# cat .sweeper
genre Film filename ~_\\d{8}_\\d{4}$ action {renamefile %asuniqfilename/%regsub:%orig:_\\d{8}_\\d{4}::/}}

humax# /mod/webif/plugin/sweeper/test `pwd`

humax# ls -1
The Boat that Rocked.hmt
The Boat that Rocked.nts
The Boat that Rocked.ts
The Boat that Rocked_2.hmt
The Boat that Rocked_2.nts
The Boat that Rocked_2.ts
The Boat that Rocked_3.hmt
The Boat that Rocked_3.nts
The Boat that Rocked_3.ts

Thanks. The new one line rule is working fine with the latest version of Sweeper. I only used the 'Test config' option with the previous version of the rule, I did not actually run the rule. With the latest version too, in test mode, the log does not show that the '_2' suffix is added when you have two files with the same base filename, it just shows that the timestamp has been removed. It does show that the suffix is added if you actually run the rule.

 

[MymsMan]

I don't need/want it myself but would it be (theoretically) possible to create a sweeper rule to use nicesplice to concatenate the subsequent file segments to the first to produce a single recording

 

[ian_j]

.

 

[Border]

Any idea what these error entries mean ?

4952        01/09/2016 22:03:15 - Unknown sweeper rule clause '::sweeper::,�h��R<0�h��R<0�h�*S�0�h�
4951        01/09/2016 22:03:15 - Unknown sweeper rule clause '::sweeper::(�*HsR

50        29/08/2016 22:17:41 -   @ /mod/webif/plugin/sweeper/auto.hook::205
49        29/08/2016 22:17:41 - Plugin error: expected integer but got ""

 

[digitl]

I have set up rules to remove 'New' from recordings but cannot find a way to remove it from the folders containing those recordings. Is it possible to do so? I've read my way through the previous 10 pages and have failed to find a reference.

 

[MontysEvilTwin]

I have set up rules to remove 'New' from recordings but cannot find a way to remove it from the folders containing those recordings. Is it possible to do so? I've read my way through the previous 10 pages and have failed to find a reference.

I don't think you can edit existing folder names with Sweeper, though I stand to be corrected on this. The Newk package is your best bet as it removes such prefixes from folder names while the recordings are in the schedule, though it does not change the name of existing folders. Packages like Newk haven't been integrated with real time scheduling yet, so if you use RTS to set up a recording you either have to then edit the folder name in the schedule yourself or let Newk do it, which will require a reboot between scheduling and the recording start.

 

[digitl]

Thanks. To keep it simple for the moment I'll edit the folder names manually.