[sweeper] Custom rules to manage recordings

[MymsMan]

And I guess it's impossible to change the recurse operation from the GUI without either resorting to the text editor or deleting and recreating the rule?

You cant add (AFAIK) a new recurse option to an existing rule easily but can edit an existing option

Changing to 0 stops recursion

 

[prpr]

Technically the Queue is not part of Sweeper, Queue is part of auto-processing.

You are of course correct. My error. Perhaps a mod. could split the relevant messages into a separate thread "Queue processing" or similar?

I have never really considered the order entries on the queue are processed, usually you would only have a couple of recordings at a time on the queue and if you are submitting a batch does it really matter what order they are processed provided they are all, eventually, processed.

Technically it's not a queue then. There'd be a riot if this happened at the checkouts. It confuses my sense of expectation at least. It probably doesn't matter, but it's not "right".

 

[prpr]

Changing to 0 stops recursion

No it doesn't. It's error trapped and prevented from being changed, as mentioned previously (<1 test).

 

[af123]

I looked at the code in /mod/webif/lib/auto/deq and didn't really understand it (proc ::auto:ending and proc ::auto::runplugins I guess).

Something like this would probably do it (I'm not in a position to test it at the moment):

@@ -155,7 +155,10 @@ proc ::auto::pending {} {
                set ba [$b get action]
                if {$aa ni $::auto::plugins} { return 0 }
                if {$ba ni $::auto::plugins} { return 0 }
-               return $($::auto::plugins($aa) - $::auto::plugins($ba))
+               set ap $::auto::plugins($aa)
+               set bp $::auto::plugins($ba)
+               if {$ap != $bp} { return $($ap - $bp) }
+               return $([$b get id] - [$a get id])
        }] [queue pending]]
}

 

[af123]

And I guess it's impossible to change the recurse operation from the GUI without either resorting to the text editor or deleting and recreating the rule?

Give sweeper beta 2.2.0 a try - I added the option to change rules between recursive and non-recursive by editing the recursion level to/from zero. I also fixed the other things you suggested regarding the rule name and giving the browser hints about the limits for the fields (although it's still down to the browser to honour them in the dialogue).

 

[prpr]

Something like this would probably do it (I'm not in a position to test it at the moment)

Yes, that looks plausible! It also works fine. Thanks.

 

[prpr]

I added the option to change rules between recursive and non-recursive by editing the recursion level to/from zero. I also fixed the other things you suggested regarding the rule name and giving the browser hints about the limits for the fields (although it's still down to the browser to honour them in the dialogue)

That all works too, with the exception of the 16 upper limit on the Edit recursion depth dialog. It should be 15 to match the "< 16" test I mentioned, or change the test to "<= 16".
And the test in auto.hook proc ::sweeper::apply_rules should probably be changed to 16 (or 15 depending on the above) to match.

 

[/df]

...
Technically [the auto "queue" is] not a queue then. There'd be a riot if this happened at the checkouts. It confuses my sense of expectation at least. It probably doesn't matter, but it's not "right".

I think the pending queue is a priority queue. Airline boarding is a better analogy than the supermarket!

# dequeue pseudo-code
while sorted pending queue [::auto::pending] is not empty
    process head item and remove
endwhile

Separately, there's priority discrimination in enqueuing auto-processable items during the auto-processing scan, to address the logical dependencies between various action types for the same recording: see /mod/webif/lib/auto/NOTES.

The pending priority queue is implemented as a list which is sorted on each dequeue operation. But the list is actually created by a sorted retrieval from the queue DB on each dequeue operation, leading one to wonder whether the dequeue priority could have been computed and stored when the item was enqueued, so that the retrieval could present an already sorted list, or even just select and return the top item.

 

[Code is a Hole]

Hello all, having been in to a coding corner, I've experimented with sweeper, first attempts years ago, well I gave up, tried a few weeks ago, what a time saver, but

I like watching 'fixing it' type videos on YouTube, but some of the videos are very long, watching on a laptop is awkward and tiresome, so downloaded (or is it installed) qtube and youtube-dl, now I have lots of .mp4 videos at the root level of 'My Video' and sweeper does not see them, so the files don't get swept away

Would it be possible to add the ability for sweeper to process youtube-dl files please

Sorry if this has been sorted already, but I tried doing a thread search for .mp4, bust the search complained .mp4 was too short a search request

 

[MymsMan]

Would it be possible to add the ability for sweeper to process youtube-dl files please

Rather than being specific to mp4 it would be quite useful to have a new rule type 'Process File Type" with the option to specify which file type and the recursion depth.
For most file types only the basic attributes (name, size, creation time, last ref time) would be available but it would be nice if for media files the meta attributes (artist, album etc) could be made enquirable and settable.

 

[Black Hole]

I would like the means to apply any arbitrary command via sweeper rules and also directly onto the auto-process queue. It might be possible to use a facility like this to implement the above.

 

[Code is a Hole]

You cant add (AFAIK) a new recurse option to an existing rule easily

For each recording :

If:	Synopsis contains	S6, ep1
And:	Recording Flagged as	Shrunk

Where it says [ For each recording ], click on the little note pad and pen, is offers a recursive number entry option, is that what you need

 

[MymsMan]

I would like the means to apply any arbitrary command via sweeper rules and also directly onto the auto-process queue. It might be possible to use a facility like this to implement the above.

Definitely.

Move/Rename is currently implemented inline during the auto scan phase since they are very fast on the same filesystem but Copy and Move to another device is slow so it would be much better if such operations could be queued.

 

[MymsMan]

For each recording :

If:	Synopsis contains	S6, ep1
And:	Recording Flagged as	Shrunk

Where it says [ For each recording ], click on the little note pad and pen, is offers a recursive number entry option, is that what you need

I think rather than a text cut and paste you needed a image snip (Shift+WindowsKey+S on a Windows system)

But since that post I had realised how to edit the recursion level but thanks for replying anyway

 

[Code is a Hole]

I sometimes get remote scheduling matches were a programme is repeated at the weekend, is there a way for sweeper to recognise a recording date as being a weekday or a weekend or even a particular day of the week

For example, some programmes get repeated at a different time on a different day, but the times may change

As it is easy to work out what day of the week it is when creating a spreadsheet based on just a date, could that ease of use be migrated to sweeper

 

[Black Hole]

You seem to want to do things in a very complicated way, ie in a way that maybe has an audience of one. Do you really have to categorise recordings according to what day of the week they were broadcast???

For example, some programmes get repeated at a different time on a different day, but the times may change

So what? The standard functions implemented without CF use the CRID codes transmitted with the programme to identify whether a transmission is a repeat of a programme already recorded (and not record a repeat unless the original recording failed). If you are trying to compensate for broadcasters cocking up that system, either you are altogether too keen on telly, or you have to record every broadcast and then sort it out afterwards, but any lengths you might go to sort them out by predefined rules are bound to come unstuck at some point.

 

[Code is a Hole]

either you are altogether too keen on telly,

Not quite that bad, more a case of Remote Scheduling finding entries you hadn't thought of being seen as a candidate to be recorded

A search in RS finds based on searching a title or synopsis rather than CRID, so attempt to record whatever matches the search criteria

There are occasions when there is programming overlap, which makes matters worse if you have five minutes of padding at the start and end of a recording, so getting a second bite helps when your first recording attempt ends before the programme actually finishes due to over-running

Not sure what you have against being too keen on telly, I just don't like recording a programme, then find the last minute or two has been missed

I have used sweeper to sort my recordings in to folders as so

Morning
S1
S2
S3
Afternoon
S1
S2
Evening

If the same programme is being or has been broadcast on different channels, then each channel also gets its own folder, as do each season, if such info is in the synopsis, yes I have been learning to use sweeper

 

[/df]

...Move/Rename is currently implemented inline during the auto scan phase since they are very fast on the same filesystem but Copy and Move to another device is slow so it would be much better if such operations could be queued.

A possible fly in the ointment. As discussed earlier in the thread, queued operations are run in a different order from that in which they were enqueued. Might it be possible for sweeper to queue, say, a move/copy for a recording and a decryption for the destination of that move/copy, and then for the decryption to fail by being run before the moved/copied recording has appeared? Or perhaps the existing dependency logic can be extended?

 

[Code is a Hole]

would a wait until the file has been shrunk be a more beneficial option, as the file being copied is smaller

 

[MymsMan]

A possible fly in the ointment. As discussed earlier in the thread, queued operations are run in a different order from that in which they were enqueued. Might it be possible for sweeper to queue, say, a move/copy for a recording and a decryption for the destination of that move/copy, and then for the decryption to fail by being run before the moved/copied recording has appeared? Or perhaps the existing dependency logic can be extended?

I don't think so - Items are enqued based on where they currently are not where they will be, Sweeper actions that move/rename a file stop further rule processing on that file
The queue priority scheme was designed to minimize the chances of a file disappearing between enquing and processing but there already some circumstances (detectads) where it can happen.

would a wait until the file has been shrunk be a more beneficial option, as the file being copied is smaller

That would be one of the advantages of having Copy and Move as queued actions - the priority could be set so that those actions happened last after shrinking and decryption etc had completed

But it is still better to be explicit about the requirements in the sweeper rules rather than rely on the priority
eg a rule to only archive shrunk and decrypted recordings might (in the future) be

# Copy to archive
flag Shrunk !flag ODEncrypted action {queue {copy /media/[NAS-Video]}}