Possible extension to newk?

[Trev]

But not all my recordings had the prefix New_
How on earth do you get that date from the number? Why is it put there and is there a token that I can use to sweep it away? Because looking in the wiki %timestamp is the start time and %etimestamp is the end time (as you are fully aware)

 

[raydon]

How on earth do you get that date from the number?

It's a Unix time, A.K.A. Epoch format number, which equates to the number of seconds elapsed since 00:00:00 hrs on 1st January 1970 (not counting leap seconds).
Plug your number into this online converter to extract the date/time.

Why is it put there... ?

It's the timestamp of when a recording was "shrunk".

 

[Trev]

OK, I now understand what it is and what puts it there, but why is it put there and why isn't it in the format yyyymmdd hhmmss?

 

[Black Hole]

No particular reason for it to be. It's just a means for the CF to differentiate between different versions of the same recording, and doesn't need to be human readable. It would take more work to make it human readable.

 

[hairy_mutley]

I just made a rule to remove the New_ prefix on the filenames and much to my surprise it worked.

Since I don't have the time at the present to experiment... care to share the rule (just the raw text would be great).
unless af123's update is imminent.

Tried this
# remove New_ prefix
filename {New_ *} action {renamefile {%orig%replace,New_ ,,}}
# remove New_ prefix
folder filename {New_ *} action {renamefile {%orig%replace,New_ ,,}}

but it doesn't seem to do nested folders (I suppose that would have been OK, if they had been run before my SeriesFiler sweeper rule was run).
It seems that the New: rule suffers with the same problem.

 

[Black Hole]

sweeper only applies to the folder the rule set is in, which is My Video by default.

 

[Trev]

# Remove New: prefix (from title)
title New: action {settitle {%orig%replace,New: ,,}}

# filenam clean (Remove New_ form filename)
filename New_ action {renamefile {%orig%replace,New_ ,,}}

# File Datestamp DD MMM YY (Make date easier to read in filename)
action {renamefile {%title %2digitdate-%2digitmonth-%2digityear %hhmm}}

All of a sudden, I'm an expert, handing out advice willy nilly.
But take note of What BH just said. I had to find out the hard way.

 

[hairy_mutley]

OK, that is do-able... just apply the rules at the appropriate folder level as a one off... OK all now renamed (filename and title).
For future recordings, they are now in my top level rule set BEFORE the SeriesFiler rule. Think that will work.

Just a few observations that I made trying to set this up.
It is obviously easier to move the rules around by cutting and pasting the file content, so I used the Text Editor button on the Sweeper page.
If you use the text Editor Button on a directory with no rules, it tells you the file doesn't exist. I had to create one rule and save it first.
Having got my dummy rule I could select test edit and replace the dummy rule with my pasted rules. Unfortunately, in this mode, it is not possible to test the rules, you have to save them before you can switch back to the interactive page to get to the Test config button.
Just to make sure that they wouldn't run, I commented them all out, saved them, switched back to the interactive page, enabled them and tested them.

I think that it would be desirable
to be able to Text Edit even if there is no rules file present.
to have a Test config button and maybe also a Run now button on the text editor page.

Shame you cant run sweeper on a USB drive as well (interactively from the sweeper page, not automatically).

Still... I am not complaining, great to have this level of functionality.

 

[hairy_mutley]

All of a sudden, I'm an expert, handing out advice willy nilly.
But take note of What BH just said. I had to find out the hard way.

Thanks for that Trev.
We are overlapping posts, I had noted BH's comment (my "OK, that is do-able" was in reference to it).

my top level rules now look like this...

# remove New_ prefix
filename {New_ *} action {renamefile {%orig%replace,New_ ,,}}
# Remove New: prefix.
title New:* action {settitle {%orig%replace,New: ,,}}
# remove New_ prefix (in folder)
folder filename {New_ *} action {renamefile {%orig%replace,New_ ,,}}
# Remove New: prefix (in folder)
folder title New:* action {settitle {%orig%replace,New: ,,}}
# Emulate Seriesfiler
folder series "" action {fileunder ""}

I will put the datestamp rule on file should I need it in the future.

 

[af123]

I've updated sweeper with a new %regsub token which works like %replace except that the first argument can be a regular expression (with some common extensions such as \s for a space character) and the second argument can include back-references to patterns matched by the first. (Don't worry if that doesn't mean a lot to you!)

It means that the New: prefix removal rules should now be written as follows.

title New:* action {settitle %orig%regsub,New:\s*,,}
folder title New:* action {settitle %orig%regsub,New:\s*,,}

and Trev, you can remove the leading spaces from your filenames with:

filename { *} action {renamefile {%orig%regsub,^\s+,,}}

 

[af123]

I think that it would be desirable
to be able to Text Edit even if there is no rules file present.

This one is fixed in the latest version of webif.

to have a Test config button and maybe also a Run now button on the text editor page.

Could be slightly harder - the file editor is part of the main web interface so I'd need to extend the API to allow sweeper to influence the buttons on that page... might not be practical.

 

[hairy_mutley]

Thanks for the first.
As for the second, maybe the benefits don't really justify the effort...
I'm just happy to have been able to automate the removal of New from title and filename so simply.

 

[af123]

sweeper only applies to the folder the rule set is in, which is My Video by default.

One thing that is worth revisiting is the way that sweeper folder rules work and a little history.
When it was first written, sweeper only managed files left in the top-level directory as a result of one-off recordings. It was conceived as a method to sweep them away into folders.
It was then extended to handle series recordings - at the time primarily to replace seriesfiler - but when it finds a folder it only looks at one randomly selected recording within it when checking if the conditions match*. If they match, then the action is applied to all recordings within.
The last major update allowed for sweeper rules to be applied to arbitrary folders and not just My Video.

* - maybe it should be looking at the most recent recording in there?

In case it helps with any ideas, my current live sweeper configuration looks like this:

# Remove New: prefix.
title {New:*} action {settitle {%orig%regsub,New:\s*,,}}
folder title {New:*} action {settitle {%orig%regsub,New:\s*,,}}
# Lock recordings from Spike
lcn 31 action lock
# Lock recordings from Spike (folders)
folder lcn 31 action lock
# Move any Children's films
lcn {>= 120} lcn {<= 129} or {duration {>= 90} genre Film } action {move Children/Films}
# Move anything recorded from a children's channel to the Miscellaneous folder
lcn {>= 120} lcn {<= 129} action {move Children/Miscellaneous}
# Move Children's series folders to Children
folder series "" lcn {>= 120} lcn {<= 129} action {fileundercreate Children}
title {Formula 1} action {move F1}
title JAG action {movecreate JAG}
folder title {Ninja Turtles} action {fileundercreate Children}
folder title {Thunderbirds are go} action {fileundercreate Children}
or {title Sex title Minchin } action lock
age {> 120} or {genre Film duration {>= 90} } action {movecreate _Films}
age {> 120} action {movecreate Misc}
folder or {foldername War title War title Nazi } action {fileundercreate War}

 

[hairy_mutley]

There is something a bit odd about the handling of the \s in the regsub.
It seems to be present on the text edit page, but the \ seems to vanish from the interactive page; both in the interactive rule and its raw text equivalent.

 

[cdmackay]

excellent updates, thanks...

In case it helps with any ideas, my current live sweeper configuration looks like this:

# Lock recordings from Spike
lcn 31 action lock
# Lock recordings from Spike (folders)
folder lcn 31 action lock

I know this ought to be in the sweeper thread, but the above reminded me: would you please consider some sort of "global" flag to allow the creation of rules that shoudl apply equally to folders, and to standalone files, so that the repetition above isn't required?

Almost all my sweeper rules are duplicated currently, for this reason (which is itself because the broadcasters seem unable to do a tidy job with series linking).

thanks...

 

[Trev]

Right then, I have a load of stuff like this in my auto.log.
1244 21/06/2015 09:45:05 - ... ERROR Target already exists
1243 21/06/2015 09:45:05 - Renaming /media/My Video/Dogs_ Their Secret Lives/Dogs_ Their Secret Lives 18-06-15 1958.ts to Dogs_ Their Secret Lives 18-06-15 1958

I assume its from my sweeper rule: action {renamefile {%title %2digitdate-%2digitmonth-%2digityear %hhmm}}
Can I stop this rule running every time sweeper kicks in without deleting it? Or better still, put a condition in?
(It's to convert date time like 20150322_2200 to something more sensible.)
Could I use something like {pseudo code} IF filename contains YYYYMMDDD-HHMM THEN?

 

[Black Hole]

I assume its from my sweeper rule: action {renamefile {%title %2digitdate-%2digitmonth-%2digityear %hhmm}}

It is. You can't rename it to something which already exists, and if it's been processed already it does already exist.

Can I stop this rule running every time sweeper kicks in without deleting it? Or better still, put a condition in?
(It's to convert date time like 20150322_2200 to something more sensible.)
Could I use something like {pseudo code} IF filename contains YYYYMMDDD-HHMM THEN?

It's more trouble than it's worth. If there's a flag you don't need, for example if you never use the Genre details, you could use that as an indication that the file has already been processed and test it before processing again. Alternatively assemble all the files that have been processed into a specific folder after processing.

 

[af123]

Could I use something like {pseudo code} IF filename contains YYYYMMDDD-HHMM THEN?

You could explicitly look for the right pattern of numbers - this should be enough (for the next 85 years at least!):

filename {*20[0-9][0-9][0-9][0-9][0-9][0-9]-*}

or in the GUI - filename matches *20[0-9]....etc.*

You could make it simpler if you want - *_20[0-9][0-9][0-9]* should be enough to match - filenames look something like New_ Brooklyn Nine-Nine_20150528_2059

Edit: I've updated the sweeper documentation on the Wiki with more details on string matching.

http://wiki.hummy.tv/wiki/Sweeper#String_matching

 

[Black Hole]

That could break if the system time string happened to contain "20xx" as a sub-string.

 

[af123]

That could break if the system time string happened to contain "20xx" as a sub-string.

Good point - updated in-place.