[sweeper] Custom rules to manage recordings

The WebIF media browser only lists files of recognised types. To be sure a folder really is empty one would have to look via a Telnet ls -a command ("list all") or get a directory listing via FTP or a network mount.
 
thanks, yes I will check in via ssh next time it happens.

Whilst I'm here, I had a question, please...

I find I am often creating two identical rulesets, one for files, and one for folders.

e.g. I want to move any prog whose title contains "Cycling" to a Cycling folder elsewhere in my hierarchy. So I create a file rule, and then another folder rule, to catch recordings that are part of a series and so have been put into a folder.

Is there any way to create a single rule that will catch both top-level non-series files and also progs in top-level series folders?

thanks much...
 
Here's an example of sweeper failing to remove an empty top-level series dir, after having filed its contents elsewhere:

Code: 
08/07/2014 01:00 - move /media/My Video/Tour De France 2014 Highlights/Tour De France 2014 Highlights_20140707_2358.ts to Calum/Sport/Cycling
08/07/2014 01:00 - moveset(/media/My Video/Tour De France 2014 Highlights/Tour De France 2014 Highlights_20140707_2358.ts) -> /media/My Video/Calum/Sport/Cycling
08/07/2014 01:00 - Failed to remove directory
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_2358.hmt
08/07/2014 01:00 - Blocking file: Tour de France 2014 Live_20140707_1159.ts
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_2358.nts
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_1859.nts
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_2358.ts
08/07/2014 01:00 - Blocking file: Tour de France 2014 Live_20140707_1159.nts
08/07/2014 01:00 - Blocking file: Tour de France 2014 Live_20140707_1159.thm
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_1859.thm
08/07/2014 01:00 - Blocking file: Tour de France 2014 Live_20140707_1159.hmt
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_1859.hmt
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_2358.thm
08/07/2014 01:00 - Blocking file: Tour De France 2014 Highlights_20140707_1859.ts
08/07/2014 01:00 - Resetting unwatched recording flag for /media/My Video/Calum/Sport/Cycling
08/07/2014 01:00 - Resetting unwatched recording flag for /media/My Video/Tour De France 2014 Highlights

Yet we can see the dir is empty; this via SSH:

Code: 
humax1# pwd
/media/My Video/Tour De France 2014 Highlights
humax1# ls -al
drwxr-xr-x    2 root    root          4096 Jul  8 01:00 .
drwxr-xr-x  10 root    root          4096 Jul  7 23:00 ..

The "blocking" files do exist, of course, but in the destination of the moveset.
 
Fixed, thanks for the detail, I was trying to remove the destination rather than the source!
 
I am having a similar problem with Sweeper. It is still doing what it should, but I am now getting error messages:
Code: 
15/07/2014 11:00 - Renaming /media/My Video/Headline News_20140715_1025.ts to Keiser Report_20140715_1025
15/07/2014 11:00 - Setting title for /media/My Video/Keiser Report_20140715_1025.ts to Keiser Report
15/07/2014 11:00 - move /media/My Video/Keiser Report_20140715_1025.ts to /media/My Video/Keiser Report
15/07/2014 11:00 - moveset(/media/My Video/Keiser Report_20140715_1025.ts) -> /media/My Video/Keiser Report
15/07/2014 11:00 - Failed to remove directory
15/07/2014 11:00 - Blocking file: Girl with a Pearl Earring.hmi
15/07/2014 11:00 - Blocking file: Comedy Fest Live_20140622_0301.nts
15/07/2014 11:00 - Blocking file: Captain America Trailer.hmi
15/07/2014 11:00 - Blocking file: Jack Taylor_ The Priest_20140528_2159.ts
15/07/2014 11:00 - Blocking file: Jack Taylor_ The Priest_20140528_2159.thm
15/07/2014 11:00 - Blocking file: Bob and Frankie's Cookery.hmi
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140604.nts
15/07/2014 11:00 - Blocking file: Line of Duty
15/07/2014 11:00 - Blocking file: Jack Taylor_ The Priest_20140528_2159.nts
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140611.hmt
15/07/2014 11:00 - Blocking file: Keiser Report
15/07/2014 11:00 - Blocking file: The Road to Rio
15/07/2014 11:00 - Blocking file: Vera
15/07/2014 11:00 - Blocking file: Black Swan.thm
15/07/2014 11:00 - Blocking file: Babylon_20140209.hmt
15/07/2014 11:00 - Blocking file: DCI Banks
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140604.hmt
15/07/2014 11:00 - Blocking file: The Sage Of Cricklewood.mp4
15/07/2014 11:00 - Blocking file: Black Swan.nts
15/07/2014 11:00 - Blocking file: The Chronicles of Narnia_ Prince Caspian.thm
15/07/2014 11:00 - Blocking file: Taken_20140419_2100.hmt
15/07/2014 11:00 - Blocking file: Jonathan Creek
15/07/2014 11:00 - Blocking file: The Night Watch_20140713_2102.hmt
15/07/2014 11:00 - Blocking file: The History Boys.ts
15/07/2014 11:00 - Blocking file: Bob and Frankie's Cookery.mp4
15/07/2014 11:00 - Blocking file: Babylon_20140209.ts
15/07/2014 11:00 - Blocking file: Stargate SG-1
15/07/2014 11:00 - Blocking file: Downfall.ts
15/07/2014 11:00 - Blocking file: Inspector De Luca
15/07/2014 11:00 - Blocking file: Comedy Fest Live_20140622_0301.hmt
15/07/2014 11:00 - Blocking file: Jack Taylor_ Shot Down_20140604_2201.nts
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140611.ts
15/07/2014 11:00 - Blocking file: The Chronicles of Narnia_ Prince Caspian.ts
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140604.ts
15/07/2014 11:00 - Blocking file: Comedy Fest Live_20140622_0301.thm
15/07/2014 11:00 - Blocking file: The Night Watch_20140713_2102.nts
15/07/2014 11:00 - Blocking file: [Deleted Items]
15/07/2014 11:00 - Blocking file: Jack Taylor_ Shot Down_20140604_2201.hmt
15/07/2014 11:00 - Blocking file: Wild China
15/07/2014 11:00 - Blocking file: Downfall.hmt
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140604.thm
15/07/2014 11:00 - Blocking file: RT Documentaries_20140713_1730.hmt
15/07/2014 11:00 - Blocking file: Jack Taylor_ The Priest_20140528_2159.hmt
15/07/2014 11:00 - Blocking file: Limitless.thm
15/07/2014 11:00 - Blocking file: The Fantastical World of Hormones with Professor John Wass.mp4
15/07/2014 11:00 - Blocking file: Limitless.hmt
15/07/2014 11:00 - Blocking file: Babylon_20140209.thm
15/07/2014 11:00 - Blocking file: Taken_20140419_2100.ts
15/07/2014 11:00 - Blocking file: Comedy Fest Live_20140622_0301.ts
15/07/2014 11:00 - Blocking file: RT Documentaries_20140713_1730.nts
15/07/2014 11:00 - Blocking file: Captain America Trailer.mp4
15/07/2014 11:00 - Blocking file: Top of the Pops_ 1979
15/07/2014 11:00 - Blocking file: The History Boys.thm
15/07/2014 11:00 - Blocking file: The Night Watch_20140713_2102.ts
15/07/2014 11:00 - Blocking file: Limitless.nts
15/07/2014 11:00 - Blocking file: House of Fools
15/07/2014 11:00 - Blocking file: Jack Taylor_ Shot Down_20140604_2201.ts
15/07/2014 11:00 - Blocking file: [ModSettings]
15/07/2014 11:00 - Blocking file: Babylon_20140209.nts
15/07/2014 11:00 - Blocking file: RT Documentaries_20140713_1730.thm
15/07/2014 11:00 - Blocking file: Downfall.thm
15/07/2014 11:00 - Blocking file: Girl with a Pearl Earring.mp4
15/07/2014 11:00 - Blocking file: Black Swan.hmt
15/07/2014 11:00 - Blocking file: RT Documentaries_20140713_1730.ts
15/07/2014 11:00 - Blocking file: Louis Theroux's LA Stories
15/07/2014 11:00 - Blocking file: Taken_20140419_2100.nts
15/07/2014 11:00 - Blocking file: Inside No 9
15/07/2014 11:00 - Blocking file: Stewart Lee's Comedy Vehicle
15/07/2014 11:00 - Blocking file: Taken_20140419_2100.thm
15/07/2014 11:00 - Blocking file: [edit]
15/07/2014 11:00 - Blocking file: Put Your Money Where Your Mouth Is 1.mp4
15/07/2014 11:00 - Blocking file: Black Swan.ts
15/07/2014 11:00 - Blocking file: Downfall.nts
15/07/2014 11:00 - Blocking file: Limitless.ts
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140611.nts
15/07/2014 11:00 - Blocking file: The Chronicles of Narnia_ Prince Caspian.hmt
15/07/2014 11:00 - Blocking file: 24 Hours to Go Broke_20140611.thm
15/07/2014 11:00 - Blocking file: W1A
15/07/2014 11:00 - Blocking file: Jack Taylor_ Shot Down_20140604_2201.thm
15/07/2014 11:00 - Blocking file: Mammon
15/07/2014 11:00 - Blocking file: The History Boys.hmt
15/07/2014 11:00 - Blocking file: Salvage Hunters
15/07/2014 11:00 - Blocking file: The Night Watch_20140713_2102.thm
15/07/2014 11:00 - Blocking file: The Chronicles of Narnia_ Prince Caspian.nts
15/07/2014 11:00 - Blocking file: The History Boys.nts
15/07/2014 11:00 - Blocking file: The Honourable Woman
15/07/2014 11:00 - Resetting unwatched recording flag for /media/My Video/Keiser Report
15/07/2014 11:00 - Resetting unwatched recording flag for /media/My Video
Sweeper seems to be behaving in such a way that once it has moved the recording from 'My Video' it is trying to delete this folder. It lists everything in 'My Video' as a blocking file in the auto log. My HDR-FOX is set to auto-update packages, so everything should be up to date.
 
MontysEvilTwin said:
Sweeper seems to be behaving in such a way that once it has moved the recording from 'My Video' it is trying to delete this folder.
Click to expand...

Something changed recently regarding Sweeper moving items and deleting folders ... I think it was to satisfy those who wanted folders to disappear from 'My Video' once the contents had been moved (similar to the seriesfiler process)

But trying to delete the 'My Video' folder itself is presumably an unintended consequence and this also seems to have brought other troublesome consequences: e.g. I had a sub-folder set up with Sweeper rules (to rename and then move contents to elsewhere) which functioned exactly as intended for several weeks, but suddenly it began deleting itself once the contents were moved, which meant that the Sweeper rules were lost each time.

Rules were:
# Re-name recording files with date and time at front
!title 201 action {renamefile {%yyyymmdd_%hhmm_%title_%shortday %date %shortmonth %year}}
# Re-name Media List title with day and date at end
!title 201 action {settitle {%title_%shortday %date %shortmonth %year}}
# Move decrypted and shrunk files to transfer folder
flag Shrunk now {> 0300} now {< 0500} action {movecreate {04_Transfer Folder}}

Used to work fine ... but now the .sweeper file has gone all hara-kiri and deletes itself.

Somehow needed a rule to tell it 'Do move files' but 'Don't delete folder'.

Hence, after re-setting up the rules for the third time, I resorted to putting a small 'blocking' file in the folder (named 'Do Not Delete' :)) which doesn't get moved, so the folder isn't deleted, and the .sweeper file is preserved - but this is a workaround - and I preferred the original functionality.

So last line is now:
# Move decrypted and shrunk files to transfer folder
flag Shrunk now {> 0300} now {< 0500} !title {Do Not Delete} action {movecreate {04_Transfer Folder}}

Maybe the source folder's deletion/non-deletion action needs to be set by the user, as part of the rule set?
 
The issue is OK in my example, because deletion of 'My Video' is never going to happen: it just fills up auto.log with junk. I think it would be best if Sweeper reverted to how it used to be, i.e. non-deletion of source folders by default, but with a means of enabling deletion if required.
 
This is actually the way it used to be. It was designed for sweeping automatically created folders away and it always removed the source. A bug introduced during development of the web interface broke this for a while.
I'm uploading a new version which doesn't try to delete the source directory if it is the media root or if it has sweeper rules. Hopefully that will make it work in the "right way" for everyone.
 
Apologies, I came to the sweeper party late. The functionality for me never changed, I just noticed the auto log file growing more than usual. Your solution sounds ideal. Thanks for making these changes.
 
I have seen another post regarding sweeper running vigorously but I just wanted to double check that my own setup isn't going to damage my hard drive.

Humax HDR Fox T2 (humax).png

If you follow the My First Planet recording it begins to error on the rename function after the 1st rename, this continued every 10 minutes for 2 days until we listened to and removed the file (I've cropped the log file as it was far too long to post fully).

Is there any way sweeper can be set to not run again unless a new file has been moved over to the rename folder?
 
ian_j said:
If you follow the My First Planet recording it begins to error on the rename function after the 1st rename, this continued every 10 minutes for 2 days until we listened to and removed the file.
Is there any way sweeper can be set to not run again unless a new file has been moved over to the rename folder?
Click to expand...

There may be a more elegant solution, but one way around this is to have a condition which only applies for the Sweeper's first pass.

e.g. If you want to rename the files and the Media List title, then you could set a condition:

"If Media List title does not contain (a year or a hyphen or whatever you're going to add) then rename file names"
and
"If first rule matches then rename Media List titles" (to a name which includes a year or hyphen etc).

Then on the second pass the condition isn't met and no re-naming is attempted.

The .sweeper file would read:

# Re-name recording files with date at front
!title 201 action {renamefile {%year %2digitmonth %2digitdate - %title}}
# Re-name Media List title with day and date at end
lastrule "" action {settitle {%title_%shortday %date %shortmonth %year}}

In addition, you could also use Sweeper to move 'processed' files out of the folder to somewhere else, so no further actions are attempted by default, e.g.

# Move renamed files to 'Processed' folder.
title 201 action {movecreate {Processed}}
 
Many thanks DelftBlue I added in some If rules to the folder & master sweeper commands and that worked.
 
Is there any way to replace/remove characters in a filename.

I've had a look at the wiki but can't see anything on the Sweeper overview.
 
You must log in or register to reply here.
Back
Top