Sweeper / TheTVDB Int. / Rename - Approach and pointers

[rapple]

Blather
After years of manually renaming downloaded files to contain SxxEyy and brief description, copied from the WebIF screen, then editing with TS-doctor to remove ads and fix up errors in files before xferring these to Plex I got to a point recently trying to dump a load we'd recorded during lockdown that we haven't got through yet. Bored of that process I finally got to "there must be a better way and it must have been done already". At least to get 90% done automatically with the rest flagged.

Didn't know about sweeper but found it searching through the forum, and while it's not hard in concept it's a little unclear from the instruction pages that I've found the best way to cope with real world issues such as "I've already processed these files and renamed them"

Thought I'd already got TVDB integration running but clearly didn't understand it was a manual process - I just thought it didn't do much.

Long Question
So... before I go trying to reinvent stuff has anyone got a canned solution to either the first 2 steps of my actual workflow or more if it's bombproof-

From a fully decoded set of files -

1. Rename all files to be Plex compliant (i.e. TV showname (sometimes year) SxxEyy Ep description or Film filmname (year of publishing) optionally Any other comments such as IMDB genre
2. Download all files to a local disk. I currently pull but push would be a good option. Optionally to create the text portion of the programme description to a similarly named file in the same directory as the main program.
3. Fix files and strip ads
4. Load to Plex.

Note, I've always been wary of auto ad stripping, the amount of time I have to spend making sure it's only stripping ads seems to negate the time spent doing it manually and we're not watching on the Humax boxes any more, they're headless in a room up in the attic. So bookmarks to flag potential ads don't seem to add much.

Shortened Question
What's the best method to make the decoded, recorded files available on an SMB file server with files renamed to reflect content incl show/film name, series, ep nos, ep name/description. Plus all text of the description avail to the humax in a text file with same name.

Thanks.

 

[Luke]

Didn't know about sweeper but found it searching through the forum, and while it's not hard in concept it's a little unclear from the instruction pages that I've found the best way to cope with real world issues such as "I've already processed these files and renamed them"

One way to not repeat a rule that renames the title, or filename, is to include a check as part of the renaming rules.
E.g. If you've added the serial and episode numbers then test to see if the title already complies with the renamed format.

!filename ~S[0-9]*E[0-9] !title ~S[0-9]*E[0-9] action {renamefile %title_S%seriesE%format:%02d:%episode:%yyyymmdd%hhmm}
!title ~S[0-9]*E[0-9] action {settitle %title_S%seriesE%format:%02d:%episode:}

 

[Black Hole]

What's the best method to make the decoded, recorded files available on an SMB file server with files renamed to reflect content incl show/film name, series, ep nos, ep name/description. Plus all text of the description avail to the humax in a text file with same name.

Use a HD-FOX as the client!

 

[rapple]

Thanks for those. Been playing a little. I have a blindspot on regex, always have, so trying to use the higher level bits for now.

Renaming itself is relatively simple. Remaining issues and questions are:
For the humax media details box in WebIF (from b'cast info) I can't see any obvious way to create a separate file to dump the synopsis info. Presumably this util is meant to alter name and content of all existing files only so this isn't something it can do?

For %regsub, from the wiki page I expected it to be e.g. %regsub:%orig:<pattern>:<replacement pattern with parameters based on groups> but in the "rename new" example code (which works) it seems to be used as a suffix to the string which is being modified. What's the actual usage instruction for %regsub please?

I'm not 100% on how multiple rules are processed and applied to matched files.
The wiki page suggests that multiple steps can be done on a single rule, and that you can halt rules (presumably for this recording). However that doesn't seem to be in the current version of sweeper.




So I presume that all rules are processed sequentially for all recordings, but that doesn't make sense either as each rule could have a different scope.
Basically if I am trying to rename a file and then move it somewhere it seems like I need two steps and therefore two rules, so the question is, what (if any) context remains between rules or do I have to do the whole matching thing for each file in both rules and do I need new matching if I've changed the name already. Is there any context that I can assume between rules or are they all individual?

Finally on the theme of automating moving this off the Humax, happy to use Humax as a client to write to an SMB server but it's now headless so this needs to run in the background somehow.
I can't see anything in sweeper to do this as Move is relative to the the "my video" base. Any undocumented features that might achieve this? Triggering of a shell script for example to copy from a to b.
Or any other utilities?

TIA

 

[Black Hole]

The wiki page suggests that multiple steps can be done on a single rule

No. "And: Stop processing rules" is only there as a reminder, there is no alternative. The rule sequence is like a case statement, so the action for the condition which matches first is executed and no later conditions get tested. Consequently you do not need to test for negatives, and you can have a final "catch all" condition at the end to act on anything which fails all the previous tests. You can filter out recordings not to be processed at the start using a rule with a "do nothing" action.

Basically if I am trying to rename a file and then move it somewhere it seems like I need two steps and therefore two rules

So you move it first with a Sweeper non-recursive rule set in My Video, then rename it with a rule set in your destination folder.

happy to use Humax as a client to write to an SMB server but it's now headless so this needs to run in the background somehow.
I can't see anything in sweeper to do this as Move is relative to the the "my video" base.

By using network-shares-automount, you can make an external share visible under My Video (not that I recommend it, the Humax firmware will then treat the share as space within its local drive and get confused). An alternative is some command line stuff: put rsync in a script and run it periodically with crontab.

 

[rapple]

Ah, that's clear on the rule processing. Ta. I can work with that. So yes a two step, but I'd move and isolate first as well.

I'm much happier mounting in the OS as and if required and then playing with rsync to xfer than temp adds etc. If I follow this route and was to use Rsync to delete source on completion I assume that I need to clean up the other files as well (.hmt .nts etc) and does this then cause any issues not deleting through the webif or front end? The synopsis I'm after is in the HMT file I believe, is there a doc describing the hmt file format anywhere and/or any decode code around for it? My coding is less than ideal but maybe it's time for the 10 year practice...

Cheers.

 

[/df]

There's an unofficial spec in the Wiki and the hmt utility that is used by WebIf, as well its source code here, and another implementation in Perl.

 

[Black Hole]

Why can't you leave the recordings on the HDR-FOX and use it as the server?

 

[rapple]

Why can't you leave the recordings on the HDR-FOX and use it as the server?

If there's a Plex server available for it then possibly! Basically all our video is on a XigmaNAS NAS and accessed through a separate Plex Server. There's recorded stuff from god knows when using MythTV v5 I think, through a few other boxes and the Humax Fox boxes. I've got 400 + DVD and BluRay rip collection all in H.265 and things from a few other sources. Plus many home recorded videos. It's just nice and easy for those that don't want to redo the sort order of the DLNA server each time they want to watch and it's got all the nice things like restart from the point you stopped no matter which device you view from, download to tablet/phone transcoding as necessary (although to be fair until recently the tablet was the best resolution display we had!) and complete integration with IMDB and theTVDB for as much metadata as you could want.
No, we don't watch most things more than once but occasionally we go back and view a few things and actually during lockdown we've revisited a few series from 10 years ago that we never quite finished or missed a few. ..and there's a few things that we watch with the kids when they visit - sort of spread out over months. It's just very convenient in that environment.

thanks /df. Don't suppose there's a Pascal or Fortran version kicking around is there? C was becoming popular when I stopped coding and Perl was something you got with an Oyster if you were lucky when scarfing them in a restaurant in one of the E.E.C. countries. [Free Beer token to anyone who want's a punt at my birth year. And apologies to those seeking tech advice as I wander far from the point, I've had too many glasses of Mild tonight. (Not Banks' unf. can't find the stuff these days)]

 

[Black Hole]

If there's a Plex server available for it then possibly!

No, but it will do SMB or NFS – you're not restricted to DLNA. There's also Mediatomb as an alternative DLNA server.

Don't suppose there's a Pascal or Fortran version kicking around is there?**

Sorry, but get real! Who's going to go to the effort of porting a working C build to those ancient clapped-out languages? Might not be even possible*.

* By which I mean a 1:1 translation by substituting the appropriate syntax and function calls. Obviously (almost) anything is possible if you're prepared to write a whole new suite of functions for the target language which are already built into the original compiler.

** I realise the comment was probably tongue-in-cheek, but I didn't read it as such initially - that's what emoticons are for.

 

[prpr]

Free Beer token to anyone who want's a punt at my birth year. And apologies to those seeking tech advice as I wander far from the point, I've had too many glasses of Mild tonight.

1948. Are you Welsh? Or a grocer?

Don't suppose there's a Pascal or Fortran version kicking around is there?

 

[/df]

...
thanks /df. Don't suppose there's a Pascal or Fortran version kicking around is there?...

Your best bet is to create shell scripts to run on the Humax box (calling hmt). Unix shell ("Bourne") scripting hasn't evolved very much from SRB's 70s original. Using SSH you should be able to control the scripts from another machine, such as your Plex server.