[seriesfiler] Series Auto-filer

[Black Hole]

I strongly recommend unburdening your HDR. There is only so much out-of-date telly that can ever be watched with new stuff coming along all the time (says he, with three 500GB HDRs and four USB drives of various capacities...).

 

[DelftBlue]

I strongly recommend unburdening your HDR. There is only so much out-of-date telly that can ever be watched with new stuff coming along all the time (says he, with three 500GB HDRs and four USB drives of various capacities...).

You are absolutely right and I agree with you completely .... Just need to persuade everyone else in this house of this wisdom.

60GB of 'unburdening' occurred today, just need to get the elves to delete another 500GB or so, then all will be peachy ( ... until the next time!)

But, hey - isn't it useful to have the custom firmware and all its various options, stress-tested under a wide range of conditions?

 

[prpr]

BH, your logic is impeccable. You would indeed think that as filing requires no new disk space, then a filing process would work regardless of disk space. Unfortunately this situation is counter-intuitive ...

Agreed. I find Jimsh code hard to read, but I think there's a superfluous call that stops the whole automatic processing from working when it doesn't need to (therefore including sweeper).
You could try editing /mod/webif/lib/bin/auto and change line 114 from "dsc" to "#dsc".

Everything that does consume disk space already has a separate disk space check elsewhere in the code.

 

[af123]

Sweeper can consume disk space.

 

[prpr]

You're gonna have to explain that one to me because I can't see how. I've reviewed the Wiki info. and the [sweeper] thread and can't see anything obvious.
Given that it almost never consumes disk space, it seems silly to force the check 100% of the time. It would be much better to check specifically on whatever operation it is that does consume space.

 

[prpr]

60GB of 'unburdening' occurred today, just need to get the elves to delete another 500GB or so, then all will be peachy ( ... until the next time!)

Have you done a "tune2fs -m 0 /dev/sda2" to gain another 5% of your disk that the Humax software needlessly wastes?

 

[Black Hole]

I'm guessing it's a lot easier to suspend everything when the safety limit is reached than to keep bits limping on. It is, after all, not good practice to run your disk space to the max, and dropping facilities is a good incentive to jettison the dross.

 

[prpr]

There is no point suspending things which don't consume space. It's just annoying as the OP has already demonstrated. Suspend the things that matter otherwise leave it alone. I really don't know why you argue things like this.

 

[Black Hole]

Maybe because I look for good, even in you?

 

[prpr]

What is good about denying a function that doesn't need to be denied?
It isn't good, it's bad.
Now, as for the unwarranted personal attack, do I have to complain about you as well?

 

[af123]

You're gonna have to explain that one to me because I can't see how. I've reviewed the Wiki info. and the [sweeper] thread and can't see anything obvious.

That will teach me to post when I'm out and about. The development version of sweeper that I have on my boxes can consume disk space with various copy and link actions, but the currently released version doesn't have those. My mistake, sorry for the confusion.
The current version of sweeper should not cause anything other than small increases in disk usage, and those would be due to the creation of directories.

The disk space check is in the automatic processing code - sweeper is a plugin to that. Automatic processing is suspended once there is less than 1GiB of free disk space remaining (that's the check you mentioned at line 114). Each function that requires disk space performs an additional check to make sure there is still 1GiB + 3 * <size of file being processed> available as disk space can come and go. The 3* is because the worst case is that you need the original file + temporary file + new temporary file in the video filesystem at the same time.
Those checks could be tweaked to reduce or eliminate that 1GiB buffer but if you're down to your last GiB then it feels right to suspend processing.

 

[DelftBlue]

Have you done a "tune2fs -m 0 /dev/sda2" to gain another 5% of your disk that the Humax software needlessly wastes?

Thank you for this suggestion. Not yet done, because I haven't grasped where I would put that, or the consequences of doing so.

Is that something that goes in via telnet or via File Editor?

Does this just make more space available? (Sort of useful, but doesn't ultimately solve the in-house issue here of too much recording and not enough deleting )
Or does this make the Humax on-screen calculation of free space and the Web-IF calculation of free space correlate? Which would be very useful! (i.e. the rule for auto-processing is 'freespace of 1GB plus 3 times size of file in question' - but this doesn't match the Humax on-screen displayed free space. There can be 30GB or more of free space but auto-processing still won't run)

If you have the time (and patience ) please could you explain further ... where to put the code/instruction and what it'd do? Are there any downsides to doing this?

(P.S. I was posting here about seriesfiler vs. sweeper processing but I think this cross-relates to cloud9's puzzlement over auto-dedup also: http://hummy.tv/forum/threads/insufficient-disk-space-for-what.5097/#post-64919)

 

[Black Hole]

The quoted command is what you might I put on the command line from a Telnet session. Personally I would free up space by deleting dross rather than play around with the file system (unless you know what you're doing).

 

[cloud9]

Somewhere on here I've had a previous discussion about the discrepancy between the humax reported free space and the web-if free space and I think the conclusion was that the web-if (which uses the free space that df reports) ignores the 5% that the disk has reserved for root use. Whereas the humax s/w calculates it (and removes the ~5G it presumably needs for pausing live tv etc). Note that all the processes are root anyway, so the reserved space isn;t special, nor is it wasted except in terms of the web-if reporting and decision making. And also note that af123 (and others) formatted his drive with no reserved space (which means he's not going to see a discrepancy himself).

I assume the tunefs command that prpr gave changes it so there is no unreserved space. You would need to double check and make your own decision about running it.

 

[hairy_mutley]

The sweeper rule you need to emulate seriesfiler is:

folder action {fileunder ""}

If that is all that is required, then maybe it should appear in the Wiki under Series Filer and/or Sweeper.
Although I don't see how that does the job?
Personally, I am still having trouble converting the Sweeper instruction list into usable commands, but maybe that would be easier if I had a particular objective in mind.
For now, all I need is this Series Filer functionality, so I will give the above a try.

 

[af123]

Somewhere on here I've had a previous discussion about the discrepancy between the humax reported free space and the web-if free space and I think the conclusion was that the web-if (which uses the free space that df reports) ignores the 5% that the disk has reserved for root use. Whereas the humax s/w calculates it (and removes the ~5G it presumably needs for pausing live tv etc).

Sounds right, although the TSR buffer is on a separate disk partition so shouldn't come into play. I will experiment and see if the reservation is relevant and adjust the automatic processing accordingly.

 

[af123]

Although I don't see how that does the job?

The format is:

folder [zero or more conditions] <action>

The first word 'folder' says that this is a rule which should apply to folders. There are no conditions in this rule so it always matches and the action taken is to search for a folder of the same name down from the top level of the media tree. If one is found, the recordings are moved.

Compare with:

folder genre Children action {fileunder "Children's Stuff"}

which only processes folders which contain recordings of genre 'Children' and then looks for a folder of the same name somewhere under the Children's Stuff folder, moving the top-level folder if it finds it. Happy to discuss further or provide examples in the sweeper thread or a new one.

Personally, I am still having trouble converting the Sweeper instruction list into usable commands, but maybe that would be easier if I had a particular objective in mind.

The GUI should make this much easier once it's finished.

 

[Ezra Pound]

If that is all that is required, then maybe it should appear in the Wiki under Series Filer and/or Sweeper.

Series Filer and Sweeper WiKi notes now include Example 5 which covers Series Filer emulation

 

[hairy_mutley]

Thanks for that clarification.
That makes the "" some sort of tree search for the folder already found? Not a syntax that immediately leaps to mind. Anyway, I know where to come if I have a particular use in mind and want help

However, I think that your answer adds some background that is missing from the Wiki; which discusses the keywords, but without the starting context. I think the above would be a useful addition.

 

[af123]

Thanks for that clarification.
That makes the "" some sort of tree search for the folder already found?

It's the fileunder that is the tree search for the folder already found. The argument is where in the tree to start looking and "" means at the top level.