undelete package - deletes files too early

[Matthew]

Does anyone know why this package deletes files from its bin much earlier then in accordance with its settings?

 

[Black Hole]

I don't think we've ever heard of this. Please give specific examples, including your settings. Don't forget it will get more aggressive if you are low on disk space.

 

[Matthew]

With these settings files get deleted after only a couple of days when space drops below 80GB

The space on the drive has never dropped below 30GB!

 

[Black Hole]

That's as maybe, but you've still not provided any data - such as what it says in the undelete log (WebIF >> Diagnostics >> empty_dustbin.log).

 

[MymsMan]

That's as maybe, but you've still not provided any data - such as what it says in the undelete log (WebIF >> Diagnostics >> empty_dustbin.log).

You didn't ask for the log before! Not everyone is a mind reader

I've never experienced problems but with only a 512GB disk I cant afford to keep so much in the dustbin and have never needed to restore older recordings

My settings are much more aggressive

1Gib is less than an hours recording time and undelete is only run once per day.
So to get it to free up space before you run critically low on storage you need to start reducing the retention period well before you actually run out, 10Gib is far to low

 

[Matthew]

Didn't know it has a log.
That points to the error, it has messages such as:
Free disk space: -6.68215 GiB
Free disk space: 10.2838 GiB

The last when the actual free space is 81GB, so something is calculating things wrong
Its a 2TB drive

 

[Matthew]

You didn't ask for the log before! Not everyone is a mind reader

I've never experienced problems but with only a 512GB disk I cant afford to keep so much in the dustbin and have never needed to restore older recordings

My settings are much more aggressive
View attachment 6195
1Gib is less than an hours recording time and undelete is only run once per day.
So to get it to free up space before you run critically low on storage you need to start reducing the retention period well before you actually run out, 10Gib is far to low

Thanks, I set things low as I was trying to stop it deleting early, didn't work though

 

[Black Hole]

The last when the actual free space is 81GB, so something is calculating things wrong

So there's the problem then. Maybe you should run a fixdisk, and if you have any external file shares mounted in My Video I recommend you don't (because they're sure to confuse file space calculations).

You didn't ask for the log before! Not everyone is a mind reader

For christ sake. How much of a mind reader do you have to be to realise that NOBODY can diagnose a fault unless they have some data to go on? I went looking to see what log files might be available, and so could anybody else.

 

[/df]

The Jim "class methods" system diskspace and system diskfree use a command like /mod/bin/busybox/stat -f -c {%S %b %f} /mnt/hd2 (adjust mount point for HD). It would be interesting to see the result of that on the problem system. Perhaps the undelete package uses its own less robust tactic for calculating the free space.

Also, tthere is a difference, when time-shift is enabled, between the actual free space and the free space available for recordings, which is less if the time-shift buffer has not yet used its reserved space. At one point the disk space display in the WebIf top bar could show negative values because of this.

 

[sine24]

For christ sake. How much of a mind reader do you have to be

Pretty massive one if you don't even know there is a log

 

[Matthew]

The Jim "class methods" system diskspace and system diskfree use a command like /mod/bin/busybox/stat -f -c {%S %b %f} /mnt/hd2 (adjust mount point for HD). It would be interesting to see the result of that on the problem system. Perhaps the undelete package uses its own less robust tactic for calculating the free space.

Also, tthere is a difference, when time-shift is enabled, between the actual free space and the free space available for recordings, which is less if the time-shift buffer has not yet used its reserved space. At one point the disk space display in the WebIf top bar could show negative values because of this.

the command /mod/bin/busybox/stat -f -c {%S %b %f} /mnt/hd2
gives the result:

stat: can't read file system information for '%b': No such file or directory
stat: can't read file system information for '%f}': No such file or directory
{4096

the hd is mounted as hd2

the command humax# /mod/bin/busybox/stat -f /mnt/hd2
gives the result:

File: "/mnt/hd2"
ID: 0 Namelen: 255 Type: ext2/ext3
Block size: 4096
Blocks: Total: 477874319 Free: 22274267 Available: 0
Inodes: Total: 121380864 Free: 121365701


Does that tell us anything?

 

[Black Hole]

/mod/bin/busybox/stat -f -c {%S %b %f} /mnt/hd2

{%S %b %f} was supposed to be interpreted as a format string, from the BusyBox stat command information being:

%S: Fundamental block size (for block counts)​

%b: Total data blocks in file system​

%f: Free blocks in file system​

They were not supposed to be passed to the command as file system pointers, so I guess the { } syntax is incorrect. This works for me:

/mod/bin/busybox/stat -f -c "%S %b %f" /mnt/hd2

File: "/mnt/hd2"
ID: 0 Namelen: 255 Type: ext2/ext3
Block size: 4096
Blocks: Total: 477874319 Free: 22274267 Available: 0
Inodes: Total: 121380864 Free: 121365701


Does that tell us anything?

Have you run fixdisk yet, and if not why not?

 

[prpr]

I guess the { } syntax is incorrect

It is Jim script speak rather than Bash script speak. Bit of an oversight from /df there.

Have you run fixdisk yet, and if not why not?

Quite so. Available (0) should be the same as Free, so I would guess that means the filesystem on that partition is read-only. ...wrong guess. See below.

 

[xyz321]

The Jim "class methods" system diskspace and system diskfree use a command like /mod/bin/busybox/stat -f -c {%S %b %f} /mnt/hd2 (adjust mount point for HD). It would be interesting to see the result of that on the problem system. Perhaps the undelete package uses its own less robust tactic for calculating the free space.

It uses df and find -mtime to determine which files to delete from the dustbin (see /mod/sbin/empty_dustbin - called by anacron).

The free space will be affected by the ext3 reserved block count if that has been changed from its default.

 

[MymsMan]

undelete logs its view of free space at the end of run, it would be nice if it also logged the space at start of run to show how much space had actually been freed up during that run of undelete.

My guess is that it deleting around 20-30GB per day - I don't record/watch that much daily because there are all the copies produced by autodecrypt, detectads, shrink etc.

 

[MymsMan]

Available (0) should be the same as Free, so I would guess that means the filesystem on that partition is read-only.

But from the format string %S %b %f undelete is not not looking at the available blocks so why is it thinking @Matthew is running so low on space and if the partition was read-only the deletes would be failing which is not what he is complaining of

On my system Free != Available but not hugely different - should I find a time to run fix-disk?

/mod/bin/busybox/stat -f  /mnt/hd2
  File: "/mnt/hd2"
    ID: 0        Namelen: 255     Type: ext2/ext3
Block size: 4096
Blocks: Total: 117335874  Free: 39530806   Available: 33570290
Inodes: Total: 29860704   Free: 29826131

 

[Black Hole]

why is it thinking @Matthew is running so low on space

Shirley the first thing to do is find out if there are any file system irregularities? Otherwise the system is not clean and all bets are off whether things operate correctly.

 

[prpr]

On my system Free != Available but not hugely different

It's 15% different. As xyz321 alluded to, I presume your reserved space is set to the standard 5% , so can't account for the difference. Edit: My calculation error.

should I find a time to run fix-disk?

Probably try tune2fs -m 0 /dev/sda2 instead, and then re-run the stat command.
All the partitions on all my boxes have had this done, and they all have Free=Available.

 

[MymsMan]

It's 15% different. As xyz321 alluded to, I presume your reserved space is set to the standard 5%, so can't account for the difference.

Probably try tune2fs -m 0 /dev/sda2 instead, and then re-run the stat command.
All the partitions on all my boxes have had this done, and they all have Free=Available.

I needed to specify sdb2 instead of sda1 but it worked for me

humax /mnt/hd2/mod/git # tune2fs -m 0 /dev/sdb2
tune2fs 1.42.13 (17-May-2015)
Setting reserved blocks percentage to 0% (0 blocks)
humax /mnt/hd2/mod/git # /mod/bin/busybox/stat -f  /mnt/hd2
  File: "/mnt/hd2"
    ID: 0        Namelen: 255     Type: ext2/ext3
Block size: 4096
Blocks: Total: 117335874  Free: 35989718   Available: 35989718
Inodes: Total: 29860704   Free: 29826081

I probably should have run tune2fs -l /dev/sdb2 first to check what the previous reserved count was

It may be worth @Matthew trying to reset the reserved space

 

[Black Hole]

Is that something fixdisk fixes, and if not should it be?