undelete package - deletes files too early

OP
Matthew

Matthew

Active Member
/mod/bin/busybox/stat -f -c "%S %b %f" /mnt/hd2
That gives
4096 477874319 21323806

No time to run fixdisk right now
Drive is working and is not in read-only mode. Its a Seagate ST2000VM003-1CT164

SMART says

IDNameFlagsRaw ValueValueWorstThresholdLife LeftNotes
1Raw_Read_Error_RatePOSR--20331608108099006-
3Spin_Up_TimePO----0096095000-
4Start_Stop_Count-O--CK828209209202090%-
5Reallocated_Sector_CtPO--CK0100100036100%-
7Seek_Error_RatePOSR--649725653087060030-
9Power_On_Hours-O--CK3309706306300063%-
10Spin_Retry_CountPO--C-0100100097100%-
12Power_Cycle_Count-O--CK177509909902099%-
184End-to-End_Error-O--CK0100100099-
187Reported_Uncorrect-O--CK0100100000-
188Command_Timeout-O--CK0100100000-
189High_Fly_Writes-O-RCK173001001000-
190Airflow_Temperature_Cel-O---K43057 (43°C)051 (49°C)045 (55°C)-
191G-Sense_Error_Rate-O--CK0100100000-
192Power-Off_Retract_Count-O--CK1241100100000100%-
193Load_Cycle_Count-O--CK828709609600096%-
194Temperature_Celsius-O---K43043049000-
197Current_Pending_Sector-O--C-0100100000-
198Offline_Uncorrectable----C-0100100000-
199UDMA_CRC_Error_Count-OSRCK0200200000-
Self-test logs
No.DescriptionStatusRemainingWhenFirst Error LBA
# 1Short offlineCompleted without error00%33097-
# 2Short offlineCompleted without error00%20705-
# 3Short offlineCompleted without error00%13592-
# 4Short offlineCompleted without error00%13592-
# 5Short offlineCompleted without error00%13562-
# 6Short offlineCompleted without error00%13533-
# 7Short offlineCompleted without error00%13504-
# 8Short offlineCompleted without error00%13474-
# 9Short offlineCompleted without error00%13445-
#10Short offlineCompleted without error00%13416-
#11Short offlineCompleted without error00%13386-
#12Short offlineCompleted without error00%13357-
#13Short offlineCompleted without error00%13327-
#14Short offlineCompleted without error00%13298-
#15Short offlineCompleted without error00%13269-
#16Short offlineCompleted without error00%13239-
#17Short offlineCompleted without error00%13210-
#18Short offlineCompleted without error00%13181-
#19Short offlineCompleted without error00%13151-
#20Short offlineCompleted without error00%13122-
#21Short offlineCompleted without error00%13092-
 
OP
Matthew

Matthew

Active Member
It's 15% different. As xyz321 alluded to, I presume your reserved space is set to the standard 5%, so can't account for the difference.

Probably try tune2fs -m 0 /dev/sda2 instead, and then re-run the stat command.
All the partitions on all my boxes have had this done, and they all have Free=Available.
Thanks

After running tune2fs -m 0 /dev/sdb2
/mod/bin/busybox/stat -f /mnt/hd2 gives:

File: "/mnt/hd2"
ID: 0 Namelen: 255 Type: ext2/ext3
Block size: 4096
Blocks: Total: 477874319 Free: 21323806 Available: 21323806
Inodes: Total: 121380864 Free: 121365657

Which is different to the original above of
File: "/mnt/hd2"
ID: 0 Namelen: 255 Type: ext2/ext3
Block size: 4096
Blocks: Total: 477874319 Free: 22274267 Available: 0
Inodes: Total: 121380864 Free: 121365701

Could that have fixed something or what has it done?
 

prpr

Well-Known Member
Could that have fixed something or what has it done?
It's just made the reserved 5% of the disk available without screwing up the space calculations. Looks like there's a bug to be fixed somewhere if this solves your problem, and I expect it will.
it's a file system problem.
No, it's normal operation.
Is that something fixdisk fixes
No, because it's not broken.
and if not should it be?
Possibly by something (not quite sure what), but probably not by fixdisk.
 

Black Hole

May contain traces of nut
No, it's normal operation.
No, because it's not broken.
So let's get this straight: somehow, something has taken the reserved space out-of-spec, but you say it's not broken?

Okay, so there's a bug of some sort so that if the reserved space is out of spec undelete gets its knickers in a twist, but that seems to me to be besides the point.

Possibly by something (not quite sure what), but probably not by fixdisk.
Why not just put it in fixdisk? If it's a correction which is otherwise harmless, it might as well. Then, if anybody is having trouble of any sort (and this is a disk thing after all), running fixdisk sorts it.
 

prpr

Well-Known Member
somehow, something has taken the reserved space out-of-spec, but you say it's not broken?
It's not out-of-spec. It's exactly as expected and is the filesystem default.
Okay, so there's a bug of some sort so that if the reserved space is out of spec undelete gets its knickers in a twist, but that seems to me to be besides the point.
No it's exactly the point. The calculation is screwed up - it takes the disk size, multiplies it by 0.96 and then subtracts the used space to give the free space. This goes negative when you get close to filling the disk, as the OP has. I have no idea what the original intent was, but it seems weird.
I've released an update to undelete on the Beta packages channel.
Why not just put it in fixdisk? If it's a correction which is otherwise harmless, it might as well. Then, if anybody is having trouble of any sort (and this is a disk thing after all), running fixdisk sorts it.
Because why should someone setting things up initially not 'benefit'? And, as I said, it's not broken. And it probably shouldn't matter. The reserved space thing was a red herring in the space calculations made by undelete.
 
OP
Matthew

Matthew

Active Member
It's not out-of-spec. It's exactly as expected and is the filesystem default.

No it's exactly the point. The calculation is screwed up - it takes the disk size, multiplies it by 0.96 and then subtracts the used space to give the free space. This goes negative when you get close to filling the disk, as the OP has. I have no idea what the original intent was, but it seems weird.
I've released an update to undelete on the Beta packages channel.

Because why should someone setting things up initially not 'benefit'? And, as I said, it's not broken. And it probably shouldn't matter. The reserved space thing was a red herring in the space calculations made by undelete.
This sounds good thanks, I have upgraded to the beta package, will see what the results are tommorow.
 
Last edited:

MymsMan

Ad detector
BTW How much free space do you think you have?
What is the Humax webif showing for Total, Used, Free and Dustbin
 

Black Hole

May contain traces of nut
And, as I said, it's not broken. And it probably shouldn't matter.
I'm totally confused.

OK, so undelete is getting its calculations wrong. Fair enough. But the "available" thing is reported by stat independently of undelete. Somebody explain to me why, if there isn't anything wrong/broken, running tune2fs is appropriate/necessary!
 

MymsMan

Ad detector
I'm totally confused.

OK, so undelete is getting its calculations wrong. Fair enough. But the "available" thing is reported by stat independently of undelete. Somebody explain to me why, if there isn't anything wrong/broken, running tune2fs is appropriate/necessary!
I may also be confused and wrong!

The file system, by default, reserves 5% of the disk space for use by the root user so that if the system is running low on space the system admins can still get in and fix things. This design decision was made when disk sizes were much smaller and systems actually had administrators.

On PVR systems there is no real need to reserve so much space, by my calcs on @Matthew's system that was 91GiB

I don't know if the reserved amount actually stops the Humax recording since I think it runs as root but if affects undeletes space calculation (the multiply by .96) it would cause undelete to think the system is low on space even if the reserved space has been removed,
It is not obvious why undelete used .96 rather than .95 or more accurately using the Available field from stat
 
Last edited:

xyz321

Well-Known Member
It is not obvious why undelete used .96 rather than .95 or more accurately using the Available field from stat
I think it was done that way to try to match the values given by the Humax app. but I am not sure if it was successful.
 

prpr

Well-Known Member
undelete uses df not stat.
It doesn't use the Available field (as that is affected by the Reserved percentage figure). It calculates Free (as Total - Used) because it (df) doesn't have Free, unlike stat.
I don't know why stat wasn't used, but it wasn't.
I think it was done that way to try to match the values given by the Humax app. but I am not sure if it was successful.
Bodgery like this just causes more problems than it solves. The Humax app. is stupid in that it tries to convert GB into GiB (or is it the other way round?), with fudge factors/offsets and just confuses everybody. It's not worth trying to match it.
Now, undelete matches the free space on the disk, and almost matches what the WebIf says (there's a minor fudge to do with the TSR buffer).
Tested and working OK on both HDR and HD.
 

MymsMan

Ad detector
So it is doing conversions to GiB because the undelete setting specifies the limits in GiB rather than GB :poop:
As if anyone cares about or notices the difference!
 

MikeSh

Well-Known Member
I never cease to be amazed. How big is this space and how much is still available must be about the most fundamental things to know about storage, disc or shipping container.
And yet ... :dunno:
 

prpr

Well-Known Member
Are you measuring in yards or metres? It's the same principle. Is 1 GB 2^30 bytes or 10^9 bytes?
People always want you to think you're getting more than you are (because bigger is better and sells more), but not less than you are paying for. Then things get fudged to suit their own ends.
 

Black Hole

May contain traces of nut
The file system, by default, reserves 5% of the disk space
OK, I misunderstood. Earlier something about a 15% error was bandied about, and I thought the tune2fs command was to fix that – instead of which the tune2fs command simply removes any reserved space and adds the 5% back as recording space (as if running that low is a good idea!).

Blocks: Total: 477874319 Free: 22274267 Available: 0
Confirmed as 4.7%.
 
OP
Matthew

Matthew

Active Member
BTW How much free space do you think you have?
What is the Humax webif showing for Total, Used, Free and Dustbin

The issue appears fixed, yesterday with approx 80GB free undelete log shows Free disk space: 8.45538 GiB today it shows Free disk space: 85.76 GiB

I think the stated free space was correct at that time (2am)

Webif states:
Total space: 1.78 TiB
Used: 1.7 TiB (95%)
Free: 80.6 GiB (5%)
Dustbin: 4.37 GiB (0%)

Thankyou all for your help with this, especially to prpr for the fix.
 

prpr

Well-Known Member
I think that must have been a mistake in his arithmetic because redoing the calculation myself shows it was as expected 5%
Yes, my error somewhere. Typo on the calculator I expect, but I can't immediately see where I went wrong. Thanks for checking as I was confused by it.
The issue appears fixed
Thanks for the feedback. Will move the package to the main repository shortly.
 

MymsMan

Ad detector
I don't know if the reserved amount actually stops the Humax recording since I think it runs as root
It doesn't, as an experiment I set the reserved space to 34% to use nearly all of my free space an to run with and started some recordings.
Normal Humax operation (Recording, Decryption, Detectads) all continued normally after the Available blocks reported by df and stat reached 0, the webif continues to display the full free amount as available.
With @prpr's fix empty_dustbin also ignored the reserved space and reported the full free space. :)

The reserved space does affect use of the disk by non privileged uses for example when mapped as a network drive windows reports 0 free space
 
Top