undelete package - deletes files too early

Black Hole said:
/mod/bin/busybox/stat -f -c "%S %b %f" /mnt/hd2
Click to expand...
That gives
4096 477874319 21323806

No time to run fixdisk right now
Drive is working and is not in read-only mode. Its a Seagate ST2000VM003-1CT164

SMART says

IDNameFlagsRaw ValueValueWorstThresholdLife LeftNotes
1Raw_Read_Error_RatePOSR--20331608108099006-
3Spin_Up_TimePO----0096095000-
4Start_Stop_Count-O--CK828209209202090%-
5Reallocated_Sector_CtPO--CK0100100036100%-
7Seek_Error_RatePOSR--649725653087060030-
9Power_On_Hours-O--CK3309706306300063%-
10Spin_Retry_CountPO--C-0100100097100%-
12Power_Cycle_Count-O--CK177509909902099%-
184End-to-End_Error-O--CK0100100099-
187Reported_Uncorrect-O--CK0100100000-
188Command_Timeout-O--CK0100100000-
189High_Fly_Writes-O-RCK173001001000-
190Airflow_Temperature_Cel-O---K43057 (43°C)051 (49°C)045 (55°C)-
191G-Sense_Error_Rate-O--CK0100100000-
192Power-Off_Retract_Count-O--CK1241100100000100%-
193Load_Cycle_Count-O--CK828709609600096%-
194Temperature_Celsius-O---K43043049000-
197Current_Pending_Sector-O--C-0100100000-
198Offline_Uncorrectable----C-0100100000-
199UDMA_CRC_Error_Count-OSRCK0200200000-
Self-test logs
No.DescriptionStatusRemainingWhenFirst Error LBA
# 1Short offlineCompleted without error00%33097-
# 2Short offlineCompleted without error00%20705-
# 3Short offlineCompleted without error00%13592-
# 4Short offlineCompleted without error00%13592-
# 5Short offlineCompleted without error00%13562-
# 6Short offlineCompleted without error00%13533-
# 7Short offlineCompleted without error00%13504-
# 8Short offlineCompleted without error00%13474-
# 9Short offlineCompleted without error00%13445-
#10Short offlineCompleted without error00%13416-
#11Short offlineCompleted without error00%13386-
#12Short offlineCompleted without error00%13357-
#13Short offlineCompleted without error00%13327-
#14Short offlineCompleted without error00%13298-
#15Short offlineCompleted without error00%13269-
#16Short offlineCompleted without error00%13239-
#17Short offlineCompleted without error00%13210-
#18Short offlineCompleted without error00%13181-
#19Short offlineCompleted without error00%13151-
#20Short offlineCompleted without error00%13122-
#21Short offlineCompleted without error00%13092-
 
prpr said:
It's 15% different. As xyz321 alluded to, I presume your reserved space is set to the standard 5%, so can't account for the difference.

Probably try tune2fs -m 0 /dev/sda2 instead, and then re-run the stat command.
All the partitions on all my boxes have had this done, and they all have Free=Available.
Click to expand...
Thanks

After running tune2fs -m 0 /dev/sdb2
/mod/bin/busybox/stat -f /mnt/hd2 gives:

File: "/mnt/hd2"
ID: 0 Namelen: 255 Type: ext2/ext3
Block size: 4096
Blocks: Total: 477874319 Free: 21323806 Available: 21323806
Inodes: Total: 121380864 Free: 121365657

Which is different to the original above of
File: "/mnt/hd2"
ID: 0 Namelen: 255 Type: ext2/ext3
Block size: 4096
Blocks: Total: 477874319 Free: 22274267 Available: 0
Inodes: Total: 121380864 Free: 121365701

Could that have fixed something or what has it done?
 
Matthew said:
Could that have fixed something or what has it done?
Click to expand...
It's just made the reserved 5% of the disk available without screwing up the space calculations. Looks like there's a bug to be fixed somewhere if this solves your problem, and I expect it will.
Black Hole said:
it's a file system problem.
Click to expand...
No, it's normal operation.
Black Hole said:
Is that something fixdisk fixes
Click to expand...
No, because it's not broken.
and if not should it be?
Click to expand...
Possibly by something (not quite sure what), but probably not by fixdisk.
 
prpr said:
No, it's normal operation.
No, because it's not broken.
Click to expand...
So let's get this straight: somehow, something has taken the reserved space out-of-spec, but you say it's not broken?

Okay, so there's a bug of some sort so that if the reserved space is out of spec undelete gets its knickers in a twist, but that seems to me to be besides the point.

prpr said:
Possibly by something (not quite sure what), but probably not by fixdisk.
Click to expand...
Why not just put it in fixdisk? If it's a correction which is otherwise harmless, it might as well. Then, if anybody is having trouble of any sort (and this is a disk thing after all), running fixdisk sorts it.
 
Black Hole said:
somehow, something has taken the reserved space out-of-spec, but you say it's not broken?
Click to expand...
It's not out-of-spec. It's exactly as expected and is the filesystem default.
Black Hole said:
Okay, so there's a bug of some sort so that if the reserved space is out of spec undelete gets its knickers in a twist, but that seems to me to be besides the point.
Click to expand...
No it's exactly the point. The calculation is screwed up - it takes the disk size, multiplies it by 0.96 and then subtracts the used space to give the free space. This goes negative when you get close to filling the disk, as the OP has. I have no idea what the original intent was, but it seems weird.
I've released an update to undelete on the Beta packages channel.
Black Hole said:
Why not just put it in fixdisk? If it's a correction which is otherwise harmless, it might as well. Then, if anybody is having trouble of any sort (and this is a disk thing after all), running fixdisk sorts it.
Click to expand...
Because why should someone setting things up initially not 'benefit'? And, as I said, it's not broken. And it probably shouldn't matter. The reserved space thing was a red herring in the space calculations made by undelete.
 
prpr said:
It's not out-of-spec. It's exactly as expected and is the filesystem default.

No it's exactly the point. The calculation is screwed up - it takes the disk size, multiplies it by 0.96 and then subtracts the used space to give the free space. This goes negative when you get close to filling the disk, as the OP has. I have no idea what the original intent was, but it seems weird.
I've released an update to undelete on the Beta packages channel.

Because why should someone setting things up initially not 'benefit'? And, as I said, it's not broken. And it probably shouldn't matter. The reserved space thing was a red herring in the space calculations made by undelete.
Click to expand...
This sounds good thanks, I have upgraded to the beta package, will see what the results are tommorow.
 
Last edited:
BTW How much free space do you think you have?
What is the Humax webif showing for Total, Used, Free and Dustbin
 
prpr said:
And, as I said, it's not broken. And it probably shouldn't matter.
Click to expand...
I'm totally confused.

OK, so undelete is getting its calculations wrong. Fair enough. But the "available" thing is reported by stat independently of undelete. Somebody explain to me why, if there isn't anything wrong/broken, running tune2fs is appropriate/necessary!
 
Black Hole said:
I'm totally confused.

OK, so undelete is getting its calculations wrong. Fair enough. But the "available" thing is reported by stat independently of undelete. Somebody explain to me why, if there isn't anything wrong/broken, running tune2fs is appropriate/necessary!
Click to expand...
I may also be confused and wrong!

The file system, by default, reserves 5% of the disk space for use by the root user so that if the system is running low on space the system admins can still get in and fix things. This design decision was made when disk sizes were much smaller and systems actually had administrators.

On PVR systems there is no real need to reserve so much space, by my calcs on @Matthew's system that was 91GiB

I don't know if the reserved amount actually stops the Humax recording since I think it runs as root but if affects undeletes space calculation (the multiply by .96) it would cause undelete to think the system is low on space even if the reserved space has been removed,
It is not obvious why undelete used .96 rather than .95 or more accurately using the Available field from stat
 
Last edited:
MymsMan said:
It is not obvious why undelete used .96 rather than .95 or more accurately using the Available field from stat
Click to expand...
I think it was done that way to try to match the values given by the Humax app. but I am not sure if it was successful.
 
undelete uses df not stat.
It doesn't use the Available field (as that is affected by the Reserved percentage figure). It calculates Free (as Total - Used) because it (df) doesn't have Free, unlike stat.
I don't know why stat wasn't used, but it wasn't.
xyz321 said:
I think it was done that way to try to match the values given by the Humax app. but I am not sure if it was successful.
Click to expand...
Bodgery like this just causes more problems than it solves. The Humax app. is stupid in that it tries to convert GB into GiB (or is it the other way round?), with fudge factors/offsets and just confuses everybody. It's not worth trying to match it.
Now, undelete matches the free space on the disk, and almost matches what the WebIf says (there's a minor fudge to do with the TSR buffer).
Tested and working OK on both HDR and HD.
 
So it is doing conversions to GiB because the undelete setting specifies the limits in GiB rather than GB :poop:
As if anyone cares about or notices the difference!
 
I never cease to be amazed. How big is this space and how much is still available must be about the most fundamental things to know about storage, disc or shipping container.
And yet ... :dunno:
 
Are you measuring in yards or metres? It's the same principle. Is 1 GB 2^30 bytes or 10^9 bytes?
People always want you to think you're getting more than you are (because bigger is better and sells more), but not less than you are paying for. Then things get fudged to suit their own ends.
 
MymsMan said:
The file system, by default, reserves 5% of the disk space
Click to expand...
OK, I misunderstood. Earlier something about a 15% error was bandied about, and I thought the tune2fs command was to fix that – instead of which the tune2fs command simply removes any reserved space and adds the 5% back as recording space (as if running that low is a good idea!).

Matthew said:
Blocks: Total: 477874319 Free: 22274267 Available: 0
Click to expand...
Confirmed as 4.7%.
 
MymsMan said:
BTW How much free space do you think you have?
What is the Humax webif showing for Total, Used, Free and Dustbin
Click to expand...

The issue appears fixed, yesterday with approx 80GB free undelete log shows Free disk space: 8.45538 GiB today it shows Free disk space: 85.76 GiB

I think the stated free space was correct at that time (2am)

Webif states:
Total space: 1.78 TiB
Used: 1.7 TiB (95%)
Free: 80.6 GiB (5%)
Dustbin: 4.37 GiB (0%)

Thankyou all for your help with this, especially to prpr for the fix.
 
MymsMan said:
I think that must have been a mistake in his arithmetic because redoing the calculation myself shows it was as expected 5%
Click to expand...
Yes, my error somewhere. Typo on the calculator I expect, but I can't immediately see where I went wrong. Thanks for checking as I was confused by it.
Matthew said:
The issue appears fixed
Click to expand...
Thanks for the feedback. Will move the package to the main repository shortly.
 
MymsMan said:
I don't know if the reserved amount actually stops the Humax recording since I think it runs as root
Click to expand...
It doesn't, as an experiment I set the reserved space to 34% to use nearly all of my free space an to run with and started some recordings.
Normal Humax operation (Recording, Decryption, Detectads) all continued normally after the Available blocks reported by df and stat reached 0, the webif continues to display the full free amount as available.
With @prpr's fix empty_dustbin also ignored the reserved space and reported the full free space. :)

The reserved space does affect use of the disk by non privileged uses for example when mapped as a network drive windows reports 0 free space
 
You must log in or register to reply here.
Back
Top