brand new HDD but EXTENTS_FL errors

[/df]

aha! found the answer, just on terminal (in the freefilesync folder) I need to write: sudo ./FreeFileSync.
...

You were specifically advised against that, FWIW because of possible unwanted side-effects, such as configuration files of your Ubuntu user account becoming owned by root. Sadly this solution was also recommended by a poster to the product forum.

 

[rodp]

On the command line:

> sudo <command> executes that command with root privilege, and you will be prompted for the admin password if sudo has not been used in that session within the last few minutes.

> sudo -i can be run on a line on its own to give all subsequent commands root privilege until "exit".

You were specifically advised against that, FWIW because of possible unwanted side-effects, such as configuration files of your Ubuntu user account becoming owned by root. Sadly this solution was also recommended by a poster to the product forum.

@/df Where was I advised not to do that? Black Hole just spoke about the process

Well I guess I'll leave it copying and then plug it back in and see what happens. What suitable test (if any) can I do to be sure there are no problems?

 

[Trev]

Would it really have taken 5 days+ to bung it in, format it and then copy the old stuff?

 

[rodp]

Hi all, as promised...

I've just plugged my old HDD back in to get the SMART data and a list of the packages I had installed. Sorry firefox is not playing ball when copying pasting it in so sorry about the formatting).

Some people reviewing the data below may so "oh there is nothing wrong with your HDD, the count of reallocated sectors is really low". However, there is still something wrong with it that fixdisk in the humax isn't wanting to fix. There is also the LBA 0 error (and as suggested in a previous post will only probably go away if you do a security delete [I think], which invariably means you've got to copy off the data anyway as it will delete the drive content [i think]). Also whenever you play a recording or stop it, it takes about 45 seconds to come back to life (during this time you can hear the HDD head cycliing (cyclic redundancy thing) trying to read/write. Fixdisk did fix some things but obviously not enough in order to resolve the problems. Oh and it also continues to reboot after about every 5 minutes at the moment too. The display counts you down saying it's going to reboot. It doesn't matter what channel you are on and I've also disabled the rewind tv feature but not improvement.

Thanks

Rodp

SMART data read from device /dev/sdb Disk Information SMART Status PASSED Model Family Seagate Video 3.5 HDD Device Model ST1000VM002-1CT162 Serial Number W1G0EJH0 LU WWN Device Id 5 000c50 05c5de4b9 Firmware Version SC23 User Capacity 1,000,204,886,016 bytes [1.00 TB] Sector Sizes 512 bytes logical, 4096 bytes physical Rotation Rate 5900 rpm Form Factor 3.5 inches Device is In smartctl database [for details use: -P show] ATA Version is ATA8-ACS T13/1699-D revision 4 SATA Version is SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is Sat Sep 12 14:28:32 2020 BST SMART support is Available - device has SMART capability. SMART support is Enabled Attributes ID Name Flags Raw Value Value Worst Threshold Life Left Notes 1 Raw_Read_Error_Rate POSR-- 213690035 107 084 006 - 3 Spin_Up_Time PO---- 0 097 097 000 - 4 Start_Stop_Count -O--CK 9730 091 091 020 89% - 5 Reallocated_Sector_Ct PO--CK 488 100 100 036 100% - 7 Seek_Error_Rate POSR-- 609702513 087 060 030 - 9 Power_On_Hours -O--CK 27997 069 069 000 69% - 10 Spin_Retry_Count PO--C- 0 100 100 097 100% - 12 Power_Cycle_Count -O--CK 9727 091 091 020 89% - 184 End-to-End_Error -O--CK 0 100 100 099 - 187 Reported_Uncorrect -O--CK 5632 001 001 000 - 188 Command_Timeout -O--CK 4295032833 100 099 000 - 189 High_Fly_Writes -O-RCK 1219 001 001 000 - 190 Airflow_Temperature_Cel -O---K 28 072 (28°C) 034 (66°C) 045 (55°C) In_the_past 191 G-Sense_Error_Rate -O--CK 0 100 100 000 - 192 Power-Off_Retract_Count -O--CK 9725 096 096 000 96% - 193 Load_Cycle_Count -O--CK 9731 096 096 000 96% - 194 Temperature_Celsius -O---K 28 028 066 000 - 197 Current_Pending_Sector -O--C- 104 100 098 000 - 198 Offline_Uncorrectable ----C- 104 100 098 000 - 199 UDMA_CRC_Error_Count -OSRCK 0 200 200 000 - Self-test logs No. Description Status Remaining When First Error LBA # 1 Short offline Completed: read failure 90% 27921 0 # 2 Short offline Completed: read failure 90% 27921 0 # 3 Short offline Completed: read failure 90% 27921 0 # 4 Short offline Completed: read failure 90% 27921 0 # 5 Short offline Completed: read failure 90% 27921 0 # 6 Short offline Completed: read failure 80% 27921 0 # 7 Short offline Completed: read failure 90% 27921 0 # 8 Short offline Completed: read failure 90% 27921 0 # 9 Short offline Completed: read failure 90% 27921 0 #10 Short offline Completed: read failure 90% 27921 0 #11 Short offline Completed: read failure 90% 27921 0 #12 Short offline Completed: read failure 90% 27921 0 #13 Short offline Completed: read failure 90% 27921 0 #14 Short offline Completed: read failure 90% 27921 0 #15 Short offline Completed: read failure 90% 27921 0 #16 Short offline Completed: read failure 80% 27921 0 #17 Short offline Completed: read failure 90% 27921 0 #18 Short offline Completed: read failure 90% 27921 0 #19 Short offline Completed: read failure 90% 27921 0 #20 Short offline Completed: read failure 90% 27921 0 #21 Short offline Completed: read failure 90% 27921 0

 

[rodp]

Would it really have taken 5 days+ to bung it in, format it and then copy the old stuff?

Been err busy working also I had so much trouble copying the data off it in the first place I didn't just want to start it copying and walk away. It probably wouldn't have copied the hidden . files anyway if I'd plugged the old HDD into the humax. I needed to use something lie freefilesync to be sure everything had copied over.

 

[Trev]

It's only telly.

 

[rodp]

However, there is still something wrong with it that fixdisk in the humax isn't wanting to fix

Just did a little investigation and spotted / remembered there is a safe mode option in the diagnostics page. So I tried this with the old HDD in and it's now stable. no crashing! this is off thread now so I'll create a new thread to ask how I start trouble shooting to find out what part of the CF is making it reboot. (link: https://hummy.tv/forum/threads/troubleshooting-rebooting-all-ok-in-safe-mode.9866/ if anyone is interested)

it takes about 45 seconds to come back to life

Again with the humax in Safe mode it responds nice and quickly now.

Oh well, I've wanted 2TB HDD for sometime so don't feel I've wasted my money.... just my time (But I guess bad sectors were starting to appear)

 

[rodp]

It's only telly.

to me it's not just about telly, it's a media server, Remote management, FTP, downloader and advert skipper.

 

[MartinLiddle]

Some people reviewing the data below may so "oh there is nothing wrong with your HDD, the count of reallocated sectors is really low". However, there is still something wrong with it that fixdisk in the humax isn't wanting to fix.

The issue with hard drive is that fix-disk isn't managing to reduce the current_pending_sector count to zero.

 

[rodp]

The issue with hard drive is that fix-disk isn't managing to reduce the current_pending_sector count to zero.

Correct - it's not sorting it out. Maybe a later version of e2fsck may (ie on Ubuntu 16.04) but not got to that point yet. I want to trouble shoot the rebooting - it started to reboot itself after I ran fixdisk but what it's done I don't know. I'm reading through https://hummy.tv/forum/threads/steps-for-resolving-hdr-fox-crash-reboot-issues.5320/ at the moment, will try deleting the DNLA database first -

delete the file dms_cds.db from the root of the second partition on the disk

.

 

[MartinLiddle]

Correct - it's not sorting it out. Maybe a later version of e2fsck may (ie on Ubuntu 16.04) but not got to that point yet. I want to trouble shoot the rebooting - it started to reboot itself after I ran fixdisk but what it's done I don't know.

This is a link to fixing bad sectors https://www.amolak.net/fix-hard-disk-bad-sectors-in-linux/ I haven't tried it but you haven't got a lot to lose with the old hard drive.

 

[/df]

This is a link to fixing bad sectors https://www.amolak.net/fix-hard-disk-bad-sectors-in-linux/ ...

Surely with a SMART-enabled disk the approach proposed there would only be applicable when the disk has exceeded its built-in pool of spare sectors, by which time it would probably be failing further sectors all the time. So you'd get a bit of operation, then hours of software-based disk checking, then a bit more operation, ...

This page describes the approach used by fix-disk, which is based on the use of the disk firmware, SMART in particular, to detect bad sectors and force them to be remapped to spare good sectors, and then fixing the affected filesystem(s).

This page gives more details of the approach and tools used in the quoted link, which dates back to the era when disk sectors were actually enumerated by sector numbers and not virtualised by on-board controller firmware, and so it was necessary to record bad sectors as part of the filesystem.

Personally, I'd back up everything from the suspect disk (if a filesystem won't mount, or there are too many errors to copy by file, I'd take an image backup using ddrescue).

Then, in order:

1. Secure Erase and test with a long SMART test
2. Try to zero the disk with dd if=/dev/zero ... and test with a long SMART test
3. Try badblocks per the quoted link
4. Let the drive's components become reused via the electronics recycling bin.

 

[rodp]

Personally, I'd back up everything from the suspect disk (if a filesystem won't mount, or there are too many errors to copy by file, I'd take an image backup using ddrescue).

Then, in order:

1. Secure Erase and test with a long SMART test
2. Try to zero the disk with dd if=/dev/zero ... and test with a long SMART test
3. Try badblocks per the quoted link
4. Let the drive's components become reused via the electronics recycling bin.

backup / image backup might need to be done out of the box but are you suggesting to do the rest whilst the HDD is in the Humax?

no. 4...... They make good paper weights!!

How do you execute a long SMART test? is that a normal fixdisk?

All these options will remove the current setup of the webif and it's pacakages (not forgetting the recordings) and there is no easy way it seems other than writing down what you had installed and then going through it once re installed and set it all back up. If someone knows an easier way that would be great to know.... I am assuming you are not sugesting to copy the image back over to the HDD after using ddrescue - that might bring back corrupted files etc.?

Thanks

Rodp

 

[/df]

backup / image backup might need to be done out of the box but are you suggesting to do the rest whilst the HDD is in the Humax?

#2 and #3 will be much faster with the disk in a caddy attached to a more capable machine.

...How do you execute a long SMART test? is that a normal fixdisk?

It's an option there, but otherwise smartctl -t long /dev/sdx (from memory) where sdx has to be replaced by the appropriate value for the disk. Then after some period that it will suggest check the result with smartctl -l selftest /dev/sdx, and again later if the test hasn't finished, and so on.

All these options will remove the current setup of the webif and its packages (not forgetting the recordings) and there is no easy way it seems other than writing down what you had installed and then going through it once re installed and set it all back up. ...

Some settings and packages live in the flash storage and will persist regardless of the disk.

Everything else lives in /mod. Therefore you can copy over your saved /mod tree (exclude /mod/boot, which is actually the flash storage) to restore your system. In case some package files might be corrupted, you can reinstall all the packages, either manually or with a script; as the list of packages is in flash storage, you could actually do that without copying the original package files, but not all package settings are in the proper places, namely root's home directory and /mod/etc (eg, Mediatomb), so it's easier to copy everything.

As for the recordings, one benefit of ddrescue is that it tells you (like fix-disk which files might be affected by disk errors, so saving you the many hours you would otherwise have to use to watch each programme to check for errors.

 

[rodp]

update...

How do you execute a long SMART test? is that a normal fixdisk?

I think I figured it out.... I did fixdisk with the options -l -y and let is go - for 2 days!

Attached is the logs. It seems the long run picked up a lot more bad stuff than the normal one. My SMART log in terms of re-allocated sectors has jumped up dramatically. So I'm glad I did buy new HDD. I suspect if I run the long process again I might find more issues but it's probably time to install the new one and make sure that after having done the copying over of the recordings using a PC Live Ubuntu CD and running Freefilesync under SUDO mode, the files are still accessable, otherwise I'll be asking about I use chown

chown

and @/df will feel all smug and reply saying I told you so!!


Old HDD after Normal and Long fix-disk process (just noticed after a reboot it's saving the drive is sdb - perhaps it needs a cold boot)






line 197 and 198 is building.

So I'm thinking the HDD after a few more runs detecting bad blocks (in a caddy) I'l like to use it as a temporary drive. (of non prescious stuff). What is the process? I'll probably convert if over to NTFS for what I have in mind and split it possibly into two partitions. Would that remove all the current bad blocks and I'd be back to square one again or would the HDD remember? Does formatting mean I don't need to worry about doing a security delete?

Any thoughts on the logs attached would be most welcome - some constructive comments wouldn't go a miss in amongst the - "yep - time to change it!")

Thanks

Rodp
PS. To stop the rebooting (yes still rebooting after about 5-10 minutes even though mediatomb is now off) I have turned off the recursive autoshrink and auto decrypt as suspect that is causing issues too.

 

[MartinLiddle]

Would that remove all the current bad blocks and I'd be back to square one again or would the HDD remember?

As long as there are spare sectors then the bad blocks should be mapped out by the hard drive firmware and will be remembered.

Any thoughts on the logs attached would be most welcome - some constructive comments wouldn't go a miss in amongst the - "yep - time to change it!")

The latest run still hasn't reduced the Current_pending_sector count to zero. To be honest, I wouldn't be spending any more time on this hard drive but if you want to carry on, then good luck and interested to see what you find.

 

[Black Hole]

(just noticed after a reboot it's saving the drive is sdb - perhaps it needs a cold boot)

That's not an issue, don't worry about it.

 

[prpr]

So I'm thinking the HDD after a few more runs detecting bad blocks (in a caddy) I'l like to use it as a temporary drive. (of non prescious stuff). What is the process?

For <deity>'s sake:
Copy anything of it you want to keep. Security delete if there's anything sensitive on it. Put it in the WEEE bin. Stop trying to use it. It's broken beyond help.