Does lots of multiply-claimed blocks mean that my drive is dying?

Dark Eyes

New Member
I bought a used HDR-FOX T2 about 10 months ago, installed CF, and followed the tips about checking the drive condition (2 recordings & timeshift together, etc). The SMART report looked good, and still does, if I'm reading it correctly (see attached). However, there regularly seems to be breakups in some of the recordings, which I don't think are down to the ariel feed. Looking on the forums, I came across fix-disk (-y), which I first ran a few days ago, and it found about 830 multiply-claimed blocks, which it started to fix. Unfortunately it is our main Freeview recorder (the other one regularly missing the first few minutes or worse), so I can only let it run for about 20 hours, before I have to exit maintenance mode. I have been doing this process for the last few nights, but the number of MC blocks isn't going down, and from the logs, it seems to be trying to fix some of the same files each run; it also doesn't manage to get through many actual recordings after it has done all the other checks. Is there a way to run just the multiply-claimed block repairs?

Given the number of multiply-claimed blocks, does this seem to indicate that the drive needs replacing;, or would I be better copying the recordings off onto a backup drive and, and reformatting the drive - if so do I need to do a 'full' format to get each sector read?
 

Attachments

  • SMART data 13-09 Run.txt
    2.5 KB · Views: 9

MymsMan

Ad detector
First try running fix-disk in maintenance mode before considering anything more drastic,
AFAIK multiply chained blocks is a software corruption rather than a hardware issue.
 

Black Hole

May contain traces of nut
Freeview recorder (the other one regularly missing the first few minutes or worse), so I can only let it run for about 20 hours, before I have to exit maintenance mode.
You are doing no good that way, you must let fix-disk finish. If you really can't, your only option is to backup and reformat (or just bite the bullet and replace the disk then reclaim stuff from the old one), but with disk errors you have no idea how good the data is you're backing up (or reclaiming).
 
OP
D

Dark Eyes

New Member
"You are doing no good that way" - I thought that was the case, which is which why I asked. The problem is 830+ MC blocks is probably going to take a few days, I seen reports of up to 10 days, and what is the chances of them happening afterwards? It seems a gamble, that will blacken my domestic reputation, yet further!

Do we know why these occur, especially if they might be software corruptions?

We've had 2 other Youview boxes in the past, one of which is still running (albeit slowly and with delayed recording starts) and I haven't experienced these breakups, especially the regularity of them, on either. Is this something that Humaxs are more prone to?
 

prpr

Well-Known Member
Given the number of multiply-claimed blocks, does this seem to indicate that the drive needs replacing;,
No.
or would I be better copying the recordings off onto a backup drive and, and reformatting the drive
Probably, if you you want to keep it available for recording instead of running fix-disk.
do I need to do a 'full' format to get each sector read?
A what?
First try running fix-disk in maintenance mode
He already said he did.
or just bite the bullet and replace the disk
When there's nothing wrong with it?
what is the chances of them happening afterwards?
Probably low, but you might need a re-format regardless if it's not something that can be fixed by fix-disk.
Do we know why these occur, especially if they might be software corruptions?
Usually a dirty filesystem shutdown - i.e. a crash or power outage or similar. Or maybe good old software bugs.
Is this something that Humaxs are more prone to?
Not necessarily. Any non-journaled filesystem (on any computer system) is susceptible to corruption unless it's closed down properly (and sometimes journaled ones too).
I can't speak for other Humax boxes, but the HDR Fox T2 uses the ext3 filesystem which is non-journaled.

So, you either need to let fix-disk run to completion, or you need to backup and re-format. It's easy to do using the telnet interface from maintenance mode. Replacing the disk seems over-the-top to me.
 
OP
D

Dark Eyes

New Member
Firstly, thanks for all the replies and advice, I'm grateful.

Looking at the recent fix-disk logs, it looks like new recordings have MC blocks, so it seems to be an ongoing problem.
Given that it is likely to take days to try and fix all the MC blocks, I think copying off the data and reformatting is probably the best option for us.
What should I backup to make restoring it back to its current setup? I know I can use Backup & Restore for the recordings schedule.
Can I backup packages & their settings, or do I just make a note of those and manually reinstall?
I can FTP from my Windows PC with root access and copy all the folders in My Video onto a USB3 drive, is this the fastest method to copy off the recordings and associated files?
(There are currently 221 recordings with Webif reporting that 416GB Used - I presume that includes the system files)
After I have reformatted the drive, in the Humax, and reinstalled CF, should I run fix-disk to see if the system is free of errors, before I copy the recordings back?
 

Black Hole

May contain traces of nut
I know I can use Backup & Restore for the recordings schedule
The recording schedule is not stored on the HDD.

and reinstalled CF
Neither is the CF, although the packages and settings are ("F" stands for firmware). However, to preserve the packages and settings, all you have to do is copy the /mod tree.

I can FTP from my Windows PC with root access and copy all the folders in My Video onto a USB3 drive, is this the fastest method to copy off the recordings and associated files?
There are soooo many misunderstandings in that statement. First of all: do you intend to connect your "USB3 drive" to the PC or the HDR-FOX?

If the former, network transfers proceed at roughly 400MB/min. If the latter, first of all the HDR-FOX does not have a USB3 interface and USB transfers proceed at about 200MB/min but if you use FTP not FXP you will be moving the data across the network twice before it ends up on the UPD (and I don't think the FTP server on the HDR-FOX supports FXP, not even the betaftpd package).

Even if you manage to achieve a sustained 400MB/min without the transfer crashing, 400-odd GB of data will take in excess of 1000 minutes (yes, that's right: at least 17 hours). Plus, "multiply claimed blocks" means you could have a lot of unrecoverable data in there (a multiply claimed block is when the file system thinks a particular block of data on disk belongs to two or more different files... which obviously it can't, so the file system therefore cannot reconstruct either file).

I think copying off the data and reformatting is probably the best option for us.
Are you sure about that? I still think your best option is to give up your dependence on telly and let fix-disk do its thing, before you even consider a backup.

For all prpr's reluctance to agree that a new drive might be a solution, it actually solves the main problems at a stroke if you're willing to stump up some cash to solve them:
  1. fit new drive (at which point you are immediately ready to record again, in a matter of minutes);
  2. install minimal packages (betaftpd);
  3. connect old drive via USB adapter;
  4. retrieve /mod;
  5. retrieve wanted (non-corrupt) recordings in slow time (or just watch them from the USB-connected drive).
Bingo.

HDR-FOX HDD Replacement
 
Last edited:

MikeSh

Well-Known Member
For all prpr's reluctance to agree that a new drive might be a solution, it actually solves the main problems at a stroke if you're willing to stump up some cash to solve them:
As the machine seems to be mission critical for your family this does seem like a reasonable solution.
 
OP
D

Dark Eyes

New Member
I can FTP from my Windows PC with root access and copy all the folders in My Video onto a USB3 drive
The USB3 drive would be attached to my PC (on a USB3 port)
Plus, "multiply claimed blocks" means you could have a lot of unrecoverable data
So as it is unrecoverable data, and the recordings will remain corrupt (as I expected), is fix-disk really worth me running (I presume that it is fixing the disk errors to try and stop it happening in the future), if I can copy the recordings off and reformat the drive, or does a reformat still leave the possibility of MC blocks existing/occurring with this regularity?
I could probably take it out of service for a weekend, so I would prepared to try this if it is going to fix the problem(s); but letting fix-disk complete its repairs seems like it is going to take much longer, given the number of errors and files involved, and I'm still not sure whether they are going to happen afterwards with this regularity. I'm struggling to understand, given my lack of hardware/Unix knowledge, whether these MC blocks existed before, or after the recordings were made.

I'm not expecting a definitive answer, I'm just trying to gather information to make an informed choice.

P.S. If I do replace the drive, are 2TB drives more likely to have any issues than 1TB due to size?
 

Luke

Well-Knwοn Мember
The USB3 drive would be attached to my PC (on a USB3 port)

So as it is unrecoverable data, and the recordings will remain corrupt (as I expected), is fix-disk really worth me running (I presume that it is fixing the disk errors to try and stop it happening in the future), if I can copy the recordings off and reformat the drive, or does a reformat still leave the possibility of MC blocks existing/occurring with this regularity?
I could probably take it out of service for a weekend, so I would prepared to try this if it is going to fix the problem(s); but letting fix-disk complete its repairs seems like it is going to take much longer, given the number of errors and files involved, and I'm still not sure whether they are going to happen afterwards with this regularity. I'm struggling to understand, given my lack of hardware/Unix knowledge, whether these MC blocks existed before, or after the recordings were made.

I'm not expecting a definitive answer, I'm just trying to gather information to make an informed choice.
I have just gone through a similar issue with a faulty HDR-FOX T2 that I knowingly bought as faulty. I had no intention of using the drive that came with it but, for fun decided to try and get it up and running with the corrupt drive. First using Fixdisk and then, and when that was looking unsuccessful then made a backup, zeroing out the HDD, let the HDR reformat, and then restored.

With the initial very lengthy runs of fixdisk the smart data did look good, and a long test was also OK. Issues quickly came back, probably hastened in there appearance by decrypting all the recordings which meant most of the recording data was re-written to the internal HDD.
As fixdisk had mentioned a couple of files by name I deleted the 2 recordings, but fixdisk and long test were still taking their time.

Accepting that fixdisk was unlikely to be a log term solution for this particular HDD in the state it was in I switched to the backup and reformat approach. Using a USB3 HDD attached to a USB2 port on a low speed laptop, in turn attached via Ethernet to a router, with the HDR also attached by Ethernet to the same router I used FTP to backup the 205GB of recordings. This took 4 hours.
After that telnetted into maintenance mode and used command "dd if=/dev/zero of=/dev/sda bs=1M" to wipe the drive. I don't know how long that took to complete and report its counts, but it would have been less than the time the backup took.
After that it was a reformat by the HDR.
The HDR is currently performing fine with no noticeable issues that I can spot in recording or playback, and fixdisk is remaining clean. I've restored the backup and sampling a few and they appear fine. Even if there are some corruptions, the TS format is designed to work with some degree of corruptions.
 

Black Hole

May contain traces of nut
I'm struggling to understand, given my lack of hardware/Unix knowledge, whether these MC blocks existed before, or after the recordings were made.

I'm not expecting a definitive answer, I'm just trying to gather information to make an informed choice.
It helps if you have a grasp of how filing systems work, or at least what they have to do. It seems very easy to just keep adding data to a disk and providing an index to that data in a directory structure, but there's a lot more to it than that: if you delete a file from somewhere in the middle, how is the vacated space reclaimed and made available to accommodate a new file? What if none of the available contiguous spaces is large enough to accommodate the new file in its entirety?

What you end up with is a database tracking all the bits which make up any particular file, and another database of all the chunks of free space for new files. A multiply-claimed block is where one particular block has somehow been included in the series of indexes for more than one file, or in the index for a file and in the index of free blocks.

One can envisage such a situation occurring if the system crashes in the process of updating the databases, but not very often. Large quantities of MCBs should not occur (unless they've been built up over a long period of time by successive improper shutdowns). Large quantities of MCBs sounds to me like some kind of major corruption in the databases themselves rather than the actual data (although the former leads inevitably to the latter), possibly disk faults, and the potential that fixdisk can't fix it or fixes it only temporarily. fixdisk can only recover the situation where there is sufficient redundancy to guess the correct entries; where the errors go beyond that fixdisk can only de-corrupt the indexes by separating out the dodgy data into "reclaimed data" files.

MCBs cannot exist until a file is written to create the MCB, which then arises later as an error in the indexing (but that error could be induced by hardware).

are 2TB drives more likely to have any issues than 1TB due to size?
I see no reason they should. The primary difficulty with increasing size of storage is an increasing overhead managing it, backing it up, whatever.
 
OP
D

Dark Eyes

New Member
I have just gone through a similar issue
Thanks for that information, that sounds like something to try before replacing the drive. That dd command line apparently fills the disk with zeros, which is sort of what I meant by a 'full format', as I think the drive reallocates any bad sectors during this process. Out of interest, how did you know when the process had finished, do you get a new command prompt when connecting via Telnet?

The primary difficulty with increasing size of storage is an increasing overhead managing it, backing it up
Apart from spending money unnecessarily, that is another concern; keeping the current sized drive forces regular housekeeping.
 

Luke

Well-Knwοn Мember
Out of interest, how did you know when the process had finished, do you get a new command prompt when connecting via Telnet?
I did not note exactly what was displayed when it had finished but it was very like:
dd: writing '/dev/sda': No space left on device​
476935+0 records in​
476933+1 records out​
There was no command prompt following that.
 

/df

Well-Known Member
...
After that telnetted into maintenance mode and used command "dd if=/dev/zero of=/dev/sda bs=1M" to wipe the drive. ...
Some sources suggest adding the oflag=direct option to the dd command line. It may be faster, anyway. More important is iflag=direct when reading back a write to verify it, so that you actually get the data from the disk (or the disk's cache) and not the OS cache.
 
OP
D

Dark Eyes

New Member
Some sources suggest adding the oflag=direct option to the dd command line. It may be faster, anyway. More important is iflag=direct when reading back a write to verify it, so that you actually get the data from the disk (or the disk's cache) and not the OS cache.
So can I use the oflag & iflag parameters in the same command, and will an iflag error cause the process to stop, or do I need to add noerror to that part of the command?
 

bottletop

Active Member
I've not used the other flags before. If trying to duplicate misbehaving drive I'll use something like dd bs=1M if=/dev/sdX of=/dev/sdY conv=noerror,sync status=progress.
 

/df

Well-Known Member
So can I use the oflag & iflag parameters in the same command, and will an iflag error cause the process to stop, or do I need to add noerror to that part of the command?
You can, but iflag options won't be relevant when reading from /dev/zero. conv=noerror is useful when trying to copy sectors that may be unreadable, as above.
 

Luke

Well-Knwοn Мember
I've not used the other flags before. If trying to duplicate misbehaving drive I'll use something like dd bs=1M if=/dev/sdX of=/dev/sdY conv=noerror,sync status=progress.
Does "status=progress" work on a Humax? I wouldn't have thought it would. ALthough next time I needed to wipe a drive on a Humax I was going to give it a try.
 

/df

Well-Known Member
dd 8.11 from the coreutils package knows about status=noxfer but, from its help and a small test, not status=progress. The two busybox versions don't support status=....
 
Top