Fixdisk keeps on totally stalling on one specific drive

[Luke]

I'm having an issue with running fixdisk to its conclusion as it keeps on stopping with everything frozen, and I'm after ideas about what I can try next.

All runs have been with

Custom firmware version: 3.10 (build 2734)
Humax Version: 1.03.12 (kernel HDR_CFW_3.10)
No fixdisk options were included when initiating fixdisk​

I initially wanted to run fixdisk against a couple of 2tb drives but, annoyingly now, first wanted to check I understood how to run fixdisk. Turned out to be simple (thanks af123 and anyone else who assisted) and the 2TB drive runs are now complete.
The drive which is problematic is 1TB and has been running fine with clean smart data and Humax UI check disk OK. It is empty of recordings. The c/f was reinstalled a few days ago following a reset and so is barren of additional packages apart from fix-disk/temenu for some runs of fixdisk.

Initially fix-disk and tmenu were not installed and when running fixdisk the drive looked reasonable until it reached an inode with an address of the order of 22300000. It then went wild but unfortunately it got accidentally interrupted at around inode 33500000. Restarting and after 70 minutes it carried on shortly after the address that it got interrupted on but I think it must have inadvertently been interrupted again (by the PC auto-locking?) at about 44700000 and then again at 50300000.

I then installed fix-disk (and tmenu).
What happened when ever fixdisk was run on the 1TB disk after that was that the output to the putty session kept on freezing after it had progressed about 120000 of address increment. When it was running 'smoothly' I estimate it would have taken about 60 hours to complete but with the freezing this would now be months.

When the putty window freezes if I leave it alone for a few hours then terminate, reboot and start again it still has only progressed about 120000. In other words the HDR itself has been doing nothing.

I've tried uninstalling fix-disk and tmenu but it still just progress about 12000 with each fixdisk run.

The only theory I have is that something to do with the HDR logging is causing an issue when it reaches a certain quantity of output and that uninstalling fix-disk and tmenu doesn't remove the issue. But that's purly based on logging not being visibly produced when fix-disk is installed and does not really make any sense as uninstalling fix-disk/tmenu fixes the logging visibility issue.

Sample output from the end of a frozen execution below. It appears to terminate on a random issue each time so I guess that what ever is causing it to freeze is not appearing in the output:

Inode 52889850, i_blocks is 3960879305, should be 0.  Fix? yes

Inode 52889981, i_size is 9269156067431821477, should be 0.  Fix? yes

Inode 52889981, i_blocks is 1284290374, should be 0.  Fix? yes

Inode 52890093, i_size is 8014684842474254315, should be 0.  Fix? yes

Inode 52890093, i_blocks is 2591489609, should be 0.  Fix? yes

Inode 52890095, i_size is 4060923659119093225, should be 0.  Fix? yes

Inode 52890095, i_blocks is 1616549222, should be 0.  Fix? yes

Inode 52889869, i_size is 6217708061731797821, should be 0.  Fix? yes

Inode 52889869, i_blocks is 2117231132, should be 0.  Fix? yes

Inode 52889823 has compression flag set on filesystem without compkression support.  Clear? yes

Inode 52889823, i_size is 18323036454650159557, should be 0.  Fix? yes

Inode 52889823, i_blocks is 3471522269, should be 0.  Fix? yes

Inode 52890170 has compression flag set on filesystem without compression support.  Clear? yes

Inode 52890170 has illegal block(s).  Clear? yes

Illegal block #0 (3568927373) in inode 52890170.  CLEARED.
Illegal block #1 (1099767888) in inode 52890170.  CLEARED.
Illegal block #2 (2623612171) in inode 52890170.  CLEARED.
Illegal block #3 (1002302815) in inode 52890170.  CLEARED.
Illegal block #4 (1304960572) in inode 52890170.  CLEARED.
Illegal block #5 (3437696117) in inode 52890170.  CLEARED.
Illegal block #6 (956579610) in inode 52890170.  CLEARED.
Illegal block #7 (2186152892) in inode 52890170.  CLEARED.
Illegal block #8 (4244722261) in inode 52890170.  CLEARED.
Illegal block #9 (3335608061) in inode 52890170.  CLEARED.
Illegal block #10 (2481883650) in inode 52890170.  CLEARED.
Too many illegal blocks in inode 52890170.
Clear inode? yes

Inode 52890192 has INDEX_FL flag set but is not a directory.
Clear HTree index? yes

Inode 52890192, i_size is 2121864064518211004, should be 0.  Fix? yes

Inode 52890192, i_blocks is 83478672, should be 0.  Fix? yes

Inode 52889987 has compression flag set on filesystem without compression support.  Clear? yes

 

[MontysEvilTwin]

I can't tell you where the problem is, but as you have no recordings/ minimal packages installed I would reformat. However, in the past I had a disk that had a lot of inode problems that was taking hours and hours to run fixdisk. I aborted fixdisk, copied the recordings I wanted to keep and reformatted the disk using the remote control handset. After doing this I ran fixdisk again and still had lots of inode problems. My conclusion is that when the HDR-FOX reformats a disk it does not necessarily completely rewrite the filesystem. If I am wrong about this and anyone has a better explanation for my observations, please feel to post. In the end I did a security erase to overwrite the disk before reformatting: this worked but was probably overkill. I suggest the following:

Reformat (remote control) and rerun fixdisk.
If there are still problems, boot into maintenance mode and reformat using the 'GPTF' option. You don't need GPT for a 1TB drive, but this scheme uses fewer inodes on the main partition (1/64) and will overwrite the existing inode scheme. You can keep this as it is: fixdisk will run much more quickly on a disk formatted this way as the main partition only has a fraction of the inodes compared to the standard formatting. Or you could reformat again (remote control) if you don't want a disk with GPT.

I am not aware of a downside to using GPT on a 1TB disk, though there may be one. With fewer inodes, there is a risk of having too few inodes though thus is unlikely since the recordings tend to be quite large.

 

[Luke]

However, in the past I had a disk that had a lot of inode problems that was taking hours and hours to run fixdisk. I aborted fixdisk, copied the recordings I wanted to keep and reformatted the disk using the remote control handset. After doing this I ran fixdisk again and still had lots of inode problems. My conclusion is that when the HDR-FOX reformats a disk it does not necessarily completely rewrite the filesystem. If I am wrong about this and anyone has a better explanation for my observations, please feel to post.
I suggest the following:

Reformat (remote control) and rerun fixdisk.

Thanks. I had already tried that (but forgot to mention) and got the same result as you.

If there are still problems, boot into maintenance mode and reformat using the 'GPTF' option. You don't need GPT for a 1TB drive, but this scheme uses fewer inodes on the main partition (1/64) and will overwrite the existing inode scheme. You can keep this as it is: fixdisk will run much more quickly on a disk formatted this way as the main partition only has a fraction of the inodes compared to the standard formatting. Or you could reformat again (remote control) if you don't want a disk with GPT.

Ah yes. That's worth a bash rather than to take out the drive and reformat externally.
I think I'll try that tonight after I've swapped another HDR-FOX T2 out of the rack which I was trying to see what was wrong, (green screen due to DRM and drive rarely recognsied and so can't reset; system flush made no difference; going to give up on that one and possibly use it for parts).

 

[prpr]

The only theory I have is that something to do with the HDR logging is causing an issue when it reaches a certain quantity of output

Space on /tmp is limited to about 64MB.

You could always use dd to blank the disk (and then use the Humax software to partition and format) or run mke2fs manually to format whichever is the troublesome partition (presumably /dev/sda2).

 

[Luke]

Thanks.
Using
dd if=/dev/zero of=/dev/sda bs=1M status=progress
it was just returning the syntax with no mention of a status option.

It is now running without the status option.
I'm wary that it will stall and just sit there. Is there some way I can still see the progress?

 

[Black Hole]

If fix-disk crashes because the log file runs out of space, that strikes me as a problem.

 

[af123]

I'm wary that it will stall and just sit there. Is there some way I can still see the progress?

Try pressing control-T - works on some versions of dd.
alternatively, send it a USR1 signal from another shell.

Did you try the gpt format option that MET suggested?

 

[af123]

If fix-disk crashes because the log file runs out of space, that strikes me as a problem.

True (although we haven't confirmed that it does).
Standard fix-disk logs are < 100K so it wouldn't usually be a problem, but it may be possible to increase the size of /tmp in maintenance mode.

 

[Luke]

Try pressing control-T - works on some versions of dd.

No luck. But thanks

alternatively, send it a USR1 signal from another shell.

OK. I'll include $PID with the fixdisk call if I need to start again (and look up how to do that).

Did you try the gpt format option that MET suggested?

Yes. I formatted gpt and then reformated via Humax. Afterwards Fixdisk then started rapidly processing finds from the beginning of the disk. The ETA of completion was about 50 hours providing it did not stall. After it self terminated 3 times with a similar error I gave up on that one. The end of the self termination log was:

Illegal block #17826829 (4294967295) in inode 1974636.  CLEARED.
Error storing directory block information (inode=1974636, block=0, num=12000990): Memory allocation failed

e2fsck: aborted
hmx_int_stor: ***** FILE SYSTEM WAS MODIFIED *****

hmx_int_stor: ***** FILE SYSTEM WAS MODIFIED *****
tee: /tmp/fix-disk.log: I/O error
Removing extra swap space.
cat: write error: No space left on device

Finished
fix-disk: session terminated with exit status 0

Press return to continue:

 

[Luke]

(Above post edited to include that I used the Humax UI to format after the gpt format.)

 

[Luke]

Is this good?

HDR2# dd if=/dev/zero of=/dev/sda bs=1M
^T
dd: writing '/dev/sda': No space left on device
953871+0 records in
953869+1 records out

It hasn't echoed the command prompt yet.

 

[af123]

Yes, looks good for a 1TB drive.

 

[Luke]

Thanks for all the assistance.
It is now formatted, has a clean run of fixdisk and running smoothly.

 

[af123]

With fewer inodes, there is a risk of having too few inodes though thus is unlikely since the recordings tend to be quite large.

I don't think that's anything to worry about. My 4TB GPT disk is about a third full and using a tiny fraction of the available inodes:

       30772 inodes used (0.81%, out of 3804288)
        1852 non-contiguous files (6.0%)
          32 non-contiguous directories (0.1%)
             # of inodes with ind/dind/tind blocks: 4313/1947/16
   324373833 blocks used (33.31%, out of 973870801)
           0 bad blocks
          95 large files

       26679 regular files
        2299 directories
          86 character device files
         118 block device files
           0 fifos
           2 links
        1581 symbolic links (1579 fast symbolic links)
           0 sockets
------------
       30765 files

 

[MontysEvilTwin]

I don't think that's anything to worry about. My 4TB GPT disk is about a third full and using a tiny fraction of the available inodes:

       30772 inodes used (0.81%, out of 3804288)
        1852 non-contiguous files (6.0%)
          32 non-contiguous directories (0.1%)
             # of inodes with ind/dind/tind blocks: 4313/1947/16
   324373833 blocks used (33.31%, out of 973870801)
           0 bad blocks
          95 large files

       26679 regular files
        2299 directories
          86 character device files
         118 block device files
           0 fifos
           2 links
        1581 symbolic links (1579 fast symbolic links)
           0 sockets
------------
       30765 files

That is good to know. So with the 'largefile' option, and a full disk, only about 3% of the available inodes would be used. As I have just ordered a 4TB PVR drive, I will be using the drive it will replace in another machine. Is it OK to reformat this drive (2TB) with the GPTF function available in maintenance mode? If it is better to keep this drive as MBR, can the main partition be reformatted with the 'largefile' option easily from the command line? As a suggestion, if there were a wider demand for this, could the GPTF function be modified to reformat a disk with MBR, and use the 'largefile' option for the main partition?

 

[af123]

So with the 'largefile' option, and a full disk, only about 3% of the available inodes would be used.

With my recording pattern, yep. Lots of short children's programmes though so I don't think there's any danger of running out of inodes.

Is it OK to reformat this drive (2TB) with the GPTF function available in maintenance mode?

It's fine - the only downside is the inability to revert to the stock firmware or stock kernel for troubleshooting, but the 'Safe Mode' introduced with CFW 3.10 was exactly for this - it boots without starting up the CFW components.

If it is better to keep this drive as MBR, can the main partition be reformatted with the 'largefile' option easily from the command line?

Yes, exactly what I did when I installed my original 2TB - see https://wiki.hummy.tv/wiki/2TB_Disk_Installation_Blog
You could format from the menus then log into maintenance mode and just:

humax# umount /dev/sda2
humax# mkfs.ext3 -m 0 -O sparse_super -T largefile /dev/sda2

As a suggestion, if there were a wider demand for this, could the GPTF function be modified to reformat a disk with MBR, and use the 'largefile' option for the main partition?

Could be done - as you can see it's pretty straightforward.