brand new HDD but EXTENTS_FL errors

[MontysEvilTwin]

If the link above is correct it is a 48 bit field split into 16 bit seek errors plus 32 bit seeks. In this example case it would be 0x000000007BE3 which is 0 errors from 31715 seeks.

Fair enough, I am not good with hex annotation and my calculation was wrong. My point was that just looking at the raw number doesn't tell you what's going on, you need to convert to hex.

 

[MontysEvilTwin]

Neither am I, but I'm not going to panic about it.

You are the only one who mentioned panicking. So you knew that a seek error rate parameter equal to 31715 means 0 errors in 31715 seeks? Without understanding what the value of that parameter means how could anyone possibly know whether it is typical or atypical?

EDIT. From Wikipedia: Seek error rate. Vendor specific raw value. Rate of seek errors of the magnetic heads. If there is a partial failure in the mechanical positioning system, then seek errors will arise. Such a failure may be due to numerous factors, such as damage to a servo, or thermal widening of the hard disk. The raw value has different structure for different vendors and is often not meaningful as a decimal number.

 

[/df]

But before Ext4, those bit positions would have been unspecified so it is entirely possible Humax did what they wanted with them, or indeed the CF, without repercussions to an Ext3 system.
...

Any unspecified bits in the inode flags should have been treated as "reserved, must be 0".

The fault is probably caused by the version of e2fsck on the PC the OP used ...

That issue came down to a UI issue in e2fsck where the 'always answer yes' option broke the filesystem because the wrong question was asked ("Do you want to enable INLINE_DATA for the filesystem to match the INLINE_DATA flag in the inode?" instead of "Do you want to treat the INLINE_DATA flag in the inode in a filesystem without INLINE_DATA support as an error?").

A change was made in v1.44.5 to assume that the presence of the inline flag in an inode is an error if the required "system.data" extended attribute is not also present. I guess that, although the "wrong question" could still be asked, this change has made it much less likely.

The EXTENTS errors seen by OP are a different case as any version of e2fsck from 1.41 on should have produced the same result.

So I suggest that either the listed inodes aren't meant to be valid inodes (Ted Ts'o theory) or the Humax formatter or other software failed to zero the storage used to populate inodes.

Both INLINE_DATA and EXTENTS are INCOMPAT features: the kernel won't mount a filesystem with such a feature unless it supports the feature; e2fsck won't process a filesystem with any feature that it doesn't support. This is why automatically enabling INLINE_DATA was a Bad Thing.

 

[prpr]

To summarise: Unless it's a dire emergency, use mkfs.ext3 and e2fsck in the Humax, not externally.

 

[rodp]

To summarise: Unless it's a dire emergency, use mkfs.ext3 and e2fsck in the Humax, not externally.

nice summary - cheers

so..... and I'm wondering if this discussion or part of it could be summarised in the page about formatting the HDD and the compatibility issues that may prevail if you have problems with the HDD and take it out, plug it into a PC (running ubuntu 16.04 - other version may be relevant) in order to not just copy off the data but also to try and run e2fsck on which is a different version to what is on the humax?

so from this, the answer to my question is that due to enhancements of the ext format over time, newer version of teh e2fsck may upset the formatting of the Humax drive and if you may have problems running the Humax e2fsck on the Humax at a later date. (I'm guessing it may still be ok on a general day to day basis, reading and writing videos but it's the maintenance part that might be a problem should there be any bad sectors or other issues starting to develop).

...Just trying to turn all your replies and comments into a succinct summary and provides a clear impact on the end usage and maintenance of the unit so that everyone else in the future can understand.

Thanks

Rod

 

[/df]

Regarding the EXTENTS diagnostics under Ubuntu, all that happened was that your disk format which had somehow become inconsistent was corrected. All good apart from understandable anxiety. It isn't the case that any formatting was upset, just that the newer version of the filesystem checker identified problems that would not have been detected by the version in the CF.

Bear in mind that a non-CF HDR user has no way of checking the filesystems (without extracting the disk): if any partition becomes unmountable, the disk has to be reformatted. The facility to check filesystems in place was a big step forward for the CF.

When a CF version is created, the EXT2FS tools are built and installed in /usr/lib/ext2. The ext2fs library is linked from /usr/lib/ext2 and the tools are linked from /sbin. All these files and directories are baked into the read-only flash installed by the CF. The effect of this is that the Humax settop program (also baked) uses these versions, whether by calling into the library or by running e2fsck -p. These are also the versions used by fix-disk.

You can also install an e2fsprogs package, which installs a slightly fuller version of the tools in /mod/sbin (which is not normally on the PATH searched for executables). Currently these are built from the same source version (1.42.13). If this package were to be built from a later version, the baked-in versions would normally still be used, and the package versions would be used only when specifically invoked.

There is also an e2recover package that installs, under different names, version 1.43.4 of a subset of tools, to aid in recovering from the INLINE_DATA problem linked in post #2 and described in my previous post.

Should a future version of CF incorporate version 1.45.x or later of the EXT2FS tools (which might be possible since the INLINE_DATA problem appears to have been fixed), you would just get the same result from fix-disk as was seen on the Ubuntu system.

 

[rodp]

Thanks @/df,

@/df .... So actually, I did the new HDD a favour by checking it out with a later version of e2fsck?? Therefore.... I don't need to reformat the drive once reinstalled in the humax? (there has been quite a bit of indecisiveness of you do and you don't so please can you confirm?

If you say it's good to go and I don't reformat, if e2fsck in the humax if run on the drive I am guessing it won't know any wiser about the flags being cleared and it therefore won't get upset? I guess the definitive answer it to actually plug it back in, go into maintenance mode and run check disk? Or.... if it mounts is that enough to say all is well?

By the way, is running the disk check from within webif terminal whilst the humax is running in normal mode the same as doing it from maintenance mode? I am guessing the answer is no but worth checking (my experience is discs need to be unmounted in ubuntu to check).

I feel the end to this conversation is near and I can crack on and my anxiety levels will drop back to normal

Thanks

Rodp

 

[Black Hole]

if it mounts is that enough to say all is well?

For heaven's sake yes. Stick it in. If it needs formatting the system will say so. If it doesn't say so, be happy.

 

[/df]

... is running the disk check from within webif terminal whilst the humax is running in normal mode the same as doing it from maintenance mode? I am guessing the answer is no but worth checking (my experience is discs need to be unmounted in ubuntu to check).
...

Normally, the filesystems (HD: filesystem) can't be unmounted because the Humax settop program is running and using files therein, and this applies:

... in general it is not safe to run e2fsck on mounted filesystems. The only exception is if the -n option is specified, and -c, -l, or -L options are not specified. However, even if it is safe to do so, the results printed by e2fsck are not valid if the filesystem is mounted. If e2fsck asks whether or not you should check a filesystem which is mounted, the only correct answer is ``no''. Only experts who really know what they are doing should consider answering this question in any other way. ...

The options -c, -l, -L are for bad block management, and -n means 'always answer no'.

Maintenance mode offers a stable environment in which the system is running without the settop program running, where, in particular, hard disk filesystem(s) can be unmounted and manipulated as desired, similar to Mac's ⌘R Recovery mode.

 

[rodp]

Hi All, next step on this process... New HDD is now formatted as per format commands above and webif now installed. Now I've taken the HDD out and put it in my caddy to plug back into PC running ubuntu 16.04 to copy the files from my backup drive to the new one. The drive mounts fine but I can't copy anything over to it. I've tried to make a directory and it's not possible. I've tried freefilesync and it says permission denied when trying to copy over files. I used Terminal and did ls -al to find out the permissions. Please can you tell me if these permissions look ok or whether I need to change anything?

If I change permissions, is this temporary or permanent? Will that change the owner and therefore upset access for the humax when it's plugged back in?

How do I do this?

Many thanks

Rodp

 

[Black Hole]

You've run into ownership issues. I'm not sure, but I think if you elevate your privileges to root you will overcome the copy problem (see below). I realise you are doing it this way for speed of transfer, but you wouldn't have run into these problems if you had simply connected the backup drive to the HDR-FOX by USB and used the CF to do the transfer.

If I change permissions, is this temporary or permanent? Will that change the owner and therefore upset access for the humax when it's plugged back in?

I'm pretty sure the Humax configuration takes no notice of ownership. The Humax app (settop) runs as root.

With regard to root privileges, recent Ubuntus (and Ubuntu-derived Linuxes) have made it very difficult (if not impossible) to log in as root (for security). A "root" user is essentially a user with admin privileges rather than root privileges, and you can temporarily elevate privileges to root.

It's easiest on the command line, and executing graphical apps with root privilege is a bit of a lottery because they might not have been built to ensure ownerships don't go awry.

On the command line:

> sudo <command> executes that command with root privilege, and you will be prompted for the admin password if sudo has not been used in that session within the last few minutes.

> sudo -i can be run on a line on its own to give all subsequent commands root privilege until "exit".

("sudo" = Super User DO)

Having obtained root privilege, there are various switches for the cp command which will copy preserving ownership (look it up).

Graphically, I'm not familiar with Ubuntu, but I think you might be able to use the file manager to open a drive or folder with root access by right-clicking and selecting "open as root".

 

[/df]

In the relevant version of Ubuntu, it's recommended to use gksudo to run a graphical program, like the file sync tool, as root; later versions want pkexec. Otherwise you have to do additional work with sudo: just the command with no parameters other than the command line to be elevated isn't enough.

 

[MontysEvilTwin]

In Ubuntu 16.04, if you open a terminal window and send the command 'gksudo nautilus' it will open the usual file manager program but with root privileges.

 

[Black Hole]

Corrections acknowledged, I was illustrating general principles. Such are the problems of doing stuff outside the native environment!

 

[rodp]

Thanks, I'll give this a go. Surprised that I didn't find any any other threads about this issue with a solution as would have thought I wasn't the first to have to copy files back over to new HDD outside the box so that it's quicker AND controlled (ie use program like freefilesync that will report back any problems whilst copying), or did I not look far/long enough?! I think from this thread a new thread could be created listing all the watchouts and tips and solutions to setting up a new HDD.

Will report back shortly

Thanks

Rodp

 

[MartinLiddle]

Surprised that I didn't find any any other threads about this issue with a solution as would have thought I wasn't the first to have to copy files back over to new HDD outside the box so that it's quicker AND controlled (ie use program like freefilesync that will report back any problems whilst copying), or did I not look far/long enough?!

I think most people think that doing it on the Humax avoids all the problems you have found. It is slower to transfer but tends to work first time.

 

[rodp]

next problem, gksudo is no longer available it seems.. https://itsfoss.com/gksu-replacement-ubuntu/

 

[rodp]

aha! found the answer, just on terminal (in the freefilesync folder) I need to write: sudo ./FreeFileSync.

Woo hoo, 80MB/s now insteald of 10-15MB/s now!!

Thanks everyone for guidance

Rodp

 

[/df]

Isn't yours 16.04? That version (at least in Lubuntu) still has gksudo.

It's quite usual when mounting a Unix-y drive on a different Unix-y system to have to override conflicting permissions, given that permissions have global scope. So people wouldn't be likely to ask about it here.

 

[rodp]

This is what I'm running:

ubuntu@ubuntu:/media/ubuntu/509bdd16-1fa7-4cf0-a555-bb4501605ba3$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.2 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.2 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

This is a live disc so at the time I got it maybe it would have been possible to install it but not now. Anyhow, found a way round it.