Very Large External Drive - Seagate Backup Plus 8TB

[prpr]

I've got a 120GB SSD attached now by USB. I created a single partition with gdisk and formatted it ext3.
The Humax software sees it as "Available Size: 0.0GB", "Used Size: 0.0GB", "Total Size: 500.0GB" on the Data Storage page.
The latter is the size of the internal disk, so that's obviously yet another bug.
Everything looks normal on the command line:

humax ~ # fdisk -lu /dev/sdb

Disk /dev/sdb: 120.0 GB, 120034123776 bytes
256 heads, 63 sectors/track, 14536 cylinders, total 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes

  Device Boot  Start  End  Blocks  Id System
/dev/sdb1  1  234441647  117220823+ ee EFI GPT
humax ~ # gdisk -l /dev/sdb
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdb: 234441648 sectors, 111.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 8AFAE9FC-C567-46A3-A1CA-9D1000F15FA7
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 234441614
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)  End (sector)  Size  Code  Name
  1  2048  234441614  111.8 GiB  8300  Linux filesystem

I can mount the filesystem OK and "df -h" reports:

/dev/sdb1  111.8G  187.7M  111.6G  0% /media/drive1

and I can copy files to it on the command line or using the WebIf. It's just the stupid Humax software which doesn't want to use it.

 

[prpr]

However, installing the virtual-disk2 package does make the storage usable.
I had to mount the GPT disk manually though:

humax ~ # mkdir /media/drive2
humax ~ # mount /dev/sdc1 /media/drive2

And now the Humax UI sees the disk and can copy to/from and play from the storage.

The Data Storage page is still screwy, but in a different way. The figures are completely wrong on the virtual-disk (roughly double). The Total size of the GPT disk is right if you look at it after the virtual, but wrong if you look at it after the internal. Avail/Used are both still 0 in both cases.

 

[af123]

prpr, is with the new 3.10 kernel? Could you please run

sg_readcap -H /dev/sdc
sg_readcap -H --16 /dev/sdc​

(with sg3utils installed)

I'm interested to see if the kernel can properly handle the extended capacity enquiry (not needed for a drive that size but would be for larger ones).

rag.log should show what happened when the disk was connected. It should be possible to add automount support for this in the CFW, but like EXFAT another disk or virtual-disk2 will still be required for it show up in the on-TV interface.

 

[MontysEvilTwin]

However, installing the virtual-disk2 package does make the storage usable.
I had to mount the GPT disk manually though:

humax ~ # mkdir /media/drive2
humax ~ # mount /dev/sdc1 /media/drive2

And now the Humax UI sees the disk and can copy to/from and play from the storage.

The Data Storage page is still screwy, but in a different way. The figures are completely wrong on the virtual-disk (roughly double). The Total size of the GPT disk is right if you look at it after the virtual, but wrong if you look at it after the internal. Avail/Used are both still 0 in both cases.

What happens if you have a 'compatible' USB drive (e.g. FAT32, Ext2/3 or NTFS with MBR) already connected when you plug in the GPT disk? For example, if you add ExFAT support, the standard UI still won't display an ExFAT formatted drive unless you have either another suitable USB drive already connected or install the Virtual-disk2 package. I presume there is a process running which looks for specific types of USB device before making the USB option available in the Standard UI and that this process is independent of what is supported in the kernel.

 

[prpr]

prpr, is this with the new 3.10 kernel?

Of course .

Could you please run

sg_readcap -H /dev/sdc
sg_readcap -H --16 /dev/sdc​

It's sda at the moment, 'cos I crashed it and it restarted with it still plugged in:

humax ~ # sg_readcap -H /dev/sda
00  0d f9 4b af 00 00 02 00
humax ~ # sg_readcap -H --16 /dev/sda
read capacity (16): transport: Host_status=0x05 [DID_ABORT]
Driver_status=0x00 [DRIVER_OK, SUGGEST_OK]

READ CAPACITY (16) failed: Sense category: -1, try '-v' option for more information
humax ~ # sg_readcap -H --16 -v /dev/sda
  read capacity (16) cdb: 9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00
read capacity (16): transport: Host_status=0x05 [DID_ABORT]
Driver_status=0x00 [DRIVER_OK, SUGGEST_OK]

READ CAPACITY (16) failed: Sense category: -1

rag.log should show what happened when the disk was connected. It should be possible to add automount support for this in the CFW, but like EXFAT another disk or virtual-disk2 will still be required for it show up in the on-TV interface.

Here's the filtered (PID and uniq) log:

--------- Info for Modinit PID 627 ---------
627: Date:  Sun Feb 14 17:27:21 UTC 2016
627: MDEV:  sda1
627: ACTION:  add
627: Model:  HDR
627: Device:  /dev/sda1
627: Disk:  sda
627:14/02/2016 17:27: Waiting for disk to become ready...
627:14/02/2016 17:27:  disk ready after 0 seconds
627:14/02/2016 17:27: /dev/sda1 is formatted as ext2/3
627:14/02/2016 17:27: /dev/sda1 is NOT removable.
627:14/02/2016 17:27: Waiting for the disk to mount...
627:14/02/2016 17:27:  still waiting...
627:14/02/2016 17:28:  still waiting...
627:14/02/2016 17:28:  timeout, aborting.

 

[af123]

humax ~ # sg_readcap -H --16 /dev/sda
read capacity (16): transport: Host_status=0x05 [DID_ABORT]
Driver_status=0x00 [DRIVER_OK, SUGGEST_OK]

Hmm, if that doesn't work then it isn't going to cope with drives with lots of sectors. I wonder if that's a limitation of the USB to SATA interface?
I get the same behaviour with mine with claims "Support all 3.5 inch SATA I/II hard disk, up to 3TB or more" - however I'm supposed to interpret that!

 

[prpr]

Hmm, if that doesn't work then it isn't going to cope with drives with lots of sectors. I wonder if that's a limitation of the USB to SATA interface?
I get the same behaviour with mine with claims "Support all 3.5 inch SATA I/II hard disk, up to 3TB or more" - however I'm supposed to interpret that!

I tried the same disk and USB-SATA adapter on a proper computer (4.2 kernel) and got this:

# sg_readcap -H -v /dev/sde
  read capacity (10) cdb: 25 00 00 00 00 00 00 00 00 00
 00  0d f9 4b af 00 00 02 00   
# sg_readcap -H -v --16 /dev/sde
  read capacity (16) cdb: 9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00
 00  00 00 00 00 0d f9 4b af  00 00 02 00 00 00 00 00
 10  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00

so it would appear not to be the adapter. Correct?

 

[af123]

so it would appear not to be the adapter. Correct?

It shows that the adapter is definitely capable of responding to that command (or passing it through to the drive). I wonder why it isn't working with the Humax? Suspects are the Broadcom hardware shim, the USB controller, the kernel or it could even be the way that the USB/SATA bridge is being initialised. I think it's unlikely to be the kernel but I haven't looked into it yet.

I've been reading up on these USB/SATA bridge chips too and it appears that they generally start reporting a sector size of 4096 as soon as the attached drive is over 2TB, they then translate between the two when talking to the drive. Some chips have configurable settings but they are not usually exposed. That would explain why people have had some success attaching large drives via USB.

 

[prpr]

I've tried a 2TB spinning rust with a different powered USB-SATA adapter and that generates the same results on the Humax as the other combination. Haven't got anything bigger to try I'm afraid.

 

[af123]

I've tried a 2TB spinning rust with a different powered USB-SATA adapter and that generates the same results on the Humax as the other combination. Haven't got anything bigger to try I'm afraid.

I was was wrong, this was a kernel problem. Now working with my test kernel:

gpttest# sg_readcap -H /dev/sdb
00  e8 e0 88 af 00 00 02 00
gpttest# sg_readcap -H --16 /dev/sdb
00  00 00 00 00 e8 e0 88 af  00 00 02 00 00 03 00 00
10  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00

This fix also allows many other drive commands to work properly via USB, including security erase which is the one I wanted to try.

 

[prpr]

Good stuff. Will test next week.

 

[prpr]

I wonder if changing the Humax software's view of what's in the boot sector might make it happy with GPT disks and remove the need for virtual-disk2 (at least for extN filesystems)?
Could it be filtered such that if GPT is detected, a fake MBR could be constructed and returned?

 

[af123]

It was a wild shot anyway and I can always get a refund from Argos - 30 day money back guarantee!!! - I'll keep watching this Custom Firmware Forum incase any clever people manage to suss out the issues with these very large external drives.

I believe that the next version of the custom firmware will support this drive formatted as GPT/EXT3.
I plan to get a first release candidate built in the next week or two if you still have the disk and are interested in trying it.

 

[Des in London]

.
(from a different/related thread....)

....I used Gparted on my PC and formatted a 2TB usb-hdd to 'GPT Ext3'.... I then connected it to my hdr-fox t2, but alas the hummy could not see it

(from this thread....)

I was wrong, this was a kernel problem. Now working with my test kernel

...does your own Humax actually "see" the connected usb-hdd (formatted as gpt/ext3) in the 'data storage' now? And are you able to copy recordings to and from it also?

....Perhaps in the future, the same 'on the box' formatting of the internal drive (by using a protective MBR), which allows the use of very large hard drives internally, can be used to format very large external drives too..........??

....out of curiosity, is this how you achieved it?

 

[af123]

.
...does your own Humax actually "see" the connected usb-hdd (formatted as gpt/ext3) in the 'data storage' now? And are you able to copy recordings to and from it also?

Yes, as long as there is another 'normal' drive connected too (which can be simulated with the virtual-disk2 package).

....out of curiosity, is this how you achieved it?

It required both driver and filesystem changes, but a protective MBR is always required for a GPT drive so I don't fully understand your question.

 

[Des in London]

Oh ok, i didn't know a protective MBR is always required for a GPT drive (i assume it's to make it backward compatible?) - i thought it was just a 'make-shift' solution that you created and used for the topic "Installing a Drive Larger Than 2TB - Possible" (for internal use), which i was keen on. As i said i was just curious to know

Would you suggest that we all format our external-usb drives (to gpt/ext3) in the next CFW build (even if under 2TB)? Or is it best left to those with very large drives such as this topic (8TB). Thanks.

 

[af123]

It's really to stop non-GPT-aware tools damaging a GPT drive. They see the disk as being filled with a partition type they don't understand.
I would only use GPT for drives over 2TB.

 

[Des in London]

....to stop non-GPT-aware tools damaging a GPT drive

Ahh i see, hence it being called 'protective'.
(every day is a learning day!)

In your expert opinion, are there any drawbacks in having a disk protected like this generally speaking, (as apposed to a 'non-protected' or 'full GPT' drive), such as for the purpose of connecting it to my Windows machine also - will it be 'seen' or will my drive be restricted to Humax usage only? Thanks as ever.

 

[Christian]

I believe that the next version of the custom firmware will support this drive formatted as GPT/EXT3.
I plan to get a first release candidate built in the next week or two if you still have the disk and are interested in trying it.

I'm so very sorry to you for not replying sooner. I did say I would keep an eye out on this forum but I was moving house so that took a priority. Yes I do still have the Backup Plus (I never did get round to returning it back to Argos) and I'm extremely grateful for all your help.

I'm really interested in trying it if you still have it available, or am I too late now?

 

[af123]

I'm really interested in trying it if you still have it available, or am I too late now?

No, I'll send you a link to a copy.