Problems copying to USB disk

alexp

Member
I'm trialling a disk backup in anticipation of hard disk exchange for two aging HDR-Fox T2's.
First, is the fastest transfer time via USB2 rather than LAN, or otherwise? I'm using a Freecom toughdrive.

I installed "auto-unprotect" and "NTFS 3G". Only 4 recordings on my test machine, two are HD, ranging from 1GB to 3GB (30 to 100 minutes). The USB drive was partitioned into a FAT32 and a NTFS. I copied all files to the NTFS partition via the OPT+. The "copying..." message was still there this morning (9 hours later) so I powered it down. Two files copied OK, the third only partly copied, no information on screen. Hmmm? Any ideas?
 

Black Hole

May contain traces of nut
I recently wanted to clear some space on one HDR-FOX and set up a block copy to a powered 1TB drive (NTFS). The first attempt failed miserably, but that might have had something to do with the HDR shutting itself down in the middle. Second attempt successfully transferred over 80GB in less than 8 hours.

In theory, the USB2 interface is capable of 480Mbps, while the absolute maximum data rate for the Ethernet is 100Mbps (and actually a great deal less than that when overheads are taken into account). Transferring over Ethernet will also impose some overhead at the PC end.

The USB probably isn't the limiting factor for the HDR-FOX, it doesn't have a terribly fast processor. 10GB/h translates to 80,000Mb/h, or 20Mbps - less than a twentieth of its theoretical capability! I will update this post with more accurate figures when I check how long the transfer actually took.

I recommend you try again, having tidied up the drive on a PC (run chkdsk /f from a command line to make sure the file system is clean).
 
OP
A

alexp

Member
OK.

Also, should I turn off the auto shutdown (looking at another post)?
In my case it's transferred about 6GB - so at a rate of 2GB/hour for 3 hours. That sounds a bit slow.
 

Black Hole

May contain traces of nut
My block transfer was actually 87,111,870,688 bytes in 317 minutes, which is 35Mibps (real megabits, not decimal megabits!). The most useful expression of this statistic for estimation purposes is 15GiB/h.

For comparison, I kicked off a network transfer using a file manager in Win7 to copy from HDR-FOX to the PC internal HDD, the PC connected to the router via WiFi and the HDR connected to the router by HomePlug. I aborted it, but it was averaging about 1.25MB/s (10Mbps) or 4.4GiB/h.

Finally I wired a direct link from PC to HDR-FOX, and using FTP shifted 10.5MB/s (84Mbps). That's about 37GiB/h.

The conclusion is that the HDR-FOX is capable of using its Ethernet interface far more efficiently than the USB interface, unless perhaps my external HDD is slugging it. However, that said, the transfer traffic does not usually get exclusive use of the network with no collisions, retries, etc. If one is desperate to transfer a large amount of data in the shortest possible time, setting up a point-to-point wired Ethernet connection is the way to go.
 

johnb

Member
Finally I wired a direct link from PC to HDR-FOX, and using FTP shifted 10.5MB/s (84Mbps). That's about 37GiB/h.

The conclusion is that the HDR-FOX is capable of using its Ethernet interface far more efficiently than the USB interface, unless perhaps my external HDD is slugging it. However, that said, the transfer traffic does not usually get exclusive use of the network with no collisions, retries, etc. If one is desperate to transfer a large amount of data in the shortest possible time, setting up a point-to-point wired Ethernet connection is the way to go.

That is around the speed I get when using rsync to mirror the Hummy HDD to my Linux (Ubuntu) Server. My setup has rsync and Samba or NFS (which depending on preference and OS) installed on the Hummy then I run rsync on the Ubuntu Server to mirror the shared Humax drive.
 
OP
A

alexp

Member
OK, test results:

I checked the disk with chkdsk /f. I also did a chkdsk /r just for good measure. No problems. I had only just created the partition anyway.
I turned off the 3 hour "automatic power down".
I set off a copy to USB of 5 files, total 10.5GB, overnight.
This worked fine !

I deleted all files on the FOX and then copied everything back from USB. All fine.

I tested out the speed by copying just one 2.94GiB file off and then restoring it:
USB->Fox took 3:40,
Fox->USB took 9:50

This works out to:

USB->FoxT2: 48 GiB/hour, or a full 500GiB disk in 10.4 hours.
FoxT2->USB: 18 GiB/hour, or a full 500GiB disk in 27.8 hours. Gulp

So why so slow to copy off ????????

I also tried going to standby after kicking off another copy back of all 5 files to FOX, and it completed it.
So does the"automatic power down" matter ????????

The crash I had previously remains unexplained. At a rate of 6GiB/hr it means it would have crashed after about an hour, whereas the automatic power down is at 3 hours. And the disk tested out without any problem. So, a mystery.

I could not try out my other USB disk, as it's a 3TB.
 
Last edited:
OP
A

alexp

Member
Hmmm.

That'll require me finding another disk to try.

In the meantime, I did get my maths wrong - the write rate is actually 18GiB/s, so it's a mere day to do a backup.

And I've just tried an ftp (filezilla) to exactly the same disk, connected to PC this time, and this is through the router (no other traffic), not a direct connection. This was a copy of all 5 files (12.5 GiB) in 19 minutes, which gives a rate of 39.5 GiB/hour. So that actually supports the case for the disk.

I can't do a ftp restore the other way till later, when I can watch the progress and time it...
 
OP
A

alexp

Member
The transfer back to Fox took longer, 22:18 compared with 19:00.
To double check I ran it again, with no other LAN traffic, Kaspersky AV off, Windows firewall and defender off, filezilla process priority set to to high. Same result.

So that's 39.5 GB/hr from the Fox and 33.6 GB/hr back to Fox for the FTP method.
Curious, as the PVR USB port method ran to 18 GB/hr and 48 GB/hr respectively.

Anyway, to compare the two methods for a non-power domestic user:

FTP:
Requires arranging for wired connections for PVR and PC to the router, rather than WiFi.
Any large disk already connected to PC USB port can be used without disrupting wiring.
Easier to track progress.

PVR USB port:
No dependency on LAN wiring.
Requires disk not greater than 1000GB taken to the PVR.
Requires installation of package NTFS 3G (assuming we want to do big backups, so not FAT32 disks).

I've already got a twin RJ45 socket extension at my PVR location hard wired to near my router & PCs, so FTP wins for me!
 

Black Hole

May contain traces of nut
Requires installation of package NTFS 3G (assuming we want to do big backups, so not FAT32 disks).
Not necessarily - see Things Every... (click) section 12.

FTP/SMB/NFS doesn't need a wired connection, but as my links are Powerline there is a considerable speed improvement by bypassing them with Ethernet. WiFi can be very fast, according to circumstances.
 
OP
A

alexp

Member
No it doesn't.

What's the significance of 1000GB?


I connected a 3TB disk to the PVR but it didn't recognise it. I assumed it was behaving as it would for the internal disk limit. So why did it not recognise it?
 
Last edited:
OP
A

alexp

Member
Further to my speed tests above, I also tried out a round-trip copy using a FAT32 partition.
This was using the same recording and same USB disk, with both FAT32 and NTFS partitions.

The overall results are:

Fox -> USB (NTFS): 18 GB/hour
USB (NTFS) -> Fox: 48 GB/hour
Either way (FAT32): 62 GB/hour

As well as the lower speed, I also had a crash the first time I tried Fox -> USB (NTFS) : see original post.
Is this connected to the NTFS 3G package?
I couldn't find a log for this that might help.
 

Black Hole

May contain traces of nut
Either way (FAT32): 62 GB/hour
Wow! That's interesting. It would be worth trying Ext3.

It does look like the custom extension for NTFS write is slugging the transfer rate, NTFS read is available in standard firmware but I don't know whether the extension ties itself into that as well.
 
Top