Moving Recordings between two Humax Machines

[Robobunny]

I have a more graphical alternative (assuming you also have a Windows PC on the same LAN). A piece of free software (ftpuse) allows you to mount an FTP server as a Windows drive letter.

1. Decrypt recordings as mentioned above, and install and start betaftpd on the Humax (see guidance above).
2. Install "ftpuse" from https://www.ferrobackup.com/download.html#ftpuse
3. Map a different drive letter to each Humax, e.g.

C:\Program Files\Ferro Software\FtpUse> ftpuse Z: 192.168.1.45 0000 /USER:humaxftp

(maps drive Z:, my PIN is the default "0000" and my Huxax IP is 192.168.1.45)

I only have one Humax so tested it with a separate FTP server I run, mounted to a different drive letter. Now, at this point, I expected to be able to drag and drop between the two, but it gave an error "not enough disk space" (which isn't true). Your mileage may vary.

I already use FileZilla FTP client to move files on/off my Humax - have done for years when Windows stopped wanting to support CIFS - however, it does not support server to server transfer, but what it does support is dragging files/directories between the FileZilla client window, to Windows explorer (this is how I use it). So I got it working as...:

4. Install FileZilla FTP client, open a session to your source Humax. I recommend limiting the number of simultaneous transfers to one.
5. From FileZilla, drag the files you want to the Windows drive letter & path of your target Humax (mapped in step 3 above) - Note: I was also able to drag and drop in the other direction as well. Good Luck!

 

[prpr]

On the client Humax you should be able to specify the location of the file containing the private key using the -i option to ssh or in the options to sftp.

It doesn't work. And it's /mod/.ssh/ not /mod/etc/.ssh/

sftp -o IdentityFile=/mod/.ssh/id_blah a.b.c.d
ssh: Ignoring unknown configuration option 'IdentityFile=/mod/.ssh/id_blah'

sftp -o "-i /mod/.ssh/id_blah" a.b.c.d
ssh: Ignoring unknown configuration option '-i /mod/.ssh/id_blah'

 

[prpr]

If it is the problem, do we have any access to the pre-compiled version of the Humax dropbear-ssh package, or any other way of getting an updated version?

I'm not yet convinced it is the problem. You could try the 2020.79 version in the Beta repository, but it doesn't work for me, so I guess it won't for you.
I was working on 2022.82, but things are rather stalled at the moment.

 

[Black Hole]

I have a more graphical alternative (assuming you also have a Windows PC on the same LAN). A piece of free software (ftpuse) allows you to mount an FTP server as a Windows drive letter.

Interesting. Personally I use DOpus (not free) as a replacement for Windows Explorer, and can access FTP servers from it as network drives without mapping them to a drive letter.

 

[escat]

Just to level set on the SSH/SFTP dropbear-ssh/greenend-sftp problem: perhaps prpr and XYZ321 could tell me if I'm on the right lines. My understanding is:

The SFTP extension to dropbear needs to have access to the location of the private key. The problem appears to be that it currently does not have such access. I can understand three different ways in which this might be achieved:

a) an explicit option on the sftp command, as above; my understanding is that this doesn't currently exist, and apparently some doubt as to whether it could be made to work;
b) some sort of config file, accessible to the sftp command, to indicate where the key is;
c) a fixed default location from which the sftp command can always obtain the key. My reading of the 'fix' in dropbear 2022.82 is that this is the proposed workround for the problem.

Have I got this right? This whole subject is still new to me. Thanks for all your help so far.

 

[prpr]

It sounds about right, although I'm no expert.
I tried 2022.82 and it does appear to authenticate properly, which is good, but it dies a bit later in the process.

 

[escat]

I'm not yet convinced it is the problem. You could try the 2020.79 version in the Beta repository, but it doesn't work for me, so I guess it won't for you.
I was working on 2022.82, but things are rather stalled at the moment.

I tried 2022.82 and it does appear to authenticate properly, which is good, but it dies a bit later in the process.

I've tried the 2020.79 Beta but, as you say, that doesn't work either. Is there any way that I can get access to the source code for the CF dropbear-ssh and greenend-sftp packages to see if I can figure out where the problem lies? Thanks for your help on this.

 

[escat]

My main interest has been trying to get the SSH/SFTP mechanism working, as above. However, I did invite other suggestions, and have found them very helpful. Briefly:

If this is a one-off, honestly it's quicker to use sneakernet (walking it across with a USB drive) than faff around trying to set up (reliable) electronic transfer.

I certainly agree and have used 'sneakernet' for years - especially as most of my recordings are already on external USB drives, and are already decrypted. As for 'faffing around' - read on.

Yes, betaftpd on one side, and tnftp (ftp) on the other.

This is a great combination, especially as it allows one to work outside the /media directory. It works straight out of the box. Although it hadn't been originally part of my thinking, it makes it easy to copy user-written code from one machine to another.

wget can certainly be used to make a copy directly from one machine to another. I have tried it using the ftp protocol. I had to create an extra symbolic link root under media to get access to files outside "My Video" etc

Indirect copy via a third system certainly requires much less faffing around; the choice of tools and techniques is almost limitless. Much depends on the speed and reliability of the network, of course. [I'm glad I'm not the only one blessed with a "slow and flaky home network" (but let's not digress)]. I ran timings for a variety of configurations copying a .ts file of 739MB. In summary, direct (betftpd/tnftp) connection was between two and three times faster than download/upload via Filezilla. YMMV.

I also ran performance tests with and without SSH. This was limited to Humax-PC performance, since (as above) I haven't yet got direct Humax-Humax ssh/sftp working. The comparisons are with Filezilla running over either FTP or SFTP. Again., in summary, unencrypted plain FTP was between 2 and 3 times faster than encrypted SFTP. Again YMMV.

FWIW, I can't get the greenend client to do anything useful at all, and there seems to be s*d all documentation.

Quite. In addition to the problem with key handling by SFTP (above), I have encountered other problems with Filezilla's use of the SFTP client. Not only does it display incorrect file sizes (reported on Git-Hub), but I've found it relatively unreliable. Or is that my network?

Thanks again for all the suggestions.

 

[escat]

betaftpd is the CF alternative to the standard built-in FTP server

betaftpd doesn't seem to show hidden (.) files, whereas the built-in server does - at least that's my experience using Filezilla.

 

[MymsMan]

betaftpd doesn't seem to show hidden (.) files, whereas the built-in server does - at least that's my experience using Filezilla.

See https://kinsta.com/knowledgebase/filezilla-show-hidden-files/

 

[escat]

See https://kinsta.com/knowledgebase/filezilla-show-hidden-files/

Yes, that was my first thought. But just setting Force showing hidden files doesn't actually change anything in this case. I think the problem may actually be in the relationship between Filezilla and betaftpd. To pick one odd symptom: if I use Filezilla (with the betaftpd server) to copy a directory containing a hidden file, I can see the file appear in the target directory - until the full copy has been completed. Then it disappears from view.

 

[MymsMan]

I am not a frequent Filezilla user so had never noticed this feechur!
The problem is not in filezilla since ftp command line shows the same.

ftp> cd "My Video"
250 CWD successful.
ftp> ls Countdown
200 PORT command OK.
150 Opening ASCII mode data connection for directory listing.
Countdown_20220708_1410.hmt
Countdown_20220708_1410.nts
Countdown_20220708_1410.thm
Countdown_20220708_1410.ts
226 Transfer complete.
ftp: 118 bytes received in 0.01Seconds 9.08Kbytes/sec.

compared with actual directory

 ls
total 862888
drwxr-xr-x  2 root root     12288 Jul  8 15:00 .
drwxr-xr-x 31 root root     32768 Jul  8 19:59 ..
-rw-r--r--  1 root root      1308 Mar 10  2020 .autodetectads
-rw-r--r--  1 root root       276 Jul  8 14:10 .series
-rw-rw-rw-  1 root root       170 Aug 13  2018 .sweeper
-rw-r--r--  2 root root      2072 Jul  8 21:10 Countdown_20220708_1410.hmt
-rw-r--r--  3 root root   2377856 Jul  8 15:00 Countdown_20220708_1410.nts
-rw-rw-rw-  2 root root     43681 Jul  8 14:13 Countdown_20220708_1410.thm
-rw-rw-rw-  2 root root 880242688 Jul  8 14:59 Countdown_20220708_1410.ts

I couldn't find much info about betaftpd on via google and it doesn't appear to have been updated since 2000

 

[/df]

Arguably not showing .files is a bug. Once upon a time FTP servers would respond transparently to ls options passed by the client, so it was reasonable, if not standard, to make the response behave like ls. RFC 959 4.1.3 doesn't mention hidden files. Even if LIST might return a list as a user would see it, the NLIST command ought to include the raw file list; where the server filesystem has a hidden attribute rather than using initial ., possibly also the STATUS command should be used to determine the hidden status of each file.

RFC 3659 introduced a new list command using facts (capabilities)and mentions X.hidden as an example possible fact.

If we ever build betaftpd again, the .file processing should be changed, so that it behaves like NFS .

 

[MymsMan]

While not showing/transferring hidden files is a nuisance is it a critical problem?

None of the folder flag files are crucial to viewing recording on another box.

Also not sure why all of this focus on FTP - with network file shares hidden files do work correctly, you can play recordings located on a remote file system without need to do a copy and move/copy operations can be initiated using webif or using remote control and SUI

 

[escat]

To draw this to a close, but to record the probable cause of the underlying problems in case anyone else ever encounters them, I pass on the following comments from the developer of greenend SFTP.

1) Problems in Filezilla (and Mac Transmit) over SFTP are caused by their parsing the 'longname' field for attributes, rather than using the 'attrs' field.

2) the greenend SFTP client was never intended for 'production' use; it's just there to test the server.

As has been pointed out, there are other ways of achiving what we want. But thanks again for all your help