[virtual-disk2] Virtual USB drive

[af123]

I have just published a new package in the repository called virtual-disk2.

This package contains files built and supplied by raydon and is based on work that adrianf26 has been doing on the Foxsat platform - thanks guys. I made some changes before publishing so any bugs in there were probably introduced by me (!)

I don't know how many people still use virtual-disk in earnest on the HDR, but one annoyance of that package is that is requires a physical USB disk to be plugged in before the virtual disk is visible in the interface as a copy destination.

This new package fixes that by creating a virtual USB drive that the Humax treats in exactly the same was as a real one, and then it overlays part of the hard disk on top of that so that anything copied to the virtual disk is actually being written to a directory on the same hard disk. The point is that the Humax thinks it is copying the content to an external disk and therefore will decrypt it on the way if conditions are right*.

The old package will remain in the repository so you can freely switch between virtual-disk and virtual-disk2. The contents of the virtual drive will be the same with either package installed as they use the same area of the hard disk to store files.

The package also works properly on the HD but is less useful there since there is generally a USB drive connected.

* - There are, of course, easier ways of obtaining decrypted content nowadays but the virtual-disk package is still fairly routinely downloaded and this is a nice evolution of it.

 

[Black Hole]

Can these techniques be used to make virtual network mounts visible without a real USB? Perhaps as simple as installing virtual-disk2 (although it would be handy if it were built into the relevant package)?

 

[af123]

Yes, with this package installed, the Humax thinks it has a real USB device inserted.

 

[Black Hole]

I don't need to use it, but it will be useful for advising others.

 

[4291]

Is there any benefit to installing vitual-disk2 on an HDFox? If not should it be marked HDR only in Package Management?

 

[Ezra Pound]

The point is that the Humax thinks it is copying the content to an external disk and therefore will decrypt it on the way if conditions are right*.

I presume that, as with the original Virtual-Disk, one of the conditions is that the 'copy' MUST be via the Remote Control (or ir) rather than a Web-If copy using a Telnet cp command

 

[Seabee]

I think I may have found a problem with virtual-disk2. Having installed it I can't modify the [mod-settings] remote shares with Webif because they appear locked for updates. Thus a new one can not be created because the template can't be updated.

Uninstalling virtual-disk2 allows me to manipulate the [mod-settings] folder with the remote but then the IP addresses don't appear to translate from "_" to "."

Reverting to virtual-disk1.1 doesn't help. Removing virtual-disk altogether doesn't help.

I have been rebooting between changes.

Have I missed something?

 

[JimD]

I am experiencing the same problem as Seabee and have tried similar things without success.

 

[Climber]

I only installed virtual-disk2 (thanks for that package) the other day and initially I could not get it to work. That problem was eventually overcome as it was a firewall issue.

One thing I did notice afterwards was that I could not change network-share passwords etc via the webif/rename.
I eventually managed to do this with Filezilla, so it does look like there is a problem somewhere.
At least I’ve managed to find a way of overcoming it meantime.

 

[JimD]

I am still unable to use network share auto mount for the above reasons. I have tried installing Filezilla but am unable to establish a connection to my HDR as I am not sure what user and password to use. This however would be a work round, not a solution.

 

[Climber]

Until a solution is posted, you can try Filezilla - USER=root PASSWORD=0000 (that is zero's)

 

[af123]

The standard username is humaxftp. Root works if you have the betaftpd package installed but gives access to the whole filesystem, whereas logging in as another username just shows you the media.

 

[Black Hole]

I have not actually tried it, but it is reported that virtual-disk2 elicits the USB icons on-screen at boot time. It is also reported that slow-to-start real USB drives do too (otherwise real drives connected at boot time do not, only drives plugged in after boot).

I suppose ir could inject an "Exit" soon after, but is there no way to counter this directly - make virtual-disk2 start before humaxtv, for example?

 

[xyz321]

I suppose ir could inject an "Exit" soon after, but is there no way to counter this directly - make virtual-disk2 start before humaxtv, for example?

It is not a big problem since the icons do disappear after 30 seconds anyway.

 

[Eric]

@af123

Hi, i don't know if you're aware but you can't copy the (decrypted) recording from the Virtual-USB onto a real usb-hdd for archive purposes.

I made the mistake a few years back of using the Virtual-drive to decrypt all my recordings as i didn't have a real usb-drive at the time, and the humax box (i thought) was on it's last legs.

Thankfully the hummy is still working but i'm left with approx 300GB of recordings that are stuck!

I could copy them back to the real internal drive: 'Media-Video' but it seems a painful way of doing this just so i can then copy off to a real usb-drive for long term archiving.

Do you know why i can't copy to a real usb-drive, and secondly is there anything that can be done to correct this minor flaw?

Many thanks
.

 

[Eric]

@@af123

Sorry to ask you another question but it relates the the Virtual-USB again...

...a little background... i've learnt the hard way when it comes to detaching a real external usb-hdd connected to my hummy, in that once upon a time i simply unplugged it before going to bed. I did this a few times until one day when i reconnected it there was nothing but a blank screen (indicating the hummy couldn't 'see' anything).

I used all sorts of tools to 'see' the contents, even using windows recovery software, which actually made everything worse as it relabeled everything as 1, 2, 3..... etc - which meant i couldn't tell which recording was which!

Anyway, now i always without any hesitation use the proper eject method in the Hidden Settings Menu (as per the wiki). One drawback of this method is that the hummy looses sight of the Virtual-USB also, meaning i can't connect another external usb-drive until the hummy has powered right down, ie. the hard drive clicks and stops spinning.

I'm constantly powering down the hummy which may or may not affect the unit through excessive power-cycles, but mainly it's really starting to piss me off! - especially when i have to wait hours at a time until i've finished recording for that period of the day and power down again, just to swap an external drive over to copy specific recordings onto it from the hummy!

(nb. i only have one usb-port as the front is faulty).
(nb. i have different hdd's for different topics: Movies hdd, Sport hdd, Science/History hdd etc..)
(nb. all my hdd are larger than 4TB so i use the special GPT table, hence i must have the Virtual-USB to make my hdd's visible)

Why can't there by a re-attach option (button), perhaps in the webif, similar to the 'eject' button that will allow me to swap usb-hhd's more easily? Or better still, have a solution where the Virtual-USB ignores the proper ejection method, allowing me to safely swap my hdd's over, without having to power down each time.

Thanks for listening
.

 

[MontysEvilTwin]

@af123

Hi, i don't know if you're aware but you can't copy the (decrypted) recording from the Virtual-USB onto a real usb-hdd for archive purposes.

I made the mistake a few years back of using the Virtual-drive to decrypt all my recordings as i didn't have a real usb-drive at the time, and the humax box (i thought) was on it's last legs.

Thankfully the hummy is still working but i'm left with approx 300GB of recordings that are stuck!

I could copy them back to the real internal drive: 'Media-Video' but it seems a painful way of doing this just so i can then copy off to a real usb-drive for long term archiving.

Do you know why i can't copy to a real usb-drive, and secondly is there anything that can be done to correct this minor flaw?

Many thanks
.

In Webif, if you go to 'Browse Media Files', click on the first '/' in the directory path then click on 'mnt' then 'hd2' you should see a 'virtual_disk' directory listed. The recordings discussed in this post should be in here. If you 'select all' then 'cut' and go back up to 'hd2' you should see 'My Video' listed. Open 'My Video' and then paste the recordings here or in a subdirectory of your choosing. This will 'move' them from the virtual_disk folder. As the cut and paste operation is done within '/mnt/hd2' it does not have to move the recordings, it just updates the catalogue, probably taking about 30 seconds to complete.

 

[MontysEvilTwin]

@@af123

Sorry to ask you another question but it relates the the Virtual-USB again...

...a little background... i've learnt the hard way when it comes to detaching a real external usb-hdd connected to my hummy, in that once upon a time i simply unplugged it before going to bed. I did this a few times until one day when i reconnected it there was nothing but a blank screen (indicating the hummy couldn't 'see' anything).

I used all sorts of tools to 'see' the contents, even using windows recovery software, which actually made everything worse as it relabeled everything as 1, 2, 3..... etc - which meant i couldn't tell which recording was which!

Anyway, now i always without any hesitation use the proper eject method in the Hidden Settings Menu (as per the wiki). One drawback of this method is that the hummy looses sight of the Virtual-USB also, meaning i can't connect another external usb-drive until the hummy has powered right down, ie. the hard drive clicks and stops spinning.

I'm constantly powering down the hummy which may or may not affect the unit through excessive power-cycles, but mainly it's really starting to piss me off! - especially when i have to wait hours at a time until i've finished recording for that period of the day and power down again, just to swap an external drive over to copy specific recordings onto it from the hummy!

(nb. i only have one usb-port as the front is faulty).
(nb. i have different hdd's for different topics: Movies hdd, Sport hdd, Science/History hdd etc..)
(nb. all my hdd are larger than 4TB so i use the special GPT table, hence i must have the Virtual-USB to make my hdd's visible)

Why can't there by a re-attach option (button), perhaps in the webif, similar to the 'eject' button that will allow me to swap usb-hhd's more easily? Or better still, have a solution where the Virtual-USB ignores the proper ejection method, allowing me to safely swap my hdd's over, without having to power down each time.

Thanks for listening
.

I don't have the hardware to mimic your situation exactly. All individual drives connected are shown when you press the eject button in Webif (including the virtual disk) and each drive has its own eject button in the pop-up box that appears. If you eject a physical drive does it also eject the virtual disk? If so this is a bug. If you do eject the virtual drive, pressing the eject button again gives a pop-up dialogue which gives an option to 'rescan sda'. If you do this the virtual drive is remounted. If these options don't work for you have you tried rebooting from Webif > Diagnostics? This will cause all drives to be remounted without spinning down the hard drive and putting the unit into standby.
Have you removed the front panel of the HDR-FOX in the past? Are you familiar with how to do this (there is a guide in the wiki)? I have a few spare front USB boards. If you PM me I'll send you one, if you like.

 

[/df]

Here's a strange by-product of the virtual-disk2 package. I expected the df command to resolve the device on which the filesystem containing a particular file was mounted, as usual. But no ...

This is an HDR with the virtual disk /dev/sda1 mounted on /media/drive1, backed by the Video drive /dev/sdb2 mounted on /mnt/hd2.

# df `readlink -f /mod/etc/init.d`
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1            1941367728 707377764 1233989964  36% /media/drive1
# df  /mnt/hd2
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1            1941367728 707377764 1233989964  36% /media/drive1
# df /dev/sdb2
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1            1941367728 707377764 1233989964  36% /media/drive1
# stat /dev/sdb2
  File: /dev/sdb2
  Size: 0               Blocks: 0          IO Block: 4096   block special file
Device: 1f00h/7936d     Inode: 1774        Links: 1     Device type: 8,12
Access: (0640/brw-r-----)  Uid: (    0/    root)   Gid: (    0/    root)
...
# stat /dev/sda1
  File: /dev/sda1
  Size: 0               Blocks: 0          IO Block: 4096   block special file
Device: 1f00h/7936d     Inode: 1778        Links: 1     Device type: 8,1
Access: (0640/brw-r-----)  Uid: (    0/    root)   Gid: (    0/    root)
...
#

Apparently Busybox df gets confused by the existence of another device with the same ID (7936). The coreutils version has some additional device filtering that might avoid the issue, but it's not included in the repo coreutils package.

Is it necessary for the virtual device to have the same ID as that of its backing disk?

 

[/df]

virtual-disk2: Startup script now disables this package when installed on the HD.

Is the disabling supposed to be overridden by the start action in /mod/etc/init.d/S89virtual-disk? At ll.21-2, it tests whether the drivers were loaded (disabled at xinit time) and then apparently loads them anyway.
Perhaps it should have s.t like this:

...        start)
        if [ -f /mod/boot/2/vdisk/.disable ]; then 
        	echo "Virtual Disk - disabled"
        else	
    	(
...
    	) &
    	fi

Also, the test code uses pgrep, which requires a dependency on procps, or adding a definition adequate for this case:

type pgrep >/dev/null ||
	pgrep() { 
		local x
		[ -n "$1" ] ||  { cat >&2 << EOM; return 1; }
pgrep: No matching criteria specified
Usage: pgrep [PATTERN]
EOM
		x=$(ps | tail -n +2)
		echo $x | grep -E "$1" |
			sed -r 's/^[[:space:]]*([[:digit:]]+)[[:space:]].*$/\1/'
	}