Much Gratitude and a few puzzles ...

[DelftBlue]

Much appreciation from this household for all the amazing customizing work.
The HDR is an impressive piece of kit - but you guys have doubled its usefulfulness by building so much additional and indispensible functionality AND so generously communicating it out to us.
Many Many Thanks.

Over time, I've hit a few minor quirks that have puzzled me, and forum searches haven't shed any light on them - so any insight gratefully received.
(I put them all in one post - but if they should be separate posts or appended to existing posts - please advise.)

a) Sometimes when using the 'Rename' function, further changes made to a Medialist Title and/or Synopsis don't stick - i.e. I go back make an additional change or correction, firmly click on update, it appears to go through but the item does not actually change.
It seems random, but I guess its not - is it a DLNA indexing issue, or is there a finite number of changes possible, or is it something else?

b) When using 'Shrink' from the OPT+ menu for individual files, after processing short files a panel appears detailing the action - and the work is clearly finished .... but on long files no info panel appears and it simultaneously says both 'Back to media list' and 'Please do not interrupt....'.
Its tempting to click on 'Back to media list' but looking at the files via Samba they are still changing size and it hasn't totally finished processing. Should the info panel appear eventually for all size of files? Is it safe to click on 'Back to media list' ... if not - (aside from monitoring via Samba) how to know when it's really finished and it's safe to move on?

c) My webif_auto.log sometimes shows info, but occasionally it says:
">>> Contents of /var/log/webif_auto.log 2.48 KiB Insufficient disk space, terminating."
('Clear' of the file makes no difference, and the HDR disk has at least 150GB available space.)

d) When using Web-IF 'Browse' to look at files and folders on USB drive attached to the HDR, OPT+ mostly brings up the full range of actions, which surprises me - and I'm trying to figure out which of them will work OK.
Are Web-IF OPT+ actions meant to work on files and folders on attached USB drives in a reliable way? (Q: is it preferred here to use the term 'directories' instead of 'folders'?) Does it make a difference if mvdisks is installed or not?
I may have it wrong, but my experiments seem to show that 'Rename' works (except for point a) above), 'Shrink' seems to work OK, 'Decrypt' is sometimes greyed out but is active with mvdisks installed, though does not always work (begins but leaves *.encrypted files behind) and folders on attached USB can be set to 'Auto' process, but do not activate.

e) Sometimes Auto-Decrypt and Auto-Shrink will process and sometime not. I am assuming it's something to do with available memory - but what are the exact rules?

f) I have a few files which will play fine on the HDR, but will not play on VLC whatever I do. They are marked with the icons for Decrypted and Shrunk, but my usual fix of putting them through Crop again does not revive them for VLC play.
It is as though the marker for Decrypt has been falsely added and the file is not actually decrypted. (Some of these files were processed by 'Auto-Decrypt' when it was a new feature and not yet working properly)
Is there any way of removing a DEC marker from a file so it can be decrypted correctly? If not - any clues as to making a recalcitrant file play on VLC?

Many thanks for any help.

 

[Ezra Pound]

d) Processes like decrypt etc. are greyed out when something that the process is dependant on is not in place e.g. quite a few processes require a DNLA name to be allocated by the Humax Firmware, this is indicated by this symbol . Have a look at the Flow chart HERE which indicated the steps for decryption.
In the case of files on USB drives, the mvdisks package does allow DNLA access, see notes HERE

 

[af123]

a) Sometimes when using the 'Rename' function, further changes made to a Medialist Title and/or Synopsis don't stick - i.e. I go back make an additional change or correction, firmly click on update, it appears to go through but the item does not actually change.
It seems random, but I guess its not - is it a DNLA indexing issue, or is there a finite number of changes possible, or is it something else?

I have absolutely no idea I'm afraid. There is no limit on the number of changes and it will not be related to DLNA (when you rename a file it will lose its place in the DLNA index and you need to wait for the Humax to catch up before you can stream it but nothing should stop you renaming it again). It sounds like a bug in the web interface - I will try and reproduce it based on your description.

b) When using 'Shrink' from the OPT+ menu for individual files, after processing short files a panel appears detailing the action - and the work is clearly finished .... but on long files no info panel appears and it simultaneously says both 'Back to media list' and 'Please do not interrupt....'.

That shouldn't happen. The progress bar should remain on the screen until the job is finished and then the info panel should appear at the same time as the back button. What browser are you using and how long are these long files?

c) My webif_auto.log sometimes shows info, but occasionally it says:
">>> Contents of /var/log/webif_auto.log 2.48 KiB Insufficient disk space, terminating."

The automatic process is very cautious and checks diskspace frequently while executing. The new version of the web interface that has just been released (0.11.0-2) will also log the percentage that it thinks is free when it decides there isn't enough. Can you keep an eye on it and post what it says when it happens again?

d) When using Web-IF 'Browse' to look at files and folders on USB drive attached to the HDR, OPT+ mostly brings up the full range of actions, which surprises me - and I'm trying to figure out which of them will work OK.
Are Web-IF OPT+ actions meant to work on files and folders on attached USB drives in a reliable way?

Whether the options are greyed out or not is generally derived by looking at the files themselves rather than where they are. For example, if the file is in the DLNA index then Decrypt will be available. Most options should work regardless of where the files are.

(Q: is it preferred here to use the term 'directories' instead of 'folders'?)

I don't think it matters. Folders is what the on-TV interface and documentation refer to and what Windows users are familiar with. Directories is what the underlying operating environment and we UNIX users call them.

Does it make a difference if mvdisks is installed or not?

With mvdisks installed, more operations will be possible as the files should be indexed by the DLNA server.

folders on attached USB can be set to 'Auto' process, but do not activate.

That one is a bug - the auto process only scans the internal drive (or primary drive on a HD) so the options should not be available.

e) Sometimes Auto-Decrypt and Auto-Shrink will process and sometime not. I am assuming it's something to do with available memory - but what are the exact rules?

It starts every 10 minutes and scans the disk for files to process assuming there isn't already a scan running. If it detects a shortage of disk space (< 10% remaining) then it aborts.

f) I have a few files which will play fine on the HDR, but will not play on VLC whatever I do. They are marked with the icons for Decrypted and Shrunk, but my usual fix of putting them through Crop again does not revive them for VLC play.
It is as though the marker for Decrypt has been falsely added and the file is not actually decrypted. (Some of these files were processed by 'Auto-Decrypt' when it was a new feature and not yet working properly)
Is there any way of removing a DEC marker from a file so it can be decrypted correctly?

Yes, the decrypted marker can be removed using the command line hmt utility. You actually need to use it to perform the inverse operation and set an encrypted flag.

humax# cd /media/My\ Video/Hunted
 
humax# hmt Hunted_20121122_2100.hmt | grep Flags
Flags: HD,Unlimited Copies,Guidance,
 
humax# hmt +encrypted Hunted_20121122_2100.hmt 
humax# hmt Hunted_20121122_2100.hmt | grep Flags
Flags: HD,Unlimited Copies,Guidance,ODEncrypted,

 

[af123]

e) Sometimes Auto-Decrypt and Auto-Shrink will process and sometime not. I am assuming it's something to do with available memory - but what are the exact rules?

Something came up in another thread that jogged my memory.
The auto-decrypt process will not be able to run in half-awake mode as the media server part of the Humax software is not running. auto-shrink should operate in any awake mode.

 

[Black Hole]

Ditto if you're using the TV Portal.

 

[DelftBlue]

af123, Ezra and BH: Thank you very much for your replies.

I really appreciate your help, as its always hard to know whether the quirks I find are due to my 'pilot error', to my kit's particular set-up or whether they are general to the software.

Re: 'Rename' changes to Media List Title and/or Synopsis sometimes not sticking

I have absolutely no idea I'm afraid. There is no limit on the number of changes and it will not be related to DLNA ...... It sounds like a bug in the web interface - I will try and reproduce it based on your description.

I've been testing to see if I can notice any definite patterns in the failures to change - and I can't

Via the Web-IF 'rename' functions all changes to the .ts filename work, but for Media List Title and Synopsis, an initial 'rename' usually works, and but a further rename attempt sometimes fails.

Changes made to the file name via the remote control/TV screen method always work, even when Web-IF rename has failed, but that is laborious!

In addition, I have found that some 'rename' changes to folder/directory names don't stick when done via Web-IF, but I can change them successfully via Samba, so its less of a problem.

 

[DelftBlue]

Re: b) When using 'Shrink' from the OPT+ menu for individual files, on long files no info panel appears and it simultaneously says both 'Back to media list' and 'Please do not interrupt....'.​

That shouldn't happen. The progress bar should remain on the screen until the job is finished and then the info panel should appear at the same time as the back button. What browser are you using and how long are these long files?

Internet Explorer 8 (on Windows XP Pro)

By 'short files' I meant 30 mins or less SD recording, with 'long files' being anything larger than that.

 

[DelftBlue]

Re: c) My webif_auto.log sometimes shows info, but occasionally it says:​

">>> Contents of /var/log/webif_auto.log 2.48 KiB Insufficient disk space, terminating."​

The new version of the web interface that has just been released (0.11.0-2) will also log the percentage that it thinks is free when it decides there isn't enough. Can you keep an eye on it and post what it says when it happens again?

>>> Contents of /mod/tmp/auto.log 147.91 KiB 28/01/2013 02:20 - ------------------------------------------------------- 28/01/2013 02:20 - Media scan starting, DLNA server status: 0 28/01/2013 02:20 - Insufficient disk space (94% 906.1G/808.2G), aborting...........

ditto ...... ditto .......

28/01/2013 14:40 - Media scan starting, DLNA server status: 1 28/01/2013 14:40 - Insufficient disk space (91% 906.1G/782.7G), aborting.

Then I deleted about 20GB of stuff and it started auto-working.

31/01/2013 01:40 - Media scan starting, DLNA server status: 1 31/01/2013 01:41 - dedup scan completed in 88.109 seconds. 31/01/2013 01:42 - DECRYPT: /media/My Video/02_Auto-Decrypt/Wonders of Life_20130129_2318 31/01/2013 01:47 - DLNA: http://192.168.1.14:9000/web/media/9001.TS 31/01/2013 01:47 - Done... 1005.08 MiB in 291.783 seconds - 3.44 MiB/s 31/01/2013 01:47 - DECRYPT: / etc. etc.

And this fits with

e) Sometimes Auto-Decrypt and Auto-Shrink will process and sometime not. I am assuming it's something to do with available memory - but what are the exact rules?​

​

You say it aborts if less than 10% space free ... ah I see ...would figure as the tipping point seems to be around 120GB available storage.

Many thanks for de-mystifying this for me

 

[prpr]

Say you want to decrypt a 1GB file, but the machine "only" has 100GB free on the 1TB disk. Machine says "no".
If you had a 500GB disk, then the machine would have said "yes".
The bigger the disk, the worse the problem - this is counter-intuitive.
The size of the original file doesn't change no matter what size disk you have, so percentages seem a bit stupid in this case.
Twice the file size amount would seem to be plenty to me.

 

[DelftBlue]

I've narrowed down more closely to the 'tipping point' for auto-processing - with my 1TB HD, at 113GB space available it doesn't run, but does with 118GB or more free.

So after some deleting to bump up the free space, auto-decrypt is working now - but strangely auto-shrink wasn't - but I think I've figured that out after studying the auto.log ...

there are repeated entries saying:
Media scan starting, DLNA server status: 1 31/01/2013 20:50 - dedup scan completed in 53.787 seconds. 31/01/2013 20:52 - decrypt scan completed in 64.132 seconds. 31/01/2013 20:52 - SHRINK: /media/My Video/usb-drive2/WEATHERVIEW_TEST/Skiing Weatherview_20130125_0223 31/01/2013 20:52 - Estimate 16% saving. 31/01/2013 20:52 - Shrinking... 31/01/2013 20:52 - Saved: 253936/1495611 packets, 46.50/273.92 MiB (16.98%) ...

Although the auto process only scans the internal drive - with mvdisks installed, it's treating the attached USB drive as a directory of the internal drive, and it's finding an auto-shrink flag (that I set as an experiment when I was finding out which OPT+ functions worked on USB) and it's repeatedly starting up and doing something ... that doesn't actually work and it gets stuck somehow and so nothing else further down the list gets to be auto-shrunk.
So it's time to call up the USB drive and remove that pesky auto-shrink flag

 

[Black Hole]

This is another known problem with mvdisks. In my opinion the mount folder should be given a name starting with "[", which will prevent the recursive processes trying to access it (as long as you do not set any auto flags on the folder).

 

[xyz321]

The problem with the current scheme of ignoring directories starting with '[' is they are shown at the end of the list of directories. A better option might be to skip over any leading spaces and then test for '['. That way you could have a mount point called " [usb-drive1]" and it would appear at the top of the list as well as being ignored by the scanning processes.

 

[Black Hole]

I have just realised my remaining disk space is way down (24G on a 500G drive), but my content is still being decrypted. An unexpected advantage of using the outdated unencrypt! Essential for me, as I use a remote mount a lot.

Can the threshold be made a user choice, like the undelete options?

 

[af123]

The problem with the current scheme of ignoring directories starting with '[' is they are shown at the end of the list of directories. A better option might be to skip over any leading spaces and then test for '['. That way you could have a mount point called " [usb-drive1]" and it would appear at the top of the list as well as being ignored by the scanning processes.

I could implement an xdev check easily enough if people would rather it did not process anything which is not on the internal(HDR) or primary(HD) disk.

 

[af123]

Twice the file size amount would seem to be plenty to me.

The logic does need refining although I'm still inclined to be more conservative than that - 3 times the file size adds some leeway in the case that two HD streams are being recorded at the same time as the decrypt operation.

Can the threshold be made a user choice, like the undelete options?

Is that really necessary if I adjust the logic as above?

 

[af123]

Although the auto process only scans the internal drive - with mvdisks installed, it's treating the attached USB drive as a directory of the internal drive,

It would be interesting to find out why it isn't working properly. Is it repeatedly shrinking the same file? What format is the external disk?

 

[Black Hole]

Is that really necessary if I adjust the logic as above?

Indeed not. I like the idea of a dynamic calculation of space required v. space available, but I don't think the formula is right. It would not be popular if recordings due over the next few days failed because automated processes plus recycle bin saving of original files filled up the disk.

A disk low warning could be flagged during RS sync, and the RS service used to email a warning to the user.

 

[af123]

It would not be popular if recordings due over the next few days failed because automated processes plus recycle bin saving of original files filled up the disk.

Agreed - the dustbin emptying will become more aggressive with low disk space but there's still a potential for problems.

I think we need something like:

XMiB + filesize*4

X = 10 would still allow for a fair number of hours of recording.

A disk low warning could be flagged during RS sync, and the RS service used to email a warning to the user.

Yes, another good idea for RS.

 

[DelftBlue]

Update: All auto-flags removed from USB drive and now auto-decrypt and auto-shrink working as expected, hurrah! (as long as more than 118 GB free space available ;-)

Re: processing auto-shrink flag on USB

It would be interesting to find out why it isn't working properly. Is it repeatedly shrinking the same file? What format is the external disk?

I wondered the same thing.
It appears to start up the shrink process but achieve nothing, despite the misleading message in auto.log.
The file in question does NOT have a shrunk icon by it.
The .ts file's 'date modified' matches its recording date so it hasn't been modified since its inception (25th Jan)
- but the 'date modified' for the .hmt file is today's date (1st Feb) matching the last run of auto-shrink scan before I removed the flag.

Yes - it was repeatedly attempting to shrink the same file - every 10 minutes auto.log showed exactly the same message:

SHRINK: /media/My Video/usb-drive2/WEATHERVIEW_TEST/Skiing Weatherview_20130125_0223 31/01/2013 20:52 - Estimate 16% saving. 31/01/2013 20:52 - Shrinking... 31/01/2013 20:52 - Saved: 253936/1495611 packets, 46.50/273.92 MiB (16.98%)

The external disk is NTFS format (Seagate 2TB)

 

[DelftBlue]

This is another known problem with mvdisks. In my opinion the mount folder should be given a name starting with "[", which will prevent the recursive processes trying to access it (as long as you do not set any auto flags on the folder).

'recursive processes' presumably being auto-shrink, auto-decrypt, unencrypt? Would that include seriesfiler?

I discovered an unintended consequence of having mvdisks and seriesfiler concurrently installed, when recordings started disappearing from the HDR, seemingly randomly.

Unseen 'hands' were removing recordings AND all trace of a recordings existence, which caused much bewilderment and suspicion (you can imagine the arguments: "it definitely recorded OK", "it was there yesterday", "no I definitely haven't deleted it", "the folder's gone too..."

It was an exasperating mystery for quite a while - until a close examination of the seriesfiler.log revealed some unexpected entries.
Any time a USB drive was attached, and a series folder was created or moved to the top level of 'My Video', which had an identically named folder on USB .... then seriesfiler was happily moving files over from HDR to USB and deleting the original folder from HDR.

So eventually I found all the 'lost' recordings tucked away deep in the recesses of the USB drive - which was good news, except that a few of them were corrupted or truncated, and due to 'move' instead of 'copy', none of them were decrypted.

So now I make sure that any programme name folders on USB drives are modified or appended so seriesfiler doesn't 'match' them up to HDR recordings.

I actually turned this mvdisks/seriesfiler quirk round to my advantage - making a rather cack-handed workaround so I can 'Move' things to USB when I don't want to 'Copy' and go back later to delete the originals.

I have a series folder on the HDR called 'MOVE_HDR to USB' with a 3min marker file inside named 'Do Not Delete This File'.
With this copied to USB to create an identical set-up, all I have to do is drop decrypted files into the 'Move' folder on the HDR at a time when it will be safely on but not busy with anything else and leave seriesfiler to it.
(The marker file stops seriesfiler deleting the 'Move' folder from the HDR when it's completed moving all the other files)

This trick also worked nicely when different people in the household were following the same series but watching at different times and then deleting before all had watched.
I just recorded a middle of the night repeat as well, giving it different folder name, with a matching folder set up on USB - so as soon as it had recorded, seriesfiler moved the new file to USB and deleted the folder from HDR - which saved much confusion and arguments . The files on USB could be decrypted individually in situ as necessary.
(That's one of the reasons why I was interested to see whether auto-decrypt and auto-shrink would work on folders on attached USB drives)