crashing and green screen

[MymsMan]

update 2:
I just experimented and manually 'decrypted' a file using Opt+ on webif.
it doesn't play, instead it says 'The channel is scrambled or not available'

Is the recording you used for this test viewable? I would expect this one, and any others decrypted this way before you set the custom key, to be unviewable

But the original file will have been stored in the _original folder so you can recover it easily.

 

[EEPhil]

OK. I will give my 2p worth. You can get the error "The channel is scrambled or not available" if there is a mis-match between the .hmt file byte 0x028e (bit 0) and the real state of the .ts file. If the .ts really is encrypted but bit 0 is '0' (indicating that the .hmt file "thinks" the file is decrypted) then that message comes up on-screen - well it does on my 2000T when I just flipped that bit. Is there any possibility that there is an error somewhere leaving such a mis-matched state?

 

[MymsMan]

OK. I will give my 2p worth. You can get the error "The channel is scrambled or not available" if there is a mis-match between the .hmt file byte 0x028e (bit 0) and the real state of the .ts file. If the .ts really is encrypted but bit 0 is '0' (indicating that the .hmt file "thinks" the file is decrypted) then that message comes up on-screen - well it does on my 2000T when I just flipped that bit. Is there any possibility that there is an error somewhere leaving such a mis-matched state?

The original file copy created by the Opt+ decrypt option incorrectly has the Decrypted flag set, this can be corrected by using the HMT utility but I seem to remember there was something that went round checking for mismatches and correcting the flags.
Given the problems with the Opt+ decrypt I wonder if it would be easier to retire the option rather than fix it.
We can now queue individual or sets of recordings for decryption so do we really need a while-you-wait options? Unnecessary effort maintaining two unrelated versions of the same function.

 

[prpr]

I seem to remember there was something that went round checking for mismatches and correcting the flags.

Were you thinking of the fixencflags diagnostic?

 

[Tik33]

I think this is because you had the auto-dedup option set on the My Video folder, this is great on Some series folders - especially kid series allowing me to find the correct episode of Bitz and Bob from my granddaughters description, but much less helpful for one off programmes as you have discovered.
It is an option that it well worth experimenting with the manual version via the Opt+ option to see if works sensibly for a series before using the auto version.

AFAIK the original names are lost forever and the only option is tedious renaming of each recording (opt+ menu) unless someone could think a way of using the recording date/time and channel to search some archive to find the original recording title - perhaps activity.log has enough information

thanks for your thoughts on this. I did do a back up of my 1tb drive by taking it out and connecting it to a SATA connector and using linux reader on an external HDD. However, it didn't detect all the files and I think that the 1tb has more recordings on it than my backup. When I look at the backup - I can see 3 types of file: the .ts, the .thm and the .nts - but I think there should be 4?

so here's what I'm thinking of doing:
1. install the 2tb drive into the machine
2. follow your very helpful 'set-up' guide to make sure 2tb drive is all set up
3. Copy across the backup files from the external HDD - I think you suggested using USB rather than FTP - would USB be quicker?
4. If I've understood correctly, they won't work straight away, but by having 'auto-decrypt' - those files should then decrypt over the next day or so - as they still have their 'name' attached to the file - they may work (although I believe that they're missing one of their sidecar files - not sure if that will make a difference?)
5. If those work, I can then copy any remaining files that are still on my 1tb drive (and are not on the external HDD) using SATA and the usb (or FTP) and rename those missing files.

by the way - I note your comment about keeping it plugged in via USB - but with a 4 year old in destructive mode running around the house, not sure that would last very long...

thanks!

 

[Black Hole]

I can see 3 types of file: the .ts, the .thm and the .nts - but I think there should be 4?

Yes - the .hmt is vital! That one contains metadata such as whether the HDR-FOX should treat a recording as encrypted!! This is essential for the on-box decryption mechanisms, although you can still decrypt using the off-line command line stripts method or the off-box Windows utility method.

A .ts without its .hmt is 'naked', and will be unplayable (regardless of encryption key) unless previously decrypted. They show up in the normal SUI Media listing as the .ts filename (because the 'title' normally displayed, rather than 'filename' property, is held in the .hmt)

Installing the sidecar package adds recreating the sidecars to the list of operations available via the WebIF media browser OPT+ menu.

A possible reason this may have come about is free space on the destination drive while performing USB copy. A bug in the Humax firmware results in the .ts being copied if there is enough free space, but then expecting the same amount of free space to copy the sidecars even though they are only small (maybe this only affects copying the .hmt). This is written up in Things Every... (click) section 12.

 

[Tik33]

Yes - the .hmt is vital! That one contains metadata such as whether the HDR-FOX should treat a recording as encrypted!! This is essential for the on-box decryption mechanisms, although you can still decrypt using the off-line command line stripts method or the off-box Windows utility method.

thank you Blackhole - I therefore take it that if it doesn't have the .hmt file - the rest of the recording is useless? so I may as well delete them from my external HDD?
Or should I try and decrypt them using your methods you mentioned and reinstall them?

I'm a bit confused as to what best to do next in terms of trying to install the recordings (with names) on to the new 2tb drive on to the new humax.

 

[MymsMan]

thank you Blackhole - I therefore take it that if it doesn't have the .hmt file - the rest of the recording is useless? so I may as well delete them from my external HDD?
Or should I try and decrypt them using your methods you mentioned and reinstall them?

I'm a bit confused as to what best to do next in terms of trying to install the recordings (with names) on to the new 2tb drive on to the new humax.

These files are a backup copy of your 1TB drive that you have now successfully decrypted so why do you still need these backups especially if they are encrypted and missing the hmt component? Or am I confused and missing something?

Just make sure that when you do copy from your 1TB to 2TB drive that you the hmt are copied correctly if you use the webif to do the copying it will copy a complete recording as a set.

 

[Black Hole]

I'm a bit confused as to what best to do next in terms of trying to install the recordings (with names) on to the new 2tb drive on to the new humax.

Assuming you are talking about .ts files missing their .hmt files, and that the .hmt files are not recoverable:

The .hmt contains the metadata for the recording, including the "name" of the programme (we call it the medialist title), and it is this metadata that the Humax firmware uses to display the name you see on the TV screen Media listing. Without the .hmt, all you get is the filename of the .ts file. However, at least some of the metadata can be reconstructed from the non-video/audio data within the .ts... if it was not optimised by stripping out the non-video/audio "padding" (a WebIF option).

First, you need to ensure all the files are decrypted. You have several routes to that, including modifying the encryption key of HDR2 to match the key of HDR1. However, without the .hmt you would need a dummy .hmt to fool the system into making decryption available. The best way is to export the files to a PC and use the HFODU utility for the decryption.

With decrypted .ts files, you then have a chance of using the sidecar package to rebuild the .hmt from whatever information can be gleaned in the "padding" within the .ts. Whatever comes out can be edited through the WebIF rename option.

 

[prpr]

Sidecar will recreate a dummy .hmt file to allow decryption (that's one of the suggestions I made to Raydon that he took on board).
Then you can recreate the .hmt file again once the .ts is decrypted.

 

[Black Hole]

Sidecar will recreate a dummy .hmt file to allow decryption

Yes, I knew we could (wasn't sure how), but I don't think that helps much unless you (a) ensure all existing recordings on HDR2 are decrypted, and (b) change the key on HDR2 to match HDR1. If you don't want to do that, you might as well decrypt off-line (which does not need the .hmt).

 

[Tik33]

These files are a backup copy of your 1TB drive that you have now successfully decrypted so why do you still need these backups especially if they are encrypted and missing the hmt component? Or am I confused and missing something?

Just make sure that when you do copy from your 1TB to 2TB drive that you the hmt are copied correctly if you use the webif to do the copying it will copy a complete recording as a set.

thanks everyone, and MymsMan is correct, I don't need the backup - it was just a question to see if I could recover all the names from the movies that way - but it appears as though it will be more complicated than it's worth - and I'd be better off manually renaming the file.

 

[Tik33]

before I do something stupid again - would you be able to confirm that I should do the below?

1. install the 2tb drive into the machine
2. follow MymsMan's very helpful idiot-proof 'set-up' guide to make sure 2tb drive is all set up
3. Copy across the backup files from the 1tb - I think you suggested using SATA and USB rather than FTP - would USB be quicker?
4. If I've understood correctly, since the files on the 1tb HDD are decrypted - they should work!
5. Go through and manually rename all the files.
6. once the above is done - would you recommend that I also use any other programme - eg auto-shrink or auto-dedup?

thank you!

 

[Black Hole]

Go through and manually rename all the files.

Yes, but how will you do that? If you want the recordings to play normally (with all the normal facilities) you will need the sidecar files. If you are content to rename the filename rather than the medialist title, you can simply use Windows Explorer (or whatever) to rename in bulk (much easier), but if you want normal behaviour with a medialist title displayed by Media you have to recreate the .hmt and then (tediously) go through them all one by one with the WebIF rename facility.

Whatever you are going to do, I recommend 4a: run sidecar on the files. Having run sidecar, you might not have to do any renaming (which is my point in post 149 you don't seem to have understood - why not try it?).

would you recommend that I also use any other programme - eg auto-shrink...

Shrink saves a few percent of a .ts file's space on the disk... but removes any possibility of sidecar reconstruction. I regard it as an act of desperation instead of proper back catalogue vs. storage space management (which is not to say I haven't used it myself from time to time!).

...or auto-dedup?

Are you mad? Dedup is partially responsible for getting you in this mess in the first place.

 

[Trev]

I think there is a possible confusion is between the filename and the media list programme name. I know it confused the hell out of me until I got a 'handle' on it.

 

[Black Hole]

Okay, to clarify:

1. The "medialist title" is what appears in the recordings list when you press the Media button, is constructed from the EPG data for the programme recorded, and is stored in the .hmt file along with other metadata for the recording (such as the position of any bookmarks, where to resume playback, whether the recording is protected and/or encrypted, etc etc).
   .
2. The medialist title is not the same as the filename, ie the name that appears in a file system directory listing with the ".ts" file extension.
   .
3. If the .hmt sidecar file is missing, or if the video file is not .ts but is supported (eg .mp4), the Media list displays the filename.
   .
4. Because of operating system / file system limitations, a filename cannot contain certain special characters that can be included in the medialist title, eg "&".

 

[Black Hole]

It strikes me that sweeper could be leveraged to create medialist titles if necessary.

 

[Tik33]

Yes, but how will you do that? If you want the recordings to play normally (with all the normal facilities) you will need the sidecar files.

just to clarify - my recordings (on the 1tb drive) do all seem to work - they just are named incorrectly. It's the recordings that I backed up (at the beginning of this journey!) on to the external HDD that are missing the .htm file

The "medialist title" is what appears in the recordings list when you press the Media button, is constructed from the EPG data for the programme recorded, and is stored in the .hmt file along with other metadata for the recording (such as the position of any bookmarks, where to resume playback, whether the recording is protected and/or encrypted, etc etc).
.

thank, you then I don't mean renaming the filename - I mean renaming the medialist title. I haven't tried it, but there appeared to be an option using Opt+ to rename and I was planning to use that manually (although it's not that easy as I'd have to play the recording and then manually rename it - and that could take a lot of time!)

Whatever you are going to do, I recommend 4a: run sidecar on the files. Having run sidecar, you might not have to do any renaming (which is my point in post 149 you don't seem to have understood - why not try it?).

thank you for pointing that out - must have missed it - will try that to see if it works! sounds much better!

Shrink saves a few percent of a .ts file's space on the disk... but removes any possibility of sidecar reconstruction. I regard it as an act of desperation instead of proper back catalogue vs. storage space management (which is not to say I haven't used it myself from time to time!).
thank you - I think that i installed auto-shrink and auto-dedup because I did some reading and the packages sounded like they would be helpful - however, I wasn't really aware of the downsides of theses packages and won't touch them!

 

[MymsMan]

How many recordings do you need to rename / retitle?
If it is a huge number attempting to automate the process might be worth it but for smaller numbers you can spend longer than doing it manually..
I don't think sidecar will overwrite an existing media list title but providing the recording has not been shrunk it will show the title in the process messages and you could copy and paste it into filename and title fields of the rename menu

 

[Tik33]

How many recordings do you need to rename / retitle?
If it is a huge number attempting to automate the process might be worth it but for smaller numbers you can spend longer than doing it manually..
I don't think sidecar will overwrite an existing media list title but providing the recording has not been shrunk it will show the title in the process messages and you could copy and paste it into filename and title fields of the rename menu

thanks - it's probably around 200 to 250 recordings! I will try installing sidecar and see how I get on.