• The forum software that supports hummy.tv has been upgraded to XenForo 2.3!

    Please bear with us as we continue to tweak things, and feel free to post any questions, issues or suggestions in the upgrade thread.

.hmt File Transformation on OPT+ Copy

Black Hole

Black Hole

May contain traces of nut
Scenario: 130GB of recordings transferred from Humax internal hard drive to external USB drive (NTFS using the custom NTFS driver, but that shouldn't make a difference) by SUI OPT+ to ensure decryption (the recordings had been unprotected). For reference, the copy took 21h56m (6GB/h).

As a quality check, I looked at the original directory by FTP and compared it 1:1 with the copy (now mounted on the PC). All the files are there, with the following systematic difference: all the original .hmt files are 13.5, 14.2, or 14.9 KB in size, and all the (decrypted) copies are 2.02 KB.

Curiosity: What accounts for the difference?
 
Good idea. I had thought the BYTs might know off the tops of their heads. I'll look into it later.

Incidentally, the WebIF browser shows one recording out of the while set as being StDef (it's not)!
 
Black Hole said:
Scenario: 130GB of recordings transferred from Humax internal hard drive to external USB drive (NTFS using the custom NTFS driver, but that shouldn't make a difference) by SUI OPT+ to ensure decryption (the recordings had been unprotected). For reference, the copy took 21h56m (6GB/h).

As a quality check, I looked at the original directory by FTP and compared it 1:1 with the copy (now mounted on the PC). All the files are there, with the following systematic difference: all the original .hmt files are 13.5, 14.2, or 14.9 KB in size, and all the (decrypted) copies are 2.02 KB.

Curiosity: What accounts for the difference?
Click to expand...

A "unix" like operating system verus PC (Windows?) maybe?
 
I have compared a 2K and a 13K HMT and it was not possible to determin a cause using the HMT utility, However using an HEX editor showed the 13K had two copies of the program description (instead of one copy) and about 10,000 nulls e.g. HEX 00 So it looks like the original file is created with lots of 'padding' that is lost when copied
 
How strange. I'm pleased you confirm what I saw, and thanks for poking around in it. I postulate that the nulls are areas for information currently unused, but if so it is odd that they get stripped out during decryption. At some point I will look at a "moved" hmt rather than a "copied" one.
 
I believe I see the decrypted hmt go from around 15K to a few K using the virtual disc method, but stay the same size (but altered) if I decrypt using the web-if, which places the original files in an 'original' folder. I may have this the wrong was round as it's from memory, but I did wonder why the two decyrpt methods produce different size hmt's - I guess one way via the Humax firmware copying de-enc files to a 'disc' and one via the packages direct?

Nowadays I use the web-if rather than the virtual disc method.
 
The webif decrypt function, and the unencrypt package both leave the HMT size untouched. They both flip the 'is encrypted' flag in the file but otherwise leave it alone.
 
That makes sense, but I'm sure when copying the 'virtual' disk the HMT file is much smaller once on the VD - which I assume is a consequence of the Humax firmware re-writing it during the un-encrypted copy.
 
Black Hole said:
Scenario: 130GB of recordings transferred from Humax internal hard drive to external USB drive (NTFS using the custom NTFS driver, but that shouldn't make a difference) by SUI OPT+ to ensure decryption (the recordings had been unprotected). For reference, the copy took 21h56m (6GB/h).
Click to expand...

Ye gods!

I just repeated this copy, but this time the recordings were pre-decrypted, and copied to an Ext3 external drive. The transfer only took 3h20m this time - 39GB/h! This time the .hmt files have copied without changing their sizes.

The only wrinkle I can see is that the very last .hmt did not transfer, I had to patch it up by FTP afterwards. Always worth checking the block transfer is complete!

The difference is so startling I need to repeat that copy to the NTFS drive to see whether the transfer rate is being slugged by the NTFS driver or the decryption process (although it could also be the drives themselves - how do I eliminate that factor?).

Update: the NTFS copy is proceeding at about 10GB/h.
 
Black Hole said:
Scenario: 130GB of recordings transferred from Humax internal hard drive to external USB drive (NTFS using the custom NTFS driver, but that shouldn't make a difference) by SUI OPT+ to ensure decryption (the recordings had been unprotected). For reference, the copy took 21h56m (6GB/h).

As a quality check, I looked at the original directory by FTP and compared it 1:1 with the copy (now mounted on the PC). All the files are there, with the following systematic difference: all the original .hmt files are 13.5, 14.2, or 14.9 KB in size, and all the (decrypted) copies are 2.02 KB.

Curiosity: What accounts for the difference?
Click to expand...

A quick scan of several hundred recordings copied off the Humax this way shows that the size of the .hmt file for most of those recorded before mid 2012 was about 3KB, whilst for most of those recorded after this date it was about 14KB. I realise that this doesn't explain your discrepancy; but it may be of some relevance or interest.

There also appear to be some changes to the mapping of the fields within the file from those reported by son_t in early 2012.

However, apologies if this all belongs in the realm of pre-history.
 
Recordings made in padding mode tend to have larger .hmt files than those made with AR. The size difference is due to extra EPG information from recordings either side. On decryption the extra EPG information appears to get removed.
 
af123, thanks for your prompt reply - and all your other invaluable contributions to this forum.

I should have been clearer in my original post; I was specifically responding to Black Hole's original post and the comparison between HMT files of size 2.02KB, and those of 13.5, 14.2 and 14.9KB. I have the same variety of sizes, but not - knowingly nor apparently - for the same reason.

By way of background, all my HMT files are taken from offloaded programmes - and are consequently all unencrypted. AFAIK, all my recordings are made with AR and without padding. I do not have the custom firmware installed - so perhaps I'm not entitled to use this forum!

As you suggest, the difference in size between those files greater than 10KB seems to arise from the different number of EPG blocks located at X'3000'. Only a very small minority contain EPG information for other programmes - possibly corresponding to the very small number of manual or TSR recordings that I make.

More interesting, and the reason for my post, is the difference between those - many - files of exactly 2.02KB (actually 2,072 or X'0818' bytes) and the much larger ones. The smaller files end immediately after the 128 byte block starting at X'0798'.

Black Hole wondered whether the difference arose from the move/copy or decryption process. In my case, I noticed that all the smaller files are older than mid 2012, and the larger files are newer. Whilst this doesn't rule out that the difference arose from my recording or unloading them in different ways, it also raises the possibility that the format of the HMT file was extended at some stage to include the additional EPG blocks.

This possibility gained credence in my mind when I noticed that the format of the HMT - documented on this forum in early 2012 - made no reference to these extra blocks.

Finally, by coincidence, I happened to have recordings of the same programme from early 2012 and from earlier this year, and so was able to compare the format of the two files - one short and one long. In addition to the absence / presence of the EPG information, they also differed in the Channel Name field at X'045D'. It would appear that the length of this field might have been increased to accommodate channel names longer than the then reported maximum of 10 bytes, as channels with longer names (e.g. 'True Entertainment') came on-line. Just possibly, I mused, these two changes happened at the same time.

I had planned a more in-depth analysis of these differences, when I searched the forum for words of wisdom from those who had gone before me, thereby encountering Black Hole's post.

Apologies for the lengthy explanation, and thanks again for your reply.
 
This topic is nearly two years out of date. If you have read the whole topic you will have realised that the subject has been adequately explored and explained. However, thanks for your additional input - it is now available for future reference.
 
You must log in or register to reply here.
Back
Top