Beta Offline decryption utility

EEPhil said:
Now that the Windows command line stripts is available can I take it that I needn't look into providing a Windows command line decryption utility - except for my own amusement?
Click to expand...
My command line requirement is met now with stripts. The only thing against stripts on Windows is the cygwin dependency. This is not an issue for me but I guess some people might not want to install it.

I am still hoping (as he has not said no yet!) @af123 will be able to do a similar trick to produce a Windows/cygwin version of HMT, as this is missing from the set of utilities currently and would probably be useful for me too for a utility I have in mind to organize/name my recordings that have just been dumped as is from my Humax box to various NAS locations.
 
MartinLiddle said:
Should the FC command have a /B argument to perform a binary comparison?
Click to expand...
Oops my bad, with the /B argument the files are still reported as identical. In either case (stripts or HFODU2) neither process consumed more than 6% of my CPU in task manager.
 
Last edited:
sceptic said:
You would expect it to be longer with all the Java overhead!
Click to expand...
Wouldn't have expected twice the time. I did some comparisons a while ago with a numerical simulation code written by someone else in C (or maybe C++) and my code in Java and an old version in Fortran. I don't remember there being that much of a difference. Perhaps af123 writes better code than I do.
Rather than manually timing a slightly more accurate way would be to look in the log file:
Code: 
19:38:41 INFO: Processing 1 files.
19:38:41 INFO: Decrypting: C:\[deleted]\HFODU2\enc_Pointless.ts
19:38:53 INFO: The file was decrypted successfully.
19:38:53 INFO: Output is C:\[deleted]\enc_Pointless_dec.ts
Obviously first item on the line is the time in hh:mm:ss. Best guess at the timing for this example would be 12 (+/- 1) seconds
 
sceptic said:
I am still hoping (as he has not said no yet!) @af123 will be able to do a similar trick to produce a Windows/cygwin version of HMT
Click to expand...
For what it's worth, tunefix is developed/debugged in Visual Studio and the same C++ source code compiles on the T2 without change. Any platform differences are handled in (my own) class/function library code with #ifdef sections.
Granted, using this technique would be more difficult with the decryption stuff used by stripts, but hmt would be fairly straightforward I'd have thought (if I were doing it, which I'm not!, oh, er, I did!).
 
Last edited:
EEPhil said:
118.85s - It's hidden away at the top of the image in post #255.
Click to expand...
Thanks. Those images are user hostile - too much pointless black space and text too small to see these days. I couldn't be bothered to read them.
 
I only checked them because BH pointed out the time. I thought, "where the hell did he get that from?" The only place I could find was that image.
 
Thanks @af123 From the looks of it, it should be possible to build a Windows version from that source which requires no additional depencies.

Oh and @prpr, point taken, I'll make sure my images are more user friendly in future!
 
EEPhil said:
Rather than manually timing a slightly more accurate way would be to look in the log file
Click to expand...
Ah yes I forgot about the log. FYI the actual timings are shown below. To my eye it finished at at 17:53 dead, so I don't know where the extra 36 seconds came from?-

Code: 
17:49:00 INFO: Processing 1 files.
17:49:00 INFO: Decrypting: E:\testing\The Desert Fox_20171229_1604.ts
17:53:36 INFO: The file was decrypted successfully.
17:53:36 INFO: Output is E:\testing\The Desert Fox_20171229_1604_hfodu.ts

I would have thought that the majority of the time was spent doing the 3Des decryption so maybe stripts has a better implementation of this algorithm?
 
sceptic said:
I would have thought that the majority of the time was spent doing the 3Des decryption so maybe stripts has a better implementation of this algorithm?
Click to expand...
Having given it a bit of thought there are quite a number of things that could cause the differences.
(Just for the record, where I have had full control over both the Java and other source for a numerical simulation I had 35s for a Java version and 33s for a Fortran one using the same algorithm/simulation data. Java was object oriented, Fortran was not. Less than 7% slower with Java. So taking twice as long for decryption is alarming.)
Looking at hfodu vs stripts:
  • I've no idea how stripts is implemented and whether it is possible that it could run multi-threaded. The Java version is deliberately written to decrypt in a single thread.
  • I've not implemented the read/write in the most efficient way. I read, decrypt, write one packet (192 bytes) at a time.(Actually, it's even worse than that. I read the four extra bytes separately and write them immediately and then work on the 188 byte ts packet). I thought I'd used the right routines to buffer this in the background but maybe it's not doing that.
  • The available 3Des decryption is probably wasting a lot of time. The routine seems to want to provide a new byte array each time it is called rather than reuse an existing one. That's likely to have extra overhead. In addition, I've no idea how efficient the algorithm is.
  • I have included some extra testing to catch various problems. This is within a loop (ie. the tests apply to each packet). Also likely to slow the routine down.
  • I only had one small Fox file for testing (I don't have a Fox!)
I could experiment with hfodu to see if I can speed it up. But, as there now exists another method for offline decryption in Windows, is it worth it [rhetorical]?
All that matters is that should people with a Humax Fox-T2 need to decrypt off the box there are a couple of tools to do this. :)
 
EEPhil said:
I could experiment with hfodu to see if I can speed it up. But, as there now exists another method for offline decryption in Windows, is it worth it [rhetorical]?
All that matters is that should people with a Humax Fox-T2 need to decrypt off the box there are a couple of tools to do this. :)
Click to expand...
Yep, not worth the candle. As far as I can see this is just a theoretical discussion about different implementations.

EEPhil said:
I've no idea how stripts is implemented and whether it is possible that it could run multi-threaded.
Click to expand...
I wouldn't think so. The OS allocates tasks to processors or virtual processors, unless the code is specifically written to launch another thread (which is then serviced by the OS according to available resources). Why would stripts be written with multi-threading in mind, when that's not available on the HDR-FOX processor/OS?
 
Black Hole said:
Yep, not worth the candle. As far as I can see this is just a theoretical discussion about different implementations.
Click to expand...
Which is what I hoped. That doesn't mean that I don't find the longer execution time odd to say the least. I might look at it - but only for my own (spit) continued professional development. :roflmao:
Black Hole said:
Why would stripts be written with multi-threading in mind, when that's not available on the HDR-FOX processor/OS?
Click to expand...
I don't know and, since I don't have one, I didn't know.
 
Black Hole said:
... Why would stripts be written with multi-threading in mind, when that's not available on the HDR-FOX processor/OS?
Click to expand...
But it is -- perhaps you were thinking of hyper-threading? The Linux/uClibc combination provided by Humax has the Native POSIX Threads library back-ported to uClibc (Humax version 0.9.29, NPTL appeared in 0.9.32) and the SoC features 2 CPUs, making threaded applications very worthwhile for Humax devs, even if the Opera browser components didn't need threading anyway:
Rich (BB code): 
humaxhdr# top
top - 01:37:01 up 14:22,  0 users,  load average: 4.94, 3.21, 2.18
Tasks:  86 total,   4 running,  82 sleeping,   0 stopped,   0 zombie
Cpu0  : 24.5%us, 36.9%sy, 35.3%ni,  0.0%id,  0.7%wa,  0.7%hi,  2.0%si,  0.0%st
Cpu1  : 19.9%us, 43.1%sy, 35.9%ni,  0.3%id,  0.7%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    125016k total,   118952k used,     6064k free,     8868k buffers
Swap:   131064k total,      204k used,   130860k free,    49728k cached
...
humaxhdr# cat /proc/739/status
Name:   humaxtv
State:        S (sleeping)
SleepAVG:     78%
Tgid: 739
Pid:  739
PPid: 659
TracerPid:    0
Uid:  0       0       0       0
Gid:  0       0       0       0
FDSize:       256
Groups:       
VmPeak:         354940 kB
VmSize:          76512 kB
VmLck:               0 kB
VmHWM:           46180 kB
VmRSS:           44252 kB
VmData:          40916 kB
VmStk:              84 kB
VmExe:            6292 kB
VmLib:           26416 kB
VmPTE:             356 kB
Threads:      113
SigQ: 0/2048
humaxhdr#
 
Last edited:
stripts, however, is single threaded. It does read packets in chunks of 10,000 though which may explain some of the speed difference.
 
I may look at manually buffering the input/output just to see if this makes a difference.
I am the author of my own salvation as the re-encryption utility has come in useful for creating a large Fox-type encrypted file for speed testing.
Already I've identified a cause of some slowness - the decryption process. I was using the function that returns a new byte array each time it's called. Very slow (426.88s) . Counter intuitively, at least to me, returning the result to the same output array each time is even slower (465.31s). Returning the result to the input array - much quicker. (343.20s). That is the result of just one set of tests - so take with a big pinch of NaCl.
 
You must log in or register to reply here.
Back
Top