Beta Offline decryption utility

[prpr]

That seems to have got it, thanks... but I say "seems" because although stripts didn't bomb out and (apparently) produced an output file (I presume it would have complained if I had got the key wrong), the product won't play in VLC.

Just try running stripts without any parameters. If you get a help screen you have it running correctly. One of the options thereon will tell you how to test the key against the recording. Or try deliberately using wrong keys and see what happens.

For context, I have a couple of recordings made on HD-FOX which are too large to decrypt locally (or at least, they fail),

There shouldn't be any size limit. It just takes bl@@dy ages. So how are they failing?

I put the original files on a UPD and plugged it into a HDR-FOX (same key as the HD-FOX), and ran DLNA decryption via the WebIF media browser OPT+ button. Having decrypted by that means, they play in VLC on my Linux system.

Interesting.

How do you suggest I go about finding out what the difference is?

You could use dd to crop off the first 192 bytes of the file, then feed it to hd and compare the outputs.
e.g. dd if=myfilename.ts bs=192 count=1|hd

 

[Black Hole]

There shouldn't be any size limit. It just takes bl@@dy ages. So how are they failing?

Just by taking bl@@dy ages, probably!

You could use dd to crop off the first 192 bytes of the file, then feed it to hd and compare the outputs.
e.g. dd if=myfilename.ts bs=192 count=1|hd

Weird!

I redirected the hd output to files, and got this:

Working:

00000000  06 e2 ea f6 47 03 2b 35  07 10 16 e0 89 5c 7e e2  |....G.+5.....\~.|
00000010  66 93 a2 cd 16 07 24 f3  45 7a 24 21 e9 14 7a e2  |f.....$.Ez$!..z.|
00000020  fb 05 1e 01 ec 9c b3 8d  38 8b 68 5a 90 70 00 f6  |........8.hZ.p..|
00000030  f5 24 ff 27 c5 9e 7f c7  99 16 45 22 dd b1 f8 1d  |.$.'......E"....|
00000040  69 2a 61 3a 4a d4 28 e8  b5 01 d1 d6 85 1e 56 3a  |i*a:J.(.......V:|
00000050  48 a3 00 e6 8b 22 e5 f9  26 0a 32 af e5 9a 1f 80  |H...."..&.2.....|
00000060  3a 93 89 01 dc f9 23 c4  69 20 7e fe 93 30 8f c3  |:.....#.i ~..0..|
00000070  95 25 94 2c 0e aa 2c f3  c1 44 00 a4 91 c7 51 1f  |.%.,..,..D....Q.|
00000080  e9 92 65 f3 e2 f8 e3 95  af e9 22 01 ab 38 2c 8b  |..e......."..8,.|
00000090  67 59 c6 2c 0e df f9 23  ef f2 75 98 ab b0 f9 02  |gY.,...#..u.....|
000000a0  2c 0e 49 22 01 ab 47 5a  56 eb f7 41 f6 23 85 c5  |,.I"..GZV..A.#..|
000000b0  bf c8 fb 1d cf 92 40 93  7f 48 8b b2 2b 45 8f 1d  |......@..H..+E..|
000000c0

Failed version:

00000000  06 e2 ea f6 47 03 2b 35  07 10 16 e0 89 5c 7e e2  |....G.+5.....\~.|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000000c0

The actual file size in bytes are the same.

What is truncating this output? I tried again with the hd output piped through more:

$ hd Delta\ 8-3\[stripts-failed\].m2ts | more
00000000  06 e2 ea f6 47 03 2b 35  07 10 16 e0 89 5c 7e e2  |....G.+5.....\~.|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000000c0  06 e3 76 bb 47 03 2b 16  00 00 00 00 00 00 00 00  |..v.G.+.........|
000000d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000180  06 e4 16 77 47 03 2b 17  00 00 00 00 00 00 00 00  |...wG.+.........|
00000190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000240  06 e4 a2 59 47 03 2b 18  00 00 00 00 00 00 00 00  |...YG.+.........|
00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000300  06 e5 2e 1e 47 03 2b 19  00 00 00 00 00 00 00 00  |....G.+.........|
00000310  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000003c0  06 e5 af 13 47 03 2b 1a  00 00 00 00 00 00 00 00  |....G.+.........|
000003d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000480  06 e6 4e cf 47 03 2b 1b  00 00 00 00 00 00 00 00  |..N.G.+.........|
00000490  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000540  06 e6 da 94 47 03 2b 1c  00 00 00 00 00 00 00 00  |....G.+.........|
00000550  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

What???

PS: Instead of using dd, this works: hd -n 192 myfilename.ts (I'm reading up on hd to find out what's causing the asterisks).

PPS: OK, the asterisk just means the intermediate lines are identical to the one above the asterisk. Got it.

 

[prpr]

00000000  06 e2 ea f6 47 03 2b 35  07 10 16 e0 89 5c 7e e2  |....G.+5.....\~.|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Ignore the first 4 four bytes, then 47 is the sync byte, 03 2b means Talking Pictures TV Video PID, then the 3 of 35 means it has an Adaptation field, length 07, followed by the Adaptation field data (which is not encrypted), followed by the video payload (all 00s, obviously invalid).
I have to say that I think this looks like a bug in the x86 version. Not sure how though. Is this a one-off file, or repeatable?
I presume you checked the key as suggested earlier (using -/ rather than -@)?
If all looks OK, can you send me the encrypted 192 byte file and the key?
It may well be your environment though, but mine's essentially the same so it would be useful to check.

Instead of using dd, this works: hd -n 192 myfilename.ts

Thanks. That's a useful one to know.

 

[Black Hole]

Is this a one-off file, or repeatable?

A second, much larger, recording file also displays this playable/unplayable behaviour, although the first 192 bytes are identical (and contain a large block of FF, except for 0000-000f and 00bc-oobf).

I presume you checked the key as suggested earlier (using -/ rather than -@)?

Yes. I was pretty sure stripts produces an error message if presented with an incorrect key, and it does. I don't know how though, I presume there is some kind of recognisable pattern in the plain data which does not exist if decrypted incorrectly.

If all looks OK, can you send me the encrypted 192 byte file and the key?

You mean the video file truncated to 192 bytes? Sure. The key is simply all zeros (my default key for all machines, not difficult to remember...).

Encrypted:

00000000  06 e2 ea f6 47 03 2b 35  07 10 16 e0 89 5c 7e e2  |....G.+5.....\~.|
00000010  ce 8f 64 12 d5 88 06 bc  65 18 79 91 fa 21 e4 ad  |..d.....e.y..!..|
00000020  65 07 fb 41 b1 44 88 0a  c9 cc 47 0b 77 93 56 ce  |e..A.D....G.w.V.|
00000030  d6 32 88 c9 5f c3 73 3e  50 68 73 6b a1 3c fa ed  |.2.._.s>Phsk.<..|
00000040  17 94 d2 13 1f a8 18 80  d3 4e 04 bc b6 a3 fd 13  |.........N......|
00000050  3b 71 5c 3b 22 50 fa 11  16 1f 8a 15 5e c4 c0 0c  |;q\;"P......^...|
00000060  e1 a3 a6 be fc 55 1c b7  5f 9c 90 77 64 f2 be 6b  |.....U.._..wd..k|
00000070  7b 47 58 25 04 79 90 d1  07 9e f0 29 79 6c da 75  |{GX%.y.....)yl.u|
00000080  23 ec c1 1f f9 55 f4 d8  07 2d d5 03 7e 3e 16 8f  |#....U...-..~>..|
00000090  48 e1 77 1f 5f 7d 1f f9  1e 41 7c 67 5a 3f 42 4b  |H.w._}...A|gZ?BK|
000000a0  be 08 22 d7 9f 7c fb 47  62 2e b6 30 d4 f2 f4 b1  |.."..|.Gb..0....|
000000b0  bd de 09 67 ce 0c d1 ef  d1 60 87 43 74 e3 60 a5  |...g.....`.Ct.`.|
000000c0

Decrypted (playable):

00000000  06 e2 ea f6 47 03 2b 35  07 10 16 e0 89 5c 7e e2  |....G.+5.....\~.|
00000010  66 93 a2 cd 16 07 24 f3  45 7a 24 21 e9 14 7a e2  |f.....$.Ez$!..z.|
00000020  fb 05 1e 01 ec 9c b3 8d  38 8b 68 5a 90 70 00 f6  |........8.hZ.p..|
00000030  f5 24 ff 27 c5 9e 7f c7  99 16 45 22 dd b1 f8 1d  |.$.'......E"....|
00000040  69 2a 61 3a 4a d4 28 e8  b5 01 d1 d6 85 1e 56 3a  |i*a:J.(.......V:|
00000050  48 a3 00 e6 8b 22 e5 f9  26 0a 32 af e5 9a 1f 80  |H...."..&.2.....|
00000060  3a 93 89 01 dc f9 23 c4  69 20 7e fe 93 30 8f c3  |:.....#.i ~..0..|
00000070  95 25 94 2c 0e aa 2c f3  c1 44 00 a4 91 c7 51 1f  |.%.,..,..D....Q.|
00000080  e9 92 65 f3 e2 f8 e3 95  af e9 22 01 ab 38 2c 8b  |..e......."..8,.|
00000090  67 59 c6 2c 0e df f9 23  ef f2 75 98 ab b0 f9 02  |gY.,...#..u.....|
000000a0  2c 0e 49 22 01 ab 47 5a  56 eb f7 41 f6 23 85 c5  |,.I"..GZV..A.#..|
000000b0  bf c8 fb 1d cf 92 40 93  7f 48 8b b2 2b 45 8f 1d  |......@..H..+E..|
000000c0

Decrypted using stripts v1.4.6 (1/7/23) in Linux Mint 21.2 (not playable)

00000000  06 e2 ea f6 47 03 2b 35  07 10 16 e0 89 5c 7e e2  |....G.+5.....\~.|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000000c0

The attached file is the above binaries in a zip.

 

[prpr]

I don't know how though, I presume there is some kind of recognisable pattern in the plain data which does not exist if decrypted incorrectly.

There's a byte in the PAT that is guaranteed to be 0, so if it isn't the file's encrypted.

You'll be pleased to know it's broken here too. It turns out that your choice of key is your undoing...

Openssl have declared certain ancient ciphers as legacy because they are so insecure (!), and the decrypt function returns an error when support is not loaded for them (you can enable it, but it's generally a bad idea).
When the first half of the key is the same as the second half, then stripts uses this legacy stuff, but unfortunately it doesn't check the return code from the decrypt function and bail out - it just assumes success and carries on, so you get a buffer full of 00s from it.

Pending a proper fix, you can fudge the key by making the second character a '1' instead of a '0'. This defeats the check for using the old cipher decryption function and instead uses one which is supported (this just makes it work, it doesn't make it any more secure!). Amazingly it still decrypts properly with this wrong key.

 

[af123]

When the first half of the key is the same as the second half, then stripts uses this legacy stuff,

It does this because it's much faster to use DES in this case. If it used 3DES, it would be doing two extra operations that just cancel out.

Amazingly it still decrypts properly with this wrong key.

That's because you're just flipping one of the parity bits here, not one of the key bits. Nice way to defeat the "just use DES" optimisation though!

in each byte, the least significant bit is an odd-parity generated bit, it is discarded when forming the effective 56-bit keys

 

[prpr]

I wondered how you got from 128 down to 112...

It does this because it's much faster to use DES in this case.

It's much faster to do nothing at all, which is basically what it's doing. But it's not entirely useful!
Sounds like not using the Openssl implementation under such circumstances is the best way to go then if a DES key is detected.
Good job you left the old code there!

 

[Black Hole]

Ye Gods! Trust me to find an edge case.

So what are we going to do? Basically you're saying the choice of user key is not unrestricted if the user might want to use stripts on the desktop with mainstream OS libraries.

Will flipping a parity bit to force OpenSSL to ignore that the key is symmetrical always work, or will a future update close that back door? Too much risk.

I suppose we could update the documentation to ensure users are aware of this restriction and not to choose symmetrical keys if they might want this use-case.

Alternatively, could we not just build stripts with the encryption implemented internally?

What I have not mentioned so far, because I thought it would cloud the issue, is that the second (much larger) recording is also unplayable but the first 192 bytes do not mismatch – they're all FF except for the first 16 bytes and the last 4 bytes. Does that mean they are not actual payload?

 

[af123]

Sounds like not using the Openssl implementation under such circumstances is the best way to go then if a DES key is detected.

Yes, except when running on the Humax where we want all the speed up we can get.

 

[af123]

Alternatively, could we not just build stripts with the encryption implemented internally?

It does have an internal implementation, it was just turned off in favour of openssl at some point, presumably because it was faster. The answer here is probably to fall back to that if the openssl library doesn't support DES, but use openssl if possible.

Will flipping a parity bit to force OpenSSL to ignore that the key is symmetrical always work, or will a future update close that back door? Too much risk.

Yes, stripts does a check that the first and second half of the key are byte for byte identical and uses the faster DES if that's the case. Checking a key's parity is a separate operation that we never do so as long as you change the key so that the two halves no longer match but the key is still the same then it will work.

 

[prpr]

Yes, except when running on the Humax

Indeed.

 

[prpr]

This version (64 bit only) is now current: https://dl.dropbox.com/s/vu3whg47ph0teiu/stripts.zip

The file has been refreshed with a new build - 1.4.7 - which works for me.

 

[prpr]

they're all FF except for the first 16 bytes and the last 4 bytes. Does that mean they are not actual payload?

Yes. Padding. You just got unlucky.

 

[Black Hole]

The file has been refreshed with a new build - 1.4.7 - which works for me.

Works for me. Definitely slower, 8s extended to 11s.

 

[Black Hole]

Let me see whether I've got this straight:

stripts for HD/HDR-FOX calls a crypto library regardless of whether the key is symmetrical, and the library supports DES as well as 3DES so we get the speed advantage of using an optimised library function both ways, and DES is faster than 3DES.

stripts for desktop can't call the library for DES because that has been obsoleted in recent versions of the library and just returns an error code if a DES operation is requested, so 1.4.7 implements DES internally and only calls the library for 3DES.

The question I have is: speed is not a significant issue on desktop, so (for the desktop version) why not simply ignore the DES optimisation and call 3DES for all keys rather than splitting off symmetrical keys? [See next post]

-----------------------------------

Explanation for new readers:

The hardware encryption used in HDR-FOX/HD-FOX is known as Triple-DES ("3DES"), "DES" being Data Encryption Standard. Once af123 devised a software implementation for decryption (useful for HD-FOX in particular, because it has the encryption/decryption hardware but it can't easily be accessed for decryption exploits), and the ability to alter the crypto key for a unit, he observed that choosing a "symmetrical" key string enables faster software decryption because it made the 3DES encryption equivalent to DES and decryptable using DES rather than the slower 3DES routines.

Hardware decryption is unaffected.

 

[Black Hole]

The question I have is: speed is not a significant issue on desktop, so (for the desktop version) why not simply ignore the DES optimisation and call 3DES for all keys rather than splitting off symmetrical keys?

Ouch! I just found out why: a factor of 2.5x processing time!!

I tested by comparing decryption using the symmetrical key and using a parity bit flip to force 3DES.

 

[EEPhil]

Explanation for new readers:

Not just new readers! It's probably five years since I looked at the Java version of the decryption algorithm. (Where did that time go?). I'd forgotten most of the information!

FWIW the method I coded always used 3DES decryption.

 

[prpr]

stripts for HD/HDR-FOX calls a crypto library regardless of whether the key is symmetrical, and the library supports DES as well as 3DES

Yes.

so we get the speed advantage of using an optimised library function both ways,

Er, see below.

and DES is faster than 3DES.

Yes.

stripts for desktop can't call the library for DES because that has been obsoleted in recent versions of the library and just returns an error code if a DES operation is requested, so 1.4.7 implements DES internally and only calls the library for 3DES.

Yes, although there is a way of enabling the library for DES without affecting anything else, if you are interested.

I've been doing some benchmarking, and on my desktop Mint machine, I see the internal functions are marginally faster on both DES and 3DES than the OpenSSL functions - it varies on the run, but on average it probably works out about 1%. (The DES is faster than 3DES by 2.5-3 times, as you found.)

On the Humax (with a mode switch added), I am rather staggered to see this (with a ~10MB test file cropped from a recording):

+ Total size: 9600000
dc d3 21...
Using OpenSSL decrypt.
Key appears correct.
Processed in: 29.69s

+ Total size: 9600000
dc d3 21...
Using Internal decrypt.
Key appears correct.
Processed in: 10.38s

+ Total size: 9600000
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
DES key detected.
Using OpenSSL decrypt.
Key appears correct.
Processed in: 12.31s

+ Total size: 9600000
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
DES key detected.
Using Internal decrypt.
Key appears correct.
Processed in: 3.91s

So it seems the internal functions are faster then the OpenSSL library ones by a factor of about 3, so I'm rather perplexed about the previous decision to switch.
Any insight @af123 ?

 

[prpr]

I see the internal functions are marginally faster on both DES and 3DES than the OpenSSL functions - it varies on the run, but on average it probably works out about 1%.

Actually since I've optimised how it works this morning, it's probably nearly 10% faster using the internal functions.