ITV1 Failed Recordings

[Owen Smith]

If you tried to design something which accounts for and reacts sensibly to every possible invalid input, you would be there for eternity.

Ignoring invalid data is standard procedure for network connected devices, we do it all the time at work.

 

[Black Hole]

Sure, but in hardware?

 

[Owen Smith]

Sure, but in hardware?

Often yes, using something like a CRC on the data. Ethernet works that way for example. But surely the EPG data is handled in firmware?

 

[prpr]

But surely the EPG data is handled in firmware?

It is on the T2 - in the humaxtv binary. We don't know what it does with the data except by observation of its output in the EPG and inferences drawn from its behaviour.

 

[Black Hole]

Do we really know that? So far as I know, it's implemented in the system-on-chip, and it seems to me any function which might be time-critical could equally have at least some hardware preprocessing for accelleration.

Often yes, using something like a CRC on the data.

Is there CRC on the EPG stuff? I doubt it. The transmitted data is supposed to conform to specification, and is protected from transmission error by the transmission standards. There is no reason for a receiver to check for the broadcasters doing something stupid at their end. Building in sanity checks where they shouldn't be required means extra hardware/software in millions of receivers when the sanity check ought to be at the few data sources.

And, like I said, once you start down that route, you can't trust anything.

 

[Owen Smith]

Do we really know that? So far as I know, it's implemented in the system-on-chip, and it seems to me any function which might be time-critical could equally have at least some hardware preprocessing for accelleration.

It seems extremely unlikely to me that such a niche function as digital terrestrial TV EPG reception would be handled in hardware. Also the data rate and volume is not significant, so unlike audio and video decoding there is no benefit to doing this in hardware.

 

[Owen Smith]

And, like I said, once you start down that route, you can't trust anything.

Correct. I spend half my life writing firmware to handle error cases. Testing them is a pain in the neck as you have to inject errors.

 

[prpr]

Do we really know that?

We don't, but the data rate is so slow (relatively speaking) to be easily decoded in a weak CPU. What would hardware do with the data? The thing it probably does do is the PID filtering and routing.

So far as I know, it's implemented in the system-on-chip

And how do you know? What is your information source?

it seems to me any function which might be time-critical could equally have at least some hardware preprocessing for accelleration.

It isn't time-critical, not in micro-processor timescales anyway. It's several orders of magnitude slower. (And there's only one 'L' in...)

Is there CRC on the EPG stuff? I doubt it.

Yes. All the service information is protected by a CRC check, considering it is critically important to the operation of the system. Why don't you read the spec.?

The transmitted data is supposed to conform to specification, and is protected from transmission error by the transmission standards. There is no reason for a receiver to check for the broadcasters doing something stupid at their end. Building in sanity checks where they shouldn't be required means extra hardware/software in millions of receivers when the sanity check ought to be at the few data sources.

I mostly agree with you on that bit, but the old adage applies - be rigorous in what you transmit and tolerant of what you receive.

 

[Black Hole]

And how do you know? What is your information source?

Even the firmware is running in the SoC, so...

My point is we don't actually know where the hardware/firmware boundaries are.

CRC is irrelevant when incorrect data is injected rather than corrupted in transmission. There is no way to detect a valid data packet contains deliberate errors, except by assuming some idiot might do that and then running extensive sanity checks on the data (which could itself be an error-prone process). I repeat: it's not economic.

 

[Owen Smith]

running extensive sanity checks on the data (which could itself be an error-prone process). I repeat: it's not economic.

And yet we do it all the time at work

 

[prpr]

My point is we don't actually know where the hardware/firmware boundaries are.

You can read the spec. for the Broadcom BCM7405 chip.
Essentially it says the PID filter and SI Section de-packetising/extraction/CRC check happens in what I would call the 'hardware' part of the chip and then dumps the data into a RAM buffer for further processing by software in the MIPS general purpose processor (what I would call the 'software' part, by the 'firmware' as the humaxtv binary is in flash rather than on disk).
There is no decode of the SI Section data in hardware.

 

[/df]

Is this not happening at Xtal Palace? ...

...
I would expect it is, but I don't have a transport stream from there, so unless you want to record me 10 minutes from anything on PSB2 and strip out all the PIDs apart from 18 and send me the zipped result, then I don't know. ...

https://de.skysend.ch/download/b6f7ac97757d9f94/#1JcUHW_jSOZRU1oqvKmbVg (74MB, expires 10 Nov)

 

[prpr]

Looks encrypted, so either tell me what the key is or decrypt and re-upload.

 

[/df]

Looks encrypted, so either tell me what the key is or decrypt and re-upload.

The zip isn't encrypted: zip < EFL20251102.ts >EFL20251102.ts.zip.

Should I have decrypted the recording (all 0s key) first?

 

[prpr]

The zip isn't encrypted:

Sorry, I meant the .ts was encrypted, not the .zip
That command generates a file name of "-" inside the .zip, which is slightly annoying on extraction.

Should I have decrypted the recording (all 0s key) first?

I thought that would be obvious for a man of your calibre! How do I read it unless it is decrypted, unless I know the key (which I now do)?

Anyway, it confirms the list of services with broken tables as:
4287, 4672, 4736, 5696, 8500, 8600, 8601, 13024, 13144, 15920, 16016, 16170, 16188, 16194,
16280, 16322, 16394, 22400, 24448, 27232, 27744, 27776, 27904, 28032, 33856, 34240, 34496
which is broadly similar to mine (I found it varies with time).