Couple of weird issues surrounding disable-OTA/DSO and Decrypt-in-place

[tahrey]

OK, I've had a look through the stickies and back about 12 pages of posts and not seen anything that directly addresses this.

I was quite happily using the custom firmware and entirely pleased with the facilities (although I was only making relatively simple use of them) up until a couple weeks ago. Then for some reason, the onboard Decrypt function has completely stopped working other than for copying to USB (real or virtual) using remote control commands - regardless of whether decrypt is attempted manually, via single folder automation or recursive - and I've started experiencing mysterious recording failures that seem related to the disable OTA thing. Basically, unless I manually change the channel away from 200, it'll stick on that PERMANENTLY even if two other recordings on different non-BBC muxes are scheduled, meaning that it'll be uselessly transmitting the Red Button holding screen to a powered-off TV, whilst failing to record one of the actually-wanted programmes (producing an orphaned .HMT file, almost like an AR failure even though AR is turned off on all channels). I lost several days worth of material before I figured out what the common feature was between days where this happened, and days where it didn't. Soon as it's changed off of ch.200, it's business as usual...

It's probably worth noting up-front that the box doesn't get rebooted very often, but I have tried doing it (via shutting down with remote / front panel button, using the Web-IF option, and removing AC power both intentionally (pulling plug after going to standby) and unintentionally (local area power cut)) and it hasn't made any difference.

The lack of decrypt-in-place isn't a completely terminal issue, but it IS a massive pain as I was relying on it to be able to have recordings automatically shrunk (shrinker still works btw) and decrypted ready for directly copying off to PC via the network. Now I have to make sure there's enough space left and that I have enough free time to manually trigger and then chaperone it copying the files over to the virtual USB drive (or a real one when I happen to have a spare with enough space), as it's much, much slower than doing it in-place (even using virtual USB rather than real), and worse still will often silently chop a longer recording short (losing typically 10 to 60% of the entire programme) if the system experiences too heavy a load due to other things whilst it's copying that way. It's messing with my whole system and I'm starting to run out of drive space.

I get the feeling from having read another thread that I may have misunderstood something, and that I can use the OPT+ command from the WebIF itself to directly copy files to PC with them being decrypted along the way? I have no problem with using that, at least temporarily, if it'll actually work. My only other option at this point is doing a mass dump of unencrypted files to the PC to clear some space (I'll be going away for a few days soon...) then copy them back later in order to actually decrypt.

Anyone got any idea what this is all about and how to fix it? Everything is up to date as far as I can tell, and I can't see any missing or rogue packages - though I'm actively considering doing an RMA prep on the box then reinstalling the CFW from scratch. The HDD passes all the tests thrown at it, though SMART does report it being held at too high a temperature and this being a failure risk (I have turned up the minimum fan speed to 40% so far to keep it from peaking too high...). I got it secondhand from Cash Converters, FWIW, but it doesn't seem to have any actual physical faults...

Cheers,
T

 

[af123]

There's a lot there but the first symptom you describe sounds as if you are possibly receiving channels from more than one transmitter with the result that the start/stop information for your recordings isn't being received on 200 when the box is waiting for it. Can you check that first? The channels page in the web interface diagnostics screen is useful for that.

For the lack of routine decryption, the first thing is to check that the content sharing server (DLNA) is still turned on in the standard on-TV menus. If it is, what does the auto.log log file show? You can increase the log level for the background decryption process through the web interface settings screen. One thing that might help here is a DLNA database reset (again through the web interface diagnostics).

 

[Black Hole]

Things Every... (click) sections 2, 5...

 

[tahrey]

Things Every... (click) sections 2, 5...

Yeah, already read that I'm afraid, and apart from the small parts af123 refers to, there's not really anything relevant to my situation or of use in fixing it. Cheers for the link at least.

There's a lot there but the first symptom you describe sounds as if you are possibly receiving channels from more than one transmitter with the result that the start/stop information for your recordings isn't being received on 200 when the box is waiting for it. Can you check that first? The channels page in the web interface diagnostics screen is useful for that.

I'll double check it, but I'll be very surprised. I have literal line-of-sight on the local transmitter (Sutton Coldfield) and could walk to it without too much difficulty, certainly cycle there and back in under an hour. Even so, reception in this area is pretty rubbish, I get enough for a full-size digital receiver to work reliably (TV, DVD recorder, PVR, etc, with about 50-75% strength and usually 100% "quality" rating) by using antenna power boost, but smaller, unboosted things like laptop TV cards have no luck, and both FM and DAB radio can be hit and miss. Highly doubt there's any interference from neighbouring transmitter cells as we're very nearly as far away from them as it's possible to get, and there's all kinds of hills and such in between. The local channel frequencies tally up with those for SC as well (UHF 39, 40 (DVB-T2 only), 42, 43, 45, 46, and 51 (only a few channels)) and I don't appear to be missing anything. The "8xx" numbers ring a bell however, though I don't think I've ever successfully tuned into them.

For the lack of routine decryption, the first thing is to check that the content sharing server (DLNA) is still turned on in the standard on-TV menus. If it is, what does the auto.log log file show? You can increase the log level for the background decryption process through the web interface settings screen. One thing that might help here is a DLNA database reset (again through the web interface diagnostics).

Derp. I think I might actually have turned that off deliberately because I had no reason to use it - or at least no obvious one! - and so figured I may as well save a bit of CPU load and network traffic in the interest of reducing the incidence of errors. I'll make sure it's turned back on as my first act and see if that ends up largely fixing everything - possibly even the tuning problems somehow

Thanks

 

[prpr]

figured I may as well save a bit of CPU load and network traffic in the interest of reducing the incidence of errors.

Flawed logic. I guess you haven't any way of quantifying 'before' and 'after', so you have no idea what you are 'saving'. All you are doing by your random fiddling is generating problems for yourself and wasting your own and other people's time in trying to solve problems so created.

 

[tahrey]

Wow, prpr, do you work at being an arse, or does it come naturally? If you didn't want to help, all you had to do was not reply, instead of getting personal about it.

Also, I already admitted having made a mistake there, but it's one I can't fix / double check until I get home. There's no need to rub it in.

Oh, and I DO have ways to quantify that stuff, both within the CFW and via my computer, power meter plugs etc. But you didn't bother asking about that did you, just assumed. There's a measurable difference in consumed power between power-on-idle and standby, which may only add up to a tenner or so over the course of a year if properly exploited, but, hey, that's a round of drinks. And the errors I was seeing - especially file copy operations silently aborting partway through - always seemed to coincide with logged (and, indeed, directly observed) periods of heavy load on the PVR system, whether CPU, disc, or network...

 

[Black Hole]

Yeah, already read that I'm afraid, and apart from the small parts af123 refers to, there's not really anything relevant to my situation or of use in fixing it. Cheers for the link at least.

Really? Section 2 explains why you shouldn't have multiple transmitters tuned and how to avoid it, while section 5 (in the further information) explains the conditions necessary for decryption (ie DLNA server turned on, etc).

 

[Ezra Pound]

The local channel frequencies tally up with those for SC as well (UHF 39, 40 (DVB-T2 only), 42, 43, 45, 46, and 51 (only a few channels)) and I don't appear to be missing anything. The "8xx" numbers ring a bell however, though I don't think I've ever successfully tuned into them.

Your Web-If >> Diagnostics >> Channel Information (for Sutton Coldfield) should look like THIS, DVB-T2 on UHF 33, 35 and 40, with DVB-T on UHF 39, 42, 43, 45, 46 and 51

 

[tahrey]

First up, before the down and dirty discussion, can I say a big thank you to you all for your help.
My box was bulging, overflowing full and missing a lot of recordings, and trying to offload them in a fashion that was playable on other devices was eating up all my spare time, particularly as it often didn't copy them complete so each one had to be double checked.

Within 48 hours of fixing the problem, all of the existing files were shrunk, decrypted, and offloaded successfully and in full to an external HDD attached to a networked laptop. Over the last couple weeks I've also managed to setup a simple but effective Sweeper system to automatically (and in stages) shrink, decrypt, and then file each new recording by date, so that all I need to do on coming home each day is to move the previous day's folder from the Humax to the external drive using drag-and-drop, over the network and the Samba server (which runs at a fairly decent 5 to 11mb/s, vs the paltry 2-3 of the USB port).
At 7.45am this morning, I actually managed to empty the Humax drive entirely, as there were surprisingly few things of interest on BH Monday and they all moved across in less than 20 minutes. I've also managed to copy a bunch of "emergency clearance" folders back onto the device (moved off before without shrinking or decryption in order to free up a few score gigabytes) and have it automatically process them, and rename a bunch of mistitled recordings inclusive of their sidecars and the information contained within. And it's only missed maybe 3 or 4 recordings that entire time, all of them seeming to be because of signal drop outs, power failures or other spurious BS.

This was super valuable over the past few weeks as I've not had a weekend to myself until this one (and even then, not completely so), between stag do's, birthday long-weekenders, weddings and such, so managing to clear down some space and have the box automatically and reliably deal with the things queued up for later copying was a massive help - it was that or just give up on the whole affair.

Basically, SCORE.
It's just SO much better now. Massive respect in the house innit.

Though I do now need to deal with the whole power down / auto reboot thing... figure out why the Portal doesn't work (never has, still doesn't, tends to bugger up whatever you're currently doing if you accidentally hit the button)... update some of the packages... learn just how you tune into the "streaming" freeview stations like Motors, Community and so-on... and re-learn how to turn a recording channel into a paused timeshift one in order to start recording another (which I used to be able to do, but now screw up every time I attempt it).

Aaaaaaaaaanyway.

Black Hole: Section 5 on that thread mentions that recordings are decrypted when streamed to another device using DNLA, but it doesn't say anything about the DNLA service being a requirement for on-box decrypting.

The full detailed method, behind a further link, does say it - in a very roundabout manner! - but I don't know if I actually got as far as that, or understood what it meant given that "DNLA Server" and "Content Sharing" may not have converged in my mind there, as they are entirely different words and I've never successfully DNLA'd a thing in my life so far. It's moderately clear once you know what the terms refer to (and which ones mean the same as each other), but not really noob-friendly. I'm sure it's explained in the footnotes if you have time and the mental strength to drag through it all, but I had neither when I came in desperation to ask these questions of you all. (Feeling a bit better now after having finally had a rest, but could do with one more long weekend of doing nothing...)

As for the multiple transmitter thing, I dismissed it out of hand (in fact, probably skipped it altogether) and forgot about it at that point because I've never even seen a sniff of a transmission from another area on our TV (or home FM radio), even back in the 4-channel analogue days, given our proximity to the local Tx and considerable distance from the next nearest (and much lower powered) rival one. I mean... point taken, I suppose, but I still don't think it's relevant to this issue, other than that the sometimes questionable signal quality could cause an occasional lost recording, and it probably wasn't behind the chronic drop-out I experienced given that it started when I turned off DNLA and ended when I turned it back on.

Ezra: Hmm, well, that confirms something I was suspecting after looking up the local transmitter coverage ... our rooftop aerial needs realignment or upgrading. I've had words with our block manager over this and she maintains it's fine, the fault is at my end and no-one else has reported any problems. Funny, then, how it suddenly improved a few days after I last mentioned it, then... with a man being seen up a ladder onto the roof... and the Ch46 mux recently started giving very borderline signal after a spate of high winds...

I literally don't see channels 33 or 35 at all - no tuner on any connected device even pauses when going past them - and I definitely don't get any of the T2 services carried by them. The others are generally OK but, as mentioned, 46 (and sometimes 51, though I rarely tune into it) can be a bit ragged round the edges (though this has not yet been worse than an occasional blocky line across the screen or a brief freeze-up, so it could be interference from something else? It seems to be much worse during ad breaks than the programmes/films themselves, weirdly). They all report as being "West Midlands" in the WebIF diag screen, but it doesn't go into any further detail. I might see if the other (non-T2) DVB devices I have on the same aerial line (TV, DVD recorder) can reveal down whether it is actually receiving from Sutton, or maybe the Malvern/Bromsgrove repeaters or something daft like that... though I'm pretty certain they use different frequencies anyway.

I don't know why reception should be so terrible here, like I say we have literal line of sight on the mast and there's not really anything around that would cause multipath issues. FM radios I can kind of understand, and my USB/PCMCIA tuner cards being completely hopeless when connected to a passive portable aerial... but digital TV via a properly specced high-gain roof aerial that's properly aimed should be OK, right? So the only thing I can think of is bad aim or knackered cabling - neither would surprise me here - and I'm seriously considering investing in a basic signal booster (plus a boosted aerial for the TV card) just to see if that helps any.

Still, that's a minor concern at this point, on the list with the streaming channels et al. The big thing was getting the automatic services working again, and I'm happy as a in about that...

 

[tahrey]

BTW, bonus tip in return ... even after shrinking etc, there seems to be a lot of spurious data / inefficient coding in the produced files (which might vanish if converted to pure MPG instead of TS, IDK). If like myself you're grabbing stuff for professional archive purposes (off-air recordings for sharing via an educational library with an ERA license), instead of timeshifting, or picking out and watching at random, you can still save a fair bundle of space by compressing them into a RAR archive. I've actually regained at least 200~300gb of space so far if not a whole lot more. The downside is you need an up to date computer to do it with any reasonable speed (my old laptop which I still use for some auxiliary file-copying purposes & such can't manage even 1x speed; the new one gets up to about 4-5x), so if you have a backlog then it may take a few days to process them all. But on a day by day basis it's only an hour or two of background grinding, and that'll gradually decrease as CPU power continues to improve.

Make sure it's RAR5 with a big dictionary, though. 128mb if you can manage it, or 256mb if you have the RAM and a 64-bit OS, it improves the compression ratio measurably (though it does slow the process slightly and chows up a lot of memory). Using "solid" mode (and "detect duplicate files") across multiple episodes of the same show. This also works very nicely for DVDRs... though the compression ratio is much less guaranteed and it's mainly more useful for letterboxed material recorded in CBR on a deck with built-in DVB (where a fair proportion of each frame is wasted with all-zeros), things that overrun into dead air and can't be edited (loooots of all-zero), or discs recorded in multiple resolutions (ie SP and LP on the same disc) where the same physical data gets represented as multiple logical files.

(Aside: if you're compressing something with PCM audio, or just PCM by itself, and you can't use FLAC... (demux the stream to separate video & audio first then) use RAR4 instead for that part. 5 works great on video and combined streams, but 4 is usually more effective on raw audio for some reason)

Whilst I'm thinking of it - is there a version of the Shrinker program available for desktop computers? I've got a bunch of files recorded using a USB DVB capture device which crunch down even harder using RAR (typically shrinking by an entire 25 to 33%!), which suggests to me that they have all that excess EPG data and the like in them. But short of copying them onto the Humax and hoping its own Shrinker can handle them, I'm not sure how to strip that crap out...

 

[Black Hole]

Black Hole: Section 5 on that thread mentions that recordings are decrypted when streamed to another device using DNLA, but it doesn't say anything about the DNLA service being a requirement for on-box decrypting.

The full detailed method, behind a further link, does say it - in a very roundabout manner!

The heck it does. Things Every... is a resource for non-CF users, and refers out to things for modders. The linked details say quite clearly (for every on-the-box method) that content sharing is required, and refer the reader to footnote 3 (how to turn it on).

 

[Ezra Pound]

tahrey : I literally don't see channels 33 or 35 at all - no tuner on any connected device even pauses when going past them - and I definitely don't get any of the T2 services carried by them

Channels 33 and 35 are problematic because they are not transmitted at the same power as the other MUXs, however if you are getting CH51 O.K. (which is also transmitted at a lower power than the main MUXs) then it may be worth checking that you have a wideband aerial. Sutton Coldfield traditionally had all it's UHF channels in a group that covered only one third of the 21 -69 UHF band and the best aerial for this was a Group B (Yellow band) aerial, however CH35 is on the very edge of this band and CH33 is outside it, so now you need a wideband aerial (Black Band) or the new reduced wideband group T (White Band) introduced after CH61-69 were sold off. Info HERE

 

[tahrey]

Ah... that might explain things. I have no knowledge of what aerial is installed and probably no influence over having it changed, as I'm in a block of flats. Certainly, it makes sense. I think we went for a yellow one back when I was living with my parents and we upgraded the loft aerial...

Maybe I'll borrow a boosted portable aerial from someone and see if there's a sniff of signal on those channels.

BH: When I clicked on the thing to look at footnotes before, I think I just got a "list of things the CF does", but I might be getting confused with something else. It could just be that I was having a bad brain fortnight, but it just didn't seem particularly clear or obvious either time I looked at it. However, you've all done a decent job of pointing out where I went wrong with my fiddling since I asked, so that's cool