'Deep Ping' for Hummy?

[mike_m]

There’s nothing more annoying than coming back from a long holiday to find that one of your Humax boxes threw a wobbly just after you went away, and failed to record anything!

So I’m using a Raspberry Pi running a cron task that regularly pings my Hummies (all HDR Fox T2’s running CF). If the ping fails, then the RPi (which is also doubling as an mqtt server) runs a couple of mqtt commands to power-cycle the appropriate socket on a smart power strip, and the dead Hummy reboots. So far, so good!

Once, however, a Hummy locked up in such a way that ordinary pings still succeeded, even though the box was otherwise completely unresponsive, and trying to login via Webif failed.

So my question is whether there is some sort of ‘deep ping’ that will enable my script to detect this condition – when presumably the OS is still running but the Humax software has crashed?

 

[MontysEvilTwin]

There’s nothing more annoying than coming back from a long holiday to find that one of your Humax boxes threw a wobbly just after you went away, and failed to record anything!

So I’m using a Raspberry Pi running a cron task that regularly pings my Hummies (all HDR Fox T2’s running CF). If the ping fails, then the RPi (which is also doubling as an mqtt server) runs a couple of mqtt commands to power-cycle the appropriate socket on a smart power strip, and the dead Hummy reboots. So far, so good!

Once, however, a Hummy locked up in such a way that ordinary pings still succeeded, even though the box was otherwise completely unresponsive, and trying to login via Webif failed.

So my question is whether there is some sort of ‘deep ping’ that will enable my script to detect this condition – when presumably the OS is still running but the Humax software has crashed?

Probably should be in the customised section.

The easiest solution is to use the remote scheduling service (RS). You can see when the box last contacted the server, there is an accessible log which tells you when a recording has been completed and you can get the server to e-mail you if the unit has not contacted it for a user-configurable amount of time.
In addition to the above, I use a wifi controllable socket to turn any misbehaving units on and off again when I am away. I also leave an expired series link in the schedule. If you do this and have to reboot by switching the mains supply off and on again, the device does not go into standby but stays half awake. This keeps the unit in contact with the RS server enabling you to send new recording requests and have them in the schedule within 10 minutes or so from any location with internet access. If you then use RS to schedule a reboot the box will go into full standby after rebooting.

 

[Trev]

Daily time switch off/on.

 

[mike_m]

@MontysEvilTwin:

By 'customised section' , do you mean the custom firmware forum? I'll happily move this post over there if necessary.

I'm not sure I fully understand your solution - but it seems to involve emails, internet connection, and my active participation. I have to confess that I’m really old-fashioned, (and really old too!) and I don’t use a smart phone as such. Although I take a laptop with me to get emails and Google stuff, I don’t ever connect via internet to anything in my home. I’m just super-paranoid about all that, and I don’t go anywhere near it. And yes, I’m the laughing stock of all my younger relatives!

Anyway, at the moment the system runs by itself, and it's not connected to the internet – just my local network - and I don't have to participate. I was hoping to keep it that way, but I guess maybe I’m being optimistic!

@Trev:

Now that’s an easy possibility. I could run a cron task on the Rpi to power-cycle the boxes at, say 3am every morning, when nothing’s going on. But would that have any detrimental effect on the hardware?

 

[Black Hole]

Now that’s an easy possibility. I could run a cron task on the Rpi to power-cycle the boxes at, say 3am every morning, when nothing’s going on. But would that have any detrimental effect on the hardware?

That seems like a hard way to do what a cheap timeswitch could do. The technique is to ensure the unit is supposed to be in standby for a period at the same time every day (night), and interrupt the power briefly during that period. Then, if it had crashed in the previous 24 hours, the reboot revives it and the worst case is you lose one day's worth of recordings.

Yes, the is a little extra voltage stress, but that's the price for extra dependability.

If you want to go "complicated", you could have your RPi ping it periodically and interrupt the power only if it gets no response. There are problems with that though:

• How does the RPi know when the reason it isn't getting a response is because it is correctly in standby and doesn't need a poke?
• Just because it does respond to a ping doesn't necessarily mean the rest of it hasn't crashed (it is multitasking);
• What if the RPi crashes?

All in all, a straightforward daily timeswitch reboot seems the better plan.

 

[prpr]

the RPi (which is also doubling as an mqtt server) runs a couple of mqtt commands to power-cycle the appropriate socket on a smart power strip

I'd be interested in knowing more detail about what hardware and software you are using to do this.

So my question is whether there is some sort of ‘deep ping’ that will enable my script to detect this condition – when presumably the OS is still running but the Humax software has crashed?

I detect lack of connection to port 80 for that condition. Sometimes Ping works but everything else is dead. This seems to catch it in that condition.

 

[mike_m]

@Black Hole:
Er... I think you're suggesting that I do exactly what I said I was already doing in my original post! Which is why, since I already have the system in place (and working), it's easier for me to have the RPi power off the Humaxes once a day than it is to chuck away my whole system and buy another piece of hardware instead.

In relation to the problems:
a) my Humax boxes are never in standby - they've been fully on more or less continuously for many years (i.e the front light is blue or red, never orange). So when a ping fails, I know the Humax is down.
b) Yes, that was the question I was asking in my original post.
c) Well, I used an RPi to trigger a still camera (every 15 mins for 3 years) on a time lapse project. I found it pretty reliable - I haven't had a crash yet. If there's a power cut, it simply reboots automatically when the power's back, and carries on as before. But of course it's always a possibility!

 

[mike_m]

@prpr:

Hardware: Raspberry Pi 3 Model B+ 64-Bit Quad Core Processor

Power strip:

Innens WiFi Smart Power Strip, Surge Protector Extension Lead Voice Control Compatible with Alexa Google Home and IFTTT, App Remote Control Timing Function, 4 AC Outlets 4 USB Ports, No Hub Required: Amazon.co.uk: Electronics

Innens WiFi Smart Power Strip, Surge Protector Extension Lead Voice Control Compatible with Alexa Google Home and IFTTT, App Remote Control Timing Function, 4 AC Outlets 4 USB Ports, No Hub Required: Amazon.co.uk: Electronics

www.amazon.co.uk

The power strip needed flashing to replace the firmware - details on this forum thread:

Using RPi to power off/on PVRs - Raspberry Pi Forums

www.raspberrypi.org

I can control the 4 sockets independently. Happy to share scripts, etc, if anyone's interested.

Originally I'd wanted to use a hardware solution - relays, etc - but I was persuaded to go the mqtt route. Much safer! The RPi acts as both the client and mqtt server, running mosquitto.

How do you check for connection on port 80? That sounds like the solution for me!

 

[Black Hole]

Er... I think you're suggesting that I do exactly what I said I was already doing in my original post!

I realise that.

Which is why... it's easier for me to have the RPi power off the Humaxes once a day than it is to chuck away my whole system and buy another piece of hardware instead.

Chuck away? No, redeploy maybe. I wonder which is cheaper to run and most reliable: an always-on RPi and WiFi-controlled power strip, or a timeswitch? In any case, I don't intend to repress your spirit of adventure - my comments are offered as background for other readers who might not be comfortable with the technicalities of programmed status monitoring and intervention.

I have been considering solutions to this kind of thing for some time, but other demands get in the way.

 

[mike_m]

@prpr:
How do you check for connection on port 80? That sounds like the solution for me!

After a bit of Googling, I'm assuming I can use nc -z [IP] [PORT No.]

 

[mike_m]

I wonder which is cheaper to run and most reliable: an always-on RPi and WiFi-controlled power strip, or a timeswitch?

Yes,of course you're right... a timeswitch would be much cheaper and more reliable. I cannot deny that this project was driven by my need to buy new gadgets and write code! It's what keeps me amused these days...

 

[MartinLiddle]

I cannot deny that this project was driven by my need to buy new gadgets and write code! It's what keeps me amused these days...

Absolutely nothing wrong with that approach to life.

 

[rpb424]

Yes,of course you're right... a timeswitch would be much cheaper and more reliable. I cannot deny that this project was driven by my need to buy new gadgets and write code! It's what keeps me amused these days...

Absolutely - that's how we learn isn't it? Where's the challenge in popping down to Argos and buying a timeswitch when you can craft a beautiful piece of code instead?!

I was going to suggest knocking something simple up in Python using the 'requests' package to periodically try and connect to the WebIF, but if you've now got a way in Bash as well then that's just as good.

 

[mike_m]

I was going to suggest knocking something simple up in Python using the 'requests' package to periodically try and connect to the WebIF, but if you've now got a way in Bash as well then that's just as good.

I must admit I was intrigued by your 'requests' suggestion, so I read a bit about it. It certainly seems interesting - though I'll come clean and say I'm not really a Pythonite - I've used it when I needed to, but deep down I'm a Bourne-and-vi kind of a person, so Bash comes naturally. Wrote my first shell script about 35 years ago on a VAX 11-780...

But still, that's definitely off-topic and should be relegated to the old-guy-boring-everybody-with-tales-of-steam-driven-computers forum...

 

[MymsMan]

That seems like a hard way to do what a cheap timeswitch could do.

Not all cheap timeswitches are reliable - I have just put a lamp on one it it keeps switching itself off for a few minutes during its ON period, that would be disastrous for the Humax but fortunately that one (same model) is running smoothly

 

[prpr]

I'm assuming I can use nc -z [IP] [PORT No.]

Yes, that's one way. There are probably almost an infinite number of other similar ways with whatever software suits you or is to hand.

(41 years, in a language called CECIL, which I remember absolutely nothing about apart from it was designed for learning about computers... we soon moved on to Basic. Hmm.... a quick Google tells me it was actually CESIL)

 

[damianiw]

An interesting thread I guess some tort of heart beat from the Humax that the Pi checks for us what you’re looking for, both run Linux you maybe could write a script on the humax to ssh to the pi and touch a file then put that in a cron for once a minute - then check that On the pi- after so many missed heart beats you trigger your reboot ?

Either that or you try to telnet / ssh from the pi as that may not work if it’s gone rouge


Sent from my iPhone using Tapatalk

 

[Black Hole]

maybe could write a script on the humax to ssh to the pi and touch a file then put that in a cron for once a minute - then check that On the pi- after so many missed heart beats you trigger your reboot ?

That would be no better than pinging it. The problem is that Linux is multi-tasking, so your script on the Humax may well continue to operate even though settop (the Humax process) has crashed.

 

[/df]

The script could check that the humaxtv process is running.

However there is a failure mode, at least on HDRs, where this is true but the system is broken because the necessary filesystems have been remounted as read-only; moreover the system ignores software reboot invocations. The on-screen symptom is a frozen picture with live sound and no response to the remote.

So a time-switch seems like the only answer: perhaps a networked time-switch that could be rescheduled to avoid recordings and remote control activity?

 

[damianiw]

That would be no better than pinging it. The problem is that Linux is multi-tasking, so your script on the Humax may well continue to operate even though settop (the Humax process) has crashed.

Hmm I hadn’t thought of that, I’ve seen the post after as well, just a really out there thought but when the humax is hung it wouldn’t change channel would it ? So if we could check the channel then use the shell to change channel and check if it changed (maybe the IR package if there’s no shell command) would that give an indication ? I know it’s a bit clunky sounding ..


Sent from my iPhone using Tapatalk