Strange crash and alerting user when there is a problem

[rodp]

Hi All,

A strange crash with the following log output from humaxtv log just occurred. It started to record The A Team on SPIKE at 17:00 today and 3 minutes into it, the box hung. The only way I new that it had hung was just by chance I was playing an mp3 file from it to my PC and it stopped playing. There was no response from webif or pinging the box and the ring remained unresponsive either via remote or via touching it. It just stayed on Red but there was no normal clicking noises from the HDD so I guessed something was up. Nevertheless I left it until the recording time had passed to see if it did anything but no, it just stayed on red ring.

So I power cycled and inspected the logs. To my surprise, Webif didn't mentioned anything about crashing, nor non of the logs mentioned anything. The only thing I could find was the aftermath when the humaxtv is adding things to the DB and adding in the file (putting it to a failed recording) plus the last boot reason 'Unkown (4)'

75 DB InPut Success =/mnt/hd2/My Video/Strictly - It Takes Two/Strictly - It Takes Two_20171011_1829.ts (I:1/F:0)
74 [DI_MEDIA_Probe] Error - Fail in P_MEDIA_ProbeOpen
73 Error>> Cannot support video format
72 DB InPut Success =/mnt/hd2/My Video/The A-Team/The A-Team_20171011_1700.ts (I:0/F:0)
71 [RR]: Switched to persistent log file.
70 [ifss_start] +++++​

Web interface version: 1.4.2-7
Custom firmware version: 3.10 (build 2734)
Humax Version: 1.03.12 (kernel HDR_CFW_3.10)
Loader Version: a7.33
System ID: ....
Serial Number: ....
Last Boot Reason: Unknown (4)​

So...... after checking all the logs, where do I go from here and do other people experience issues like this? (It's recorded many of the other programs in the series over the last few weeks on SPIKE so I don't think it's the channel).

Is there a log that I'm missing that captures all the times that there are crashes /missed recordings or at least when recording fail? (perhaps humaxtv.log is it?!). Alternatively, is there a way for the humax to send out a 'I'm still alive' signal whilst recording / on standby to the rs site and if the right shut down sequence isn't transmitted with no 'I'm alive' signal sent, then can the RS site send an email?

Any guidance most welcome.

Thanks

Rodp

PS. Has anyone thought about adding an LED to the HDD so that you can at least see if there is any read / write activity? (I know you'd have to slice through the warranty tab to do that to gain access). (I know you could tap into the ribbon cable in IDE days (pin 38?) but can you do that with SATA drives?!)

 

[Black Hole]

(I know you'd have to slice through the warranty tab to do that to gain access).

What warranty?

Have you looked at Commissioning an HDR-FOX (click)?

do other people experience issues like this?

Yes. See Steps for Resolving HDR-FOX Crash/Reboot Issues (click).

Is there a log that I'm missing that captures all the times that there are crashes /missed recordings or at least when recording fail?

No. If it crashes inside the humaxtv process, there is no opportunity for the CF to record anything other than an unexpected reboot.

Alternatively, is there a way for the humax to send out a 'I'm still alive' signal whilst recording / on standby to the rs site and if the right shut down sequence isn't transmitted with no 'I'm alive' signal sent, then can the RS site send an email?

The HD/HDR-FOX rs package phones home every ten minutes; IIRC you can set RS to email you if the unit isn't seen for a selectable period of time.

I'm thinking about some hardware to monitor a network ping, and power-cycle the mains if it drops out - but it's in a long list of "to dos".

 

[rodp]

Thanks for the reply BH. Will have a read. I came across something I thought was going to help me just now.... https://hummy.tv/forum/threads/crash-diag-log.5451/ as it talked about the crashdiag log. I found I could access this via the diagnostics page and running the crashdiag one form the drop down list (to save me having to use telnet / ftp). I was getting very excited as it listed through all the different crashes in chronological order, however it stopped at the last but one crash! doh!

*********** Core file /mod/core/humaxtv.1505895608.290 ***********
Wed Sep 20 09:20:08 BST 2017​

So it seems that the crash I experienced today wasn't a crash that the humax could handle or even new about otherwise it would have wirrten a log.


So this is the log of the last crash for those out of interest. What does Segmentation fault mean?

*********** Core file /mod/core/humaxtv.1505895608.290 ***********
Wed Sep 20 09:20:08 BST 2017
Core was generated by `/usr/bin/humaxtv'.
Program terminated with signal 11, Segmentation fault.
#0 0x008a18a8 in ?? ()
[Current thread is 1 (Thread 2467)]
Thread
#0 0x008a18a8 in ?? ()

Thread 112
#0 0x2c58ae3c in recvmsg () from /lib/libpthread.so.0
#1 0x2c58ae20 in recvmsg () from /lib/libpthread.so.0

Thread 111
#0 0x2c59d6bc in ioctl () from /lib/libc.so.0
#1 0x2c07924c in ?? () from /usr/lib/libnexus.so

Thread 86
#0 0x2c58ae3c in recvmsg () from /lib/libpthread.so.0
#1 0x2c58ae20 in recvmsg () from /lib/libpthread.so.0

Thread 74
#0 0x2c59ec20 in select () from /lib/libc.so.0
#1 0x2aadddf8 in _ftext () from /var/lib/humaxtv_backup/mod/libnugget.so
#2 0x00000020 in ?? ()

Thread 73
#0 0x2c5791ec in mq_timedreceive () from /lib/librt.so.0
#1 0x2c5791cc in mq_timedreceive () from /lib/librt.so.0

Thread 70
#0 0x2c58a5dc in read () from /lib/libpthread.so.0
#1 0x2c58a5c0 in read () from /lib/libpthread.so.0

Thread 68
#0 0x2c59ec20 in select () from /lib/libc.so.0
#1 0x008fd36c in ?? ()

Thread 5
#0 0x2c58aa6c in accept () from /lib/libpthread.so.0
#1 0x2c58aa50 in accept () from /lib/libpthread.so.0​

Thanks

Rodp

 

[af123]

That's a crash in the DLNA streaming code.
A Segmentation Fault means that the program tried to access memory that it didn't own - it's a programming bug.

 

[Black Hole]

IIRC a segmentation fault is when a process tries to access a memory area outside its permitted range.

Edit: snap.

 

[Black Hole]

As I have described elsewhere, crashes appear to be most often associated with the network. What I believe happens is that some network communications error occurs which wasn't tested for in development (it is very time consuming to set up test examples of every possible external error condition that could occur, so no surprise that they didn't), and the code either bombs out because it doesn't handle the error correctly, or it fails to trap the error at all and passes out-of-bounds data into code that only expects good data.

Network traffic is the only real uncontrolled variable in the HDR-FOX's operating environment, and disconnecting it from the network substantially reduces the incidence of crashes. My boxes used to crash quite frequently until I updated their homeplug networking firmware (improving the reliability of the network), and my "supported user" isn't connected to the network at all and never reports to me that her box has crashed (although, to be fair, she might just turn it off and on again).

 

[rodp]

AF123..... so this error log could well be the moment the android app I use called Hi-Fi Cast accesses the Humax DLNA server / service. The humax crashes and reboots if I refresh the list of DLNA servers available on my network and then go into Mediatomb on my phone. Once I'm in the directory list (ie after the humax reboots) I'm fine - it's just at the initial entering of the directory list I have the problem. I can't at present test my theory as on Monday my phone decided to brick itself (completely stock item, no custom stuff and I can't even get the factory reset to work - rubbish! LG G3 by the way! Sorry I digress!!)

Thanks

Rodp

 

[Black Hole]

We have had something similar reported before, I'll see if I can find the reference. IIRC the poster wanted to get the app developer to avoid crashing the Humax - but that seemed unlikely to happen to me!

 

[af123]

I'm sure I could patch the Humax software for this but it would take a few days of work and it's probably not the only place at which crashes occur..

 

[Black Hole]

Even that one instance seems like one heck of a lot of work to me. You would need to work out what circumstances lead up to that particular fault, and what the code should do under those curcumstances.

 

[rodp]

@BH, that similar report before might have been me! Not progressed it any further with the developer as I need to grab some packet data no doubt but I'd be interested to know if the issue occurs with other people's setup / phone etc. As you say, I would imagine it's alot of work to pin down the cause, hence I have in some sense waited to see if anyone else confirms the same issue. clearly people aren't using Hi-Fi Cast but something else like BubbleUPNP.

 

[Black Hole]

Or maybe not using the dicky DLNA service at all and opting for SMB or NFS file sharing instead. Okay, so DLNA doesn't require pre-decryption, but...

 

[rodp]

I suppose what you're saying is to use some other form of media server then as apps like BubbleUPNP (which can then push the music etc. to a Chromecast for example) need DLNA. DLNA as I understand it is the main way to search by track / album / genres etc. too.... or am i behind the times?!

Thanks

Rodp

 

[Black Hole]

Do you really need that kind of search capability when accessing recordings on the HDR-FOX? Just putting them in appropriate folders is sufficient for me. In any case, I'm not aware that the Humax DLNA server can provide indexing of that nature - all you get is a list.

BTW: I can play HDR-FOX .ts files perfectly well via DLNA using the VLC app on my iPad.

 

[rodp]

The search capability is helpful but to be honest the main reason I use these apps is to cast music to my Chromecast audio. I have to use BubbleUPNP or HiFi Cast which are DLNA based (I don't think they let you navigate SMB folders. Local folders maybe on the phone but not network based devices). There is also another reason to use the DLNA service on the humax as you can control realtime conversion. Chromecast won't play MP3 Layer 2. It only likes Layer 3 but I've not gone down the route of converting all my files to MP3 Layer 3. So the Mediatomb converts the mp3 files to WAV in realtime and streams it to the Chromecast. Complicated but it works (most of the time!).

 

[robman84]

Sorry for the necro-thread but my HDR Fox-T2 has been exhibiting the exact same behaviour. I tried every fix I could find and eventually installed the custom firmware. Now the crash is *every* time I restart the hummy, and almost exactly after an uptime of 75 each time. After the crash it works fine all day until the next day when I turn it back on again. It is set to automatically come on at 04:20 to disable OTA as well and that crashes after uptime of almost always 230. The crashdiag is almost exactly the same as the one posted above. Not necessarily hoping for a suggestion, but thought my info might help those who might be fixing problems. :

>>> Beginning diagnostic crashdiag

Running: crashdiag
1.03.12
*********** Core file /mod/core/humaxtv.1514487787.246 ***********
Thu Dec 28 19:03:07 GMT 2017
Core was generated by `/usr/bin/humaxtv'.
Program terminated with signal 11, Segmentation fault.
#0  0x008a18a8 in ?? ()
[Current thread is 1 (Thread 1636)]
Thread
#0  0x008a18a8 in ?? ()

Thread 112
#0  0x2c55de3c in recvmsg () from /lib/libpthread.so.0
#1  0x2c55de20 in recvmsg () from /lib/libpthread.so.0

Thread 111
#0  0x2c5716bc in ioctl () from /lib/libc.so.0
#1  0x2c04d24c in ?? () from /usr/lib/libnexus.so

Thread 86
#0  0x2c55de3c in recvmsg () from /lib/libpthread.so.0
#1  0x2c55de20 in recvmsg () from /lib/libpthread.so.0

Thread 74
#0  0x2c572c20 in select () from /lib/libc.so.0
#1  0x2aacadf8 in _ftext () from /var/lib/humaxtv_backup/mod/libnugget.so
#2  0x00000020 in ?? ()

Thread 73
#0  0x2c54c1ec in mq_timedreceive () from /lib/librt.so.0
#1  0x2c54c1cc in mq_timedreceive () from /lib/librt.so.0

Thread 70
#0  0x2c55d5dc in read () from /lib/libpthread.so.0
#1  0x2c55d5c0 in read () from /lib/libpthread.so.0

Thread 68
#0  0x2c572c20 in select () from /lib/libc.so.0
#1  0x008fd36c in ?? ()

Thread 4
#0  0x2c55da6c in accept () from /lib/libpthread.so.0
#1  0x2c55da50 in accept () from /lib/libpthread.so.0





>>> Ending diagnostic crashdiag

 

[robman84]

update: Installed DLNA filter and added the IP of my WD MyCloud - seems to be not crashing now (fingers crossed). When I left the IP address of DLNA filter blank, all DLNA traffic on my network stopped working. But when I added in the WD, I could access its server. Hopefully, it'll all carry on working!