Is this a Humax failing, or more fundamental?

I noticed several months ago that UTV (Free view) and TV3 (Irish saor view) had the exact same series crid for their series. I had a screenshot of the webif schedule at the time but I deleted it.

These series had the same series crid and multiple events were listed in webif but the HDR only scheduled recordings related to the original recording.

Interesting.
The HDR has some additional logic that caters for not recording identical broadcasts from other channels (but not the prime channel). When the web-if scheduling was first being used IIRC there was a lot f talk concerning Home and Away recording too many times if the web-if was used. At the time Eastenders and Formula 1 were being given a change of series CRID when broadcast on a different channel despite them being the same series. IIRC with the unreliability of both C5 and BBC the Web-if was changed not to look for episodes on other channels.
Now that the BBC are getting is act together more often and that c5 are at least attempting to use CRIDs in the same way as the other channels what was expedient and appropriate then for the web-if is not necessarily the best solution now.

So far the closest I can theorise about what the logic the HDR-FOX uses is that if the HDR-FOX is scheduling a series it identifies all other channels that have an entry with the same series CRID (including the series CRID prefix). Then if any of the programmes ids for that series CRID on another channel matches any of the programmes ids (in the epg) for the series on the initial channel it ignores all episodes from that other channel. That logic would have the same result as I have obtain on my HD-FOX and HDR-FOXs when I have noticed series CRIDs across channels for either repeats or channel jumping. Whether it is the same logic as the HDR-FOX and HD-FOX uses I didn't know.

A question for you Border: What I found interesting about your freeview/saorview example is that it demonstrates that the HDR-FOX and HD-FOX may be using some other logic to decide what it schedules rather than my latest theory, unless you were only comparing part of the CRID. When the HDR-FOX reports the series CRID on the i-plate it also includes the prefix. Were you including all of the series CRID in your comparison or just the last part?
E.g. A BBC CRID of 'FP.BBC.CO.UK /R1A72N' is not the same as a channel 5 CRID of 'WWW.FIVE.TV /R1A72N'.
If you were comparing just the last part the that would account for why the HDR-FOX was able to differentiate between the two initially looking similar CRIDs in your example.
 
I have deleted the screenshot so I can't be 100% sure but I think they were exact matches. I thought it was unusual that I tried scheduling series and set up conflicting schedules to see if the humax would shift channels in subsequent scheduled events but it didn't.
 
The broadcasters set the SCRID and CRID info. If that is wrong, then the machine is going to do the "wrong" thing.
The Humax keeps a list of CRIDs it has already recorded in the schedule booking info. for the series link recording
e.g. for the previously mentioned Cycling stuff, szRecordedProgCrid is now "1FP.BBC.CO.UK/7A3B3P|1FP.BBC.CO.UK/7A3AY3|" which were yesterday's two programmes and szEventToRecord is now just "1FP.BBC.CO.UK/7A3AY4|" which is today's.
This is presumably what stops it recording duplicates.

Anyway, I have verified that the cross-channel series links are correctly put into /var/lib/humaxtv/rsvp.db, so it must be something else that is stripping them out.
My guess is that it's /mod/boot/rsvsync, which runs at startup and moves the entries to /var/lib/humaxtv/rsv.db and as this is a compiled app. it's going to need af123 to investigate any further.
 
My guess was wrong. /mod/boot/rsvsync is not responsible for this, as taking a copy of the database after it has run shows.
It appears to be the Humax software that is doing it.
Here are some database dumps (lines split to aid readability).

Webif scheduled:
Code:
INSERT INTO "pending"         VALUES(0,3,655553,1412297400,'00000000000000',
6600,0,52487,'§Formula 1: The Japanese Grand Prix -...',0,0,0,0,0,0,0,4,50,
'FP.BBC.CO.UK/TTOKMT','§Formula 1: The Japanese Grand Prix -...','',
'1FP.BBC.CO.UK/7A3ACU|1FP.BBC.CO.UK/7A3ACV|1FP.BBC.CO.UK/7A3ACW|1FP.BBC.CO.UK/7A3ABD|1FP.BBC.CO.UK/7A3ABI|1FP.BBC.CO.UK/7A3AB4|1FP.BBC.CO.UK/7A3ABK|',
NULL,
0,0,30,0);
Webif scheduled, reboot, Humax modified:
Code:
INSERT INTO "TBL_RESERVATION" VALUES(1,3,655553,1412297400,'00000000000000',
6600,0,52487,'§Formula 1: The Japanese Grand Prix -...',0,0,0,0,0,0,0,4,50,
'FP.BBC.CO.UK/TTOKMT','Formula 1: The Japanese Grand Prix -...','',
'1FP.BBC.CO.UK/7A3ACU|1FP.BBC.CO.UK/7A3ACV|1FP.BBC.CO.UK/7A3ACW|',
X'C100 0A00 B8F2 2D54 800C 2E54 07CD 0000
  C100 0A00 242C 2E54 9443 2E54 08CD 0000
  C100 0A00 7453 2F54 DC63 2F54 09CD 0000',
0,0,30);
Humax scheduled:
Code:
INSERT INTO "TBL_RESERVATION" VALUES(1,3,655553,1412297400,'000000000000000☺',
6600,0,52487,'§Formula 1: The Japanese Grand Prix -...',0,0,0,0,0,0,0,4,50,
'FP.BBC.CO.UK/TTOKMT','§Formula 1: The Japanese Grand Prix -...','',
'1FP.BBC.CO.UK/7A3ACU|1FP.BBC.CO.UK/7A3ACV|1FP.BBC.CO.UK/7A3ACW|1FP.BBC.CO.UK/7A3ABD|1FP.BBC.CO.UK/7A3ABI|1FP.BBC.CO.UK/7A3AB4|1FP.BBC.CO.UK/7A3ABK|',
X'C100 0A00 B8F2 2D54 800C 2E54 07CD 0000
  C100 0A00 242C 2E54 9443 2E54 08CD 0000
  C100 0A00 7453 2F54 DC63 2F54 09CD 0000
  C400 0A00 C070 2F54 E893 2F54 1ECE 0000
  C400 0A00 40E1 2F54 58F6 2F54 D6DD 0000
  C400 0A00 50D0 3054 04FE 3054 DEDD 0000
  C400 0A00 4436 3154 6452 3154 DFDD 0000',
0,0,30);

It looks like there are a couple of tiny changes to strings, but the major one is the aulEventToRecordInfo field. No idea what any of that means or whether it can be fabricated in advance.
 
Last edited:
aulEventToRecordInfo consists of 16-byte fields. Each of the fields contains four 32-bit numbers which are service, start time, end time and event ID. They're stored in system byte order so, for example, the first entry in the Humax scheduled list has:

Start: 0x542df2b8 = Fri Oct 3 01:50:00 2014
End: 0x542e0c80 = Fri Oct 3 03:40:00 2014
 
For documentation reference purposes, the "service" field is a combination of svcIdx and tsIdx from TBL_SVC in the /var/lib/humax/channel.db database.

I have tried supplying all the info. in rsvp.db and that does indeed keep the Humax software happy and allows it to work, so it's 'just' a matter of someone fixing the Jim script.
 
...so it's 'just' a matter of someone fixing the Jim script.
So, for a complete Jim novice, I've had a go and it works, although it doesn't handle backup/restore (yet), and there are probably better ways of doing some things...
Code:
humax# diff -u /mod/webif/lib/rsv.class~ /mod/webif/lib/rsv.class
--- /mod/webif/lib/rsv.class~
+++ /mod/webif/lib/rsv.class
@@ -255,7 +255,7 @@
  set fields [lsort [$self vars]]
-  foreach field {aulEventToRecordInfo szSvcName usLcn sort} {
+  foreach field {szSvcName usLcn sort} {
  set df [lsearch $fields $field]
  set fields [lreplace $fields $df $df]
  }
@@ -269,8 +269,11 @@
  foreach field $fields {
  # Escape any quotes embedded in the data.
  regsub -all {'} [$self get $field] {''} f
-  lappend vals "'$f'"
-  #lappend vals "'[$self get $field]'"
+  if {$field eq "aulEventToRecordInfo"} {
+  if {$action == 0} {
+  lappend vals "X'$f'"
+  } else { lappend vals "''" }
+  } else { lappend vals "'$f'" }
  }
  set query "insert into ${table}("
@@ -484,13 +487,24 @@
  # Series
  set args(ucCRIDType)  50
  set args(ucRecKind)  4
-  set args(szCRID)  "$ccrid[$event get series_crid]"
+  set scrid  "[$event get series_crid]"
+  set args(szCRID)  "$ccrid$scrid"
  set args(szFPBRecPath)  "$args(szevtname)"
-  set progs [lmap i [epg fetch dump -scrid [$event get series_crid]] {
+  set epgdump [epg fetch dump -scrid $scrid]
+  set progs [lmap i $epgdump {
  if {[set ecrid [$i get event_crid]] eq ""} { continue }
  list "1$::ccrid$ecrid"
  }]
  set args(szEventToRecord)  "[join $progs "|"]|"
+  set eventinfo [lmap i $epgdump {
+  if {[set ecrid [$i get event_crid]] eq ""} { continue }
+  $i get_channel_info
+  set sttime [$i get start]
+  list "[rsv ultohex [list \
+  [$i get channel_hsvc] $sttime \
+  $($sttime + [$i get duration]) [$i get event_id]]]"
+  }]
+  set args(aulEventToRecordInfo)  "[join $eventinfo ""]"
  }
  return [rsv new $args]
@@ -664,3 +678,14 @@
  close $fd
 }
+proc {rsv ultohex} {ulVals} {
+  set hex ""
+  foreach ulVal $ulVals {
+  for {set i 0} {$i < 4} {incr i} {
+  append hex [format %02X $($ulVal & 255)]
+  set ulVal $($ulVal >> 8)
+  }
+  }
+
+  return $hex
+}
 
So, for a complete Jim novice, I've had a go and it works, although it doesn't handle backup/restore (yet), and there are probably better ways of doing some things...
Nice! I'll incorporate that. I'm not sure there are many better ways of doing things. I'd use the pack extension rather than your ultohex function but it looks good to me.
 
Here's the patch for backup/restore:

Code:
@@ -516,8 +533,11 @@
     puts -nonewline $fd "event\t"

     foreach f $fields {
-       if {$f eq "aulEventToRecordInfo"} { continue }
-       puts -nonewline $fd "[$event get $f]\t"
+       set ret [$event get $f]
+       if {$f eq "aulEventToRecordInfo"} {
+         binary scan $ret H* ret
+       }
+       puts -nonewline $fd "$ret\t"
     }
     puts $fd ""
   }
@@ -572,7 +592,6 @@
     set vars {}
     set i 0
     foreach f $fields {
-       if {$f eq "aulEventToRecordInfo"} { continue }
       incr i
       lappend vars $f [lindex $vals $i]
     }
 
I'm not entirely convinced. Haven't you effectively just changed the format of the file by doing that? Surely aulEventToRecordInfo now needs to go on the end of the line to preserve backwards compatibility?
Trying a manual backup and looking at it in WebIf certainly gives some strange results.
e.g.
Listing scheduled events from auto-2014-Oct-07-02:08...
--- Auto Update --- ()
BBC ONE West (BBC ONE West)


Listing scheduled events from backup-2014-Oct-08-00:16...
--- Unnamed reminder --- ()
BBC ONE West ()

TBH, position dependence of fields in files like this is rather a long way up the horrid scale in my book.
 
Last edited:
I'd use the pack extension rather than your ultohex function but it looks good to me.
Fine. I wish I'd known about it. It's not exactly easy to spot in the docs. and they are the usual terse prose.
When you find the real Jim documentation page, it has a page for extensions but pack isn't one of them. <sigh>
 
Restore also needs to fix up the hsvc field in the aulEventToRecordInfo data. All this is why I ducked it first time round!!!
 
I'm not entirely convinced. Haven't you effectively just changed the format of the file by doing that?
Oh, I see you've just updated it all with 1.0.17 and new backups now work, but it still doesn't retain backwards compatibility on the "View Backup" on Webif:

Listing scheduled events from auto-2014-Oct-07-02:08...
20141007043000 ()
20150105220000 (BBC ONE West)


I don't know what a restore from an old backup might do either...
 
I don't know what a restore from an old backup might do either...
Restore from old backup will be fine - I forgot to add the backwards compatibility to view though.
Fixing up hsvc during restore is going to be nasty... good point... I will address that.

There were a couple of other things I had to fix too such as the routines that copy schedule entries to create things like pending folder change events.
 
I have the restore working now. Will just wait to see how mine records over the coming few days.
 
In terms of just comparing the schedule when set via the remote and when set by the web-if I selected 3 series for the different ways that their broadcaster has used/misused the series crid. The result is spot on with Quest+1 not being picked up, and the HDR making the most of the channel 5 oddity. I like that the web-if shows the channel jumping but on the box's own schedule display when the I-plate is used within the schedule it displays the wrong channel for those (regardless of how the schedule was set).

Difficult CRIDs.png
 
I like that the web-if shows the channel jumping
Me too!
when the I-plate is used within the schedule it displays the wrong channel for those (regardless of how the schedule was set).
It doesn't matter how the schedule was set any more. The Webif is functionally equivalent to the Humax remote now.
 
It doesn't matter how the schedule was set any more. The Webif is functionally equivalent to the Humax remote now.
Yes, I know that. I was just trying to emphasise that, plus also make clear that the bug in the HDR-FOX T2's schedule display is nothing to do with the web-if and is there "regardless of how the schedule was set". I obviously failed!
 
I've been thinking about this a bit more and this seems like the best thread to put some thoughts in..

There is a field in the scheduling database called aulEventToRecordInfo (the aul probably means array-of-unsigned-longs or something similar). It stores a list of matching episodes for a series recording. Before the latest update anything scheduled through the web interface or RS left this field empty and so did restoring from a backup.

What happened was that the Humax software filled it in at some point. For those with long memories, the presence of this field was what triggered the accepted lightning flash on the schedule page that was removed some time ago. At the time I didn't know what was in the field - just a sequence of binary characters. At some point I did work this out (revision controls says December 2011) and updated the schedule display page to show all upcoming episodes based on it.

With the latest webif version, based on prpr's patch, anything scheduled through the web interface and RS does populate this field in the same way that the Humax would. I have a working patch for backup restore that fixes up the field too but I'm concerned that it isn't doing enough (@prpr the event IDs might need updating too...).

Now the thing is that when the Humax was left to fill in that field, it would not track the recording across channels which is what caused the disparity between the two scheduling methods. What I'm left wondering is what happens to the schedule as new matching events come into the EPG data? Does it continue to track recordings across channels after the initial series recording is set or does it only jump channels for the first 7 days?

If the nightmare neighbour series continues with the odd jump to 5* then it would be a good test for this or if anyone has the entire formula1 series set we could tell in November.
 
Back
Top