Split transmissions not correctly handled by Humax scheduler

[/df]

This thread in the other place discusses the problem that, whereas split transmissions are identified by a Programme CRID followed by an Instance Metadata Identifier (IMI), such as www.itv.com/1001281561#1, and owing to a brain-dead specification the IMI is always the same for all parts and repeats of the split show, but repeats must be detected by starting more than 3 hours from the end of the previous set of split transmissions, the Humax software (HD/R-Fox T2 and some others) treats all occurrences of the P-CRID+IMI as parts of the same show regardless of the 3 hour rule. This results in false conflicts and possibly unwanted recordings.

Assuming that the on-box UI is a lost cause in this regard, could such situations be handled in WebIF or related functions so that (a) split repeats don't show as conflicts (b) if the scheduled split show has been recorded any repeats are marked as already recorded?

Or does this happen already?

Separately, are there ever 3-part splits, as rsv.class appears to assume not?

 

[mihaid]

Assuming that the on-box UI is a lost cause in this regard, could such situations be handled in WebIF or related functions so that (a) split repeats don't show as conflicts (

but surely you would want webif and rs TO SHOW conflicts because that repeat might knock off a genuine recording.

 

[MymsMan]

This thread in the other place discusses the problem that, whereas split transmissions are identified by a Programme CRID followed by an Instance Metadata Identifier (IMI), such as www.itv.com/1001281561#1, and owing to a brain-dead specification the IMI is always the same for all parts and repeats of the split show, but repeats must be detected by starting more than 3 hours from the end of the previous set of split transmissions, the Humax software (HD/R-Fox T2 and some others) treats all occurrences of the P-CRID+IMI as parts of the same show regardless of the 3 hour rule. This results in false conflicts and possibly unwanted recordings.

Assuming that the on-box UI is a lost cause in this regard, could such situations be handled in WebIF or related functions so that (a) split repeats don't show as conflicts (b) if the scheduled split show has been recorded any repeats are marked as already recorded?

Or does this happen already?

Separately, are there ever 3-part splits, as rsv.class appears to assume not?

One of the problems is that 'already recorded' is determined by crid and all of the parts have the same crid.

Annoyed by the moved episodes of Pointless I have started work on a schedule checker to hopefully identify and correct errors in the recording schedule so splt recording problems would be a candidate for fixing - whether they are fixable remains to be seen, when is another issue!

 

[hairy_mutley]

Sounds like the same thing...? I set Sony Movies, Kill Bill 1 & 2 to record; they have recorded twice already and showed a couple more to record (until I cancelled them). I have seen the problem on several recordings on that channel recently, so it may be a good place to look for repeatable examples.

 

[/df]

The conflict issue is that, for example with a split show broadcast on Sunday and repeated with the same split on Monday, the Humax conflict detector thinks that both the original and the repeat need to be recorded, so a conflict could be indicated incorrectly if other shows need to be recorded at the same time on Monday. Whereas with a non-split show, it "knows" that the EPG entries for the same P-CRID are alternatives.

It's possible that the Monday repeat could be appended to or duplicate the Sunday recording -- I haven't tested that.

One possible fix would be to remove or in some way disable the split recording once a good recording of one split set has been made.

Also, I don't see that the 3-hour rule is implemented in the conflict processing of rsv.class, but then I don't really understand how that code anywsy.

 

[hairy_mutley]

Try this... Pulp Fiction is on Sony Movies twice in the next 7 days, each time it is transmitted in 2 parts, all 4 parts have the same CRID.
If I schedule the first instance to record (from the WebIf), the WebIf schedule and the EPG only show the first part as recording.
Just in case... I rebooted, but it didn't change anything.
However, the RS now shows all 4 parts as scheduled.
From my experience with Kill Bill, the RS is probably showing what will actually happen. So , although the OP may not want it to do this, it can only show what the box will do. However, the discrepancy between RS & WebIf is a problem that needs looking at.

:edit: it turns out that the WebIf eventually shows all parts as scheduled.

 

[hairy_mutley]

Also being discussed on the Standard Firmware forum

 

[MymsMan]

edit: it turns out that the WebIf eventually shows all parts as scheduled.

That is BAU, for any recording the webif only puts in the first episode and the Humax fills in all the subsequent episode, the same happens with Refresh events
Quite how long the time before the Humax catches up seems variable

 

[Black Hole]

BAU

Business As Usual (now added to the Glossary, section "Netiquette").

 

[/df]

Also being discussed on the Standard Firmware forum

That is, the other place.

As suggested above I used these showings of Pulp Fiction for some tests.

1. Testing on HD (simpler as only one recording at a time)

After scheduling PF with the on-screen UI, the show appeared in the schedule with a green icon and a tell-tale instruction to use the left arrow for more episodes; once the EPG had refreshed several days into the future, left arrow showed the two Thursday parts plus the repeat Sat-Sun parts (initially showed only today's parts).

Trying to schedule a show that overlapped with the Sat-Sun parts produced a conflict in the on-screen UI.

The Webif showed all 4 parts of PF for recording. Trying to schedule a show as a manual event in Webif that overlapped with the Sat-Sun parts resulted in a conflict (marked in "pink" but actually mauve or lilac).

I cancelled this recording as space is tight on the attached disk.

2. Testing on HDR

Scheduling using the on-screen UI gave the same results as with the HD, except that (i) the 4 parts were listed from the left arrow immediately (the box had been on for some time) and (ii) two overlapping shows had to be scheduled to produce the conflict, as expected.

The Webif display was the same as with the HD and scheduling using Webif gave the same results, except that two overlapping manual event shows had to be scheduled to produce the conflict, as expected. The conflict colour was still wasn't pink; conflict resolution took 0.823s.

I left this recording in place for further observation.

 

[/df]

So of course it just recorded the two repeat parts. At least that gives a chance to cut off the end of the first part at a different point.

Considering that we have detectads, perhaps the Right Thing would be to transform such a split recording into a single recording starting at the beginning of the first part and ending at the end of the second part. Is there a way of hacking the reservation parameters to do that?

 

[MymsMan]

Not without creating manual recordings which would lose any AR tracking and the intervening programme would normally be too long to be detected as an ad break

 

[Black Hole]

Is there a way of hacking the reservation parameters to do that?

No, processing is required post-recording, but recordings can be merged (see join option).

 

[/df]

I think it would generally be better if the system could be persuaded to record each set of parts as one programme when the intervening programme is sufficiently short; Sony Movies aka MovieMix typically inserts 5 minute intermissions. Even if detectads wouldn't normally treat the intermission as an ad it would still be possible to have the programme bookmarked and edit the intermission bookmarks out.

One clear result is that the WebIf schedule display is misleading, if not incorrect, for split transmissions. It lists all the other parts (both initial showing and repeats) as "also:" when they are not actually alternatives, and it doesn't clarify why a conflict might be flagged, whether according to Humax's defective interpretation or not.

Tomorrow we are being offered "Layer Cake" (21:00-21:55, 22:00-23:10), which I scheduled for recording using WebIf EPG:

# sqlite3  /var/lib/humaxtv/rsv.db
SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
sqlite> .mode line
sqlite> select * from TBL_RESERVATION where ulslot = 1; /* manually edited binary data to \x format below */
              ulslot = 1
            ersvtype = 3
                hsvc = 65555
             nsttime = 1586203200
            szsttime = 00000000000000
           nduration = 3300
             erepeat = 0
             usevtid = 6688
           szevtname = \x15Layer Cake
         ulPreOffset = 0
        ulPostOffset = 0
         ulProgramId = 0
          ulSeriesId = 0
            ucVolume = 0
         ucInputMode = 0
             usChNum = 0
           ucRecKind = 2
          ucCRIDType = 49
              szCRID = WWW.MOVIEMIX.CO.UK/X4558980
        szFPBRecPath = 
  szRecordedProgCrid = 
     szEventToRecord = 1WWW.MOVIEMIX.CO.UK/X4558980#01|1WWW.MOVIEMIX.CO.UK/X4558980#01|
aulEventToRecordInfo = \x13\x00\x01\x00@\x8a\x8b^$\x97\x8b^ \x1a\x00\x00
           bRecomRsv = 0
 usLastRecordedEvtId = 0
              eReady = 30
sqlite>

This test is to see what happens after setting nduration to the total duration of the two parts including intermission, ucRecKind to show a normal EPG-based recording, szEventToRecord to show the first p-CRID with no IMI suffix, and updating the endTime field (offset 8) of aulEventToRecordInfo to the epoch value of nsttime+nduration.

sqlite> update TBL_RESERVATION set nduration=7800, ucRecKind=1, szEventToRecord='1WWW.MOVIEMIX.CO.UK/X4558980|', aulEventToRecordInfo =x'13000100408a8b5eb8a88b5e201a0000' where ulslot = 1;

The on-screen schedule shows no change until the HDR is restarted, but presumably RTS could fix that. After restarting it appears as a single EPG-scheduled recording from 21:00-23:10. What will happen tomorrow night? Stay tuned.

 

[Black Hole]

Yeah, but you also need a test case where the EPG item changes transmission time.

 

[/df]

in fact what happens is that if you select the modified item in the on-screen schedule list it gets updated to the length of the first part. Presumably the start_time and duration fields of the EPG information for "Sony Movies" in the Event Information Table from the transmission stream are being used to update the schedule. Presumably also there is some monitoring of the transmitted EPG data that will confound this scheme by updating the scheduled duration, if only just before the recording starts, or, as suggested, if the start time changes.

These are the relevant D-Book v7 requirements for a PVR. For an HD PVR there is an additional requirement to persist the EPG data across shutdowns.

A programme may consist of multiple EIT events as signalled by the broadcaster (for example, a film divided into two parts by a news programme would be three events). Signalling carried in the SI (see Section 8.5.3.12) allows the [PVR] to identify and record the events containing the parts of a single programme. A recorder shall implement this function.

The referenced section 8.5.3.12 specifies how each event is linked to one p-CRID or several CRIDs of other types by a Content Identifier Descriptor (CID).

When the programme selected [comprises] more than one event, all constituent events shall be selected so that the complete programme is recorded.
...
Once selected, the appropriate programme(s) shall be marked as selected on the EPG display.

When the [PVR] is not in passive standby [the state in which the PVR is inactive as far as the user is concerned and no broadcast signal is being decoded] and a schedule change occurs, the affected programmes in the schedule of recordings and any recordings in progress shall be updated.

In standby, a [PVR] shall monitor the EITp/f sufficiently frequently and for sufficient duration to allow a programme to be recorded successfully even when the start time is brought forward by up to ten minutes and the schedule information is updated at least 5 minutes before the new start time.

Hence the need to wake the PVR at least 15 minutes before a scheduled event. The implication is that when not in standby the PVR has to check the EPG at least every 5 minutes thereafter and possibly update the schedule if the EITp/f (ie, Now/Next) data indicates a timing change.

As a minimum, the [PVR] shall incorporate a default mechanism for controlling the starting and stopping of a recording based on the broadcast EITp/f. Additional mechanisms may be incorporated as an option.

The mechanism used shall allow complete events, as defined by the broadcaster in the EITp/f, to be recorded (less up to 10s for channel changing and service acquisition) and shall track changes to the start time and end time of the event.

The start of an event is indicated by its transition to the present event for the specified service. The end of an event is indicated by the event being replaced by a different event as the present event for that service[.] It is permissible for a recording to start before the start of an event and to finish after the event, but this may create unnecessary conflicts with the requirement for a back-to-back recording capability.

Note
The information about the now/next events and the transitions between them is the only broadcast information sufficiently accurately related to the broadcast events to give accurate control of event recording. See Section 8.10.1.

The referenced section 8.10.1 describes the use of Now/Next data for timing and that the EITp/f:Other data transmitted in one mux may signal the start of a programme carried by another mux before the EITp/f:Actual data sent with that mux has been updated.

The [PVR] shall be able to record back-to-back events on the same service without registering this as a conflict[.]

A conflict which is detected at the time of making a booking shall be indicated immediately, together with details of the cause, so that the user can take appropriate action.

The default action taken by the [PVR] (with no user interaction) shall be made clear to the user in the manual and there shall be a mechanism for informing the user of failed or incomplete recordings.

When scheduled recordings overlap, the [PVR] shall use the alternate instance information, when provided, to record one or more of the programmes at their alternate times thereby minimising the conflict, subject to any device limitations (e.g. available space).

And of course the underlying problem requirement specifying how an event's CID may indicate a split transmission:

A CRID in the CID shall be a programme CRID (crid type 0x31) with an IMI extension. Where two events have the same CRID and IMI value and the gap between each event is less than 3 hours (measured from the end of the preceding event to the start of the next event) then they shall be considered to be segments of a single item of content. An item of content may be split across more than two events as long as the gap between each event remains less than 3 hours.

One might think that this IMI should have been organised as <split-instance>.<split-part> so obviating the need for the 3-hour rule and enabling the PVR to identify the actual programme and its repeats and potentially resolve conflicts by (eg) recording Part 1 from the initial showing and Part 2 from a repeat.

Whereas the same show on the +1 channel has the same p-CRID with a #02 IMI, distinguishing it from the +0 version but also ruining any chance for the recorder to mix-and-match parts from +0 and +1 to resolve a conflict. But they can't use the same IMI because the convention doesn't distinguish between parts and alternates. Dohhhh.

 

[Black Hole]

One might think that this IMI should have been organised as <split-instance>.<split-part> so obviating the need for the 3-hour rule

One would indeed, but you know what standards committees are like. Maybe whoever proposed that particular version wasn't well liked.

However, there is one small glimmer of hope: Humax haven't implemented the 3-hour rule, so maybe they haven't been diligent in following some other rule either.

 

[mihaid]

Does this not amount to a massive hammer to crack the tiniest of nuts? I definitely don't dismiss EVERYTHING you guys do but is this taking it to extremes when the following occurs:
notified that an extra recording of a split prog already recorded is causing a conflict by the excellent email service invented by you. go to the rs and DELETE the focker.
done.

 

[hairy_mutley]

notified that an extra recording of a split prog already recorded is causing a conflict

Except that you might be getting that notification for several days before the initial recording takes place?

What about the situation where it is the second instance that you want to record because the first causes the conflict?
Ok, there may not be a suitable automated solution, but, given that someone is willing, it seems worth investigating.

 

[mihaid]

What about the situation where it is the second instance that you want to record because the first causes the conflict?

then delete the first one. simples