Media List Sort Order

SD channels from HD muxes also seemed to have this prefix, but I can't be sure which those were at the time of recording

That sounds intriguing. Are all the above historic recordings? As far as I can find out, the only SD channel currently carried on the HD Mux (PSB3), is channel 66 TBN. On a sample of one recording, this looks to follow the normal SD conventions for strings.

I'm not familiar with the internal structure of the custom firmware. Is it described anywhere? Is the insertion of the X'15' in the Title field all down to one shared routine?

Thanks again for your help.
 
That sounds intriguing. Are all the above historic recordings? As far as I can find out, the only SD channel currently carried on the HD Mux (PSB3), is channel 66 TBN. ...
In London we have (for now) 2 HD muxes. The SD channels I mentioned were all on one or other at some point, in particular when the recordings were made.
I'm not familiar with the internal structure of the custom firmware. Is it described anywhere? Is the insertion of the X'15' in the Title field all down to one shared routine?
Essentially, yes: see https://hummy.tv/forum/threads/media-list-sort-order.10539/post-160886.

proc {ts renamegroup} at l.466ff in /mod/webif/lib/ts.class is called with prefix octal 25 == 0x15 at the end of the block containing /mod/webif/html/browse/rename.jim l.30:
Code:
# find /mod/webif -name '*.jim' -exec grep -Hn '\\025' "{}" \;
/mod/webif/html/browse/rename.jim:30:                           set new "\025$new"
/mod/webif/html/browse/index.jim:114:   if {[string index $bfile 0] == "\025"} {
/mod/webif/html/m/browse.jim:47:        if {[string index $bfile 0] == "\025"} {
#
 
Last edited:
The current version of hmt reads from either 0x17F or 0x180, although I think that's just a fortuitous bug.
Sometimes you see the same character at 0x17F as at 0x180, which is probably the result of two things writing in different places. This can be detected and corrected for the vast majority of cases though.
So, when modifying the name here, should it write to 0x17F if it found it there previously, thus perpetuating the (bad?) behaviour, or should it clean up by clearing 0x17F to 00 and writing at 0x180? My preference is for the latter.
 
I don't think the field is used anywhere by the standard firmware.

By now we are probably (over-)confident in our understanding of the issue enough to write to 0x180 regardless, unless running on < OEM1.03.
If it's not used by the standard F/W then why does it matter? Old recordings made on old firmware will just get moved to the new standard when modified. The hmt utility will still read either (discarding the char at 0x17F if it's the same as the one at 0x180, which is most likely an error - you can't be 100% certain).
 
I thought @/df had posted an example like this, which is what I was seeing with the Les Mis stuff, but I can't find it now.
This is what the Humax software created:
Code:
humax /mnt/hd2/My Video/Les Miŝ‘rables # ls -ld ../Les*
drwxr-xr-x 2 root root 4096 Fri 18 Mar 2022 01:02:50 ../Les Mis?????rables
humax /mnt/hd2/My Video/Les Miŝ‘rables # ls -ld ../Les*|hd
00000000  64 72 77 78 72 2d 78 72  2d 78 20 32 20 72 6f 6f  |drwxr-xr-x 2 roo|
00000010  74 20 72 6f 6f 74 20 34  30 39 36 20 46 72 69 20  |t root 4096 Fri |
00000020  31 38 20 4d 61 72 20 32  30 32 32 20 30 31 3a 30  |18 Mar 2022 01:0|
00000030  32 3a 35 30 20 2e 2e 2f  4c 65 73 20 4d 69 73 cc  |2:50 ../Les Mis.|
00000040  82 e2 80 98 72 61 62 6c  65 73 0a                 |....rables.|
0000004b
humax /mnt/hd2/My Video/Les Miŝ‘rables # pwd|hd
00000000  2f 6d 65 64 69 61 2f 4d  79 20 56 69 64 65 6f 2f  |/media/My Video/|
00000010  4c 65 73 20 4d 69 73 cc  82 e2 80 98 72 61 62 6c  |Les Mis.....rabl|
00000020  65 73 0a                                          |es.|
00000023
Interesting that 'ls' doesn't show the decoded names, but it does if you 'cd' to it.
The on-screen UI is also wrong for the folder name.

After renaming with the latest WebIf, we get this:
Code:
humax /mnt/hd2/My Video/Les Misérables # ls -ld ../Les*
drwx------ 2 root root 4096 Sun 27 Mar 2022 19:20:30 ../Les Mis??rables
humax /mnt/hd2/My Video/Les Misérables # ls -ld ../Les*|hd
00000000  64 72 77 78 2d 2d 2d 2d  2d 2d 20 32 20 72 6f 6f  |drwx------ 2 roo|
00000010  74 20 72 6f 6f 74 20 34  30 39 36 20 53 75 6e 20  |t root 4096 Sun |
00000020  32 37 20 4d 61 72 20 32  30 32 32 20 31 39 3a 32  |27 Mar 2022 19:2|
00000030  30 3a 33 30 20 2e 2e 2f  4c 65 73 20 4d 69 73 c3  |0:30 ../Les Mis.|
00000040  a9 72 61 62 6c 65 73 0a                           |.rables.|
00000048
humax /mnt/hd2/My Video/Les Misérables # pwd|hd
00000000  2f 6d 6e 74 2f 68 64 32  2f 4d 79 20 56 69 64 65  |/mnt/hd2/My Vide|
00000010  6f 2f 4c 65 73 20 4d 69  73 c3 a9 72 61 62 6c 65  |o/Les Mis..rable|
00000020  73 0a                                             |s.|
00000022
and everything displays correctly (WebIf, on-screen and 'cd') apart from 'ls' (probably expected).

So what coding is generating "cc 82 e2 80 98" for the é which results in ŝ‘ ?
 
Last edited:
The current version of hmt reads from either 0x17F or 0x180, although I think that's just a fortuitous bug.
Sometimes you see the same character at 0x17F as at 0x180, which is probably the result of two things writing in different places. This can be detected and corrected for the vast majority of cases though.
So, when modifying the name here, should it write to 0x17F if it found it there previously, thus perpetuating the (bad?) behaviour, or should it clean up by clearing 0x17F to 00 and writing at 0x180? My preference is for the latter.

If it's not used by the standard F/W then why does it matter? Old recordings made on old firmware will just get moved to the new standard when modified. The hmt utility will still read either (discarding the char at 0x17F if it's the same as the one at 0x180, which is most likely an error - you can't be 100% certain).


Yes, the whole thing's bizarre. Here's my reading of it - but I stand to be corrected.

The Humax base firmware up to 1.02.32 stored it at offset 0x017F, from 1.03.12 it stored it at offset 0x0180. So, from then on, the base firmware had to allow for either.

hmt will find the filename in either location. But the hmt +setfilename command will store it at offset 017F, regardless of where the old filename started. The Web-if rename option does the same.

Conversely, the remote control rename command will store the new name at offset 0x0180, regardless of where the old filename started.

So, by renaming a programme first with hmt/Web-if rename and then with the remote control one can indeed end up with a spurious character at offset 0x017F. Except, that, of course, it's impossible for the system to tell whether it's actually spurious or not. Why doesn't this throw things? Well, actually, one can delete the whole filename field, and the programme will still play correctly.

As af123 and Raydon noted several years ago this field isn't actually used. So why does hmt support it?

As I say, bizarre - at least to me.

BTW, thanks for fixing the X'15' at offset 0x029A problem - my real interest!
 
Conversely, the remote control rename command will store the new name at offset 0x0180, regardless of where the old filename started.
Which version are you talking about? I'd expect that from 1.03.12, as that's where it creates it.
I'd also expect 1.02.32 to rename at 0x17F, as that's where it creates it. Are you saying it doesn't?
 
Which version are you talking about? I'd expect that from 1.03.12, as that's where it creates it.
I'd also expect 1.02.32 to rename at 0x17F, as that's where it creates it. Are you saying it doesn't?
Sorry, poor explanation on my part. No, it is exactly as you say. The real surprise was to find that it didn't matter what was in this field, just as it doesn't matter what is in the path name at offset 0x'0080'. Raydon included the latter point in his Wiki documentation for the hmt format, but not the former.

Thanks for your help.
 
I thought @/df had posted an example like this, which is what I was seeing with the Les Mis stuff, but I can't find it now.
...
So what coding is generating "cc 82 e2 80 98" for the é which results in ŝ‘ ?
That was just using the same Glums example that you posted.

"cc 82 e2 80 98" is a valid UTF-8 encoding. Unfortunately it's U+0302 COMBINING CIRCUMFLEX ACCENT followed by U+2018 LEFT SINGLE QUOTATION MARK. In Unicode, a combining diacritic applies to the preceding character. Thus s\u0302\u2018 should display as ŝ‘. NB: in my browser's monospace font the circumflexes are rendered horizontally offset to the right: ŝ vs .

Presumably some coding muddle has led to this.

The UTF-8 encoding of é is 0xc3a9.

In ISO6937 0xc3 is a (forward) combining circumflex accent and 0xa9 is LEFT SINGLE QUOTATION MARK. This looks suspicious. Maybe a transformation where UTF-8 is interpreted as ISO6937 which is inaccurately encoded as UTF-8 is the problem?
 
Maybe a transformation where UTF-8 is interpreted as ISO6937 which is inaccurately encoded as UTF-8 is the problem?
Maybe. Or maybe it's the Humax software just assuming it's 6937 when it isn't i.e. it just ignores the coding.
I'm struggling to find anything to convert to 6937 reliably. My Mint box complains about "s\xcc\x82s" being invalid using iconv, converting to anything. Any ideas?
 
Some discussion and Python code here. Hence a recent Python 3 should be able to do the conversion, for Minty testing, or for on-box conversion the linked source file from 2016-12-04 could be Jim-ified, or C-ified if that's too slow. The encoder is only about 20 lines with a giant encoding table that could be translated with a script.
 
Some discussion and Python code here.
I don't think I'm sufficiently interested to port it. Getting off 6937 seems to be a good thing in general, and we already have that.
I was more interested why the CC 82 stuff won't apparently decode from UTF-8 to anything using iconv (either on or off-box). 8859-3 has s circumflex but still no go.
 
... why the CC 82 stuff won't apparently decode from UTF-8 to anything using iconv ...

I think, because it's already UTF-8 as the result of a non-invertible transformation.
 
I'm sure we could come up with a Jim script that would retrospectively remove any 0x15 from the Title in a .hmt, and so to all in a folder, etc
Being completely ignorant of TCL/Jim, it's taken a while to produce a routine to remove the 0x15 character from my library of existing recordings. Although Jim's string-handling does make it easy simply to remove the 0x15, I did wonder what Jim technique could be used to actually set a specific hexadecimal value in a particular location in the hmt file. The binary function seems to be the natural way of doing it in TCL, but it doesn't seem to be present in the Humax Jim implementation. Of course, there are other ways to do this, for example using C cross-compliers or using the hmt routine itself. But is there a natural way to do it in Jim itself? Thanks for any suggestions.
 
You have to say package require binary; also package require pack.

Then if hmt is your writable open .hmt file (untested):
Code:
proc write {fd data {fpos ""}} {
	if {$fpos ne ""} {
		$fd seek $fpos start
	}
	return [$fd puts -nonewline $data]
}

proc patch {hmt width offset value} {
	pack data $value -intle $width
	write $hmt $data $offset
}
(adapted from my still WIP Jim version of hmt)
 
Last edited:
The hmt command has a patch option, from the help text
Code:
Generic patch commands:
    +patch8=offset:value     patch 8-bit value
    +patch16=offset:value    patch 16-bit value
    +patch32=offset:value    patch 32-bit value
  Offset and value can be preceded with 0x to indicate hex.
 
Back
Top