prpr
Well-Known Member
For further discussion of anything previously in these threads:
or the associated beta hmt utility (2.0.10-2).
or the associated beta hmt utility (2.0.10-2).
Last edited:
Beta [webif] 1.4.9-8
- Thread starter prpr
- Start date
- Status
Black Hole
May contain traces of nut
I've carried out some preliminary tests of the Beta Web-if and hmt packages using a spare machine with Humax firmware 1.03.12 and custom firmware 3.13.
As per the instructions, I installed Auto-update 2.0.5 and opkg-beta 1.1. I initially experienced some difficulty in that, although the installations seemed to run successfully, all the package lists (Installed, Available and Upgrades) were empty. Reboot and power off/on failed to fix this problem, but then I received two notifications: that hmt 2.0.10 and webif 1.4.9 had been installed. After acknowledging these notifications, the package lists were then displayed correctly. (This may be normal and expected behaviour).
There were no existing recordings on this machine. I made four new ones, two at standard definition and two at high definition. On each of these, I carried out a selection of the decrypt, crop and rename functions that I apply to all my recordings. The primary interest was whether the problem with the 0x15 prefix in the title field at offset 029A had been resolved. But I also made a note of the other fields in the hmt file that have been referenced in the thread. In summary:
1) Both Webif decrypt and crop were successful without creating the 0x15 prefix at offset 029A.
2) However, the 0x15 prefix at offset 029A was still inserted by the webif rename function. Rename via the remote control did not insert the prefix.
3) Webif rename and crop both moved the filename from offset 0180 to offset 017F, but decrypt left it at 0180. Rename via the remote control also left it at offset 0180.
4) For the standard definition recordings, Webif rename still changed the iPlate prefixes at offsets 0516 and 0616 from 0x106937 to 0x15. Rename via the remote control did not change them.
I realise that this is just the start, so please let me know what other tests might be useful.
As per the instructions, I installed Auto-update 2.0.5 and opkg-beta 1.1. I initially experienced some difficulty in that, although the installations seemed to run successfully, all the package lists (Installed, Available and Upgrades) were empty. Reboot and power off/on failed to fix this problem, but then I received two notifications: that hmt 2.0.10 and webif 1.4.9 had been installed. After acknowledging these notifications, the package lists were then displayed correctly. (This may be normal and expected behaviour).
There were no existing recordings on this machine. I made four new ones, two at standard definition and two at high definition. On each of these, I carried out a selection of the decrypt, crop and rename functions that I apply to all my recordings. The primary interest was whether the problem with the 0x15 prefix in the title field at offset 029A had been resolved. But I also made a note of the other fields in the hmt file that have been referenced in the thread. In summary:
1) Both Webif decrypt and crop were successful without creating the 0x15 prefix at offset 029A.
2) However, the 0x15 prefix at offset 029A was still inserted by the webif rename function. Rename via the remote control did not insert the prefix.
3) Webif rename and crop both moved the filename from offset 0180 to offset 017F, but decrypt left it at 0180. Rename via the remote control also left it at offset 0180.
4) For the standard definition recordings, Webif rename still changed the iPlate prefixes at offsets 0516 and 0616 from 0x106937 to 0x15. Rename via the remote control did not change them.
I realise that this is just the start, so please let me know what other tests might be useful.
OP
prpr
Well-Known Member
I don't think anyone said that, but never mind. It's always good to have installed.
They should have been hmt 2.0.10-2 (2.0.10-4 now) and webif 1.4.9-8.
The latest standard webif is 1.4.9-6.
It is crucial to have the whole string correct and report it as so.
I don't think so, but I have no explanation.
The results are inconclusive until you can confirm exactly what versions of the above are installed.
Thanks for the detailed comments. As you probably realise, I'm on a steep learning curve. So I won't get everything right first time. I now realise that I should have upgraded the Beta packages after I installed them. Sorry.
I am now at hmt 2.0.10-4 (as above) and Webif 1.4.9-8 (presumably that's since replaced 1.4.9-6).
I realise that the results of the test are of no use. But are the test themselves any use, or should I do something different?
Thanks again.
I am now at hmt 2.0.10-4 (as above) and Webif 1.4.9-8 (presumably that's since replaced 1.4.9-6).
I realise that the results of the test are of no use. But are the test themselves any use, or should I do something different?
Thanks again.
Black Hole
May contain traces of nut
OP
prpr
Well-Known Member
Yes, the tests are OK. Do one at once and observere what happens and where, and whether stuff looks right on the WebIf and on-screen and in the .hmt file itself. If you can exercise things with non-ASCII characters too then all the better. If you know how to drive the command line hmt program effectively then look there as well. Hopefully a few others will chime in too.
I think I've solved all the character encoding/decoding problems of previous versions.
Bear in mind that things like nicesplice (crop) and sidecar possibly do their own thing, and are out of my reach and scope of what's been done recently.
Last edited:
Thinking about this problem in general, is there a file equivalent of the Webif installed packages list? If so, this could be attached to any post where the information might be important, especially in Beta test mode?
(I'd welcome your comments as to whether this is a useful way to present my results - too long for a start, I suspect).
Run on Humax 1.03.12, CFW 3.13 (Build 4028) Webif 1.4.9-8, hmt 2.0.10-4. (Thanks for the pointer to the opkg functions above, just received).
Taking one step at a time. In the first instance, I ran tests that replicate how I normally process recordings, and that first showed up these problems. This post records my findings with respect to decrypt/rename. I'll cover other tests later.
At this stage, I'm specifically looking at changes to the .hmt file in the fields of interest: the file name (offset 017F/0180); the title (029A); the iTitle (0516); the iSynopsis (0516); and the iGuidance, if any (0741). Where appropriate I ran hex comparisons on the relevant hmt files. As the standard for comparison, I used files that had not yet been processed by Webif.
I had a number of clean recordings made over the last 24 hours, both SD and HD. I performed decryption using Webif, both by explicit action and by the use of autodecrypt-enabled folders. I performed rename either by Webif or by using the remote control.
Summary of results:
1) Decrypt, however performed, had no affect on the hmt file other than the relevant flags at offsets 028E and 03DC. No surprise.
2) Rename using the remote control changed neither the position of the filename (0180) nor the prefixes used in the strings. No surprise.
2) In none of my Webif rename tests so far was the filename moved from offset 0180 to 017F. This is a change from the current situation. I assume that this is the result of a deliberate objective to standardise on this location.
3) In none of my Webif rename tests was the title at offset 029A preceded by 0x15, again a change from the current situation. In itself, this change should fix the problem with the sort order that I originally raised - once I've changed all the affected files in my library!
4) The results of my Webif rename tests were more problematic with regard to the iPlate string prefixes. (To recap: the Humax firmware uses 0x106937 for SD and 0x15 for HD. The current firmware replaces these with 0x15 throughout.) In all the cases that I tried, the HD prefixes remained as 0x15, which is consistent with the Humax firmware. For SD recordings, however, the iTitle at offset 0516 was still being changed from 0x106937 to 0x15, whilst the iSynopsis at offset 0616 is not changed from 0x106937. Is this difference deliberate? (None of my recordings had any Guidance information, so I haven't been able to test the field at offset 0741 yet.)
5) Finally, although I don't normally look at the upper reaches of the hmt file (i.e. offsets above 0818), my hex comparisons threw up another anomaly. I've not seen a definitive map of these regions, but I assume that part of it is the EPG data for the previous, current and following programmes on the current channel. If this is the case, then offsets 30FE, 33BE and 367E contain the respective titles of each of these three programmes. with the appropriate SD or HD prefix. In the case of Webif rename, these have all been replaced by the iPlate title (as at offset 0516) of the current recording, with the 0x15 prefix. This doesn't seem appropriate.
I've no experience of non-ASCII characters. Others seem more exercised by this problem than me. But I'm happy to have a go. Did you have any particular scenarios that you wanted checking? I'll look into the crop problem next, since crop is part of my normal processing. I'm aware of the sidecar (and conflicting AV2HDR) routine, but will look into it again when we've got the basics agreed.
Thanks for the help, and apologies for the length.
OP
prpr
Well-Known Member
It's OK. Bit long, but alright to start with.
One of those 516's is meant to be 616, but I know it's just a typo. Also check 3E2 with 741.
It is.
This is good too.
By design. Everything is converted to UTF-8 (0x15 prefix). This works OK on HD recordings. I haven't got round to testing it on SD ones, but don't really see why it shouldn't work.
What does it show on-screen? Is it still correct?
It should be.
No. Everything should be 0x15. If it's not, then repeat the test to confirm, and then tell me the exact command you are using.
It's documented here: https://wiki.hummy.tv/wiki/HMT_File_Format
Not necessarily. One of my old recordings contains 8 blocks. I've never looked at it before so haven't worked out what's there or why. I guess the intent was to set every block regardless, and then it'll never display the wrong thing. Nothing has changed in how this works, apart from fixing the encoding.
Offsets seem correct based on what I was looking yesterday.
What do you think is appropriate and why, given the above? Again, exact command and "what you see" vs "what you expect" is useful.
I'll try and write some instructions later. You need to be slightly creative if you're doing this from the command line. If using the WebIf you should just be able to copy'n'paste e.g. "Les Misérables" and such like.
Not really, apart from the SD/HD thing mentioned above.
OK.
No problem. Thanks for the feedback.
Correct - typo, sorry. Will also check 03E2 when I test a programme with guidance.
That's at the heart of many of my comments: the pros and cons of Webif preceding all relevant strings with 0x15 versus sticking with the convention apparently used by the Humax firmware, that strings in the hmt files for HD recordings are preceded by 0x15, whilst those for SD recordings are preceded by 0x106937. I don't have strong views either way, but just want to understand the arguments.
I confirm that the Synopsis string prefix at offset 0616 is not changed from 0x106937 to 0x15 in any of my SD recordings. I'm not using any command, I'm examining the hex representation of the hmt file.
It needs more confirmation, but so far I haven't found any text on screen that displays incorrectly, whatever the prefix.
You're right, I'd forgotten it was in there - but beware the ambiguity between 0x15 and 0x106937 prefixes is not fully explained. In practice, they follow the HD=0x15 and SD=0x106937 convention.
It turns out to be more interesting. I followed your advice to look on the screen. It would appear that what's on the screen is dictated by what's in the transmission stream, not what is in the hmt file, regardless of what I have done to the Title or you have done to the prefix. I guess it may be another example, like the folder and filename, where the hmt field is written, but not read. More investigation needed perhaps. (Or we could just ignore it.)
Again, I'm really just flagging up the difference between what Webif produces and the conventions that the Humax firmware itself appears to be using. Apologies for language that appears to prejudge the matter. As before, this is all based on examining the hex representation of the file directly.
I guess what would be really useful is some examples where using the 0x15 and 0x106937 would cause the system to do different things. But I'm mindful of the 'sort problem' that created this thread in the first place. There was no problem when the system treated the field as a text string. The problem only showed up when it treated it as a hexadecimal sort string. Given that the two string prefixes have different lengths, it's possible that some other internal routine may trip over them in some way. But in the sheer nature of things, that may be difficult to conceive in advance.
I've tried to get more information in fewer words.
bottletop
Active Member
Retest of my post #30 https://hummy.tv/forum/threads/media-list-sort-order.10539/post-160895
Testing beta WebIF & hmt
Web interface version: 1.4.9-8
Custom firmware version: 3.13 (build 4028)
Humax Version: 1.03.12 (kernel HDR_CFW_3.13)
Loader Version: a7.30
System ID: 80bc.7e00
Series recording scheduled
Pokémon Master Journeys:...
Series CRID SONYKIDS.CO.UK/S573500
Channel 206 - POP
Event Name Pokémon Master Journeys:...
Start Mon 28 Mar 2022 16:30 BST
telnet (via Linux Mint 20.2 Cinnamon terminal)
ftp access to Pokemon directory is fine Linux Mint 20.2 Cinnamon/Nemo 5.0.5/ftp
ftp access to Pokemon directory fails using FileZilla Client Version 3.46.3
The Humax GUI displays the recording names, directories and synopsis correctly.
Testing beta WebIF & hmt
Web interface version: 1.4.9-8
Custom firmware version: 3.13 (build 4028)
Humax Version: 1.03.12 (kernel HDR_CFW_3.13)
Loader Version: a7.30
System ID: 80bc.7e00
Series recording scheduled
Pokémon Master Journeys:...
Series CRID SONYKIDS.CO.UK/S573500
Channel 206 - POP
Event Name Pokémon Master Journeys:...
Start Mon 28 Mar 2022 16:30 BST
telnet (via Linux Mint 20.2 Cinnamon terminal)
Code:
Humax HDR-Fox T2 (humax2324) 1.03.12/3.13
To return to the menu, type: exit
humax2324# cd /media/My\ Video/
humax2324# cd Pokémon\ Master\ Journeys____/
humax2324# pwd
/media/My Video/Pokémon Master Journeys____
humax2324# /mod/bin/busybox/pwd
/mnt/hd2/My Video/Pokémon Master Journeys____
humax2324# /mod/bin/ls -al *.hmt
-rw-r--r-- 1 root root 2072 Mar 30 17:01 Pok??mon Master Journeys_____20220330_1630.hmt
humax2324# /mod/bin/busybox/ls -al *.hmt
-rw-r--r-- 1 root root 2072 Mar 30 17:01 Pok??mon Master Journeys_____20220330_1630.hmt
humax2324# hmt -list *.hmt
Format:SD
Title:Pokémon Master Journeys:...
ITitle:Pokémon Master Journeys:...
Channel:206 (POP)
Episode:0,0/0
Folder:/mnt/hd2/My Video/Pokémon Master Journeys____/
Filename:Pokémon Master Journeys_____20220330_1630
Genre:Children (80)
EPG:...The Series: Night and Day, You are the Ones!: An encounter with some remarkable people and Pokémon gives Chloe and Eevee some valuable life lessons! (S1 Ep31)
Recording Status: Valid (OK)
Flags: SD,New,Unlimited Copies,Shrunk,Addetection,
Copy count:0
Scheduled start:1648654200 (Wed Mar 30 16:30:00 2022)
Scheduled duration:1860
Recording start:1648654203 (Wed Mar 30 16:30:03 2022)
Recording end:1648656060 (Wed Mar 30 17:01:00 2022)
Duration:1857
Stored duration:1857
Play resumes at:0 seconds in.
Timezone offset:3600
Service ID (SID):27424
Event ID:45540
Transport Stream ID (TSID):24640
Originating Network ID (ONID):9018
Programme Map Table PID (PMTPID):1012
Video PID:631
Audio PID:632
Bookmarks:5 = 3 570 863 1476 1857ftp access to Pokemon directory is fine Linux Mint 20.2 Cinnamon/Nemo 5.0.5/ftp
ftp access to Pokemon directory fails using FileZilla Client Version 3.46.3
Status: Connecting to 192.168.2.24:21...
Status: Connection established, waiting for welcome message...
Status: Insecure server, it does not support FTP over TLS.
Status: Server does not support non-ASCII characters.
Status: Logged in
Status: Retrieving directory listing...
Status: Directory listing of "/" successful
Status: Retrieving directory listing of "/My Video"...
Status: Directory listing of "/My Video" successful
Status: Retrieving directory listing of "/My Video/Killing Eve"...
Status: Directory listing of "/My Video/Killing Eve" successful
Status: Retrieving directory listing of "/My Video/Would I Lie to You_"...
Status: Directory listing of "/My Video/Would I Lie to You_" successful
Status: Retrieving directory listing of "/My Video/Pokémon Master Journeys____"...
Command: CWD /My Video/Pokémon Master Journeys____
Response: 550 No such file or directory
Error: Failed to retrieve directory listing
Status: Connection established, waiting for welcome message...
Status: Insecure server, it does not support FTP over TLS.
Status: Server does not support non-ASCII characters.
Status: Logged in
Status: Retrieving directory listing...
Status: Directory listing of "/" successful
Status: Retrieving directory listing of "/My Video"...
Status: Directory listing of "/My Video" successful
Status: Retrieving directory listing of "/My Video/Killing Eve"...
Status: Directory listing of "/My Video/Killing Eve" successful
Status: Retrieving directory listing of "/My Video/Would I Lie to You_"...
Status: Directory listing of "/My Video/Would I Lie to You_" successful
Status: Retrieving directory listing of "/My Video/Pokémon Master Journeys____"...
Command: CWD /My Video/Pokémon Master Journeys____
Response: 550 No such file or directory
Error: Failed to retrieve directory listing
# opkg list-installed
7zip - 9.20.1
abduco - 0.6-1
anacron - 2.3-2
at - 3.1.18
auto-schedule-restore - 1.3
auto-unprotect - 2.0.2
badnts - 1.0.1-1
bash - 4.3-30
betaftpd - 0.0.8pre17-5
binutils - 2.21-1
boot-settings - 1.0.4
bsed - 1.0.0
busybox - 1.20.2-1
bzip2 - 1.0.4
ca-bundle - 3.41
chaseget - 0.1.2-1
cifs - 2.6.18
coreutils - 8.11
cpulimit - 0.1.1-0
crashdiag - 1.1
cron-daemon - 1.18.3-4
curl - 7.63.0
curl-command - 7.21.2
dbupdate - 1.0.0
detectads - 0.2.5-4
disable-ota - 1.0.3-4
dlna-servername - 1.0.3
dropbear-ssh - 2020.79-3
e2fsprogs - 1.42.13
epg - 1.2.8
exfat - 1.2.6
fan - 1.0.0
ffmpeg - 4.1-1
file - 5.0.4
fix-disk - 0.5
forcedate - 1.1
fuse - 2.7.6-1
gawk - 3.1.8
gcc - 4.5.2-3
gdb - 7.1-1
gmake - 3.82
header-files - 1.9
hmt - 2.0.10-4
hwctl - 1.0.0
id3v2 - 0.1.11-1
inotify-tools - 3.14
ir - 1.21
jim - 0.79
jim-binary - 0.78
jim-cgi - 0.7-2
jim-oo - 0.78
jim-pack - 0.79
jim-sqlite3 - 0.78
jim-xconv - 1.0.0
lamemp3 - 3.98.4
libgmp - 5.0.1
libiconv - 1.13.1-1
libmpc - 0.8.2
libmpfr - 3.0.0
libparted - 3.1
libpcre - 8.37-1
libreadline - 6.2-1
libsndfile - 1.0.25-2
libutil - 0.9.29
libxconv - 1.0.2
lighttpd - 1.4.53-1
lsof - 4.87
multienv - 1.7-1
ncurses - 5.9
network-shares-automount - 1.4-2
nfs-utils - 1.2.3-2
nicesplice - 1.8
nicesplice-magic-folders - 1.2
ntfs-3g - 2013.1.13-5
ntfsprogs - 2013.1.13-1
ntpclient - 2010-365-5
nugget - 1.0
openssl - 1.1.1.d-1
openssl-command - 1.0.0.d
opkg-beta - 1.1
parted - 3.1-1
portmap - 6.0-1
procps - 3.2.8-3
python - 2.7.1-3
qtube - 0.1.0-1
recmon - 2.2.1-1
rsvsync - 1.1.13
rsync - 3.0.8
samba - 3.6.25-1
service-control - 2.5
smartmontools - 6.4
sqlite3 - 3.27.2
ssmtp - 2.64
stripts - 1.4.5
swapper - 1.0.1
sysmon - 1.2.6
tcpfix - 1.0.0
tcpping - 1.1
tempmon - 1.0.2
tmenu - 1.22
trm - 1.2
tweak - 1.0
uclibc-devel - 0.9.31-2
undelete - 1.6-4
vim - 7.3
virtual-disk2 - 2.0-4
webif - 1.4.9-8
webif-channelicons - 1.1.27
webif-charts - 1.3
webshell - 1.0.4
wget - 1.20.3
wireless-helper - 1.1.1
wireless-tools - 29-1
wpa-supplicant - 0.6.10
youtube-dl - 2021.12.17b
zip - 3.0-1
7zip - 9.20.1
abduco - 0.6-1
anacron - 2.3-2
at - 3.1.18
auto-schedule-restore - 1.3
auto-unprotect - 2.0.2
badnts - 1.0.1-1
bash - 4.3-30
betaftpd - 0.0.8pre17-5
binutils - 2.21-1
boot-settings - 1.0.4
bsed - 1.0.0
busybox - 1.20.2-1
bzip2 - 1.0.4
ca-bundle - 3.41
chaseget - 0.1.2-1
cifs - 2.6.18
coreutils - 8.11
cpulimit - 0.1.1-0
crashdiag - 1.1
cron-daemon - 1.18.3-4
curl - 7.63.0
curl-command - 7.21.2
dbupdate - 1.0.0
detectads - 0.2.5-4
disable-ota - 1.0.3-4
dlna-servername - 1.0.3
dropbear-ssh - 2020.79-3
e2fsprogs - 1.42.13
epg - 1.2.8
exfat - 1.2.6
fan - 1.0.0
ffmpeg - 4.1-1
file - 5.0.4
fix-disk - 0.5
forcedate - 1.1
fuse - 2.7.6-1
gawk - 3.1.8
gcc - 4.5.2-3
gdb - 7.1-1
gmake - 3.82
header-files - 1.9
hmt - 2.0.10-4
hwctl - 1.0.0
id3v2 - 0.1.11-1
inotify-tools - 3.14
ir - 1.21
jim - 0.79
jim-binary - 0.78
jim-cgi - 0.7-2
jim-oo - 0.78
jim-pack - 0.79
jim-sqlite3 - 0.78
jim-xconv - 1.0.0
lamemp3 - 3.98.4
libgmp - 5.0.1
libiconv - 1.13.1-1
libmpc - 0.8.2
libmpfr - 3.0.0
libparted - 3.1
libpcre - 8.37-1
libreadline - 6.2-1
libsndfile - 1.0.25-2
libutil - 0.9.29
libxconv - 1.0.2
lighttpd - 1.4.53-1
lsof - 4.87
multienv - 1.7-1
ncurses - 5.9
network-shares-automount - 1.4-2
nfs-utils - 1.2.3-2
nicesplice - 1.8
nicesplice-magic-folders - 1.2
ntfs-3g - 2013.1.13-5
ntfsprogs - 2013.1.13-1
ntpclient - 2010-365-5
nugget - 1.0
openssl - 1.1.1.d-1
openssl-command - 1.0.0.d
opkg-beta - 1.1
parted - 3.1-1
portmap - 6.0-1
procps - 3.2.8-3
python - 2.7.1-3
qtube - 0.1.0-1
recmon - 2.2.1-1
rsvsync - 1.1.13
rsync - 3.0.8
samba - 3.6.25-1
service-control - 2.5
smartmontools - 6.4
sqlite3 - 3.27.2
ssmtp - 2.64
stripts - 1.4.5
swapper - 1.0.1
sysmon - 1.2.6
tcpfix - 1.0.0
tcpping - 1.1
tempmon - 1.0.2
tmenu - 1.22
trm - 1.2
tweak - 1.0
uclibc-devel - 0.9.31-2
undelete - 1.6-4
vim - 7.3
virtual-disk2 - 2.0-4
webif - 1.4.9-8
webif-channelicons - 1.1.27
webif-charts - 1.3
webshell - 1.0.4
wget - 1.20.3
wireless-helper - 1.1.1
wireless-tools - 29-1
wpa-supplicant - 0.6.10
youtube-dl - 2021.12.17b
zip - 3.0-1
OP
prpr
Well-Known Member
That all seems good to me.
This is just directory name confusion, which we already know the Humax software gets wrong.
Because BetaFTPD on the Humax doesn't support the FEAT command to show UTF8 support, and FileZilla is probably set to auto-detect, it fails and defaults to something else (seems to be assuming ISO8859-1 / CP1252).
You can set an option to force it. Details here apparently: https://hosting.xyz/wiki/hosting/ftp/clients/filezilla/utf-8/
I've run Webif crop on a range of SD and HD recordings. In all cases, the file name remains at offset 0180 and the title field at offset 029A no longer has the 0x15 prefix inserted. It looks as though my original concerns (with the sort order in the media list) have been resolved. Thanks to you all for your help.
- Status