[tunefix] Automatic channel organisation and maintenance

My point is that tunefix has a life of its own without being subservient to tunefix-update.
I don't disagree.
This would make more sense, I think (assuming the demands of tunefix-update are compatible with that policy).
They are.
If you can turn it around, why was it necessary to change it in the first place?
The change was from nothing to something, not from something to something else. I picked what I thought was sensible at the time. I should have checked what the GUI did, but I didn't. So, tunefix itself will now do the same as the GUI editor does.
All I need is an option to prevent tunefix rearranging the config file and then execute in order. How about a flag at the start of the file, eg tunefix-update=no?
It just doesn't work like that any more. FORCE, FORCESVC, LCN, NAME is what will happen.
Philosophically I don't think it is right to rewrite a config file, but if you must rewrite the config file to suit tunefix-update either provide an option to protect the config file as is or disable the rewriting when it is not required (ie when tunefix-update is not active).
It's not just tunefix-update. tunefix itself sometimes changes the config. file e.g. on Freeview retune day last year.
 
tunefix 1.7.2 released:
  • Fix bug in mux. move code which was removing the wrong region's MUX config. settings
  • Promote FORCE and FORCESVC above LCN, NAME and MUX in command execution order
  • Extend range of removed LCNs from 787-799 down to 778-799
 
Thanks for this; I'm not 100% a fan of this new approach to tunefix, but it's your package so...

I'll take a look at the instruction manual wording.
  • Extend range of removed LCNs from 787-799 down to 778-799
Do these get added to .conf automatically, or are they removed regardless?

To confirm: I take it the REGION command doesn't delete National muxes?
 
Do these get added to .conf automatically, or are they removed regardless?
The former, on package upgrade.
To confirm: I take it the REGION command doesn't delete National muxes?
Correct. If you specify at least one REGION command, it acts as if you'd also specified "Com78 National" and then removes anything but those.
Retune completed successfully.
TGFT!
 
This xinit.d log entry looks unsatisfactory:
Code:
...
[/var/lib/humaxtv/mod/xinit.d/tunefix]
/var/lib/humaxtv/mod/xinit.d/tunefix: can't resolve symbol 'sqlite3_errstr' in lib '/var/lib/humaxtv/mod/xinit.d/tunefix'.
...
Also:
Code:
# ls -l /var/lib/humaxtv/mod/xinit.d/tunefix
-rwx------ 1 root root 87492 Apr 26 01:23 /var/lib/humaxtv/mod/xinit.d/tunefix
# nm -D /usr/lib/libsqlite3.so | grep sqlite3_err
0002beb8 T sqlite3_errcode
0002c45c T sqlite3_errmsg
00036ce8 T sqlite3_errmsg16
# nm -D /mod/lib/libsqlite3.so | grep sqlite3_err
00107280 T sqlite3_errcode
00106f1c T sqlite3_errmsg
001070d4 T sqlite3_errmsg16
00107470 T sqlite3_errstr
#
So this looks like a secret API change, as both libs are labelled 0.8.6.
SQlite changelog said:
2012-12-12 (3.7.15)
Added the sqlite3_errstr() interface.
...
The SQLite build files don't seem to mention the soname/version for shared libraries. I wonder if 0.8.6 is derived from a TCL version?

Let's check further:
Code:
# cat sqliteinfo.c
/* printf, scanf, stdin, stderr */
#include <stdio.h>
/* sqlite3_* */
#include <sqlite3.h>

int main(int argc, char *argv[]) {
    /* unused */
    (void)argc; (void)(argv[0]);
    printf( "sqlite3_version: %s\nlibversion: %s\n" 
                "sourceid: %s\nlibversion: %d\n",
                sqlite3_version, sqlite3_libversion(),
                sqlite3_sourceid(), sqlite3_libversion_number());
    return 0;
}
# echo $LD_LIBRARY_PATH
/mod/lib:/lib:/usr/lib
# ./sqliteinfo
sqlite3_version: 3.27.2
libversion: 3.27.2
sourceid: 2019-02-25 16:06:06 bd49a8271d650fa89e446b42e513b595a717b9212c91dd384aab871fc1d0f6d7
libversion: 3027002
# LD_LIBRARY_PATH=/usr/lib:/mod/lib:/lib
# ./sqliteinfo
sqlite3_version: 3.7.5
libversion: 3.7.5
sourceid: 2011-01-28 17:03:50 ed759d5a9edb3bba5f48f243df47be29e3fe8cd7
libversion: 3007005
Based on this sample it looks as if there's no attempt to version the ABI as opposed to the implementation. For instance, the addition of sqlite3_errstr() after 3.7.14 didn't cause a change to ABI 3.8. Also, the firmware library looks by the date as if it came from the Humax source distribution and has been rebuilt for CFW 3.13.

There are some 60 new functions in the 3.27.2 library compared with 3.7.5. This is only a problem for programs that are linked against the shared library and that have to run when the /mod filesystem may not be available (like tunefix), or when WebIf (which depends on sqlite3) may not be installed.

One solution would be to link such programs statically, but that could produce executables too big to live happily in RW flash.

A short-term solution would be to avoid or replace the missing function. The missing function can be acquired from main.c in the SQLite source (lines 1546 ff).

A longer-term action would be to modify the headers in the sqlite3 package, and in any cross-compilation environments, so that only the 3.7.2 APi is recognised, unless some #define is set (or vice versa).

Although a newer library could be included in the CFW, that might have unwanted effects for programs like the settop binary that were built against 3.7.2, which might even call functions like those in the test program and fail to run if reported version numbers are unexpected; and anyway a repo package can't rely on a particular CFW version.
 
Last edited:
Although a newer library could be included in the CFW, that might have unwanted effects for programs like the settop binary
On the plus side, the settop binary is statically linked with sqlite (which is a frustration for certain reverse engineering challenges..)
 
Doh!
Well, that leaves /sbin/swifi and /sbin/sqlexec (which I seem to have re-implemented) as CFW programs that might be affected by upgrading /usr/lib/libsqlite3.so. Are there any more?
 
Last edited:
What's this got to do with tunefix? Should it not be in a separate sqlite thread?
Or are you saying tunefix isn't working any more because something else changed?
 
Possibly it has digressed from tunefix but if you check the log quoted at the head of post #230 it should become clear that something is awry there; it's not obvious whether the functionality is being affected. Maybe some test channel.db schema tweak is exposing the issue, as I haven't seen a similar entry before?
 
Possibly it has digressed from tunefix but if you check the log quoted at the head of post #230 it should become clear that something is awry there; it's not obvious whether the functionality is being affected.
What's in /tmp/xinit.log relating to tunefix? Is the banner there?
The missing function is only called if it throws an exception, but it seems like it might not be running at all and that message is generated at load time rather than run time?
It runs here of course without the same error.
 
that message is generated at load time rather than run time?
Definitely run time, having tested it.
it's not obvious whether the functionality is being affected.
Definitely will be.
Maybe some test channel.db schema tweak is exposing the issue, as I haven't seen a similar entry before?
I would expect so. Upgrade tunefix and try again. I've nobbled the use of sqlite3_errstr() if it doesn't exist in 1.9.3.
Then we will see what the error message is and what it's trying to do.
 
Tragically I get the same error message in xinit.log, and:
Code:
# ls -lH /mod/boot/xinit.d/tunefix 
-rwx------ 1 root root 87492 Jun 20 01:13 /mod/boot/xinit.d/tunefix
# nm -D /mod/boot/xinit.d/tunefix | grep 3_err
         U sqlite3_errcode
         U sqlite3_errmsg
         U sqlite3_errstr
#
What I did to remove this dependency in a similar program was to paste in the code from SQLite with some #defines, but this would have to be reorganised a bit for multiple source files using sqlite3_errstr:
Code:
/* provide missing functions from newer ABI if needed */
#ifndef USE_SHARED_ERRSTR
#ifndef ArraySize
#define ArraySize(a) ((int)(sizeof(a) / sizeof(*(a))))
#endif
#ifndef ALWAYS
#define ALWAYS(a) (a)
#endif

/*
** Return a static string that describes the kind of error specified in the
** argument.
*/
static const char *sqlite3ErrStr(int rc){
  static const char* const aMsg[] = {
    /* SQLITE_OK          */ "not an error",
    /* SQLITE_ERROR       */ "SQL logic error",
    /* SQLITE_INTERNAL    */ 0,
    /* SQLITE_PERM        */ "access permission denied",
    /* SQLITE_ABORT       */ "query aborted",
    /* SQLITE_BUSY        */ "database is locked",
    /* SQLITE_LOCKED      */ "database table is locked",
    /* SQLITE_NOMEM       */ "out of memory",
    /* SQLITE_READONLY    */ "attempt to write a readonly database",
    /* SQLITE_INTERRUPT   */ "interrupted",
    /* SQLITE_IOERR       */ "disk I/O error",
    /* SQLITE_CORRUPT     */ "database disk image is malformed",
    /* SQLITE_NOTFOUND    */ "unknown operation",
    /* SQLITE_FULL        */ "database or disk is full",
    /* SQLITE_CANTOPEN    */ "unable to open database file",
    /* SQLITE_PROTOCOL    */ "locking protocol",
    /* SQLITE_EMPTY       */ 0,
    /* SQLITE_SCHEMA      */ "database schema has changed",
    /* SQLITE_TOOBIG      */ "string or blob too big",
    /* SQLITE_CONSTRAINT  */ "constraint failed",
    /* SQLITE_MISMATCH    */ "datatype mismatch",
    /* SQLITE_MISUSE      */ "bad parameter or other API misuse",
#ifdef SQLITE_DISABLE_LFS
    /* SQLITE_NOLFS       */ "large file support is disabled",
#else
    /* SQLITE_NOLFS       */ 0,
#endif
    /* SQLITE_AUTH        */ "authorization denied",
    /* SQLITE_FORMAT      */ 0,
    /* SQLITE_RANGE       */ "column index out of range",
    /* SQLITE_NOTADB      */ "file is not a database",
    /* SQLITE_NOTICE      */ "notification message",
    /* SQLITE_WARNING     */ "warning message",
  };
  const char *zErr = "unknown error";
  switch( rc ){
    case SQLITE_ABORT_ROLLBACK: {
      zErr = "abort due to ROLLBACK";
      break;
    }
    case SQLITE_ROW: {
      zErr = "another row available";
      break;
    }
    case SQLITE_DONE: {
      zErr = "no more rows available";
      break;
    }
    default: {
      rc &= 0xff;
      if( ALWAYS(rc>=0) && rc<ArraySize(aMsg) && aMsg[rc]!=0 ){
        zErr = aMsg[rc];
      }
      break;
    }
  }
  return zErr;
}
#define sqlite3_errstr(a) sqlite3ErrStr(a)
#endif
 
Code:
humax# cat sqltest.c
#include <stdio.h>
#include <sqlite3.h>

int main(int argc, char *argv[])
{
    const char *(*pfn)(int) = sqlite3_errstr;

    printf("%s\n", pfn ? (*pfn)(0) : "none");
    return 0;
}

humax# nm -D sqltest|grep 3_err
         U sqlite3_errstr
humax# LD_LIBRARY_PATH=/mod/lib ./sqltest
not an error
humax# LD_LIBRARY_PATH=/usr/lib ./sqltest
none
 
Last edited:
If that's the test (which would be fine if a reduced level of error diagnostic is accepted) in the latest tunefix, then it doesn't seem to be effective:
Code:
# stat /mod/boot/xinit.d/tunefix
....  Size: 87492 ...
Access: 2020-06-20 01:13:19.000000000 +0100
Modify: 2020-06-20 01:13:19.000000000 +0100
Change: 2020-06-20 12:21:28.000000000 +0100
...
# ls --full-time /tmp/xinit.log 
-rw-r--r-- 1 root root 2066 2020-06-20 13:24:39.000000000 +0100 /tmp/xinit.log
# cat /tmp/xinit.log 
...
[/var/lib/humaxtv/mod/xinit.d/rsvsync]
[/var/lib/humaxtv/mod/xinit.d/tunefix]
/var/lib/humaxtv/mod/xinit.d/tunefix: can't resolve symbol 'sqlite3_errstr' in lib '/var/lib/humaxtv/mod/xinit.d/tunefix'.
[/var/lib/humaxtv/mod/xinit.d/vdisk]
...
#
Or is it the proposal for the next version? Otherwise It's just recapitulating the test that I posted originally.
 
Sorry if this query has already been covered, but Forces TV is showing as having a frequency change this coming Monday (22nd June 2020), the epg is also showing a retune warning from about midday Monday, is [ TuneFix ] going to cope with this situation, or is this one of the exceptions

Amongst the Forces TV retune issue, there are other changes to HD channels and +! catch-up channels being removed
 
Back
Top