[tunefix] Automatic channel organisation and maintenance

prpr · Jun 21, 2020

/df said:
I can report that the failing SQL statement begins

That's not surprising given you've modified the schema, but I got the expected error message that told me what the error was and what the query was when I did it.
I'm afraid my gdb skills are almost 0 and mipsel assembler skills are exactly 0. I haven't touched a command line debugger since SYMDEB nearly 30 years ago.
So the above doesn't really help unless you can interpret.

it's not the failure of the statement that's interesting... but the consequent error handling.

Indeed, but I still don't know why I can't make it go wrong. Nor do I understand how it can be executing SQL when it hasn't even printed the Tunefix version banner.

prpr · Jun 21, 2020

prpr said:
Indeed, but I still don't know why I can't make it go wrong. Nor do I understand how it can be executing SQL when it hasn't even printed the Tunefix version banner.

Oh, I have now, after tweaking freq.db to add a column, but it did print the banner:

Code:

humax /var/lib/humaxtv # LD_LIBRARY_PATH=/usr/lib /mod/boot/xinit.d/tunefix
Tunefix V1.9.3 (c) 2015-2020 prpr
/mod/boot/xinit.d/tunefix: can't resolve symbol 'sqlite3_errstr' in lib '/mod/boot/xinit.d/tunefix'.

Wonder why my test program worked though...

/df · Jun 21, 2020

prpr said:
Oh, I have now, after tweaking freq.db to add a column, but it did print the banner:

Code:

humax /var/lib/humaxtv # LD_LIBRARY_PATH=/usr/lib /mod/boot/xinit.d/tunefix Tunefix V1.9.3 (c) 2015-2020 prpr /mod/boot/xinit.d/tunefix: can't resolve symbol 'sqlite3_errstr' in lib '/mod/boot/xinit.d/tunefix'.

Wonder why my test program worked though...

That's what it does for me too, at the telnet shell prompt (and did in the gdb session). In xinit.log I don't see the banner, maybe something to do with combining stdout and stderr (the error message presumably on stderr, then the program quits without stdout being flushed?). Maybe something about C++ causes the construct in the test program to be handled differently?.

Despite gdb unskills, you could try this, which would be much more enlightening in the presence of the source code:

Code:

gdb /path/to/tunefix
set env LD_LIBRARY_PATH=/usr/lib
b *0x0040de9c
run

That should run the program until just before the point where it failed in the session I posted. Then s (step) will take you line-by-line, and bt (backtrace) will show how it got there.

gdb's built-in help is really extensive but you could just refer to a quick reference like this.

prpr · Jun 21, 2020

Well, now I've got it to go wrong, I started pruning and ended up with this trivial thing:

Code:

#include <stdio.h>
#include "sqlite3.h"

int main(int argc, char *argv[])
{
    const char *(*pfn)(int) = sqlite3_errstr;

    printf("%08X\n", pfn);
    if (pfn)
        printf("1\n");
    else
        printf("0\n");

    return 0;
}

Now run these variations:

Code:

humax# gcc -o sqltest -O0 sqltest.cpp -lsqlite3
humax# ./sqltest
2ABBA470
1
humax# LD_LIBRARY_PATH=/usr/lib ./sqltest
00000000
0
humax# gcc -o sqltest -O1 sqltest.cpp -lsqlite3
humax# LD_LIBRARY_PATH=/usr/lib ./sqltest
00000000
1

This appears to be a compiler/optimiser bug.
So, I worked out all the options which -O1 turns on and enabled them individually with -O0. It still worked properly.
I then disabled them all individually with -O1 and it still failed.
So I have no idea what's really causing it - it's just a stupid bug in the compiler somewhere.

Code is a Hole · Jun 22, 2020

I've just done a retune, and no change, still getting a pop-up banner stating Forces TV is moving, when the scheduled events were restored I checked the restore log

Restoring Bless This House (Forces TV) No change in channel service. AUL Forces TV (721095)
Restoring Bless This House (Forces TV) No change in channel service. AUL Forces TV (721095)

Seems the broadcasters need to add a time of the day along with the retune date

Ezra Pound · Jun 22, 2020

There is a period of 'change over' lasting a few days with COM8 only being turned off at the end of this period, in my region all programmes that are closing down have already gone, but programmes that are moving (including Forces) are available without re-tune at the moment

Code is a Hole · Jun 22, 2020

It appears Forces TV is no longer on Freeview

/df · Jun 22, 2020

prpr said:
...
This appears to be a compiler/optimiser bug.
So, I worked out all the options which -O1 turns on and enabled them individually with -O0. It still worked properly.
I then disabled them all individually with -O1 and it still failed.
So I have no idea what's really causing it - it's just a stupid bug in the compiler somewhere.

It looks like the run-time resolution of the function address is confusing things. Sadly a small attempt with volatile wasn't successful, but this rather more intricate work-around gives the same result regardless for -O0 and -O3:

Code:

#include <stdio.h>
#include <sqlite3.h>
/* dynamic linking */
#include <dlfcn.h>

int main(int argc, char *argv[])
{
    void *sqlib = dlopen("libsqlite3.so", RTLD_LAZY|RTLD_NOLOAD);
    if (sqlib) {
        char *(*pfn)(int) =
                (char *(*)(int)) dlsym( sqlib, "sqlite3_errstr");

        printf("%8p\n", pfn);
        if (pfn)
            printf("1\n");
        else
            printf("0\n");

        dlclose(sqlib);
        return 0;
    }
    
    return 1;
}

All a bit specific to the HD/R on-box environment in case you were going to build a (maybe even a Windows) tunefix accessing the databases over the network.

This small change to the code in post #264 also straightens out the compiler, unless the address of sqlite3_errstr happens to be 0x80000000:

Code:

    ...
    printf("%08X\n", pfn);
    if (((unsigned int)pfn)&0x7fffffff)
        printf("1\n");
        ...

Unfortunately the optimiser at -O3 is wise to formulae like (mask & x)|(~mask & x), even broken out into two functions.

Alternatively there is the compile-time approach that I posted earlier.

Code is a Hole · Jun 22, 2020

Andrea Edwards said:
It appears Forces TV is no longer on Freeview

Correction, Forces TV is moving from COM8 to COM6, the retune has Forces TV still on channel 96 which can be viewed, but there is no accurate epg data being broadcast

prpr · Jun 22, 2020

/df said:
Sadly a small attempt with volatile wasn't successful, but this rather more intricate work-around gives the same result regardless for -O0 and -O3

Yes, I resorted to dlopen/sym/close yesterday to resolve the address to prove it worked.

All a bit specific to the HD/R on-box environment in case you were going to build a (maybe even a Windows) tunefix

I do have the latter. The whole thing is written in cross-platform code developed and tested with Visual Studio on Windows. The SQL3 class is cross-platform and I don't want to go polluting that directly with Linux specific stuff.

This small change to the code in post #264 also straightens out the compiler

Interesting. I tried casts to int and other things but never thought to do bit-fiddling.

Alternatively there is the compile-time approach that I posted earlier.

I didn't do that. I went for sqlite3_libversion_number():

Code:

    if (sqlite3_libversion_number() > 3007014)
      ; // do something with sqlite3_errstr()
    else
      ; // don't do something with sqlite3_errstr()

and probably should have done that earlier.
This works and was released last night with tunefix 1.9.5
I also fixed the query that updates freq.db so it doesn't barf any more when you modify the schema.

Code is a Hole · Jun 23, 2020

Andrea Edwards said:
Correction, Forces TV is moving from COM8 to COM6, the retune has Forces TV still on channel 96 which can be viewed, but there is no accurate epg data being broadcast

Update, checked again this morning and the epg is now showing programme details as expected, panic over

Black Hole · Jun 23, 2020

Who was panicking?

/df · Jun 23, 2020

prpr said:
...
This works and was released last night with tunefix 1.9.5
I also fixed the query that updates freq.db so it doesn't barf any more when you modify the schema.

Yes, that worked for me, thanks. xinit.log was full of deleting 5USA+1, etc.

Black Hole · Jun 27, 2020

For my supported user I would like to remove all services except a small named few. I know I can go through and mark all except those for deletion, but the list will need editing all the time as new services get added or others are renamed.

alanofcleeves · Jun 27, 2020

Black Hole said:
I would like to remove all services except a small named few.

Just a thought: You might be able to use Tunefix settings to 'force' all the channels you need to keep onto specific numbers e.g. 1-10 or however many you need, and set it to delete all channels with higher numbers, e.g. 11-799?
Not sure if it will work but maybe worth a try - depends which operation Tunefix applies first when the settings are processed.

Black Hole · Jun 27, 2020

alanofcleeves said:
Not sure if it will work

Yes, it would work... if I didn't want to keep their LCNs corresponding with those in the TV guides. The order commands are applied is governed by tunefix's built-in order of succession, and not necessarily the order commands are listed in the config file.

Thus, this wouldn't work because all the FORCE commands get executed before any of the LCN commands:

Code:

FORCE|301|BBC ONE
LCN|1-299
FORCE|1|BBC ONE

(the result would be BBC ONE missing entirely)

The easiest way to do it at the moment is with LCN commands.

prpr · Jun 30, 2020

Black Hole said:
I was messing about with the supported user's unit the other day, trying to retune and strip the dross

That's just:
LCN|6-799
then. Simples!

Black Hole · Jun 30, 2020

Well, almost! She does like the horse racing on ITV4. But actually I want to keep the services which appear in the TV listings magazines, and a few radio services.

Trev · Jun 30, 2020

Is that the +1s out of the window then? Or are they just too complicated?

Black Hole · Jun 30, 2020

Trev said:
Is that the +1s out of the window then? Or are they just too complicated?

I doubt she'll go outside any button not explicitly assigned (custom remote coming up), but I would like to think I could tell her to go to "33" (or whatever) if there was something I thought she would like. The problem with that is making sure she doesn't enter "3333".

[tunefix] Automatic channel organisation and maintenance

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

May contain traces of nut

Well-Known Member

May contain traces of nut

Active Member

May contain traces of nut

Well-Known Member

May contain traces of nut

The Dumb One

May contain traces of nut