Beta [webif] 1.5.0

[prpr]

This release fixes another couple of bugs, but is mainly used to cause an upgrade to the Jim packages, upon which the whole WebIf depends, from version 0.79 to 0.81

Please DO NOT upgrade the jim, jim-oo, jim-pack or jim-sqlite3 packages manually. These all need to be done at once, otherwise everything breaks, and you will need to fix it using the command line.
Upgrade the webif package, which will upgrade everything else via the dependency mechanism.

There is also an update to the sqlite3 package (from version 3.27.2 to 3.38.2).

Feedback welcomed.

 

[Wallace]

Packages updated. I will monitor and offer feedback if/when necessary.

Thank you for your continued support.

 

[Black Hole]

Ditto. It would be nice to have a round-up of the "bugs fixed" so we know what to look for.

 

[prpr]

Improved the SMART handling and reporting
Fixed the Clipboard paste bug with a 'naked' .TS
Both on the end of this thread.

 

[peterworks]

Thank you for your continued support.

+1

 

[prpr]

Updated WebIf to 1.5.0-1 to help with SMART handling and reporting on the HD-Fox.

 

[/df]

The Jim updates were still enabled in the Package Management>Upgrades display after installing WebIf 1.5.0-1, but disappeared on refreshing (Check Updates). I don't suppose this is new behaviour: presumably the Install button disables the package after installing it and doesn't check the recursive dependencies.

Should there be an 'Even More Advanced' flag for packages that should only be installed as dependencies?

 

[Black Hole]

Should there be an 'Even More Advanced' flag for packages that should only be installed as dependencies?

I've wondered along the same lines. "Show advanced packages" reveals a raft of components I doubt should be so easily available for fiddling with, given that people have to go to the advanced list for a lot of the interesting stuff. At least the "uninstall" button ought to be disabled while a package is a dependency.

How about a "system" list, separate from the normal/advanced list?

 

[prpr]

The Jim updates were still enabled in the Package Management>Upgrades display after installing WebIf 1.5.0-1, but disappeared on refreshing (Check Updates). I don't suppose this is new behaviour:

It isn't.

presumably the Install button disables the package after installing it and doesn't check the recursive dependencies.

Indeed. I guess the only realistic option is to refresh the whole page. But does it actually matter if you try to install something that has already been installed?
TBH, I struggle to understand why the Jim packages are separate in the first place. Anything like the above four that rely on each other for correct operation should surely be in the same package, especially as the files all come from the same build output of the same source code tree.
An option would be to amalgamate them, and then somehow clean-up the redundant packages. The only thing that depends on jim-pack, jim-oo, or jim-sqlite3 that isn't the WebIf is wireless-helper, so that should be easily fixable at the same time.

Should there be an 'Even More Advanced' flag for packages that should only be installed as dependencies?

How about a "system" list, separate from the normal/advanced list?

It quickly gets complicated classifying stuff.

At least the "uninstall" button ought to be disabled while a package is a dependency.

That sounds like a good idea. Don't know how easy it would be to do.

 

[/df]

A Jim-webif meta-package whose dependencies also depend on it, so that installing one causes them all to be installed?

 

[prpr]

So what depends on what exactly?
Doesn't that create circular dependencies (how would you ever uninstall anything where A depends on B, and B depends on A, without forcing it)?

 

[/df]

opkg >0.2 is said to understand circular dependencies, which are allowed in the Debian package scheme. I haven't tried it, but I would treat removing a package by making a visited list of packages to be removed, initialised with the target package, then recursively adding unvisited dependencies until none remain.

Some Debian tools prompt with all the affected dependencies, eg if uninstalling ubuntu-desktop, which is not unreasonable, but what can be disturbing is when upgrading, say, ffmpeg results in an offer to uninstall most of your system, because the upgrade wants to install a dependent lib that replaces one that the original system build required.

See also https://en.wikipedia.org/w/index.php?title=Dependency_hell&section=2.

 

[/df]

Today I had HD Fox T2 in a condition where the /var/lib/humaxtv/setup.db could not be opened because it was persistently locked (presumably by the settop program, the only process that had it open, or perhaps from something bad that happened before). I wondered if there might be an issue between the SQLite3 bound into settop and the new version. In any case, this could happen for whatever reason, but there is a work-around for the read-only case by trying to copy the locked DB and opening the result, as in this revision in webif/lib/system.class:

proc {system param} {param {type Value} {tbl MENUCONFIG}} {{newdb ""}} {
        if {[catch {set db [sqlite3.open /var/lib/humaxtv/setup.db]} msg]} {
                return 0                                                    
        }                                                                       
        set val 0                                                               
        loop _ 2 {                                                              
                try {                                                           
                        set ret [$db query "                                    
                                select item$type                                
                                from TBL_$tbl                                   
                                where itemName = '$param'                       
                        "]                                                      
                        break                                                   
                } on error {msg opts} {                                         
                        if {[string first "database is locked" $msg] >= 0} {    
                                try {                                           
                                        if {$newdb eq ""} {
                                                set newdb [file tempfile setupXXXXXX]
                                        }                                       
                                        file copy -force /var/lib/humaxtv/setup.db $newdb
                                        set db [sqlite3.open $newdb]            
                                } on error {msg opts} {                         
                                        file delete $newdb
                                        break                                   
                                }                                               
                                continue                                        
                        }                                                       
                                                                                
                        catch {$db close}                                       
                        return {*}$opts $msg                                    
                }                                                               
        }                                                                       
                                                                                
        if {[llength $ret] == 1} {                                              
                lassign [lindex $ret 0] x val                                   
        }                                                                       
        $db close                                                               
        return $val                                                             
}

The original problem manifested as an error message from running /mod/bin/status.

 

[prpr]

there is a work-around for the read-only case by trying to copy the locked DB and opening the result

How does it (and any subsequent copies) get deleted? Looks like you need to delete it at the bottom of the function, if you copied it, given that it's only single-use.

 

[/df]

It's at most one /tmp/setup.dbXXXXXX file per Jim process that loads system.class and encounters the issue. As it's the first time I've seen the problem I thought we could exceptionally stand a few of such files that don't survive a reboot.

It would be interesting to understand whether updates made to the DB are reflected in the copied file; possibly they live in some journal until the DB connection that made the changes is closed?

 

[prpr]

It's at most one /tmp/setup.dbXXXXXX file per Jim process that loads system.class and encounters the issue. As it's the first time I've seen the problem I thought we could exceptionally stand a few of such files that don't survive a reboot.

Seems like this can be handled using the defer command e.g.

file copy -force /var/lib/humaxtv/setup.db $newdb
defer "catch {file delete $newdb}"

It would be interesting to understand whether updates made to the DB are reflected in the copied file

Once the transaction is committed they will be.

possibly they live in some journal until the DB connection that made the changes is closed?

The connection doesn't need to be closed. Read this.

 

[/df]

Possibly related: after being switched off last night and restarted (I wasn't there then) the box exhibits the symptoms of having gone through the uncommanded factory reset; in which case maybe a routine that catches a temporary setup.db and restores it after the humaxtv process has finished would prevent this situation ...

 

[prpr]

Today I had HD Fox T2 in a condition where the /var/lib/humaxtv/setup.db could not be opened because it was persistently locked

Whilst your mod guards against a long-term lock, is it worth a guard against a short-term lock with something like:
$db query {pragma busy_timeout(50)}

 

[prpr]

Updated WebIf to 1.5.0-2 to implement the above setup.db locking enhancements and to fix the File Editor problem with apostrophes in the name

 

[prpr]

I struggle to understand why the Jim packages are separate in the first place. Anything like the above four that rely on each other for correct operation should surely be in the same package, especially as the files all come from the same build output of the same source code tree.
An option would be to amalgamate them, and then somehow clean-up the redundant packages.

Turns out that this is quite easy, now that I've had time to read the documentation and actually test it.
jim-0.81-2 and webif_1.5.0-3 released. You should be able to install the former without the latter and not break anything. Does anyone want to test it?
Installing the latter will automatically install the former because of the dependency.
The jim-oo, jim-pack and jim-sqlite3 packages are all then redundant and should be automatically removed as part of the upgrade.

Hopefully this doesn't cause any hassles and removed the blocker that has been holding up release of WebIf 1.5.0