Sysmon - latest CPU Utilisation charts not available

[hummybee]

Hi prpr

IT'S WORKING!!!!!!!!!!!!!!!!!!

Do I still need to open a second telnet session?

Can you please summarise what was wrong, how it happened and what fixed it (all for future reference)?

 

[martxw]

I think I've been having a different cause for the problem. No CPU data since 11 Feb.
Running the crontab entry "/mod/monitor/run" shows:

Runtime Error: /mod/monitor/lib/db.jim:69: database disk image is malformed
in procedure 'record' called at file "/mod/monitor/bin/smart", line 38
at file "/mod/monitor/lib/db.jim", line 69

If I run sqlite3 /mod/monitor/monitor.db "SELECT * FROM smart;" I see this at the end.

1392088682|raw|0|0|432|0|2593
1392088742|raw|0|0|432|0|2593
1392088800|5min|0|0|432|0|2593
1392088802|raw|0|0|432|0|2593
1392088862|raw|0|0|432|0|2593
1392088922|raw|0|0|432|0|2593
1392088982|raw|0|0|432|0|2593
1392089042|raw|0|0|432|0|2593
1392089100|5min|0|0|432|0|2593
Error: database disk image is malformed

To fix it I just renamed monitor.db and ran "/mod/monitor/run", and a new DB file is created.

Finally, to restore the historic data, I ran:

humax# sqlite3 /mod/monitor/monitor.db.bak ".dump" > backup.sql
humax# sqlite3 /mod/monitor/monitor.db < backup.sql
Error: near line 3: table smart already exists
Error: near line 3292: table temp already exists
Error: near line 6597: table vmstat already exists

It appears to be working for the moment now. Not sure what happened after the last good reading at "Tue, 11 Feb 2014 03:25:00 GMT"

 

[hummybee]

Just rebooted again (from webif) and it's still working fine so looks like it's fixed.

Thank you everyone for taking the time to help, particularly prpr, all very much appreciated.

 

[prpr]

Just rebooted again (from webif) and it's still working fine so looks like it's fixed.

But nothing has actually been changed, has it?
All the above were just non-permanent tests.

 

[hummybee]

Hi prpr

Now that it's on from standby, it's not working again.

As per my post #60, I completed the first and second 'tests' but stopped after that as I didn't (and still don't) know how to open a second telnet session.

Can you please pick it up from there and let me know what I need to do now?

 

[Ezra Pound]

If you are using Putty, right Click on the blue bar at the top of the window an select 'Duplicate Session'

 

[hummybee]

Hi Ezra Pound.

I'm using WinXP.

 

[Ezra Pound]

So how do you open your Telnet Sessions?

 

[Black Hole]

Probably with the Windows Telnet client.

 

[hummybee]

Hi Ezra Pound and Black Hole.

Start/run/telnet/192.168.x.x.

I have just downloaded a portable version of Putty so could use it but may need some help.

Do I start from the beginning as per prpr's post #57 or just the last part?

 

[Ezra Pound]

As long as your Telnet session opens in a window you can just run Start/run/telnet 192.168.x.x twice, you will then have two Telnet windows available, enter prpr's last two of 4 lines (starting with pkill crond) in one window and then run the 1 Minute stuff in the other window

 

[hummybee]

I thought I tried that but will try again.

Don't want to mess up recordings scheduled for tonight so will try tomorrow and hopefully prpr will be able to let me know where to start from.

Thanks for your help.

 

[prpr]

As af123 said, it looks like a Cron problem, but what exactly is proving rather elusive.
Reboot the Humax and run this in one window:

humax# sqlite3 /mod/monitor/monitor.db "select * from vmstat where dat>1393095000"

every minute for a few minutes.
Then in another window do this:

humax# pkill crond
humax# /mod/sbin/crond -f -d 1

and repeat the command in window 1 every few minutes. If you see entries being added in the database, then do this in window 2:

humax# pkill crond
humax# /mod/sbin/crond

and repeat the command in window 1 every few minutes. Do you see entries being added to the database or not?

 

[hummybee]

Hi prpr

The second session of telnet is changing/updating but not sure about entries being added to the database.

Session one (so far)

Humax HDR-Fox T2 (HumaxHDRFoxT2two) 1.03.11/2.22

HumaxHDRFoxT2two# sqlite3 /mod/monitor/monitor.db "select * from vmstat where da
t>1393095000"
HumaxHDRFoxT2two# sqlite3 /mod/monitor/monitor.db "select * from vmstat where da
t>1393095000"

1393148464|raw|2604|948|26|81028|21|46|6
HumaxHDRFoxT2two#
HumaxHDRFoxT2two# sqlite3 /mod/monitor/monitor.db "select * from vmstat where da
t>1393095000"
1393148464|raw|2604|948|26|81028|21|46|6
HumaxHDRFoxT2two# sqlite3 /mod/monitor/monitor.db "select * from vmstat where da
t>1393095000"
1393148464|raw|2604|948|26|81028|21|46|6
1393148524|raw|2268|888|32|79940|21|39|8
HumaxHDRFoxT2two# sqlite3 /mod/monitor/monitor.db "select * from vmstat where da
t>1393095000"
1393148464|raw|2604|948|26|81028|21|46|6
1393148524|raw|2268|888|32|79940|21|39|8
HumaxHDRFoxT2two# sqlite3 /mod/monitor/monitor.db "select * from vmstat where da
t>1393095000"
1393148464|raw|2604|948|26|81028|21|46|6
1393148524|raw|2268|888|32|79940|21|39|8
1393148584|raw|2080|836|36|80280|21|34|9
1393148644|raw|2092|872|39|80180|22|30|10
1393148708|raw|2576|2032|40|77964|22|27|11
1393148768|raw|2608|1776|41|78288|21|24|13
1393148825|raw|2296|884|42|79508|22|22|14
HumaxHDRFoxT2two# sqlite3 /mod/monitor/monitor.db "select * from vmstat where da
t>1393095000"
1393148464|raw|2604|948|26|81028|21|46|6
1393148524|raw|2268|888|32|79940|21|39|8
1393148584|raw|2080|836|36|80280|21|34|9
1393148644|raw|2092|872|39|80180|22|30|10
1393148708|raw|2576|2032|40|77964|22|27|11
1393148768|raw|2608|1776|41|78288|21|24|13
1393148825|raw|2296|884|42|79508|22|22|14
HumaxHDRFoxT2two#


Session two (so far)


Humax HDR-Fox T2 (HumaxHDRFoxT2two) 1.03.11/2.22

HumaxHDRFoxT2two# pkill crond
HumaxHDRFoxT2two# /mod/sbin/crond -f -d 1
crond: crond (busybox 1.20.2) started, log level 1
crond: user:root entry:0 2 * * * /mod/sbin/anacron -s -d
100000000000000000000000000000000000000000000000000000000000
001000000000000000000000
11111111111111111111111111111111
111111111111
1111111
crond: command:/mod/sbin/anacron -s -d
crond: user:root entry:* * * * * /mod/monitor/run
111111111111111111111111111111111111111111111111111111111111
111111111111111111111111
11111111111111111111111111111111
111111111111
1111111
crond: command:/mod/monitor/run
crond: user:root entry:*/10 * * * * /mod/webif/lib/bin/auto >/dev/null 2>&1
100000000010000000001000000000100000000010000000001000000000
111111111111111111111111
11111111111111111111111111111111
111111111111
1111111
crond: command:/mod/webif/lib/bin/auto >/dev/null 2>&1
crond: user:root entry:0 2 * * * /mod/sbin/anacron -s -d
100000000000000000000000000000000000000000000000000000000000
001000000000000000000000
11111111111111111111111111111111
111111111111
1111111
crond: command:/mod/sbin/anacron -s -d
crond: user:root entry:* * * * * /mod/monitor/run
111111111111111111111111111111111111111111111111111111111111
111111111111111111111111
11111111111111111111111111111111
111111111111
1111111
crond: command:/mod/monitor/run
crond: user:root entry:*/10 * * * * /mod/webif/lib/bin/auto >/dev/null 2>&1
100000000010000000001000000000100000000010000000001000000000
111111111111111111111111
11111111111111111111111111111111
111111111111
1111111
crond: command:/mod/webif/lib/bin/auto >/dev/null 2>&1
crond: wakeup dt=15
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1010 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1026 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1042 cmd /mod/monitor/run
crond: wakeup dt=10
pkill crond
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1067 cmd /mod/monitor/run
/mod/sbin/crond
crond: wakeup dt=10
humaxcrond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1116 cmd /mod/monitor/run
HumaxHDRFoxT2two#crond: wakeup dt=10
pkill crondcrond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1141 cmd /mod/monitor/run
crond: wakeup dt=10
/mod/sbin/crond
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1519 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1569 cmd /mod/monitor/run
crond: wakeup dt=11
crond: wakeup dt=49
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1583 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: job: 0 /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1597 cmd /mod/monitor/run
crond: child running /bin/sh
crond: USER root pid 1598 cmd /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1615 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1629 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1643 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1657 cmd /mod/monitor/run
crond: wakeup dt=10
crond: wakeup dt=50
crond: file root:
crond: line /mod/sbin/anacron -s -d
crond: line /mod/monitor/run
crond: job: 0 /mod/monitor/run
crond: line /mod/webif/lib/bin/auto >/dev/null 2>&1
crond: child running /bin/sh
crond: USER root pid 1671 cmd /mod/monitor/run

Session two is still changing/updating.

What do I do now please?

 

[prpr]

Let's try a slightly different approach...

You missed this step, so make sure you do it now:
Edit your /mod/var/spool/cron/crontabs/root file to change this line:
* * * * * /mod/monitor/run
to this:
* * * * * /mod/monitor/run -d

Edit your /mod/etc/init.d/S01crond file to change this:
/mod/sbin/crond & ;;
to this:
# /mod/sbin/crond & ;;
/mod/sbin/crond -f -d 1 >/mod/tmp/crond.log 2>&1 & ;;

Then reboot the Humax and use the Webif's Diag page to look at crond.log

You should see lines like this:

crond: USER root pid 25292 cmd /mod/monitor/run -d



+ Starting smart +


Creating table: (create table if not exists smart (
        dat integer not null primary key,
        scope text not null default 'raw'
    ,realloc integer,offline integer,startstop integer,pending integer,poh integer))
RECORD: (insert or replace into smart
        (dat, scope, realloc, offline, startstop, pending, poh) values(1393195861,'raw', '0', '0', '294', '0', '10079'))



+ Starting temp +


Creating table: (create table if not exists temp (
        dat integer not null primary key,
        scope text not null default 'raw'
    ,temp integer))
RECORD: (insert or replace into temp
        (dat, scope, temp) values(1393195862,'raw', '43'))



+ Starting vmstat +


Creating table: (create table if not exists vmstat (
        dat integer not null primary key,
        scope text not null default 'raw'
    ,freemem integer,buffmem integer,scpu integer,cachemem integer,ucpu integer,idle integer,wcpu integer))
RECORD: (insert or replace into vmstat
        (dat, scope, freemem, buffmem, scpu, cachemem, ucpu, idle, wcpu) values(1393195862,'raw', '7480', '16864', '8', '53056', '9', '79', '4'))
crond: wakeup dt=10

Please don't post all the cr@p from the top of the file - it's very dull and we've seen it all before!

 

[hummybee]

Hi prpr

Ok with the first two changes but not clear where this line goes:-

/mod/sbin/crond -f -d 1 >/mod/tmp/crond.log 2>&1 & ;;

Is this how it should look after the changes:-

#!/bin/sh

export TZ=GMT+0BST,M3.5.0/1,M10.5.0/2

case "$1" in
start)
#/mod/sbin/crond & ;;
/mod/sbin/crond -f -d 1 >/mod/tmp/crond.log 2>&1 & ;;

stop)
killall crond ;;
*)
exit 1 ;;
esac

Thanks for your help.

 

[prpr]

Is this how it should look after the changes:-

case "$1" in
start)
#/mod/sbin/crond & ;;
/mod/sbin/crond -f -d 1 >/mod/tmp/crond.log 2>&1 & ;;
stop)
killall crond ;;
*)
exit 1 ;;
esac

Yeah, the line with the # just stops it working, so all you need to do to put it back as it was is to remove the # and the following line. Saves trying to remember what it was before...

 

[hummybee]

Hi prpr

crond: USER root pid 984 cmd /mod/monitor/run -d



+ Starting smart +


Creating table: (create table if not exists smart (
dat integer not null primary key,
scope text not null default 'raw'
,realloc integer,offline integer,startstop integer,pending integer,poh integer))
RECORD: (insert or replace into smart
(dat, scope, realloc, offline, startstop, pending, poh) values(1393258142,'raw', '0', '0', '112', '0', '331'))



+ Starting temp +


Creating table: (create table if not exists temp (
dat integer not null primary key,
scope text not null default 'raw'
,temp integer))
RECORD: (insert or replace into temp
(dat, scope, temp) values(1393258142,'raw', '47'))
Runtime Error: /mod/monitor/bin/vmstat:20: /mod/bin/vmstat: can't load library 'libproc-3.2.8.so'
at file "/mod/monitor/bin/vmstat", line 20



+ Starting vmstat +


Creating table: (create table if not exists vmstat (
dat integer not null primary key,
scope text not null default 'raw'
,freemem integer,buffmem integer,scpu integer,cachemem integer,ucpu integer,idle integer,wcpu integer))
crond: wakeup dt=10

Looks promising but haven't run CPU Utilisation chart yet.

What next?

 

[prpr]

Runtime Error: /mod/monitor/bin/vmstat:20: /mod/bin/vmstat: can't load library 'libproc-3.2.8.so'
at file "/mod/monitor/bin/vmstat", line 20

+ Starting vmstat +

Creating table: (create table if not exists vmstat (
dat integer not null primary key,
scope text not null default 'raw'
,freemem integer,buffmem integer,scpu integer,cachemem integer,ucpu integer,idle integer,wcpu integer))
crond: wakeup dt=10

There should be a line starting with "RECORD:" after the "Creating table" bits, but there isn't. The cause of that would seem to be the Runtime Error about being unable to load the library. The cause of that is unclear.
You could try a "opkg --force-reinstall install procps" and see if that helps.

 

[af123]

That looks like it could be a missing or broken mod environment file...

Try 'cat /mod/.env' from the command line.