Monitor Network Usage for Longer

rodp

Member
Hi All,

Is there a way to view network usage for more than one month in the system monitoring page of Webif? eg a year? I believe the package is sysmon that is doing the business but I'm not sure if anyone else has previously suggested / requested a length of time greater than a month - searching doesn't pulling anything up.

Therefore, could this be easily implemented into sysmon or could I do something to record the data somehow each week / month into a file on the humax?

Thanks in advance

Rodp
 
Last edited:
OP
R

rodp

Member
Hi All,

After a little more investigation I found this page: http://wiki.hummy.tv/wiki/Custom_Firmware_Package_Notes#Sysmon and it talks about the monitor.db beign the file that contains the info such as network usage. I tried using the Database Browser but it wasn't available in the list to view. How do you view this file or at least get it into the list to view?

Thanks in advance

Rod
 

Ezra Pound

Well-Known Member
The problem is that the monitor.db file is going to be very large and although the db viewer can handle it, it will become very slow and almost unusable, it would be better to copy the file off the Humax and examine it using an sqlite viewer on a computer, here is what the data would look like in the database browser

upload_2015-11-8_11-34-12.png
 
Last edited:
OP
R

rodp

Member
Hi Ezra,

Thanks for the reply. My db says it's only about 300kb insize (assuming that ls -al command in telnet gives bytes). I've never deleted it and I've had my hummy for a few years now on custom firmware.

I'll copy it off though... Presume ftp in using root?

Could you tell me from the data you attached how i convert that into the charts we see in the normal webif interface? Ie, is the first column a timestamp? Are octects a count of 8 bytes or something?

Thanks

Rod
 

Ezra Pound

Well-Known Member
Yes you would need FPT logon = root in order to get to the /mod/monitor folder. The picture above was the result placing a copy of monitor.db file in an area that the Web-If database viewer can 'see', however I don't recommend doing this, (my 19MB file crashed the Humax). You can get a copy of a Sqlite Browser for your computer from here http://sqlitebrowser.org/ in order to view the database. The first column is an 'Epoch' time stamp where 1445695200 equates to Sat, 24 Oct 2015 14:00:00 GMT and you are correct about the octets.
 

MymsMan

Ad detector
My db says it's only about 300kb insize (assuming that ls -al command in telnet gives bytes). I've never deleted it and I've had my hummy for a few years now on custom firmware
Presumably records older than 30days are being deleted to prevent the database growing too large.

It would be nice if it produced a daily summary record before deleting so that you could go back further in time albeit in less detail (but who needs hourly details for months back)
 
OP
R

rodp

Member
Ok, I'll take a copy and see what there is in the file. If there is a command that is removing anything older than 30 days, where / what do i need to do to alter this? Failing that i imagine a cron could be scheduled to copy the file every month? Any tips on that would be most welcome.

Thanks in advance

Rodp
 

af123

Administrator
Staff member
Presumably records older than 30days are being deleted to prevent the database growing too large.
per-minute data is kept for 1 day;
5-minute aggregates are kept for 5 days;
15-minute aggregates for 10 days
hourly for 15 days.

(see /mod/monitor/lib/db.jim, {proc purge} for those who are interested)
Is there a way to view network usage for more than one month in the system monitoring page of Webif? eg a year?
For most of the stats, it doesn't really make sense to keep longer-term records but it's easy enough to implement. I'll release a sysmon update which creates and retains daily and monthly summaries with longer retention times.
 
OP
R

rodp

Member
Thanks af123,

That's great. I'll look forward to the update. Will it be a cumulative thing from when I apply the update or will the older data / usage be there straight away (assuming there is older data in the DB in the first place).

Thanks

Rodp
 

prpr

Well-Known Member
Will it be a cumulative thing from when I apply the update or will the older data / usage be there straight away (assuming there is older data in the DB in the first place).
Fairly obviously there won't be older data in the DB because it has already been purged and you can't just magic it back.
 
OP
R

rodp

Member
Hi prpr,

so if 1 octet = 1 byte, why don't they call it byte. If that's a byte then Ezra's network comms is very low - I'll have to check mine out as that should be Gigabytes!

Thanks

Rodp
 

prpr

Well-Known Member
so if 1 octet = 1 byte, why don't they call it byte.
It's historical, before the term "byte" was invented really (or at least standardised as 8 bits). Anything networky seems to use octets often.
 

Black Hole

May contain traces of nut
"Byte" is a colloquial term that became popularised and ultimately a ratified standard. It's just a play on "bit", and then from byte we get "nybble" (four bits, half a byte).

In early computer development, there was no standardisation on eight bits as a convenient 'packet' of data - the internal organisation of computers had varying groupings for data width and address width, and whatever the data width it was called a "word" - and it was necessary to be clear what a word was for any particular computer architecture. I worked on computers with 12-bit and 24-bit words.

Multiples of four bits was settled on quite early, because that was efficient for representing and manipulating BCD (Binary Coded Decimal, where decimal digits 0-9 are individually represented in four binary bits). Some computers worked specifically in BCD because it made input and output of human-readable numbers efficient, whereas modern computers can perform the transformations so fast it is better that they perform internal operations in pure binary.

Binary multiples of 8 bits (8, 16, 32 etc) became standard when chip manufacturers decided that an 8-bit data bus in and out of memory chips was the best compromise. Why binary multiples (dropping the 24-bit word)? Because logical operations frequently want to specify a single data bit within the data word, so a data field within the instruction code (again a binary word - people rarely see the nuts and bolts of computer operation these days) has to pinpoint the relevant bit, and this is most efficient if the number of bits to choose from is a power of 2.

Only then did the word "byte" (and its definition as a group of eight binary bits) become commonplace, and then gradually "word" came to mean two bytes (16 bits) and "long word" four bytes - for when the data quantities to be represented/manipulated exceed the range available in one byte (256 values).

In summary: "byte" is a relatively recently coined term, popularised when computing became available to the enthusiast/hobbyist, and originally would have been indeterminate in what data width it meant. Before that, "octet" was in circulation within the computer engineering community (although principally applied to serial data transmission rather than parallel data widths) - and it was clear what it meant by its prefix "oct".

The complication does not end there. Any form of data transmission is "lossy" - ie there is a chance of corruption over the transmission link, and what the receiver gets is not necessarily what the transmitter sent. That is the case even over a short piece of wire, although over a short piece of wire (unless you are hitting the upper limits of data rate) the chance is so slight one does not normally have to worry about it. In order to overcome the loss characteristics of any particular type of transmission link, data is encoded in a way to improve the likelihood of perfect reception. One common way of doing this is 8-5 encoding, where eight bits on the transmission represent 5 bits of actual data (used on disk drives). Thus one octet does not necessarily a byte make (in this example, it would take eight octets to make five bytes).
 
Last edited:

Black Hole

May contain traces of nut
PS: I was browsing in Tesco today, and a Sandisk device claiming to be 1GB said "1GB = 1,000,000,000 Bytes" in the small print (yep, that's why you can't make a "32GB" UPD format for recording on a HD-FOX - see Glossary entry "KiB"). Further down there was the French version of the small print, and that said "1Go = 1,000,000,000 octets".
 
Top