WebIf Browse Media Files Really Slow . . .
- Thread starter hy4boy
- Start date
OK: the df and df -i output both look very like mine - nothing amiss there that I can see. (You have a 2TB disk - you upgraded it yourself I guess?)
The top output: the text formatting isn't like what I get from top on my system, which is weird. (Can you tell me what version number you get when you run "top -v"? I get "procps version 3.2.8".)
The main thing though is that the processor is mostly idle. That surprises me, I admit (the stats were copied while the browser page was loading, yes?). Also, even the "io" percentage is small, so it doesn't seem to be waiting for disk accesses.
I'm out of ideas for now I'm afraid... Hopefully af123 will make something of your web-page timing info
(The one last thing I have thought of is kinda faffy - running "strace" on the httpd or jimsh processes while the request is live, to see what they're doing. It might not even work. Hence let's see what other suggestions people have beforehand!
)
The top output: the text formatting isn't like what I get from top on my system, which is weird. (Can you tell me what version number you get when you run "top -v"? I get "procps version 3.2.8".)
The main thing though is that the processor is mostly idle. That surprises me, I admit (the stats were copied while the browser page was loading, yes?). Also, even the "io" percentage is small, so it doesn't seem to be waiting for disk accesses.
I'm out of ideas for now I'm afraid... Hopefully af123 will make something of your web-page timing info
(The one last thing I have thought of is kinda faffy - running "strace" on the httpd or jimsh processes while the request is live, to see what they're doing. It might not even work. Hence let's see what other suggestions people have beforehand!
)
Black Hole
May contain traces of nut
These aren't forum "rules" as such, no need to get into a lather about it, just what has developed as best practice. Rather than let newcomers find out by trial and error (and put up with an untidy forum in the process), I point them at a get-you-going guide. Simple as that, and no apology necessary.
Update: I've played around with strace and I was able to get it to work for the media browser script on my machine. It just *might* help to work out what's going on with your machine, hy4boy...(The one last thing I have thought of is kinda faffy - running "strace" on the httpd or jimsh processes while the request is live, to see what they're doing. It might not even work. Hence let's see what other suggestions people have beforehand!
Soooo, if you get bored enough
, try this:
Code:
cd /mod/webif/html/browse/
strace -tt -T -o /tmp/st.out /mod/bin/jimsh /mod/webif/html/browse/index.jim > /tmp/jimout.htmlThen you can take a look at the output file /tmp/st.out and see what you see. My file is almost 3MB in size, so probably not ideal to post it on the forum
but dropbox or similar would work - that's providing a quick inspection of the file doesn't yield any clues. NB: the start of each line will hold a timestamp for the entry point to the system call, and the last bit of each line is supposed (bit iffy in practice) to hold the duration of each call.Oops: nearly forgot - you will need to add the strace package first.
Black Hole
May contain traces of nut
Isn't the energy of new blood wonderful.
The timing information says that it is the main index.jim script which is taking the time, but interestingly the channel icons come in after a minute so there may be an initial delay there. Have you by any chance ever enabled the test theTVDB integration feature? This tries to connect to the Internet when building the media list. You could run the tvdb/off diagnostic to be sure it's off.
Can you run the index command from the command line and see if there are any obvious pauses in the output? That will provide further clues as to where the delay is happening.
Code:
humax# cd /mod/webif/html/browse
humax# ./index.jim
.. lots of output ..
<font class=footnote>Rendered in: 1.115 seconds</font>This is definitely not related to your slow media browser.
Correct!neilski said:
Looks like I didn't have 'sysmon' installed after all as I got this . . .neilski said:
Code:
humax# top -V
top: invalid option -- V
BusyBox v1.20.2 (2013-01-02 19:13:09 GMT) multi-call binary.
Usage: top [-b] [-nCOUNT] [-dSECONDS] [-m]
Provide a view of process activity in real time.
Read the status of all processes from /proc each SECONDS
and display a screenful of them.
Keys:
N/M/P/T: show CPU usage, sort by pid/mem/cpu/time
S: show memory
R: reverse sort
H: toggle threads, 1: toggle SMP
Q,^C: exit
Options:
-b Batch mode
-n N Exit after N iterations
-d N Delay between updates
-m Same as 's' keyAfter installing 'sysmon' and trying again, I got this . . .
Code:
Tasks: 46 total, 1 running, 45 sleeping, 0 stopped, 0 zombie
Cpu(s): 4.3%us, 2.8%sy, 2.3%ni, 90.3%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 125016k total, 121460k used, 3556k free, 14280k buffers
Swap: 131064k total, 44k used, 131020k free, 58452k cachedIndeed they were - both first time without 'sysmon' and this time with.neilski said:
OK, so if we take a small sample of the output as below there appears to be a repeat pattern where it pauses . . .
Code:
<span class=filesize> (1.48 GiB) </span>
<img src="/img/channels/out/More 4.png" class="va browsechannel" height=21 alt="More 4" title="More 4">
<img src="/images/173_3_00_G3_News.png" class="va genre" height=21 alt="News & Factual" title="News & Factual">
<img src="/images/172_1_26_SD.png" class="va" height=21 >
------- PAUSES HERE ------
<img src="/img/compress.png" class="va" height=21 alt="Shrunk" title="Shrunk">
<a href=#>
<img class="opt va" border=0 width=45 type=ts did=28
locked=0 encd=0 def=SD new=0 bx=0
shrunk=1 rsize=1588068672
odencd=1 dlna=0 thmok=1
src=/images/181_1_00_Help5_OPT_Plus.png>
</a>
<div class="results blood" style="margin: 0 0 0 5em"></div>
</div>
<div class="va bf" id=29>
<img class=va border=0 src=/img/Video_Failed.png>
<input class="fs fsts" type=checkbox>
<a class=bf title="Jimmy Carr delivers a host of one-liners, stories and jokes, some clever, some rude and a few totally unacceptable. It's an opportunity to see Jimmy at his near-the-knuckle best. [S]"
file="/media/My%20Video//Jimmy%20Carr_%20Laughing%20%26%20Joking_20141130_2305-1419448740.ts" type=ts href=#>
Jimmy Carr_ Laughing & Joking_20141130_2305-1419448740.ts
</a>
<span class=filesize> (991.55 MiB) </span>
<img src="/img/channels/out/Channel 4+1.png" class="va browsechannel" height=21 alt="Channel 4+1" title="Channel 4+1">
<img src="/images/173_3_00_G3_Special.png" class="va genre" height=21 alt="Entertainment" title="Entertainment">
<img src="/images/172_1_26_SD.png" class="va" height=21 >
------- PAUSES HERE ------
<img src="/img/compress.png" class="va" height=21 alt="Shrunk" title="Shrunk">
<a href=#>
<img class="opt va" border=0 width=45 type=ts did=29
locked=0 encd=0 def=SD new=0 bx=0
shrunk=1 rsize=1039715328
odencd=1 dlna=0 thmok=1
src=/images/181_1_00_Help5_OPT_Plus.png>
</a>
<div class="results blood" style="margin: 0 0 0 5em"></div>
</div>
<div class="va bf" id=30>
<img class=va border=0 src=/img/Video_TS.png>
<input class="fs fsts" type=checkbox>
<a class=bf title="...Arrows 2014. Suzi Perry and expert analysts Eddie Jordan and David Coulthard reflect on the 2014 Formula 1 season and the thrilling battle between Lewis Hamilton and Nico Rosberg. [HD] [S]"
file="/media/My%20Video//Lewis%20Hamilton_%20Duel%20of%20the%20Silver____20141227_1302-1419723296.ts" type=ts href=#>
Lewis Hamilton_ Duel of the Silver____20141227_1302-1419723296.ts
</a>
<span class=filesize> (3.62 GiB) </span>
<img src="/img/channels/out/BBC ONE HD.png" class="va browsechannel" height=21 alt="BBC ONE HD" title="BBC ONE HD">
<img src="/images/173_3_00_G3_Sports.png" class="va genre" height=21 alt="Sport" title="Sport">
<img src="/images/172_1_00_HD.png" class="va" height=21 >
------- PAUSES HERE ------
<img src="/img/compress.png" class="va" height=21 alt="shrunk" title="Shrunk">So after a little more playing around, the 'slowness' / delays appear to be being caused by the individual recordings in the media view. I've grouped all single programs into a folder, so that now the 'My Video' directory only contains folders and it refreshes that list in about 1 second. However when I go into a folder to list the programs within, then the delay(s) reappear.
Black Hole
May contain traces of nut
It could be that there is a problem with one or some of the files. Do you get the delinquent half-awake state? See Things Every... (click) section 18.
prpr
Well-Known Member
Divide and conquer.
I'm getting similar problems (very slow browsing of Media Library via Webif), which (also?) started after I replaced my original drive with a 2TB Seagate PVR (AV) unit. Followed the Wiki instructions for replacement, but used the native device Format option as the firmware is recent and supposed to support the 2TB units. All working OK apart from the slow browsing via the Webif and Decrypt / AutoDecrypt isn't working on recent files. One thing (maybe linked?) is that a bunch of core system (or Custom FW?) directories now appear in the "media/My Videos" directory, along with new recordings. Tempted to start over again with the disk replacement, but wonder if any better ideas? As with comments above, no strange errors showing in the logs apart from these in the Recmon log.
Device software versions are:
Web interface version: 1.2.3-5
Custom firmware version: 3.00 (build 2137)
Humax Version: 1.03.12 (kernel HDR_CFW_3.00)
df and ls -l listing of "media/My Videos" is :
note dvbepg, mod, thumbnail, Tsr directories here.
Thoughts / suggestions / help welcome!
Code:
96 at file "/mod/etc/recmon.d/auto-unprotect", line 11
95 /mod/etc/recmon.d/auto-unprotect:11: Error: invalid command name "0"
94 at file "/mod/etc/recmon.d/auto-unprotect", line 11
93 /mod/etc/recmon.d/auto-unprotect:11: Error: invalid command name "0"
92 at file "/mod/etc/recmon.d/auto-unprotect", line 11
91 /mod/etc/recmon.d/auto-unprotect:11: Error: invalid command name "0"Device software versions are:
Web interface version: 1.2.3-5
Custom firmware version: 3.00 (build 2137)
Humax Version: 1.03.12 (kernel HDR_CFW_3.00)
df and ls -l listing of "media/My Videos" is :
Code:
humax# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 21824 21824 0 100% /
tmpfs 62508 64 62444 0% /tmp
tmpfs 62508 0 62508 0% /media
/dev/mtdblock1 2048 604 1444 29% /var/lib/humaxtv
/dev/mtdblock2 2048 408 1640 20% /var/lib/humaxtv_backup
/dev/sda1 1035684 44752 990932 4% /mnt/hd1
/dev/sda2 1941367752 651912116 1289455636 34% /mnt/hd2
/dev/sda3 10325780 806964 9518816 8% /mnt/hd3
humax# ls -l media/My\ Video/
-rw-r--r-- 1 root root 13184 May 16 23:25 Alan Partridge_ Alpha Papa_20150516_2122.hmt
-rw-r--r-- 1 root root 8270208 May 16 22:47 Alan Partridge_ Alpha Papa_20150516_2122.nts
-rw-r--r-- 1 root root 43680 May 16 22:01 Alan Partridge_ Alpha Papa_20150516_2122.thm
-rw-r--r-- 1 root root 2659565568 May 16 22:47 Alan Partridge_ Alpha Papa_20150516_2122.ts
-rw-r--r-- 1 root root 324 Jun 29 22:59 Bitter Lake.hmi
-rw-r--r-- 1 root root 2460499416 Jun 29 22:29 Bitter Lake.mp4
-rw-r--r-- 1 root root 13888 Jun 16 23:24 Dara O Briain Meets Stephen Hawking_20150616_2238.hmt
-rw-r--r-- 1 root root 4522112 Jun 16 23:24 Dara O Briain Meets Stephen Hawking_20150616_2238.nts
-rw-r--r-- 1 root root 43680 Jun 16 23:24 Dara O Briain Meets Stephen Hawking_20150616_2238.thm
-rw-r--r-- 1 root root 1643012096 Jun 16 23:24 Dara O Briain Meets Stephen Hawking_20150616_2238.ts
-rw-r--r-- 1 root root 13888 Jun 29 00:40 David Attenborough Meets President Obama_20150628_2233.hmt
-rw-r--r-- 1 root root 3911040 Jun 28 23:10 David Attenborough Meets President Obama_20150628_2233.nts
-rw-r--r-- 1 root root 43680 Jun 29 00:40 David Attenborough Meets President Obama_20150628_2233.thm
-rw-r--r-- 1 root root 1341599744 Jun 28 23:10 David Attenborough Meets President Obama_20150628_2233.ts
drwxr-xr-x 2 root root 4096 May 24 23:16 Elizabeth I
drwxr-xr-x 2 root root 4096 Jul 26 16:00 Episodes
-rw-r--r-- 1 root root 13888 Jul 6 00:40 Far from the Madding Crowd_20150705_2201.hmt
-rw-r--r-- 1 root root 15180608 Jul 6 00:39 Far from the Madding Crowd_20150705_2201.nts
-rw-r--r-- 1 root root 43680 Jul 6 00:40 Far from the Madding Crowd_20150705_2201.thm
-rw-r--r-- 1 root root 6579281920 Jul 6 00:39 Far from the Madding Crowd_20150705_2201.ts
-rw-r--r-- 1 root root 13184 May 19 18:10 Grayson Perry's Dream House_20150517_2103.hmt
-rw-r--r-- 1 root root 2852512 May 17 22:02 Grayson Perry's Dream House_20150517_2103.nts
-rw-r--r-- 1 root root 43680 May 17 22:44 Grayson Perry's Dream House_20150517_2103.thm
-rw-r--r-- 1 root root 1442140160 May 17 22:02 Grayson Perry's Dream House_20150517_2103.ts
-rw-r--r-- 1 root root 13184 Jun 15 01:03 High Fidelity_20150614_2316.hmt
-rw-r--r-- 1 root root 5155520 Jun 15 01:03 High Fidelity_20150614_2316.nts
-rw-r--r-- 1 root root 43680 Jun 15 01:03 High Fidelity_20150614_2316.thm
-rw-r--r-- 1 root root 3042471936 Jun 15 01:03 High Fidelity_20150614_2316.ts
drwxr-xr-x 2 root root 4096 Jul 19 21:03 Humans
drwxr-xr-x 33 root root 20480 May 10 19:37 My Video
drwxr-xr-x 2 root root 4096 Jun 22 22:32 New_ A Place in the Sun_ Home___
-rw-r--r-- 1 root root 13888 Jun 15 23:09 New_ Made in Chelsea_20150615_2101.hmt
-rw-r--r-- 1 root root 2821536 Jun 15 21:59 New_ Made in Chelsea_20150615_2101.nts
-rw-r--r-- 1 root root 43680 Jun 15 22:13 New_ Made in Chelsea_20150615_2101.thm
-rw-r--r-- 1 root root 1280540672 Jun 15 21:59 New_ Made in Chelsea_20150615_2101.ts
drwxr-xr-x 2 root root 4096 May 30 11:51 New_ The Hotel Inspector
-rw-r--r-- 1 root root 13184 May 11 00:26 Prometheus_20150510_2201.hmt
-rw-r--r-- 1 root root 14218496 May 11 00:26 Prometheus_20150510_2201.nts
-rw-r--r-- 1 root root 43680 May 11 00:26 Prometheus_20150510_2201.thm
-rw-r--r-- 1 root root 5085855744 May 11 00:26 Prometheus_20150510_2201.ts
-rw-r--r-- 1 root root 13184 Jun 18 22:55 The Secret World of Lego_20150614_2001.hmt
-rw-r--r-- 1 root root 2853504 Jun 14 21:00 The Secret World of Lego_20150614_2001.nts
-rw-r--r-- 1 root root 43680 Jun 14 22:30 The Secret World of Lego_20150614_2001.thm
-rw-r--r-- 1 root root 1345503232 Jun 14 21:00 The Secret World of Lego_20150614_2001.ts
drwxr-xr-x 2 root root 4096 May 9 21:07 Tsr
-rw-r--r-- 1 root root 13888 Jun 29 00:40 Walkabout_20150628_2302.hmt
-rw-r--r-- 1 root root 9449344 Jun 29 00:40 Walkabout_20150628_2302.nts
-rw-r--r-- 1 root root 43680 Jun 29 00:40 Walkabout_20150628_2302.thm
-rw-r--r-- 1 root root 5096488960 Jun 29 00:40 Walkabout_20150628_2302.ts
drwxr-xr-x 2 root root 4096 Jul 26 16:01 Witnesses
drwxr-xr-x 2 root root 4096 May 9 21:07 dvbepg
-rw-r--r-- 1 root root 1036 May 9 21:07 filebglastop.dat
drwxr-xr-x 12 root root 4096 May 9 21:07 mod
drwxr-xr-x 2 root root 4096 May 9 21:07 thumbnail
humax#note dvbepg, mod, thumbnail, Tsr directories here.
Thoughts / suggestions / help welcome!
You hero! Deleted all of the misplaced directories (which, as you say were also present, correctly, in /mnt/hd1) and Webif Media File Browsing browsing is now back to its old speedy self.
Also re-enabled Content Share (which had somehow become disabled) and Decrypt working again.
Many thanks for your help... So...
>> hy4boy - Worth checking whether you have these duplicated / wrongly copied directories and "filebglastop.dat" in your My Videos folder and deleting them?
Also re-enabled Content Share (which had somehow become disabled) and Decrypt working again.
Many thanks for your help... So...
>> hy4boy - Worth checking whether you have these duplicated / wrongly copied directories and "filebglastop.dat" in your My Videos folder and deleting them?
Last edited: