.BMP Files

[4ndy]

When archiving recordings via Filezilla I have noticed a number of .bmp files sitting amongst some of the recording sets. Obviously these are not visible in the file browser, so don't get deleted.

Are these part of the process which generates the thumbnail? I have the thumbnails package installed as my machines are generally left on.

Sent from my SM-G950F using Tapatalk

 

[Luke]

... a number of .bmp files ...
Are these part of the process which generates the thumbnail?

You could start by seeing if the BMPs you are seeing are accounted for by the first reason given previously in the following hread
https://hummy.tv/forum/threads/thum...recordings-thumbnails.7787/page-4#post-132444
or if it could be related to the second possibly reason raised in that thread.

 

[prpr]

Are these part of the process which generates the thumbnail?

Yes. It's a nuisance and I'm none the wiser about the cause, or a solution.

 

[4ndy]

One of mine is dated 27th July - Tom Jones.bmp, so they are not from a superseded version of a package as was first suggested. I also have an error in the queue relating to the thumbnail generation for that recording. the others (I have been deleting most as I find them) are too old to still be in the queue.

I will just continue to delete them as I come across them.

Thanks for the replies.

 

[andyk]

none the wiser about the cause, or a solution.

I've been modding this area (to automatically generate new nts and thm files if a video is edited off-box).

These bitmaps are left lying around because a failure is caught in ts mkbmp when exec'ing ffmeg to generate the bmp file. The catch command does not return any msg to help with diagnose the issue. The bitmap is generated perfectly.

Following such a failure, the thumbnail is not generated again from the decrypt path. Neither is it queued for generation by the daily catch-up scan whilst the failed queue entry persists, or if the Humax generates a thumbnail. In fact it's never generated, even if you use the 'force' facility, as ffmpeg refuses to overwite the bmp file without the '-y' option.
This fail happens quite often - I've not seen any pattern to the recordings that suffer in this way - ffmeg returns the (meaningless) non-zero exit result because of some minor recoverable problem with the video stream and the thumbnail sequence gives up in response to this.

There are various ways round this - probably the simplest is to get rid of the error checking. In my case I put in code in the error path to check the bmp file is present and the right size - it always has been.

The diagnostics here ('puts' the message to stdout after the caught fail, which is redirected to null in the auto process) currently aren't effective. I've switched this to output to stderr with the usual log time stamp (stderr is redirected to the auto log for the auto process). However, I've never seen it give any information even when a fatal error is forced so maybe the puts is deliberate. I don't have much idea of what Jim-TCL is supposed to be doing with this message facility.

There are a other problems with the thumbnail generation. For example they are never generated if the Humax generates them before decrytion completes. Since the thumbnail is generated during shutdown after a recording, this is quite frequent for my usage. If this wasn't the case I suspect there would be a lot more of these bitmaps lying around.

I've corrected these problems and things are working as I expect (plus changes for me so if I indulge in off-box edits, the nts file is automatically regenerated, along with the thumbnail).

Mind you, thumbnails aren't very effective really, but at least they're a little more meaningful if they're well into the recording. And no more bitmap debris.

 

[/df]

I've been modding this area (to automatically generate new nts and thm files if a video is edited off-box).

These bitmaps are left lying around because a failure is caught in ts mkbmp when exec'ing ffmpeg to generate the bmp file. The catch command does not return any msg to help with diagnose the issue. The bitmap is generated perfectly.

man ffmpeg is pretty unhelpful on return codes.

You should be able to get diagnostics from ffmpeg if you redirect stderr into the caught output (exec ... 2>@1, and perhaps change the -loglevel. It may be necessary to detect and ignore the specific error (what is it?); if ts::mkbmp fails, it should delete any bitmap that it might have started to create. With your proposal below, assuming that a new bitmap of the correct size means it worked, any other new bitmap should be deleted.

One thing to investigate Is whether there is any difference between the old (2.x) and new (4.y) versions. I have a suspicion that the more heavyweight version 4 may sometimes hang or crash the machine.

...
I've corrected these problems and things are working as I expect (plus changes for me so if I indulge in off-box edits, the nts file is automatically regenerated, along with the thumbnail).
...

Might you be able to fork the repository https://git.hpkg.tv/hummypkg/webif (after registering, if you hadn't already) and propose your generic changes to /mod/webif/lib/ts.class as a pull request?

A usable workflow is: fork, edit file in forked repo, save to a patch branch, create pull request from your patch branch to hummypkg/master.

If you are proposing any changes to the files in the thumbnails package, ie the files in /mod/webif/plugin/thumbnails, you could post diffs in the package thread linked above.

 

[andyk]

redirect stderr into the caught output (exec ... 2>@1,

Yes - I've done that.

change the -loglevel

Yes I tried that - it just shows that all video streams I looked at generate this sort of thing ...

[h264 @ 0x45e390] number of reference frames (0+5) exceeds max (4; probably corrupt input), discarding one
[NULL @ 0x45e390] illegal reordering_of_pic_nums_idc 8
    Last message repeated 1 times
[h264 @ 0x49f260] co located POCs unavailable
    Last message repeated 1 times
[h264 @ 0x49f260] reference picture missing during reorder
    Last message repeated 1 times
[h264 @ 0x49f260] Missing reference picture, default is 65383
    Last message repeated 1 times
[h264 @ 0x49f260] mmco: unref short failure

...the sort of thing you often see ffmpeg churning out. This lot didn't lead to any failure reported. I thought it was a bit busy and didn't tell me much, so returned to loglevel fatal. The various posts I saw in this area agreed it was all unsatisfactory but recommended against attempting to analyse the error stream.

Might you be able to fork the repository https://git.hpkg.tv/hummypkg/webif (after registering, if you hadn't already) and propose your generic changes to /mod/webif/lib/ts.class as a pull request?

Gulp. I suppose I ought expand my knowledge and learn more about Git, so I'll give it a go. I can certainly manage a patch. Can you confirm that the code delivered by the thumbnail package is not held in the git repository then? (That needs to change to make sure thumbnails appear even if the box has generated them before decrypting finishes.)

 

[/df]

... all video streams I looked at generate this sort of thing ...
[...]
...the sort of thing you often see ffmpeg churning out. This lot didn't lead to any failure reported. I thought it was a bit busy and didn't tell me much, so returned to loglevel fatal. The various posts I saw in this area agreed it was all unsatisfactory but recommended against attempting to analyse the error stream.

Such a pity that the MPEG standards weren't created from the ffmpeg code rather than the other way round. But Jim exec is only supposed to complain if the command has exit code != 0.

... Can you confirm that the code delivered by the thumbnail package is not held in the git repository then? ...

Defo.

The thumbnails package delivers the files in the /mod/webif/plugins/thumbnails directory, which aren't in the Webif repo. One could create a new repository containing the files from the package and modify those to generate the patches.

The Gitea server doesn't demand a lot of Git skills. You can do lots more stuff with the git command line but it isn't required.

 

[MymsMan]

Gulp. I suppose I ought expand my knowledge and learn more about Git, so I'll give it a go. I can certainly manage a patch.

When learning GIT I tried to write it down as a wiki page , it may help you get started but I am no git expert so I may have got some parts wrong

 

[prpr]

Proposed a patch myself, which has been accepted.

 

[andyk]

Proposed a patch myself, which has been accepted.

Excellent! That should stop the left-over bmp files.

 

[prpr]

Excellent! That should stop the left-over bmp files.

It does indeed. I applied it to all my boxes and haven't had a problem since.
The trouble is, I don't think it has gone anywhere apart from the git repository as there hasn't been a Beta release since, let alone a main-line one.
This does seem to be a real problem now.