[thumbnails] Package to Auto-Generate Recordings Thumbnails

[af123]

It would be quite nice to have a parameter field to be able to add extra options to the queued for commands on the browse menu

I'll have a look to see how easy that would be.

There's a bug with queuing to force new thumbnails. If you select more than one recording and queue to force an overwrite of the old thumbnail, only the first queue entry has the 'force' parameter added to the command.

This is fixed in the latest version that was published last night.

 

[4ndy]

This package seems to have stopped working. The task is not being added to the queue.

The genthumbs diagnostic scans, but fails to create any thumbnails.

Tried the usual force package install and reboot.



Sent from my iPad using Tapatalk

 

[Black Hole]

What's in auto.log? If it says nothing useful, go into settings and increase the logging level.

 

[4ndy]

Reading af123's comments in this thread, I can see that the auto thumb process is hooked to decrypt. I had turned off auto decrypt as I (naively) thought that it was now built into the a beta stripts package. I obviously misinterpreted the opening post in the offline decryption thread.

Once recursive decrypt is turned back on I can see the thumb process in the queue. I will report back if it actually creates a thumbnail or not, as the diagnostic call did not.

 

[Black Hole]

Off-line decryption is not an alternative for auto-decryption - off-line decryption makes decryption available when the standard decryption processes are not suitable (eg HD-FOX, or if an HDR-FOX has died). Obviously an encrypted recording cannot be read to extract thumbnail frames, so decryption is a prerequisite for thumbnails.

 

[prpr]

I'm seeing lots of .bmp files left in various recording directories, presumably caused by some artefact of the thumbnail generation process.
This is happening across several machines. I noticed it because it stops tidy-folders from working properly.
I've had a look at the relevant code around this process and can't see anything obviously wrong. The thumbnail seems to be generated most of the time, but the .bmp file doesn't get deleted sometimes. It doesn't happen all the time - it's sporadic.
Anyone else getting this? Here's a command to find the files: humax# find "/media/My Video/"|grep bmp

 

[Black Hole]

Yes, but not many:

HDR1: no match
HDR4: no match (not used much, mainly radio)
HDR3:



Note date of file involved!

 

[MymsMan]

I have no bmp files in the video directories.

I suspect @Black Hole 's 5 files are from the older manual Set Thumbnail option which generates samples .5 seconds apart. However when I tried it I didn't see any bmp files left behind

I do need to take a look at detectads though - I do have a bunch of orphaned -crop.thm that are not being renamed with the rest of the -crop files

 

[Ezra Pound]

I think MymsMan is correct in saying the files were created by an older program, in this case from Mar 2013 :-

Humax1# find "/media/My Video/"|grep bmp
/media/My Video/archive/Arena_ Produced by George Martin_20110425_2102.pos0.5.bmp
/media/My Video/archive/Arena_ Produced by George Martin_20110425_2102.pos1.0.bmp
/media/My Video/archive/Arena_ Produced by George Martin_20110425_2102.pos1.5.bmp
/media/My Video/archive/Arena_ Produced by George Martin_20110425_2102.pos2.0.bmp
/media/My Video/archive/Arena_ Produced by George Martin_20110425_2102.pos0.0.bmp
Humax1#

-rw-rw-rw-  1 root root      43734 Mar 12  2013 Arena_ Produced by George Martin_20110425_2102.pos0.0.bmp
-rw-rw-rw-  1 root root      43734 Mar 12  2013 Arena_ Produced by George Martin_20110425_2102.pos0.5.bmp
-rw-rw-rw-  1 root root      43734 Mar 12  2013 Arena_ Produced by George Martin_20110425_2102.pos1.0.bmp
-rw-rw-rw-  1 root root      43734 Mar 12  2013 Arena_ Produced by George Martin_20110425_2102.pos1.5.bmp
-rw-rw-rw-  1 root root      43734 Mar 12  2013 Arena_ Produced by George Martin_20110425_2102.pos2.0.bmp

 

[Black Hole]

I suspect @Black Hole 's 5 files are from the older manual Set Thumbnail option which generates samples .5 seconds apart. However when I tried it I didn't see any bmp files left behind

I can't imagine why I might have done that, unless I was just having a play. My recollection doesn't go back that far.

 

[prpr]

At the moment, deletion of the bitmap file is done using file tdelete which results in trm being executed.
This seems a bit ridiculous, as a standard rm command would appear to be perfectly adequate for small files such as this.
As a test, I've substituted this.

Poking further at trm, I see that it's actually quite needlessly resource expensive:
stat64("test.hmt", {st_mode=S_IFREG|0644, st_size=2072, ...}) = 0
open("test.hmt", O_RDWR|O_LARGEFILE) = 3
close(3) = 0
unlink("test.hmt") = 0

There is no point in opening the file if you're not going to do any operation on it (which only happens with 'large' files). It would seem there is definitely an optimisation to be made with this, regardless of what my test may or may not show up.

 

[af123]

Poking further at trm, I see that it's actually quite needlessly resource expensive:

I think expensive is a stretch, but it can definitely skip the open/close for small files. trm 1.2 is on its way up to the repository, thanks.
The threshold, btw, is 80MiB. trm was based on the logic that the Humax blob uses to remove files, using successive ftruncate() operations to spread the load across the ext2 log and reduce disk throughput issues that were experienced with just rm or Jim's file unlink. This was from before we had undelete as that help to defer deletions until a quiet time.

 

[MymsMan]

I can't imagine why I might have done that, unless I was just having a play. My recollection doesn't go back that far.

Since the date is very close to the creation of the new thumbnails package I expect you were looking at the old version before proposing the automatic version!

 

[Black Hole]

Perhaps I should get onto the Alzheimer's tablets.

 

[EEPhil]

Perhaps I should get onto the Alzheimer's tablets.

But, would you remember to take them?

 

[Black Hole]

I don't always remember to take my current pills - some mornings I find last night's still in the packet, and some evenings I find I rushed out in the morning without taking what I should. I usually know - when the antihistamines wear off I get bunged up.

 

[prpr]

Looking at my Queue log, I see 23 entries where it tried to generate thumbnails. Of these 6 completed successfully and the other 17 failed. The message is the ever so helpful "Error generating thumbnail". Sometimes I despair.

 

[/df]

I installed thumbnails and had a little walk-through review.

• "Error generating thumbnail" means that l.72 of $ts mkthm $offset in /mod/webif/plugin/thumbnails/queue.hook returned but no thumbnail was found; the return value isn't tested and could have been interesting.
• ts::mkthm can fail without an error message only if any of ts::mkbmp (it shouldn't), /bin/echo, or file::tdelete does.
• file::tdelete wraps exec in a try block but then the handler below rethrows any error (if my Jim gestalt is right), so it ought to be protected in the calling code l.378 of /mod/webif/lib/ts.class* -- this seems from walk-through to be the only way of getting the reported error with no other error message:
  

          try {
  ...
          } on error {msg opts} {                             
                  incr opts(-level)                           
                  return {*}$opts $msg                        
          }

• ts::setflag uses catch around exec hmt but hmt always returns 0 == OK and can't cause exec to throw; you have to check the output of hmt to test for an error.
• two exec constructs in ts::mkthm could be replaced by native Jim I/O (dd -> open 1,2, seek 1, copyto 1,2, close 1,2; echo -> open, puts, close).

* The file::tdelete command matches the native file::delete here: the documentation says:

... If the file or directory doesn’t exist, nothing happens. If it can’t be deleted, an error is generated. ...

And the latter means it needs a catch or try to detect when the deletion failed.

 

[Black Hole]

thumbnails was written pre-software decryption; would it be difficult to make it decrypt just the first block (or whatever) so it can generate a thumbnail regardless of the source file being encrypted?

Perhaps there could be a new general purpose on-demand decryption service which clients request media through, which either supplies existing decrypted content or processes the requested blocks using the fastest decryption available.

 

[MymsMan]

thumbnails was written pre-software decryption; would it be difficult to make it decrypt just the first block (or whatever) so it can generate a thumbnail regardless of the source file being encrypted?

Yes, it should be possible
Using the .nts file it you can work out where in the .ts file you needed and you could start decryption (hardware or software) from that offset for just a short period to give you a minimal decrypted .ts file from which you could grab the thumbnail image