Crop fails on manual use of Detect Adverts

[paultry]

No, what happened is the backup copy of the original uncropped file failed to be created as expected, so the process took the least risky option and aborted. Under normal circumstances this wouldn't happen, but the invisibility of error messages definitely doesn't help.


Not standard, but the result of meddling fingers while you were exploring. An edge case.


No need, it will happen any time you try to re-run a crop without tidying up from the previous run properly.

Given circumstances already stated, not knowing that file in _original from a decrypt process has to be deleted before a separate crop process is run isn't reasonably characterised as "meddling fingers", and to call running a separate crop process as an "edge case" is also unrepresentative. Please, moderate your tone.

 

[paultry]

Is anything shown in webif-error.log (viewable via the diagnostics page) following a crop attempt?

As an experiment try a Detectads run with cropping within detectads and see if that runs (can specify -crop y -debug 2 in "Process options" if you don't want to change the global setting)

I did check in webif-error.log previously, but IIRC it was empty. When I run your experiment (specifying -crop y -debug 2 in "Process options), without deleting the copy in _original created by the decrypt process, it does work. The cropping process results in 2 files in media/My Videos. One is the file including bookmarks, and the other is cropped.

So different result when running crop separately to running with detectads.

 

[Black Hole]

Please, moderate your tone.

That was moderate!

If you're that touchy, goodbye.

 

[paultry]

That was moderate!

If you're that touchy, goodbye.

Not touchy, discerning. Unlike you Brian.

 

[prpr]

If this is standard behaviour, then it's just a question of knowing about it. The combination of no documentation of this slightly unexpected behaviour, and the message being unreadable, made for a headache. Is it possible for a Dev to tweek crop behaviour so it keeps the message on screen a bit longer?

It doesn't do it (hide the message) if you deselect the Save Bookmarks option.
But I've tweaked it so that it doesn't hide at all now.
WebIf 1.5.3-1 is now available in the Beta area.

(It's a Javascript change, so you might need to do a Ctrl-F5 refresh, or whatever the equivalent is in any particular browser.)

 

[paultry]

It doesn't do it (hide the message) if you deselect the the Save Bookmarks option.
But I've tweaked it so that it doesn't hide at all now.
WebIf 1.5.3-1 is now available in the Beta area.

(It's a Javascript change, so you might need to do a Ctrl-F5 refresh, or whatever the equivalent is in any particular browser.)

I updated to the WebIf 1.5.3-1, and it works for me. I can't be certain but I think that the message from a successful crop has also changed - it seems much more verbose (in a good way).

Many thanks.

 

[escat]

With apologies for coming late to the party..

I've used crop most days for several years, but never run into this particular problem. Very occasionally I've noticed that it sometimes it appears to finish without giving a full 'completed' message. In these case sometimes it has finished and sometimes it hasn't started. A minor problem. More serious is the problem that Black Hole mentions:

but it does not patch up the .nts sidecar file

This is a real problem (for me at least) that happens on almost every occasion I use crop - leading to unpredictable and infuriating jumps after a pause in playback. As BH says it can be manually fixed by rebuilding the .nts file after cropping, but can't this be fixed within the crop function itself, please? I'm happy to help with testing etc.

 

[prpr]

it can be manually fixed by rebuilding the .nts file after cropping, but can't this be fixed within the crop function itself, please? I'm happy to help with testing

WebIf 1.5.3-2 is now in the Beta repository. It's only had limited testing here.

 

[escat]

That's fantastic! Many thanks for the prompt response. I'll give it a whirl when I get the chance.

 

[escat]

WebIf 1.5.3-2 is now in the Beta repository. It's only had limited testing here

That looks to have done the trick, thanks. I'll continue testing since I've only had a limited sample so far. In summary:

1) The problem occurred on different boxes, on different channels (and different multiplexes). It happened frequently, but I can't be certain it happened on every recording.
2) Most obvious symptoms were lack of progress in the time-line display from some point, and erratic behaviour on pause, fast forward or back.
3) Main difference in the nts files between those produced by crop and those generated by sidecar was in the Played time field at offset X'18' in each frame. Although both files had the same number of frames, those of crop often repeated the same time across several frames (often 3), whereas those produced by sidecar had monotonically increasing times. It may be coincidence, but the frames covering the glitch in the time-line display often had many more repeats.
4) With the new Beta version, the nts files produced by crop and sidecar are identical.

I hope that this is in line with what you anticipated. Many thanks.

A few different nits:

a) The cropped version seems to leave a non-blank character in the first bookmark position (X'31C') in the hmt file, even when there are no residual bookmarks and the bookmark counter (X'298') is 0. This results in a ghost bookmark visible from the bookmarks option.
b) During the crop operation, the progress is reported intermittently, leading to the impression that it's possibly not working.
c) The predicted length of the post-crop recording is several seconds short of the actual length.

Thanks again. It's an invaluable help.

 

[Black Hole]

With the new Beta version, the nts files produced by crop and sidecar are identical.

They would be, no doubt they use the same process. All prpr will have done is chain a sidecar operation onto the crop operation.

 

[MymsMan]

Note: AFAIK the fix only affects manual crop operations and not crop performed within detectads,
I will need to investigate how to build it into detectads itself and unfortunately I have had a shortage of round tuits for far too long

 

[prpr]

4) With the new Beta version, the nts files produced by crop and sidecar are identical.

That's because crop is now running sidecar (if installed) to regenerate the .nts file.

a) The cropped version seems to leave a non-blank character in the first bookmark position (X'31C') in the hmt file, even when there are no residual bookmarks and the bookmark counter (X'298') is 0. This results in a ghost bookmark visible from the bookmarks option.

What bookmarks option? Where? You need to be more descriptive or (preferably) include a picture. Is this with "Save new bookmarks" set to Yes or No when cropping? I can't reproduce this. Can you send me the .hmt file?

b) During the crop operation, the progress is reported intermittently, leading to the impression that it's possibly not working.

The update function runs every second and compares actual file size to expected file size to update the progress bar. I guess it's at the mercy of what the crop program is doing (I've never looked at how it works), but I can't see why it would be intermittent. Can you quantify "intermittently"? This is too vague.

c) The predicted length of the post-crop recording is several seconds short of the actual length.

Again, you need to put some numbers on things. What did you start with, what are the bookmarks, what was the predicted length and what was the actual length.

 

[escat]

Thanks for all your comments. I had guessed that you had incorporated the link to sidecar, but glad that it's confirmed. It will make testing much simpler.

I'll re-run the tests on the other points, to confirm the results and provide more detail. In any case, they are just nits, not serious.

Thanks again.

 

[escat]

What bookmarks option? Where? You need to be more descriptive or (preferably) include a picture. Is this with "Save new bookmarks" set to Yes or No when cropping? I can't reproduce this. Can you send me the .hmt file?

I always set Save new bookmarks to "no". I normally expect the Bookmarks option on the Opt drop-down to be greyed out, but in my tests it is active, resulting in:



The bookmark counter in the hmt file (X'298') is set to zero, so the bookmark address fields should be ignored.




I attach the full hmt file.

I hope that this helps.

(Sorry to be pernickety. When I started programming in the early sixties, turn round time for any run on the computer was a week. One had to look for any unusual feature in the output, just in case! The habit stuck.)

 

[escat]

The update function runs every second and compares actual file size to expected file size to update the progress bar. I guess it's at the mercy of what the crop program is doing (I've never looked at how it works), but I can't see why it would be intermittent. Can you quantify "intermittently"? This is too vague.

One normally expects the in-progress crop screen to show a progress bar like this


When I previously ran this same crop, the progress showed zero continuously until the crop actually finished. This time it behaved as expected.

Conversely on a different programme, the progress bar showed zero for the first 42 seconds.


of the operation. When I previously ran this same crop, it first showed progress as I expected, then the progress went blank for a period, then reappeared before the crop was completed.

That's what I meant by 'intermittent': a poor choice of words for which I apologise.

Again, you need to put some numbers on things. What did you start with, what are the bookmarks, what was the predicted length and what was the actual length.

Again, my words could have been better. Referring to the above images, the option was not to save bookmarks. So, strictly speaking, I wouldn't have expected any reference to 'new bookmarks'. However, that aside, I was equating the position of those new bookmarks to the predicted length of the resultant recording - as I assume they would have been had I'd opted to save bookmarks. You will see that, in both cases, the second bookmark is 8 seconds before the end of the recording.

I hope that this helps. Let me know if you need any more input. And thanks for all your valuable contributions.

 

[escat]

I guess it's at the mercy of what the crop program is doing (I've never looked at how it works)

I'm sorry: I shouldn't have bothered you with these other problems. I hadn't realised that you hadn't been looking in detail at crop. I can now see that they've long existed before now. Please feel free to ignore them.

 

[MymsMan]

I always set Save new bookmarks to "no". I normally expect the Bookmarks option on the Opt drop-down to be greyed out, but in my tests it is active, resulting in:

WHY, you might want to manually insert bookmarks into a programme that hasn't been through detectads and has no prexisting bookmarks

 

[prpr]

I normally expect the Bookmarks option on the Opt drop-down to be greyed out, but in my tests it is active,

Ah, so that's what you meant by "This results in a ghost bookmark visible from the bookmarks option". It wasn't obvious (to me anyway).

The bookmark counter in the hmt file (X'298') is set to zero, so the bookmark address fields should be ignored.

Indeed.

Sorry to be pernickety. One had to look for any unusual feature in the output, just in case!

That's fine. I'm pernickety myself, but knowing exactly what you're doing/how to reproduce/where to look stops the time waste of guessing/searching.
I'm sure you appreciate what I mean!

 

[prpr]

When I previously ran this same crop, the progress showed zero continuously until the crop actually finished. This time it behaved as expected.

I'm afraid I can't explain these at the present time, and don't use crop much myself so have never noticed.

You will see that, in both cases, the second bookmark is 8 seconds before the end of the recording.

When I was looking at the code last night I saw there was an 8 second fudge in certain circumstances. I haven't gone through it any more to try and work out what it's for/why it's there.

I shouldn't have bothered you with these other problems.

It's not a problem. I'd rather people reported them than not. In the former case they may or may not get fixed. In the latter they probably won't unless I come across it myself.

you might want to manually insert bookmarks into a programme that hasn't been through detectads and has no prexisting bookmarks

Yes, so are we all agreed this isn't an actual problem then? It's as expected.