[nicesplice] magic folders and 43000 empty files

[hairy_mutley]

My HDR-Fox-T2 has been a bit sluggish for the last few days. Media access very slow (several seconds to respond) and we have been seeing frequent picture breakups during replay.
I thought that I would take a look for orphanned sidecar files in case that was the source of the problem, so, from my PC, I did a scan of all files. There were a couple of stray files without .ts files in the deleted autodycrypt, but the main issue seemed to be 43000 zero length files. These files are not visible through SUI, web-if or rs.

It appears that on new years day, I started a magic folders [cut] on an HD file; I added a pair of bookmarks to the recording and moved it to [edit]/[cut]. The next day, I successfully performed a cut on another file and noticed that the first file cut hadn't completed, but didn't get around to investigating further.
It turns out that since starting the first cut, it has been generating a dozen pairs of empty files every second!
(filenames reflec

I have moved the recording out of [cut] to stop it, and removed the empty files, but what went wrong?

 

[jack616]

nicesplice does this from time to time.
If you find your humax slowing down - especially when trying to open [edit] this is the thing to look for.
The most I've had was over 240,000 - it took more than 40 minutes for filezila just to delete them.

I have no idea of any common factor that causes this. I just accept that it happens.

 

[hairy_mutley]

240,000 I thought mine was bad enough.
I initially tried a command line "rm *", but just got something like arguement list too long. Even more selective wildcards produced several instances of "bad address". In the end I resorted to deleting through the network share using my normal PC file tool (ZTree). Like you, it took a while!

 

[hairy_mutley]

This seems a rather nasty gotcha, any thoughts on one or more of the following...
any suggestions on how we can diagnose the source of the problem?
- not sure if anything helpful in existing logs
- not sure if it is repeatable, I can try putting the offending recording back into [cut]
could/should magic folders detect and prevent the suicidal tendencies?
could/should [web-if] and/or [rs] alert the user?
add something to the wiki to say that if a magic folders action does not complete in a reasonable time, then the user need to remove the offending file and go in under the bonnet to remove the remaining trash.

 

[jack616]

I guess a cron job might look for zero length files and flag a problem but whether its worth the effort I don't know.

 

[Black Hole]

Or just have an automatic process to delete them and forget about it.

 

[hairy_mutley]

I guess a cron job might look for zero length files and flag a problem but whether its worth the effort I don't know.

OK low probability of occurance, but the impact is potentially high.
My recordings were exhibiting frequent periodic picture and sound breakup/dropout resulting in significant complaints from the management.
Accessing the menus wqas becoming impractical.
I think that this justifies a little effort.

Or just have an automatic process to delete them and forget about it.

I got the impression that the dropouts and menu delays were getting worse... running nicesplice a dozen times a second is rather undesirable, but this suggests that the presence of the mounting number of files may be a factor, so deleting them might well help.

It would probably be better if magic folders could detect its own failed attempts and stop trying to perform the edit (as with successful cuts, the files are named according to the original filename and the date-time of the edit attempt)
Obviously it would be better to at least look for the source of the original problem.

 

[Trev]

OK low probability of occurance, but the impact is potentially high.

Sounds like a risk analysis. You know, the one where the H&S stasi take no notice of the 'likelihood of occurrence' and 'impact' factors and it doesn't matter how low they are, they still have to cook up some bloody stupid rule to cover it rather than applying a bit of common sense.

 

[prpr]

But risk analysis DOES take into account likelihood and impact factors. Low and high respectively either way round is not that much of a concern, although well worth thinking about, but high and high definitely mandates preventive measures. Nobody cares about low and low.
Better have a look on your shoulder for potatoes cooked in fat...

 

[hairy_mutley]

Sounds like a risk analysis. You know, the one where the H&S stasi take no notice of the 'likelihood of occurrence' and 'impact' factors and it doesn't matter how low they are, they still have to cook up some bloody stupid rule to cover it rather than applying a bit of common sense.

I am NOT QA, but unfortunately I have to dabble in risk analysis every now and then for product deliveries. You know how it is, just enough knowledge to be dangerous. Unfortunately, in this case it seemed an appropriate analysis. (sorry, must have drank too much coffee this morning)

In this case, the statement equates to a badly chewed ear from the management; as we all know, that overrides all other considerations.

 

[Ezra Pound]

I guess a cron job might look for zero length files and flag a problem but whether its worth the effort I don't know.

BH : Or just have an automatic process to delete them and forget about it.

I'm not sure that is the answer, If Magic Folders is generating dozens of zero length of files per second and something else is deleting them, there is still a lot of CPU activity that could carry on undetected indefinitely, there would also need to be some checks on what zero length files were being deleted as they do exist in the Custom Firmware folders for other perfectly legitimate reasons.

 

[jack616]

In this case, the statement equates to a badly chewed ear from the management; as we all know, that overrides all other considerations.

OK well if you're not using the box just for private entertainment purposes you're in a whole different ball game.
I'm sure someone would be happy to quote for a fix.
On the other hand I find it curious that business use requires the use of nicesplice.
And on my third hand - I don't think I'd trust one of these for business use. Just not reliable enough.
I used them for charitable purposes for a while but had to trash them and get sky boxes.
(for both box and broadcast signal issues)

 

[Trev]

But risk analysis DOES take into account likelihood and impact factors. {snip} Nobody cares about low and low.

I know that, but yes they do when the Elf n safty people are about. It doesn't matter how low/low they are, you MUST still do something about it. Crazy.

In this case, the statement equates to a badly chewed ear from the management; as we all know, that overrides all other considerations.

Even a mild chewing trumps ANYTHING else.

 

[hairy_mutley]

OK well if you're not using the box just for private entertainment purposes you're in a whole different ball game.

You mis-understand, we are simply talking about home/work parallels.
At work, I have been in engineering long enough to be thick skinned about ear chewing from management.
No, this is at home, this is much more serious; management = wife and chewing is not to be ignored.

 

[Black Hole]

OK well if you're not using the box just for private entertainment purposes you're in a whole different ball game.

See Glossary (click), entry Netiquette.. SWMBO.

 

[prpr]

if (chewing == ignored)
{
bank_balance = 0;
house_access = 0;
divorce = 1;
}

 

[Trev]

or in BASIC
IF SWMBO = ignored THEN death.

 

[jack616]

You mis-understand, we are simply talking about home/work parallels.
At work, I have been in engineering long enough to be thick skinned about ear chewing from management.
No, this is at home, this is much more serious; management = wife and chewing is not to be ignored.

Aha I see - my mistake - apple-ogies