[FlatView] Provide flattened view of recordings

[MontysEvilTwin]

I think this may be just something inherent in having multiple links to the same file in the filesystem. The web interface uses the 'du' command to get disk usage for a folder and presumably this is being a bit too clever here. I will have to look at the busybox source code to work out what it is doing and how to fix it.

If you run this on the command line, what do you see?

humax# cd /media/My\ Video
humax# /mod/bin/busybox/du -h -d 1 .

I'm guessing you get the smaller sizes that you see in the web interface.
Picking a folder which is smaller than it should be, what about:

humax# /mod/bin/busybox/du -h River/*

humax# /mod/bin/busybox/du -h -d 1 . [abbreviated output]
4.0K    ./Lewis                                                                
52.0K   ./The Apprentice             

humax# /mod/bin/busybox/du -h Lewis/*                                            
4.0K    Lewis/Lewis_20151006.hmt                                               
2.1M    Lewis/Lewis_20151006.nts                                               
44.0K   Lewis/Lewis_20151006.thm                                               
1.5G    Lewis/Lewis_20151006.ts                                                
4.0K    Lewis/Lewis_20151013.hmt                                               
5.4M    Lewis/Lewis_20151013.nts                                               
44.0K   Lewis/Lewis_20151013.thm                                               
1.9G    Lewis/Lewis_20151013.ts                                                
     
humax# /mod/bin/busybox/du -h Lewis/                                             
3.4G    Lewis/                                                           

humax# /mod/bin/busybox/du -h The\ Apprentice/*                                  
4.0K    The Apprentice/The Apprentice_20151014_2059.hmt                        
5.5M    The Apprentice/The Apprentice_20151014_2059.nts                        
44.0K   The Apprentice/The Apprentice_20151014_2059.thm                        
2.3G    The Apprentice/The Apprentice_20151014_2059.ts                         
4.0K    The Apprentice/The Apprentice_20151015_2059.hmt                        
5.6M    The Apprentice/The Apprentice_20151015_2059.nts                        
44.0K   The Apprentice/The Apprentice_20151015_2059.thm                        
2.1G    The Apprentice/The Apprentice_20151015_2059.ts                         
                        
humax# /mod/bin/busybox/du -h The\ Apprentice/                                   
4.4G    The Apprentice/

The first command does show incorrect folder sizes for some folders (see above).

 

[MymsMan]

No, it was only ever available on Dropbox and seemed to attract general negativity so I removed it.

I don't quite see how. I presume I'd need some hook into the inotify stuff but I'm struggling to find an example. It seems more like a sibling of recmon than something descended from it, as deletion is obviously nothing to do with recording.
Anyway, inotify doesn't look too hard to wrap in C++ and then use...

Move and delete have been added to recmon to support FlatView.

If you want to be called whenever a recording starts, stops, is moved or deleted you just need to need to put an executable file into /mod/etc/recmon.d - no need to set up your own inotify stuff.
To be called every 20 minutes as part of the auto process you just need an auto.hook file in /mod/webif/plugin/yourpackage, similar for other hook points

See flatview, detectads and others for examples

 

[af123]

I don't quite see how.

Nowadays, putting this into /mod/webif/plugin/tidyfolders/auto.hook would do the job:

proc ::tidyfolders::scan {args} {
         global root

         foreach entry [readdir -nocomplain $root] {
                 set dir "$root/$entry"
                 if {![file isdirectory $dir]} continue
                 system rmdir_if_empty $dir ".series"
         }
}

register postexpirescan ::tidyfolders::scan

 

[xyz321]

I think this may be just something inherent in having multiple links to the same file in the filesystem. The web interface uses the 'du' command to get disk usage for a folder and presumably this is being a bit too clever here. I will have to look at the busybox source code to work out what it is doing and how to fix it.

If you run this on the command line, what do you see?

humax# cd /media/My\ Video
humax# /mod/bin/busybox/du -h -d 1 .

The following seems to fix it:

du -h -l -d 1 .

However, I suspect that the grand total will not now be correct so you may need to add

du -sh .

to get the grand total if required.

 

[af123]

The following seems to fix it:

du -h -l -d 1 .

Yep, that does it, thanks. Fixed in webif 1.2.5-3.

 

[af123]

flatview 1.0.2 now available:

• Use the recording start time for age threshold checks rather than a file timestamp;
  
• New folder flag to force flatview inclusion (regardless of [ prefix or no-flatten flag).

 

[Ezra Pound]

You know very well you were advocating that flatview should replace flatten, leaving the users no choice and as if your particular usage can be the only usage. To claim otherwise is disingenuous.

You said that, in your opinion, there was no point in keeping flatten now we have flatview, I (and others) replied that is not the case, and instead of just shutting up about it you challenged it. That's the mistake - the challenge implies you do not accept that others may disagree with your opinion.

When I object to being taken to task it is because I am not even being allowed my opinion.

#58 said " In my view it also makes Flatten redundant" no more and no less, it was MY view. I was not dictating to anyone and only someone with an ego as big as yours would see it that way, You can't slap down other user's opinions with a simple 'shut-up' or 'keep quite', all users are allowed their opinion without it being swamped by you relentless posting frenzy, to date you have made YOUR opinion know some 15248 times

 

[Black Hole]

Count the likes.

 

[cdmackay]

Without seeming to take sides in the endless battle...

it seems to me that flatview cannot be considered a replacement for flatten, since the latter does something that the former does not, namely to flatten your existing hierarchy, moving everything into a flat dir. If you want that (ghastly) behaviour, then you need flatten.

I am very pleased to see the suggestion that flatten be disabled by default, on installation. That would have helped "my friend"

thanks again...

 

[cdmackay]

the age limit doesn't seem to be working, for me...

Initially, I had the default, 7 days. I then changed it to no limit. That proved unworkable for me: I have a few hundred recordings, and it was taking too long to load the FlatView folder.

I then changed the limit to what it is now: 14 days.

I did that several days ago, but the oldest file in FlatView was recorded at:

Wed Sep 30 07:58:03 2015 BST (Scheduled: Wed Sep 30 07:55:00 2015 BST)

shouldn't the limit have caused that to be removed from FlatView?

there are a few other recordings in there older than 14 days, too.


Edit: I have unwatched only==NO, if that makes a different. and yes, limit==YES

 

[cdmackay]

It does seem that the most recent entry in FlatView does not get a Location field added to Media Details, even many hours after it has been recorded.

It's not just the last entry - I now see several entries that don't have Location. Most do, though.

 

[xyz321]

Yes, first-processed wins.

I think originally flatview also ignored directories starting with '_' in its scan of the tree. This was so that recordings in '_original' directories, with the same name as a new version in the main tree, would be ignored.

 

[af123]

I have just uploaded version 1.0.3 of flatview which will now, by default, ignore folders with a leading _ in their name. This means that it will skip and _original or _duplicate folders which have been created through manual webif OPT+ operations or by the dedup process.
If, however, you use a _ prefix on real folders for any reason, you will now need to explicitly flag these for inclusion by flatview using the OPT+ menu.

 

[Wallace]

This may be a silly question but if I install this package and find that I don't like it for whatever reason, if I remove it, will my folders return to as they were before?

 

[af123]

This may be a silly question but if I install this package and find that I don't like it for whatever reason, if I remove it, will my folders return to as they were before?

Yes. It only creates one new folder at the top level and it will remove it again if you uninstall*.

* - if you have manually put anything into the folder that it creates, then it will leave those alone. I don't recommend doing that though : )

 

[Wallace]

Thanks.

 

[Black Hole]

That's the whole point, and the principle conceptual difference from flatten. flatten actually moves the recordings in the folder structure, whereas flatview creates a new folder with cross-references to the recordings in such a way that they can be accessed from the SUI, but without moving anything at all.

sweeper is flatten's best friend, both of them (can) move recordings around, while flatview keeps watch and provides short-cut access (with or without flatten and sweeper).

 

[MymsMan]

What is the intended effect if you use the Opt+ utilities on a file in the Flatview folder or set one of the Auto-xxx options on the FlatView folder?

Should they be disabled?

It is not obvious to the end user which options involve update in place and would be reflected on the original recording and which are delete & replace and would break the link to the original.

As an experiment I tried running the Opt+ Decrypt option on a file in the FlatView folder, it appeared to work but when I returned to the folder view the file was gone, the original undecrypted file popped back a few minutes later (readded by auto). Not very user friendly!

It would be better not to show options that won't work in the Opt+ menus or to link from the menu to the original file so that any changes are made to the original and are then reflected in the FlatView folder in due course.

Recmon log for the event (n.b. detectads is only logging recmon event calls it isn't doing anything)

294 18/10/2015 19:44 - FV: --> Removing The Go-Between_20150920_2100 (link count)
293 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 119
292 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 317
291 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
290 /mod/webif/plugin/flatview/auto.hook:119: Error: could not read "/media/My Video/ [FlatView]/The Go-Between_20150920_2100.ts": No such file or directory
289 18/10/2015 19:44:35 detectads: -delete {/media/My Video/ [FlatView]/The Go-Between_20150920_2100}
288 18/10/2015 19:44:24 detectads: -move {/media/My Video/ [FlatView]/The Go-Between_20150920_2100}
287 18/10/2015 19:44:24 detectads: -move {/media/My Video/ [FlatView]/The Go-Between_20150920_2100}

 

[4ndy]

I series recorded TFI Friday and watched part of it via the [Flatview] folder on the Humax UI. I have the default settings to show only unwatched and 7 day view.

Today I got an RS email stating that [Flatview]/TFI Friday/TFI Friday.... had failed. On checking tonight I can see that the .hmt file has remained in the [Flatview] folder after the watched flag removed the other files in the set.


Sent from my iPad using Tapatalk

 

[Trev]

On checking tonight I can see that the .hmt file has remained in the [Flatview] folder after the watched flag removed the other files in the set.

I've had a couple of those as well. Interestingly, and perhaps a clue is that they showed up on the CF media list (than normally only displays .ts files)
Similarly, I have some weird 'orphaned' comments in my recmon log from tonight.

668 19/10/2015 20:03 - FV: +++ Adding The Wonder of Bees with Martha Kearney 19-10-15 1929
667 19/10/2015 20:03 - FV: +++ Adding Coronation Street -Crop 19-10-15 1931
666 19/10/2015 20:03 - FV: --> Removing Coronation Street_20151019_1931-crop (orphaned)
665 19/10/2015 20:03 - FV: --> Removing Coronation Street_20151019_1931-dec (orphaned)
664 19/10/2015 20:03 - FV: --> Removing Coronation Street_20151019_1931 (orphaned)
663 Syntax: epgpatch <recording>
662 19/10/2015 20:03:40 detectads: -delete {/media/My Video/ [FlatView]/Coronation Street_20151019_1931-crop}
661 Syntax: epgpatch <recording>
660 Syntax: epgpatch <recording>
659 19/10/2015 20:03:39 detectads: -delete {/media/My Video/ [FlatView]/Coronation Street_20151019_1931-dec}
658 19/10/2015 20:03:39 detectads: -delete {/media/My Video/ [FlatView]/Coronation Street_20151019_1931}
657 Syntax: epgpatch <recording>
656 Syntax: epgpatch <recording>
655 19/10/2015 20:03:04 detectads: -move {/media/My Video/Coronation Street_20151019_1931}
654 19/10/2015 20:03:04 detectads: -move {/media/My Video/[Deleted Items]/Coronation Street_20151019_1931}
653 19/10/2015 20:03:03 detectads: -delete {/media/My Video/Coronation Street_20151019_1931-dec}
652 Syntax: epgpatch <recording>
651 Syntax: epgpatch <recording>
650 Syntax: epgpatch <recording>
649 19/10/2015 20:03:02 detectads: -move {/media/My Video/Coronation Street -Crop 19-10-15 1931}
648 19/10/2015 20:03:02 detectads: -move {/media/My Video/Coronation Street_20151019_1931-crop}
647 19/10/2015 20:02 - FV: +++ Adding Coronation Street_20151019_1931-dec
646 Syntax: epgpatch <recording>
645 19/10/2015 20:02:57 detectads: -stop {/media/My Video/Coronation Street_20151019_1931-dec}
644 19/10/2015 20:02 - FV: +++ Adding Coronation Street_20151019_1931
643 19/10/2015 20:02:46 detectads: -stop {/media/My Video/Coronation Street_20151019_1931}
642 Syntax: epgpatch <recording>
641 Syntax: epgpatch <recording>
640 19/10/2015 20:02:35 detectads: -delete {/media/My Video/ [FlatView]/The Wonder of Bees with Martha Kearney_20151019_1929}
639 19/10/2015 20:02 - FV: --> Removing The Wonder of Bees with Martha Kearney_20151019_1929 (orphaned)
638 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 119
637 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 317
636 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
635 /mod/webif/plugin/flatview/auto.hook:119: Error: could not read "/media/My Video/ [FlatView]/The Wonder of Bees with Martha Kearney_20151019_1929.ts": No such file or directory
634 Syntax: epgpatch <recording>
633 19/10/2015 20:01:43 detectads: -move {/media/My Video/The Wonder of Bees with Martha Kearney 19-10-15 1929}
632 19/10/2015 20:01:42 detectads: -move {/media/My Video/The Wonder of Bees with Martha Kearney_20151019_1929}
631 Syntax: epgpatch <recording>
630 Syntax: epgpatch <recording>
629 Syntax: epgpatch <recording>
628 19/10/2015 20:01:34 detectads: -move {/media/My Video/The Wonder of Bees with Martha Kearney_20151019_1929}
627 19/10/2015 20:01:33 detectads: -move {/media/My Video/The Wonder of Bees with Martha Kearney_20151019_1929}
626 19/10/2015 19:59 - FV: +++ Adding The Wonder of Bees with Martha Kearney_20151019_1929
625 Syntax: epgpatch <recording>
624 19/10/2015 19:59:53 detectads: -stop {/media/My Video/The Wonder of Bees with Martha Kearney_20151019_1929}
623 19/10/2015 19:32 - FV: +++ Adding The One Show 19-10-15 1857
622 Syntax: epgpatch <recording>
621 19/10/2015 19:31:34 detectads: -delete {/media/My Video/ [FlatView]/The One Show_20151019_1857}
620 19/10/2015 19:31 - FV: --> Removing The One Show_20151019_1857 (orphaned)
619 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 119
618 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 317
617 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
616 /mod/webif/plugin/flatview/auto.hook:119: Error: could not read "/media/My Video/ [FlatView]/The One Show_20151019_1857.ts": No such file or directory
615 19/10/2015 19:31:02 detectads: -start {/media/My Video/Coronation Street_20151019_1931}
614 Syntax: epgpatch <recording>
613 Syntax: epgpatch <recording>
612 19/10/2015 19:30:39 detectads: -move {/media/My Video/The One Show 19-10-15 1857}
611 Syntax: epgpatch <recording>
610 19/10/2015 19:30:39 detectads: -move {/media/My Video/The One Show_20151019_1857}
609 Syntax: epgpatch <recording>
608 Syntax: epgpatch <recording>
607 19/10/2015 19:30:33 detectads: -move {/media/My Video/The One Show_20151019_1857}
606 19/10/2015 19:30:33 detectads: -move {/media/My Video/The One Show_20151019_1857}
605 Syntax: epgpatch <recording>
604 19/10/2015 19:29:07 detectads: -start {/media/My Video/The Wonder of Bees with Martha Kearney_20151019_1929}
603 19/10/2015 19:28 - FV: +++ Adding The One Show_20151019_1857
602 19/10/2015 19:28:30 detectads: -stop {/media/My Video/The One Show_20151019_1857}
601 Syntax: epgpatch <recording>
600 19/10/2015 18:57:48 detectads: -start {/media/My Video/The One Show_20151019_1857}
599 Syntax: epgpatch <recording>
598 19/10/2015 16:16 - FV: --> Removing Sunday Night at the Palladium_20151011_2100 (orphaned)
597 19/10/2015 16:16:21 detectads: -delete {/media/My Video/ [FlatView]/Sunday Night at the Palladium_20151011_2100}
596 Syntax: epgpatch <recording>
595 Syntax: epgpatch <recording>
594 19/10/2015 16:15:28 detectads: -move {/media/My Video/Sunday Night at the Palladium/Sunday Night at the Palladium_20151011_2100-crop}
593 Syntax: epgpatch <recording>
592 19/10/2015 16:15:20 detectads: -move {/media/My Video/Sunday Night at the Palladium/Sunday Night at the Palladium_20151011_2100}
591 Syntax: epgpatch <recording>
590 19/10/2015 02:07:32 detectads: -delete {/media/My Video/[Deleted Items]/Strictly Come Dancing 10-10-15 1819}