[FlatView] Provide flattened view of recordings

af123 · Oct 12, 2015

I have just published a new package called FlatView as an alternative to flatten that might prove useful to some people. It provides both a flattened view and a hierarchical view at the same time by creating a new top-level directory (called " [FlatView]" by default). Recordings appear in both without using any additional disk space.

The idea for this package is not mine; xyz321 sent it to me, along with some working code, in 2013 so any credit should be directed that way! I took the code about 6 months ago and converted it into this package, integrating it with the background automatic processing etc. and I have finally had time to finish it. Thanks to xyz321 and Black Hole for testing this version over the past couple of weeks.

Some notes:

Recordings deleted from the flattened view area are automatically removed from the original location in the hierarchy and the original folder will be removed if it is empty;
Recordings deleted from the hierarchical view will be automatically removed from the flattened area too;
Similarly, if a recording is renamed in either area, then the change will be reflected in the other (there may be a small delay);
Since the recordings are linked together, changes in flags such as 'Unwatched' and the resume point are synchronised across the two locations;
If you move a recording out of the flattened view area, the system will think that it has been deleted and will remove the corresponding recording from the hierarchical view area (you'll still have the copy that you moved elsewhere);
If you move a recording into the flattened view area, it will be left untouched;
If you have undelete installed, then deleting a recording from the flattened view area will result in two entries in the dustbin - the one that you deleted and another copy from the hierarchical location. However, they are still linked and so both use the same disk space.

cdmackay · Oct 13, 2015

oh this is superb, thank you very much indeed

Working perfectly, for me.

one question please: I see you have it age-limited by default. Are there any performance implications to not limiting?

I would like to have a flat view of everything, e.g. for searching (although TV Diary also offers that, albeit slower), but not if it causes any perf issues...

thanks again, brilliant!

cdmackay · Oct 13, 2015

Also, would you please consider adding a "Location" field to the Media Details box?

Previously, that would have been redundant, as you had to be in the directory in order to bring up the Media Details - but now you can bring it up from [FlatView], and it would be useful to know where the recording is, in the hierarchical view.

thanks much...

xyz321 · Oct 13, 2015

cdmackay said:
one question please: I see you have it age-limited by default. Are there any performance implications to not limiting?

I would like to have a flat view of everything, e.g. for searching (although TV Diary also offers that, albeit slower), but not if it causes any perf issues...

I do run it with no age-limiting. The only performance issue I have noticed is that the Humax UI can take a second or two to display the FlatView directory when there are lots of entries.

Black Hole · Oct 13, 2015

It's worth pointing out that the default folder name (which can be changed if desired) has a leading space so that it is listed at the top of the media screen (and therefore quick to access).

flatview conforms to the convention of leaving "["-prefixed folders alone. In theory there is not much need to use flatview and flatten together, and the primary use of "["-prefixed folders is to protect them from flatten, but that's not the only use so, in this case, conforming to the convention may prove inconvenient and it may be desirable to find a way to deselect this default, or to flag such folders as "flatviewable" (or revise ones policy - in my case I have flattened or non-series recording swept into [Unclassified], all news swept into [NEWS]...).

This may already be possible, I have not checked: the flatview package also respects the "noflatten" flag, but I am not sure whether the "[" over-rides the lack of a .noflatten (I expect it to).

I have spotted a possible problematic interaction between flatview and detectads, not pinned down exactly but it seems that flatview indexes a recording out of sync with detectads processing it, which results in the flatview version being unplayable. Any further reported observations will be useful to work out what the problem actually is (if there is one at all).

af123 · Oct 13, 2015

Black Hole said:
It's worth pointing out that the default folder name (which can be changed if desired) has a leading space so that it is listed at the top of the media screen (and therefore quick to access).

Indeed - and this is the reason that over the past few months I have been making packages ignore leading space when checking if a folder has a 'special' name.

This may already be possible, I have not checked: the flatview package also respects the "noflatten" flag, but I am not sure whether the "[" over-rides the lack of a .noflatten (I expect it to).

It does. I see your point - it's almost as if a 'forceflatten' flag is needed to override the [ prefix on the folder name.

Black Hole · Oct 13, 2015

af123 said:
Indeed - and this is the reason that over the past few months I have been making packages ignore leading space when checking if a folder has a 'special' name.

Aha! So " [" is treated the same as "["?

Black Hole · Oct 13, 2015

af123 said:
If you move a recording out of the flattened view area, the system will think that it has been deleted and will remove the corresponding recording from the hierarchical view area (you'll still have the copy that you moved elsewhere)

I imagine this means there is no actual deletion and undelete is not invoked?

At what point is the flatview listing frozen - is it invoked on the end-of-recording event, or periodically refreshed on scan? To create the initial flattened view on installation there has to be a scanning process, and if there is a periodic scan then the moved entry will be found again .

af123 · Oct 13, 2015

cdmackay said:
Also, would you please consider adding a "Location" field to the Media Details box?

Previously, that would have been redundant, as you had to be in the directory in order to bring up the Media Details - but now you can bring it up from [FlatView], and it would be useful to know where the recording is, in the hierarchical view.

thanks much...

Good idea, it will be in the next version.

af123 · Oct 13, 2015

Black Hole said:
Aha! So " [" is treated the same as "["?

Yes - unless I've missed a package somewhere, all leading white-space is now ignored when checking if the directory is special (as indicated by the [)

af123 · Oct 13, 2015

Black Hole said:
I imagine this means there is no actual deletion and undelete is not invoked?

Undelete would be invoked on the copy in the original hierarchical view and it would end up in the bin. It would still be linked to the copy that you moved elsewhere though so no disk space lost.

MymsMan · Oct 13, 2015

Flatview is not working for me - the directory is empty.

I don't currently have anything in series folders since I have only just uninstalled Flatten but I do have a number of recent entries in the root My Videos folder which I would expect to see (unless I have misunderstood the intent).

I can't see anything that might be relevant in any of the logs

Black Hole · Oct 13, 2015

How long have you had it installed? Also, do you have any recordings younger than the default threshold?

Black Hole · Oct 13, 2015

If one is using the age filter, it would be quite slick to also rename the flatview folder to " [Recent Recordings]" - but it would also be handy to have a second invocation of flatview to handle another folder with the negative of the age condition, and call it " [Older Recordings]".

af123 · Oct 13, 2015

MymsMan said:
Flatview is not working for me - the directory is empty.

I don't currently have anything in series folders since I have only just uninstalled Flatten but I do have a number of recent entries in the root My Videos folder which I would expect to see (unless I have misunderstood the intent).

I can't see anything that might be relevant in any of the logs

You can run /mod/webif/plugin/flatview/run from the command line to see what it's doing. With the default settings it only includes recent recordings which have not been watched but that can be changed.

MymsMan · Oct 13, 2015

af123 said:
Good idea, it will be in the next version.

Whilst looking at the "Media Details" box could you consider splitting Bookmarks out as separate line rather than being tacked on to the end of the Flags line.

MymsMan · Oct 13, 2015

af123 said:
You can run /mod/webif/plugin/flatview/run from the command line to see what it's doing. With the default settings it only includes recent recordings which have not been watched but that can be changed.

Code:

Humax# /mod/webif/plugin/flatview/run
13/10/2015 13:33 - ORPHANS: []
Humax#

I installed Flatview around 10am and have rebooted the Humax since then.
I have a number of unwatched programs from yesterday and today but have tried the unlimited options as well

cdmackay · Oct 13, 2015

One note, that will no doubt be obvious: if you have scripts, etc, to backup your recordings to a USB device [or UPD, for BH

], remember to arrange for it to skip the FlatView dir, unless you wanted everything in it backed up twice.

e.g. if using rsync, add:

--exclude '/My Video/ \[FlatView\]/' \

(and remember the embedded leading space before the (escaped) open square bracket).

Apols if this is obvioius...

MymsMan · Oct 13, 2015

An extract from my recmon log

Code:

799 13/10/2015 19:05:19 detectads: -delete {/media/My Video/ [FlatView]/South Today_20151013_1828}
798 13/10/2015 19:00:11 detectads: -start {/media/My Video/New_ The Gadget Show/New_ The Gadget Show_20151013_1900}
797 13/10/2015 18:58:08 detectads: -stop {/media/My Video/South Today/South Today_20151013_1828}
796 13/10/2015 18:58 - FV: +++ Adding South Today_20151013_1828
795 13/10/2015 18:45:13 detectads: -delete {/media/My Video/ [FlatView]/Eggheads_20151013_1759}
794 13/10/2015 18:29 - FV: +++ Adding Eggheads_20151013_1759
793 13/10/2015 18:29:31 detectads: -stop {/media/My Video/Eggheads/Eggheads_20151013_1759}
792 13/10/2015 18:28:59 detectads: -start {/media/My Video/South Today/South Today_20151013_1828}
791 13/10/2015 18:00 - FV: --> Removing unwanted: Pointless_20151013_1715
790 13/10/2015 18:00:17 detectads: -delete {/media/My Video/ [FlatView]/Pointless_20151013_1715}
789 13/10/2015 18:00:07 detectads: -delete {/media/My Video/ [FlatView]/The Chase_20151013_1659}
788 13/10/2015 18:00 - FV: +++ Adding Pointless_20151013_1715
787 13/10/2015 18:00:06 detectads: -delete {/media/My Video/ [FlatView]/The Chase_20151013_1659-dec}
786 13/10/2015 18:00:06 detectads: -stop {/media/My Video/Pointless/Pointless_20151013_1715}
785 13/10/2015 18:00 - FV: --> Removing The Chase_20151013_1659 (inode mismatch)
784 13/10/2015 18:00 - FV: --> Removing The Chase_20151013_1659-dec (orphaned)
783 at file "/mod/webif/lib/fileops", line 80
782 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 123
781 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 303
780 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
779 /mod/webif/lib/fileops:80: Error: Cannot read /media/My Video/ [FlatView]/The Chase_20151013_1659-dec.info
778 13/10/2015 17:59:53 detectads: -delete {/media/My Video/The Chase/The Chase_20151013_1659-dec}
777 13/10/2015 17:59:53 detectads: -move {/media/My Video/The Chase/The Chase_20151013_1659-crop}
776 13/10/2015 17:59:52 detectads: -move {/media/My Video/The Chase/The Chase_20151013_1659}
775 13/10/2015 17:59:52 detectads: -stop {/media/My Video/The Chase/The Chase_20151013_1659-dec}
774 13/10/2015 17:59 - FV: +++ Adding The Chase_20151013_1659-dec
773 13/10/2015 17:59:50 detectads: -move {/media/My Video/The Chase/The Chase_20151013_1659}
772 13/10/2015 17:59 - FV: +++ Adding The Chase_20151013_1659
771 13/10/2015 17:59:45 detectads: -stop {/media/My Video/The Chase/The Chase_20151013_1659}
770 13/10/2015 17:59:45 detectads: -start {/media/My Video/Eggheads/Eggheads_20151013_1759}
769 13/10/2015 17:15:07 detectads: -start {/media/My Video/Pointless/Pointless_20151013_1715}
768 13/10/2015 16:59:27 detectads: -start {/media/My Video/The Chase/The Chase_20151013_1659}
767 13/10/2015 16:00:04 detectads: -delete {/media/My Video/ [FlatView]/Fifteen to One_20151013_1458-dec}
766 13/10/2015 16:00 - FV: --> Removing Fifteen to One_20151013_1458-dec (orphaned)
765 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 110
764 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 303
763 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
762 /mod/webif/plugin/flatview/auto.hook:110: Error: could not read "/media/My Video/ [FlatView]/Fifteen to One_20151013_1458-dec.ts": No such file or directory
761 13/10/2015 15:59:51 detectads: -delete {/media/My Video/ [FlatView]/Fifteen to One_20151013_1458}
760 13/10/2015 15:59 - FV: --> Removing Fifteen to One_20151013_1458 (inode mismatch)
759 at file "/mod/webif/lib/fileops", line 80
758 in procedure 'file' called at file "/mod/webif/plugin/flatview/auto.hook", line 123
757 in procedure '::flatview::build_flatlist' called at file "/mod/webif/plugin/flatview/auto.hook", line 303
756 in procedure '::flatview::scan' called at file "/mod/etc/recmon.d/flatview", line 38
755 /mod/webif/lib/fileops:80: Error: Cannot read /media/My Video/ [FlatView]/Fifteen to One_20151013_1458.info
754 13/10/2015 15:59:43 detectads: -delete {/media/My Video/Fifteen To One/Fifteen to One_20151013_1458-dec}
753 13/10/2015 15:59:42 detectads: -stop {/media/My Video/Fifteen To One/Fifteen to One_20151013_1458-dec}
752 13/10/2015 15:59 - FV: +++ Adding Fifteen to One_20151013_1458-dec
751 13/10/2015 15:59:42 detectads: -move {/media/My Video/Fifteen To One/Fifteen to One_20151013_1458}
750 13/10/2015 15:59:41 detectads: -move {/media/My Video/Fifteen To One/Fifteen to One_20151013_1458-crop}
749 13/10/2015 15:59:41 detectads: -move {/media/My Video/Fifteen To One/Fifteen to One_20151013_1458}
748 13/10/2015 15:59:40 detectads: -move {/media/My Video/Fifteen To One/Fifteen to One_20151013_1458-orig}
747 13/10/2015 15:59 - FV: +++ Adding Fifteen to One_20151013_1458
746 13/10/2015 15:59:05 detectads: -stop {/media/My Video/Fifteen To One/Fifteen to One_20151013_1458}

It would appear that flatview is getting upset by detectads deleting and renaming recordings at the end of recording (I am testing a new version which renames the output to be the original file name rather than keeping the -crop/-dec file names) - we need to work out the best way to prevent conflicts between the packages.

My Flatview folder is currently empty despite the FV: +++ Adding ... log entries. I did see Eggheads in the folder briefly shortly after it finished recording (no detectads processing) but it then disappeared and the log shows recmon recorded a -deleted call 15 mins later at 18:45

The auto log shows FV removed it from the FV folder as an Orphan (whatever that means)

Code:

3988 13/10/2015 18:45:13 - Completed postexpirescan plugin ::flatview::scan
3987 13/10/2015 18:45:13 - FV: --> Removing unwanted: Eggheads_20151013_1759
3986 13/10/2015 18:45:13 - ORPHANS: [Eggheads_20151013_1759]
3985 13/10/2015 18:45:12 - Running postexpirescan plugin ::flatview::scan (priority 50) [processed_files]

The recording is still in /media /My Video /Eggheads and unwatched.

Ezra Pound · Oct 13, 2015

I have to say I only loaded Flatview in order to document it for the WiKi, as I though I had no use for it, however having loaded it I find it a VERY useful new feature and I really like the fact that I can see all recently recorded files in one place.
Question : Having watched a recording by selecting it from the [Flatview] folder, I then deleted the file from [flatview], (I have Undelete installed), however when I navigated to the folder where the recording resides, the recording was still there. I presumed that there was a single file with two ways to get to it i.e. via [flatview] and the folder hierarchy, but if this is the case shouldn't both the [flatview] and the folder versions both be deleted at the same time or in my case both versions moved to the [Deleted] folder?
EDIT
This has now sorted itself out, there appears to be a period of time where the file is deleted from [Flatview] but is still present in the folder hierarchy, however the folder version also finds it's way to [Deleted] after this delay

Another file deleted from [flatview] took over 10 Mins. before it's twin folder was moved to [Deleted]

[FlatView] Provide flattened view of recordings

Administrator

Active Member

Active Member

Well-Known Member

May contain traces of nut

Administrator

May contain traces of nut

May contain traces of nut

Administrator

Administrator

Administrator

Ad detector

May contain traces of nut

May contain traces of nut

Administrator

Ad detector

Ad detector

Active Member

Ad detector

Well-Known Member