Auto-Expire only deleting .hmt files

[MymsMan]

For a while now episodes of Eggheads have been disappearing from my Flatview folder shortly after recording even though they are still present in my Eggheads folder.
If have auto-expiry set for the Eggheads folder
I finally got around to start looking at the problem and it appears the Expire code is deleting the .hmt files but leaving .ts, .nts and .thm files in place,
I have not looked at the code yet to see what is really happening. My suspicion is with the "Never delete unwatched" setting

humax /mnt/hd2/My Video/Eggheads # ls
total 8767172
drwxr-xr-x  2 root root      4096 May 24 19:00 .
drwxr-xr-x 25 root root     32768 May 24 17:59 ..
-rw-rw-rw-  1 root root         8 May 12 20:19 .autoexpire
-rw-r--r--  1 root root       276 May 24 18:29 .series
-rw-rw-rw-  1 root root   1121824 Apr 20 18:54 Celebrity Eggheads_20230420_1828.nts
-rw-rw-rw-  1 root root     43681 Apr 20 18:34 Celebrity Eggheads_20230420_1828.thm
-rw-rw-rw-  1 root root 292075008 Apr 20 18:54 Celebrity Eggheads_20230420_1828.ts
-rw-rw-rw-  1 root root   1198112 Apr 25 18:59 Celebrity Eggheads_20230425_1828.nts
-rw-rw-rw-  1 root root     43681 Apr 25 18:34 Celebrity Eggheads_20230425_1828.thm
-rw-rw-rw-  1 root root 315663744 Apr 25 18:59 Celebrity Eggheads_20230425_1828.ts
-rw-rw-rw-  1 root root   1257184 Apr  5 19:00 Eggheads_20230405_1828.nts
-rw-rw-rw-  1 root root     43681 Apr  5 18:33 Eggheads_20230405_1828.thm
-rw-rw-rw-  1 root root 324387072 Apr  5 19:00 Eggheads_20230405_1828.ts
-rw-rw-rw-  1 root root   1284064 Apr  7 18:59 Eggheads_20230407_1828.nts
-rw-rw-rw-  1 root root     43681 Apr  7 18:33 Eggheads_20230407_1828.thm
-rw-rw-rw-  1 root root 332511168 Apr  7 18:59 Eggheads_20230407_1828.ts
-rw-rw-rw-  1 root root   1102336 May  1 18:29 Eggheads_20230501_1802.nts
-rw-rw-rw-  1 root root     43681 May  1 18:06 Eggheads_20230501_1802.thm
-rw-rw-rw-  1 root root 286688256 May  1 18:29 Eggheads_20230501_1802.ts
-rw-rw-rw-  1 root root   1098048 May  2 18:29 Eggheads_20230502_1801.nts
-rw-rw-rw-  1 root root     43681 May  2 18:05 Eggheads_20230502_1801.thm
-rw-rw-rw-  1 root root 280339584 May  2 18:29 Eggheads_20230502_1801.ts
-rw-rw-rw-  1 root root   1116480 May  3 18:29 Eggheads_20230503_1801.nts
-rw-rw-rw-  1 root root     43681 May  3 18:05 Eggheads_20230503_1801.thm
-rw-rw-rw-  1 root root 286056576 May  3 18:29 Eggheads_20230503_1801.ts
-rw-rw-rw-  1 root root   1179872 May  5 18:29 Eggheads_20230505_1801.nts
-rw-rw-rw-  1 root root     43681 May  5 18:05 Eggheads_20230505_1801.thm
-rw-rw-rw-  1 root root 301163904 May  5 18:29 Eggheads_20230505_1801.ts
-rw-rw-rw-  1 root root   1105888 May  8 18:29 Eggheads_20230508_1801.nts
-rw-rw-rw-  1 root root     43681 May  8 18:05 Eggheads_20230508_1801.thm
-rw-rw-rw-  1 root root 282646656 May  8 18:29 Eggheads_20230508_1801.ts
-rw-rw-rw-  1 root root   1120224 May  9 18:14 Eggheads_20230508_1829.nts
-rw-rw-rw-  1 root root     43681 May  9 18:14 Eggheads_20230508_1829.thm
-rw-rw-rw-  1 root root 283173696 May  9 18:14 Eggheads_20230508_1829.ts
-rw-rw-rw-  1 root root   1100960 May  9 18:29 Eggheads_20230509_1801.nts
-rw-rw-rw-  1 root root     43681 May  9 18:06 Eggheads_20230509_1801.thm
-rw-rw-rw-  1 root root 282827328 May  9 18:29 Eggheads_20230509_1801.ts
-rw-rw-rw-  1 root root   1267360 May 10 18:18 Eggheads_20230509_1828.nts
-rw-rw-rw-  1 root root     43681 May 10 18:18 Eggheads_20230509_1828.thm
-rw-rw-rw-  1 root root 332032512 May 10 18:18 Eggheads_20230509_1828.ts
-rw-rw-rw-  1 root root   1098336 May 10 18:29 Eggheads_20230510_1801.nts
-rw-rw-rw-  1 root root     43681 May 10 18:05 Eggheads_20230510_1801.thm
-rw-rw-rw-  1 root root 275451648 May 10 18:29 Eggheads_20230510_1801.ts
-rw-rw-rw-  1 root root   1404576 May 11 18:08 Eggheads_20230510_1828.nts
-rw-rw-rw-  1 root root     43681 May 11 18:08 Eggheads_20230510_1828.thm
-rw-rw-rw-  1 root root 365161344 May 11 18:08 Eggheads_20230510_1828.ts
-rw-rw-rw-  1 root root   1172704 May 11 18:29 Eggheads_20230511_1801.nts
-rw-rw-rw-  1 root root     43681 May 11 18:05 Eggheads_20230511_1801.thm
-rw-rw-rw-  1 root root 301307520 May 11 18:29 Eggheads_20230511_1801.ts
-rw-rw-rw-  1 root root   1187200 May 12 18:14 Eggheads_20230511_1828.nts
-rw-rw-rw-  1 root root     43681 May 12 18:14 Eggheads_20230511_1828.thm
-rw-rw-rw-  1 root root 306095808 May 12 18:14 Eggheads_20230511_1828.ts
-rw-rw-rw-  1 root root   1103264 May 12 18:29 Eggheads_20230512_1801.nts
-rw-rw-rw-  1 root root   1318304 May 12 18:59 Eggheads_20230512_1829.nts
-rw-rw-rw-  1 root root   1107520 May 15 18:29 Eggheads_20230515_1801.nts
-rw-rw-rw-  1 root root     43681 May 15 18:05 Eggheads_20230515_1801.thm
-rw-rw-rw-  1 root root 272700096 May 15 18:29 Eggheads_20230515_1801.ts
-rw-rw-rw-  1 root root   1108288 May 15 18:59 Eggheads_20230515_1829.nts
-rw-rw-rw-  1 root root     43681 May 15 18:33 Eggheads_20230515_1829.thm
-rw-rw-rw-  1 root root 268109760 May 15 18:59 Eggheads_20230515_1829.ts
-rw-rw-rw-  1 root root   1098528 May 16 18:29 Eggheads_20230516_1801.nts
-rw-rw-rw-  1 root root     43681 May 16 18:06 Eggheads_20230516_1801.thm
-rw-rw-rw-  1 root root 278143296 May 16 18:29 Eggheads_20230516_1801.ts
-rw-rw-rw-  1 root root   1304832 May 16 18:59 Eggheads_20230516_1828.nts
-rw-rw-rw-  1 root root     43681 May 16 18:33 Eggheads_20230516_1828.thm
-rw-rw-rw-  1 root root 344725824 May 16 18:59 Eggheads_20230516_1828.ts
-rw-rw-rw-  1 root root   1101568 May 17 18:30 Eggheads_20230517_1801.nts
-rw-rw-rw-  1 root root     43681 May 17 18:05 Eggheads_20230517_1801.thm
-rw-rw-rw-  1 root root 283051968 May 17 18:30 Eggheads_20230517_1801.ts
-rw-rw-rw-  1 root root   1072192 May 18 18:29 Eggheads_20230518_1801.nts
-rw-rw-rw-  1 root root     43681 May 18 18:06 Eggheads_20230518_1801.thm
-rw-rw-rw-  1 root root 263658432 May 18 18:29 Eggheads_20230518_1801.ts
-rw-rw-rw-  1 root root   1212352 May 18 19:00 Eggheads_20230518_1829.nts
-rw-rw-rw-  1 root root     43681 May 18 18:33 Eggheads_20230518_1829.thm
-rw-rw-rw-  1 root root 307342080 May 18 19:00 Eggheads_20230518_1829.ts
-rw-rw-rw-  1 root root   1096928 May 19 18:29 Eggheads_20230519_1801.nts
-rw-rw-rw-  1 root root     43681 May 19 18:05 Eggheads_20230519_1801.thm
-rw-rw-rw-  1 root root 282835968 May 19 18:29 Eggheads_20230519_1801.ts
-rw-rw-rw-  1 root root   1114112 May 22 18:29 Eggheads_20230522_1802.nts
-rw-rw-rw-  1 root root     43681 May 22 18:05 Eggheads_20230522_1802.thm
-rw-rw-rw-  1 root root 284460480 May 22 18:29 Eggheads_20230522_1802.ts
-rw-rw-rw-  1 root root   1148160 May 22 19:00 Eggheads_20230522_1828.nts
-rw-rw-rw-  1 root root     43681 May 22 18:33 Eggheads_20230522_1828.thm
-rw-rw-rw-  1 root root 300241536 May 22 19:00 Eggheads_20230522_1828.ts
-rw-rw-rw-  1 root root   1087552 May 23 18:29 Eggheads_20230523_1801.nts
-rw-rw-rw-  1 root root     43681 May 23 18:05 Eggheads_20230523_1801.thm
-rw-rw-rw-  1 root root 282117120 May 23 18:29 Eggheads_20230523_1801.ts
-rw-rw-rw-  1 root root   1180032 May 23 19:00 Eggheads_20230523_1828.nts
-rw-rw-rw-  1 root root     43681 May 23 18:33 Eggheads_20230523_1828.thm
-rw-rw-rw-  1 root root 291260160 May 23 19:00 Eggheads_20230523_1828.ts
-rw-rw-rw-  1 root root   1105952 May 24 18:29 Eggheads_20230524_1801.nts
-rw-rw-rw-  1 root root     43681 May 24 18:05 Eggheads_20230524_1801.thm
-rw-rw-rw-  1 root root 286144704 May 24 18:29 Eggheads_20230524_1801.ts
-rw-rw-rw-  2 root root      2072 May 24 19:00 Eggheads_20230524_1829.hmt
-rw-rw-rw-  2 root root   1311264 May 24 19:00 Eggheads_20230524_1829.nts
-rw-rw-rw-  2 root root     43681 May 24 18:34 Eggheads_20230524_1829.thm
-rw-rw-rw-  2 root root 337353024 May 24 19:00 Eggheads_20230524_1829.ts

 

[prpr]

There was a change a few months ago to expire lone .hmt files (such as those from failed recordings which otherwise sat there forever).
Perhaps that is the cause? I've never really looked at Flatview, so have no idea what it does. If you have any details to add, that would be useful.

There's also a bug with expiring lone .ts files (which really means those without a corresponding .hmt) which I haven't yet got around to looking at.

 

[MymsMan]

I recreated the missing hmt and turned off the the "unwatched" flag and it now seems to be working

A couple of observations:

• It only seems to delete one recording per auto run so it will take a longish while to reduce the surplus recordings in Eggheads down to 5
• There seems to be a second attempt to delete the hmt after it has dust binned the recording files
• Is this the orphan hmt deletion that is being called even when it wasn't an orphan and even when the ts wasn't deleted because of the unwatched flag?

 3454    25/05/2023 15:15:14 - <********* ::expire::run (3.209 seconds)
3452    25/05/2023 15:15:14 - ::expire::run: could not read "/media/My Video/Eggheads/Eggheads_20230503_1801.hmt": No such file or directory
3451    25/05/2023 15:15:14 - expire:        Removing.
3450    25/05/2023 15:15:14 - expire:      Eggheads_20230503_1801.hmt
3449    25/05/2023 15:15:14 - expire:     EXPIRE: removed /media/My Video/Eggheads/Eggheads_20230503_1801.ts
3448    25/05/2023 15:15:14 - expire:        Removing.
3447    25/05/2023 15:15:13 - expire:      Eggheads_20230503_1801.ts
3446    25/05/2023 15:15:13 - expire:        Eggheads_20230523_1828.hmt
3445    25/05/2023 15:15:13 - expire:        Eggheads_20230523_1828.ts
3444    25/05/2023 15:15:13 - expire:        Eggheads_20230518_1829.hmt
3443    25/05/2023 15:15:13 - expire:        Eggheads_20230518_1829.ts
3442    25/05/2023 15:15:13 - expire:        Eggheads_20230524_1801.hmt
3441    25/05/2023 15:15:13 - expire:        Eggheads_20230524_1801.ts
3440    25/05/2023 15:15:13 - expire:        Eggheads_20230522_1828.hmt
3439    25/05/2023 15:15:13 - expire:        Eggheads_20230522_1828.ts
3438    25/05/2023 15:15:13 - expire:        Eggheads_20230517_1801.ts
3437    25/05/2023 15:15:13 - expire:        Eggheads_20230517_1801.hmt
3436    25/05/2023 15:15:13 - expire:        Eggheads_20230523_1801.ts
3435    25/05/2023 15:15:13 - expire:        Eggheads_20230523_1801.hmt
3434    25/05/2023 15:15:13 - expire:        Eggheads_20230516_1828.hmt
3433    25/05/2023 15:15:13 - expire:        Eggheads_20230516_1828.ts
3432    25/05/2023 15:15:13 - expire:        Eggheads_20230522_1802.hmt
3431    25/05/2023 15:15:13 - expire:        Eggheads_20230522_1802.ts
3430    25/05/2023 15:15:13 - expire:        Eggheads_20230519_1801.ts
3429    25/05/2023 15:15:13 - expire:        Eggheads_20230519_1801.hmt
3428    25/05/2023 15:15:13 - expire:        Eggheads_20230511_1828.hmt
3427    25/05/2023 15:15:13 - expire:        Eggheads_20230511_1828.ts
3426    25/05/2023 15:15:13 - expire:        Eggheads_20230510_1828.hmt
3425    25/05/2023 15:15:13 - expire:        Eggheads_20230510_1828.ts
3424    25/05/2023 15:15:13 - expire:        Eggheads_20230518_1801.hmt
3423    25/05/2023 15:15:13 - expire:        Eggheads_20230518_1801.ts
3422    25/05/2023 15:15:13 - expire:        Eggheads_20230515_1801.hmt
3421    25/05/2023 15:15:13 - expire:        Eggheads_20230515_1801.ts
3420    25/05/2023 15:15:13 - expire:        Eggheads_20230509_1828.hmt
3419    25/05/2023 15:15:13 - expire:        Eggheads_20230509_1828.ts
3418    25/05/2023 15:15:13 - expire:        Eggheads_20230516_1801.ts
3417    25/05/2023 15:15:13 - expire:        Eggheads_20230516_1801.hmt
3416    25/05/2023 15:15:13 - expire:        Eggheads_20230511_1801.hmt
3415    25/05/2023 15:15:13 - expire:        Eggheads_20230511_1801.ts
3414    25/05/2023 15:15:13 - expire:        Eggheads_20230515_1829.hmt
3413    25/05/2023 15:15:13 - expire:        Eggheads_20230515_1829.ts
3412    25/05/2023 15:15:13 - expire:        Celebrity Eggheads_20230425_1828.hmt
3411    25/05/2023 15:15:13 - expire:        Celebrity Eggheads_20230425_1828.ts
3410    25/05/2023 15:15:13 - expire:        Eggheads_20230510_1801.hmt
3409    25/05/2023 15:15:13 - expire:        Eggheads_20230510_1801.ts
3408    25/05/2023 15:15:13 - expire:        Eggheads_20230505_1801.ts
3407    25/05/2023 15:15:13 - expire:        Eggheads_20230505_1801.hmt
3406    25/05/2023 15:15:13 - expire:        Eggheads_20230509_1801.hmt
3405    25/05/2023 15:15:13 - expire:        Eggheads_20230509_1801.ts
3404    25/05/2023 15:15:13 - expire:        Eggheads_20230508_1829.hmt
3403    25/05/2023 15:15:13 - expire:        Eggheads_20230508_1829.ts
3402    25/05/2023 15:15:13 - expire:        Eggheads_20230508_1801.hmt
3401    25/05/2023 15:15:13 - expire:        Eggheads_20230508_1801.ts
3400    25/05/2023 15:15:13 - expire:        Eggheads_20230503_1801.hmt
3399    25/05/2023 15:15:13 - expire:        Eggheads_20230503_1801.ts
3398    25/05/2023 15:15:13 - expire:      Entries: 48
3397    25/05/2023 15:15:11 - expire:  days {} type 0 keep 5 keepnew 0
3396    25/05/2023 15:15:11 - expire:EXPIRE: [/media/My Video/Eggheads]
3395    25/05/2023 15:15:11 - expire:      Entries: 0
3394    25/05/2023 15:15:11 - expire:  days {} type 0 keep 5 keepnew 0
3393    25/05/2023 15:15:11 - expire:EXPIRE: [/media/My Video/The Chase]
3392    25/05/2023 15:15:11 - expire:      Entries: 0
3391    25/05/2023 15:15:11 - expire:  days {} type 0 keep 5 keepnew 0
3390    25/05/2023 15:15:11 - expire:EXPIRE: [/media/My Video/Tenable]
3389    25/05/2023 15:15:11 - expire:      Entries: 0
3388    25/05/2023 15:15:11 - expire:  days {} type 0 keep 5 keepnew 0
3387    25/05/2023 15:15:11 - expire:EXPIRE: [/media/My Video/Tipping Point]
3386    25/05/2023 15:15:11 - *********> ::expire::run (Priority 900)

 

[MymsMan]

I recreated the missing hmt and turned off the the "unwatched" flag and it now seems to be working

Spoke far too soon

The deletion carried on to beyond the five episodes I wanted kept and deleting hmt files for recordings

it is also leaving orphan .thm and .nts files - not deleting the full file set

humax /mnt/hd2/My Video/Eggheads # ls
total 839732
drwxr-xr-x  2 root root      4096 May 25 18:59 .
drwxr-xr-x 25 root root     32768 May 25 17:59 ..
-rw-rw-rw-  1 root root         8 May 25 16:53 .autoexpire
-rw-r--r--  1 root root       276 May 25 18:29 .series
-rw-rw-rw-  1 root root   1121824 Apr 20 18:54 Celebrity Eggheads_20230420_1828.nts
-rw-rw-rw-  1 root root     43681 Apr 20 18:34 Celebrity Eggheads_20230420_1828.thm
-rw-rw-rw-  1 root root   1198112 Apr 25 18:59 Celebrity Eggheads_20230425_1828.nts
-rw-rw-rw-  1 root root     43681 Apr 25 18:34 Celebrity Eggheads_20230425_1828.thm
-rw-rw-rw-  1 root root   1257184 Apr  5 19:00 Eggheads_20230405_1828.nts
-rw-rw-rw-  1 root root     43681 Apr  5 18:33 Eggheads_20230405_1828.thm
-rw-rw-rw-  1 root root   1102336 May  1 18:29 Eggheads_20230501_1802.nts
-rw-rw-rw-  1 root root     43681 May  1 18:06 Eggheads_20230501_1802.thm
-rw-rw-rw-  1 root root   1098048 May  2 18:29 Eggheads_20230502_1801.nts
-rw-rw-rw-  1 root root     43681 May  2 18:05 Eggheads_20230502_1801.thm
-rw-rw-rw-  1 root root   1105888 May  8 18:29 Eggheads_20230508_1801.nts
-rw-rw-rw-  1 root root     43681 May  8 18:05 Eggheads_20230508_1801.thm
-rw-rw-rw-  1 root root   1120224 May  9 18:14 Eggheads_20230508_1829.nts
-rw-rw-rw-  1 root root     43681 May  9 18:14 Eggheads_20230508_1829.thm
-rw-rw-rw-  1 root root   1267360 May 10 18:18 Eggheads_20230509_1828.nts
-rw-rw-rw-  1 root root     43681 May 10 18:18 Eggheads_20230509_1828.thm
-rw-rw-rw-  1 root root   1098336 May 10 18:29 Eggheads_20230510_1801.nts
-rw-rw-rw-  1 root root     43681 May 10 18:05 Eggheads_20230510_1801.thm
-rw-rw-rw-  1 root root   1187200 May 12 18:14 Eggheads_20230511_1828.nts
-rw-rw-rw-  1 root root     43681 May 12 18:14 Eggheads_20230511_1828.thm
-rw-rw-rw-  1 root root   1103264 May 12 18:29 Eggheads_20230512_1801.nts
-rw-rw-rw-  1 root root   1318304 May 12 18:59 Eggheads_20230512_1829.nts
-rw-rw-rw-  1 root root   1107520 May 15 18:29 Eggheads_20230515_1801.nts
-rw-rw-rw-  1 root root     43681 May 15 18:05 Eggheads_20230515_1801.thm
-rw-rw-rw-  1 root root   1098528 May 16 18:29 Eggheads_20230516_1801.nts
-rw-rw-rw-  1 root root     43681 May 16 18:06 Eggheads_20230516_1801.thm
-rw-rw-rw-  1 root root   1072192 May 18 18:29 Eggheads_20230518_1801.nts
-rw-rw-rw-  1 root root     43681 May 18 18:06 Eggheads_20230518_1801.thm
-rw-rw-rw-  1 root root   1212352 May 18 19:00 Eggheads_20230518_1829.nts
-rw-rw-rw-  1 root root     43681 May 18 18:33 Eggheads_20230518_1829.thm
-rw-rw-rw-  1 root root   1114112 May 22 18:29 Eggheads_20230522_1802.nts
-rw-rw-rw-  1 root root     43681 May 22 18:05 Eggheads_20230522_1802.thm
-rw-rw-rw-  1 root root   1148160 May 22 19:00 Eggheads_20230522_1828.nts
-rw-rw-rw-  1 root root     43681 May 22 18:33 Eggheads_20230522_1828.thm
-rw-rw-rw-  1 root root   1087552 May 23 18:29 Eggheads_20230523_1801.nts
-rw-rw-rw-  1 root root     43681 May 23 18:05 Eggheads_20230523_1801.thm
-rw-rw-rw-  1 root root   1105952 May 24 18:29 Eggheads_20230524_1801.nts
-rw-rw-rw-  1 root root     43681 May 24 18:05 Eggheads_20230524_1801.thm
-rw-rw-rw-  1 root root 286144704 May 24 18:29 Eggheads_20230524_1801.ts
-rw-rw-rw-  1 root root   1311264 May 24 19:00 Eggheads_20230524_1829.nts
-rw-rw-rw-  1 root root     43681 May 24 18:34 Eggheads_20230524_1829.thm
-rw-rw-rw-  1 root root   1103840 May 25 18:29 Eggheads_20230525_1801.nts
-rw-rw-rw-  1 root root     43681 May 25 18:05 Eggheads_20230525_1801.thm
-rw-rw-rw-  1 root root 275157120 May 25 18:29 Eggheads_20230525_1801.ts
-rw-rw-rw-  2 root root      2072 May 25 18:59 Eggheads_20230525_1829.hmt
-rw-rw-rw-  2 root root   1100736 May 25 18:59 Eggheads_20230525_1829.nts
-rw-rw-rw-  2 root root     43681 May 25 18:33 Eggheads_20230525_1829.thm
-rw-rw-rw-  2 root root 270170112 May 25 18:59 Eggheads_20230525_1829.ts
humax /mnt/hd2/My Video/Eggheads #

I think I will switch to using Sweeper rules to expire recording older than a week instead!

 

[prpr]

A couple of observations:

Says he then making three!

It only seems to delete one recording per auto run so it will take a longish while to reduce the surplus recordings in Eggheads down to 5

That's because it crashes trying to delete a non-existent file.

There seems to be a second attempt to delete the hmt after it has dust binned the recording files

That's because it's missing the path when searching for the .ts file. The .hmt is only supposed to be on the list if the .ts doesn't exist.

Is this the orphan hmt deletion that is being called even when it wasn't an orphan and even when the ts wasn't deleted because of the unwatched flag?

Yes.

it is also leaving orphan .thm and .nts files - not deleting the full file set

This is tied up with the above - it deletes the .ts and .hmt separately.

I think I will switch to using Sweeper rules to expire recording older than a week instead!

That doesn't get the bug fixed though does it? Anyway, I've committed some fixes/improvements.