LumpyTrumpet
New Member
I've also hit this fairly recently and wondered when 1.4.7 will be pushed out to general release.
I'm happy to test the TVDB integration in the 1.4.7 beta version if that helps, but I don't know how to get hold of it...
TheTVDB.com Integration
- Thread starter af123
- Start date
I'm happy to test the TVDB integration in the 1.4.7 beta version if that helps, but I don't know how to get hold of it...
Black Hole
May contain traces of nut
LumpyTrumpet
New Member
Thanks - that doesn't look too tricky....
I'll try it out and report back in this thread.
LumpyTrumpet
New Member
Great - I will give it a whirl.
Found 1.47 - I needed to update packages after installing opkg-beta.
Last edited:
LumpyTrumpet
New Member
OK - that didn't take long.
Unfortunately this does not work for me.
I have attached a record of my tests, and the web-if error log - it looks like there is a syntax error/invalid option (-4) in the wget call which probably happens after the dialog is initialised but before it gets populated by the wget results
<snip>
| mod/webif/lib/tvdb.class:79: Error: wget: invalid option -- 4 | |
| 106 | at file "/mod/webif/lib/tvdb.class", line 79 |
| 105 | in procedure 'tvdb' called at file "/mod/webif/lib/tvdb.class", line 83 |
| 104 | at file "/mod/lib/jim/oo.tcl", line 73 |
| 103 | in procedure 'tvdb _fetch' called at file "/mod/lib/jim/oo.tcl", line 52 |
| 102 | in procedure '<reference.<tvdb___>.00000000000000000000>' called at file "/mod/webif/lib/tvdb.class", line 195 |
| 101 | at file "/mod/lib/jim/oo.tcl", line 73 |
| 100 | in procedure 'tvdb searchseries' called at file "/mod/lib/jim/oo.tcl", line 52 |
| 99 | in procedure '<reference.<tvdb___>.00000000000000000000>' called at file "/mod/webif/html/browse/tvdb/search.jim", line 12 |
| 98 | |
| 97 | -Y Use proxy ('on' or 'off') |
| 96 | -U STR Use STR for User-Agent header |
| 95 | -O FILE Save to FILE ('-' for stdout) |
| 94 | -T SEC Network read timeout is SEC seconds |
| 93 | -P DIR Save to DIR (default .) |
| 92 | -q Quiet |
| 91 | -c Continue retrieval of aborted transfer |
| 90 | -s Spider mode - only check file existence |
</snip>
If I try a manual wget:
HUMAX# wget
BusyBox v1.20.2 (2013-01-02 19:13:09 GMT) multi-call binary.
Usage: wget [-c|--continue] [-s|--spider] [-q|--quiet] [-O|--output-document FILE]
[--header 'header: value'] [-Y|--proxy on/off] [-P DIR]
[--no-check-certificate] [-U|--user-agent AGENT] [-T SEC] URL...
Retrieve files via HTTP or FTP
-s Spider mode - only check file existence
-c Continue retrieval of aborted transfer
-q Quiet
-P DIR Save to DIR (default .)
-T SEC Network read timeout is SEC seconds
-O FILE Save to FILE ('-' for stdout)
-U STR Use STR for User-Agent header
-Y Use proxy ('on' or 'off')
HUMAX#
Then I get the same - it looks like the -4 switch/option is not available in this version (Busybox 1.20.2?)
Hmm - missing dependency? I have noticed that there is an updated wget version available - I will update to that and retry.
Last edited:
LumpyTrumpet
New Member
Woohoo! It works now!!
You need wget 1.20.3 installed also for the amended code to work. Thanks af123!!
UPDATE after further testing - TVDB Integration appears fixed in 1.4.7 with WGET 1.20.3 *
Last edited:
prpr
Well-Known Member
The dependencies are wrong then.
tvdiary depends on:
webif (>= 1.4.1-2)
jim-binary (>= 0.76)
sqlite3-xconv (>= 1.0.1)
redring (>= 2.20-1)
No mention of wget at all.
Because tvdiary's a different thing.
TheTVDB.com integration is a built-in option of WebIF, which only needs wget for that option. Either it should be a hard dependency, or there should be an error message to prompt the installation of wget.
Hello all,
Applogies if this should be a new thread but thought this might be a bug so adding to this existing one...
I'm struggling to get the dedup/tidyup process to go through and reapply the series and episodes numbers again after renaming some files via the rename opt+ route via webif. Is this possible or will dedup only work when there are two files with the same episode ID (irrespective if the filenames)?
I was ultimately wanting to edit the tvdb file in the mod folder for a specific series (the A team) as series 2 episode 6 got renamed incorrectly as the last 2 characters (I think) got clipped ('When You Comin' Back, Range Rider? (1)' vs 'When You Comin' Back, Range Rider? (2)'). As a result e2e6 got renamed to what e2e7 was ('The Taxicab Wars').
The reason for clipping could be due to me amending the tvdb prefix setting to include ' - A-Team - ', but at the time I didn't think that would have been a problem but it is evident that the humax media list on the tv will only display 40 characters so perhaps so?
This has had an unusual knock on effect for the rest of the episodes in series 2 somehow as each new episode has taken on the next but one title and the last one in the series ends up being called some generic name with a generic synopsis. Is this a bug?
I've since removed the extra wording in the settings but the dedup/tidy process says there is nothing to do and is not picking up the fact that the files I've renamed via opt+ need fixing. (Referring to /mod/var/tvdb/77904.db, the generic titles have come from the last 2 lines where the series number is set to 0).
Any guidance or confirmation that this is a bug and needs fixing would be much appreciated. Otherwise, a solution to re-apply the names as per the tvdb file would be appreciated.
Thanks
Rodp
Applogies if this should be a new thread but thought this might be a bug so adding to this existing one...
I'm struggling to get the dedup/tidyup process to go through and reapply the series and episodes numbers again after renaming some files via the rename opt+ route via webif. Is this possible or will dedup only work when there are two files with the same episode ID (irrespective if the filenames)?
I was ultimately wanting to edit the tvdb file in the mod folder for a specific series (the A team) as series 2 episode 6 got renamed incorrectly as the last 2 characters (I think) got clipped ('When You Comin' Back, Range Rider? (1)' vs 'When You Comin' Back, Range Rider? (2)'). As a result e2e6 got renamed to what e2e7 was ('The Taxicab Wars').
The reason for clipping could be due to me amending the tvdb prefix setting to include ' - A-Team - ', but at the time I didn't think that would have been a problem but it is evident that the humax media list on the tv will only display 40 characters so perhaps so?
This has had an unusual knock on effect for the rest of the episodes in series 2 somehow as each new episode has taken on the next but one title and the last one in the series ends up being called some generic name with a generic synopsis. Is this a bug?
I've since removed the extra wording in the settings but the dedup/tidy process says there is nothing to do and is not picking up the fact that the files I've renamed via opt+ need fixing. (Referring to /mod/var/tvdb/77904.db, the generic titles have come from the last 2 lines where the series number is set to 0).
Any guidance or confirmation that this is a bug and needs fixing would be much appreciated. Otherwise, a solution to re-apply the names as per the tvdb file would be appreciated.
Thanks
Rodp
Tried that and it doesn't think there is anything to do. Even if you try renaming it, it still ignores it.
In the pic below you can see that s2e06 remains titled as The Taxicab Wars.
s2E06, when you look at the TVDB DB list it doesn't place s2e06 with all the others, it is infact at the bottom for some reason along with a couple of others. I've tried clearing the DB and the cache using the command in diagnostics page but this made no difference.
The only thing I'm wondering is, is there a flag which needs to be removed from the files to make the dedup/tidy routine process them again?
Thanks
Rodp
Sorry, of course there is: the Dedup flag.
It can be reset by a shell command
hmt -dedup "/path/to/"s2e*.ts, or using a sweeper rule in the directory if you want to use the WebIf GUI.Tidying David Attenborough's Natural Curiosities reveals two issues with this integration.
1. A long-term bug is that any single quote in the file name breaks the integration, in that invalid JS is created and the Search button in the episode page then does nothing.
This patch makes it possible to search again:
Or maybe this escaping is done elsewhere and should be unified?
Or maybe it's also needed elsewhere and not being done yet?
2. The initial search fails because the search API endpoint is no longer working (
Fortunately, the v4 API search endpoint used by the site's own search page works without any registration or similar. The code has to
And:
1. A long-term bug is that any single quote in the file name breaks the integration, in that invalid JS is created and the Search button in the episode page then does nothing.
This patch makes it possible to search again:
Code:
--- old/webif/html/browse/tvdb/episode.jim
+++ new/webif/html/browse/tvdb/episode.jim
@@ -14,8 +14,8 @@
}
puts "
<script type=text/javascript>
-var file = '$file';
-var dir = '[file dirname $file]';
+var file = '[string map {{'} {\'}} $file]';
+var dir = '[string map {{'} {\'}} [file dirname $file]]';
</script>
"
jscss episode.jsOr maybe it's also needed elsewhere and not being done yet?
2. The initial search fails because the search API endpoint is no longer working (
No results found). Checking in the browser, https://thetvdb.com/api/GetSeries.php?seriesname=The+Hunt returns an empty XML result <Data/>.Fortunately, the v4 API search endpoint used by the site's own search page works without any registration or similar. The code has to
POST magic stringified JSON containing the search term (::tvdb::v4_search_data) to https://api4.thetvdb.com/web/search/queries (``):- the wget web access work-around has to be extended for `POST`ing
- the JSON response has to be extracted like the old XML one
- we also have to deal with full rather than path-only banner image URLs
- fortunately the old ZIPped XML details download API is still working, once a series has been selected.
Code:
--- old/webif/lib/tvdb.class
+++ new/webif/lib/tvdb.class
@@ -1,6 +1,8 @@
source /mod/webif/lib/setup
+source /mod/webif/lib/escape
package require cgi
+package require json
if {![exists -command class]} { package require oo }
if {![exists -command sqlite3.open]} { package require sqlite3 }
require system.class xml.class
@@ -12,6 +14,14 @@
set ::tvdb::icache "/mod/webif/cache/tvdb/img"
set ::tvdb::cacheage 86400
set ::tvdb::debug 0
+set ::tvdb::v4_search_data [join {
+ "{" {"requests":}
+ "[{" {"indexName":"TVDB","params":}
+ "{" {"query":"%s","maxValuesPerFacet":10,"page":0,"filters":"NOT is_official=0","facets":}
+ {["type","year","network","status"],"tagFilters":""} "}}]}"
+} ""]
+set ::tvdb::v4_search_url "https://api4.thetvdb.com/web/search/queries"
+
if {![file isdirectory $::tvdb::cache]} {
file mkdir $::tvdb::cache
@@ -72,15 +82,19 @@
# return $ret
#}
-proc {tvdb fetchfile} {url} {
+proc {tvdb fetchfile} {url {headers {}} {args data}} {
set cmd {wget -q -4 -O-}
+ lappend cmd {*}[concat {*}[lmap k_v $headers {list "--header" [join $k_v ": "]}]]
+ if {[llength $data] == 1} {
+ lappend cmd {--post-data} [lindex $data 0]
+ }
lappend cmd $url
::tvdb::dlog "Executing $cmd"
return [exec {*}$cmd]
}
-tvdb method _fetch {path} {
- return [tvdb fetchfile "https://$::tvdb::mirror$path"]
+tvdb method _fetch {path {headers {}} args} {
+ return [tvdb fetchfile "https://$::tvdb::mirror$path" $headers {*}$args]
}
tvdb method _parse {xml vars {end "XX"}} {
@@ -192,10 +206,25 @@
tvdb method searchseries {series} {
::tvdb::dlog "Searching TVDB for $series"
- set seriesxml [$self _fetch \
- "/api/GetSeries.php?seriesname=[cgi_quote_url $series]"]
- return [$self _extract $seriesxml Series \
- {seriesid SeriesName banner Overview}]
+ set post [format $::tvdb::v4_search_data [jescape $series]]
+ set referer [format "https://%s/search?query=%s" $::tvdb::mirror [cgi_quote_url $series]]
+ set results [tvdb fetchfile $::tvdb::v4_search_url [list \
+ {Content-Type application/json} [list Referer $referer]
+ ] $post]
+ set results [json::decode -index $results]
+ # emulate result of _extract method above
+ set extract [lambda {args} {
+ set attrs {}
+ foreach {k v} $args {
+ set attrs([string tolower $k]) $v
+ }
+ return $attrs
+ }]
+ return [lmap d [dict values [dict getdef $results "results" "0" "hits" {}]] {
+ dict with d {
+ $extract seriesid $id seriesName $name banner $banner Overview $overview
+ }
+ }]
}
tvdb method findseries {series} {
@@ -402,18 +431,28 @@
}
proc {tvdb cachebanner} {banner} {
+ set banner [regsub \
+ "^https?://[string map {"." {\.}} $::tvdb::artwork](/.+)$" \
+ $banner {\1}]
+
+ if {[string index $banner 0] != "/"} {
+ set banner "/$banner"
+ }
if {![string match {/banners/*} $banner]} {
- set _banner "/banners/$banner"
+ set _banner "/banners$banner"
} else {
set _banner $banner
}
- if {![file exists "$::tvdb::icache/$banner"]} {
+ set cached_banner "$::tvdb::icache$banner"
+ if {![file exists "$cached_banner"]} {
set ret [tvdb fetchfile "https://$::tvdb::artwork$_banner"]
- if {[string length $ret] > 50} {
- system mkdir_p "$::tvdb::icache/[file dirname $banner]"
- file write "$::tvdb::icache/$banner" $ret
- }
- }
+ if {[string length $ret] <= 50} {
+ return
+ }
+ system mkdir_p "$::tvdb::icache[file dirname $banner]"
+ file write "$cached_banner" $ret
+ }
+ return [regsub "^/mod/webif(/.+)$" $cached_banner {\1}]
}
proc {tvdb bannerurl} {banner} {
Code:
--- old/webif/cgi-bin/tvdbimg.jim
+++ new/webif/cgi-bin/tvdbimg.jim
@@ -7,8 +7,6 @@
set file [cgi_get file "-"]
if {$file eq "-"} { exit }
-if {![file exists "$::tvdb::icache/$file"]} {
- tvdb cachebanner $file
-}
+set file [tvdb cachebanner $file]
-httpredirect "/cache/tvdb/img/$file"
+httpredirect "$file"