[youtube-dl] Download files from youtube.com or other video platforms

BBC Sounds is supposed to be supported without work-arounds, so any examples would be helpful.
 
Hmm... easy (or so I thought).

But I just picked a couple at random and they did indeed start downloading (.m4a). I'm sure I've had to make the substitution in the past, but that might have been some time and several releases ago.
 
@/df
This doesn't work:
Code: 
humax ~ # youtube -v https://www.bbc.co.uk/iplayer/episode/l003p40w
[debug] System config: [u'--restrict-filenames', u'--prefer-ffmpeg', u'-f', u'best[height<=?1080][fps<=?60]', u'-o', u'/mnt/hd2/My Video/%(title)s.%(ext)s']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.bbc.co.uk/iplayer/episode/l003p40w']
[debug] Encodings: locale ASCII, fs ASCII, out ASCII, pref ASCII
[debug] youtube-dl version 2021.12.17
[debug] Python version 2.7.1 (CPython) - Linux-2.6.18-7.1-7405b0-smp-with-libc0
[debug] exe versions: ffmpeg 4.1, ffprobe 4.1
[debug] Proxy map: {}
[bbc] l003p40w: Downloading webpage
ERROR: Unable to extract playlist data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/bbc.py", line 1255, in _real_extract
    webpage, 'playlist data'),
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/common.py", line 1012, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract playlist data; please report ...
 
Spin the JS Framework wheel of fortune: today's framework (Orbit JS) assigns the hydration JSON to window.__IPLAYER_REDUX_STATE__, and it's yet a different structure. Actually, that seems to have appeared before, but it wasn't seen noticed because normal iPlayer shows use a different code path. This one, though, has an initial l, not previously seen (live?), and so isn't treated like an iPlayer show. Simple solution is to change l.43 of extractor/bbc.py:
Code: 
-    _ID_REGEX = r'(?:[pbm][\da-z]{7}|w[\da-z]{7,14})'
+    _ID_REGEX = r'(?:[pbml][\da-z]{7}|w[\da-z]{7,14})'
 
Last edited:
Many thanks. I tried with yt-dlp as well, but it didn't work there either (not surprising, given the relevant code is the same).
 
Hi All,

Came across this thread as was having problems with itv hub (namely: https://www.itv.com/hub/the-ipcress-file/10a1209a0001 ( The Ipcress File episode 1). Had a read through and came across the post (#421, around page 20-22) which mentioned about setting up a fakehttp.py. I had a go at this but didn't seem to make a difference. The error I'm getting seems very similar though

Code: 
36  12/03/2022 21:46:29 - -code 1 -level 0 -errorinfo {{} /mod/webif/plugin/qtube/queue.hook 29} -errorcode {CHILDSTATUS 19644 1}
35    12/03/2022 21:46:29 - Caught error: ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
34    12/03/2022 21:46:28 - [ITV] 10a1209a0001: Downloading JSON metadata
33    12/03/2022 21:46:26 - [ITV] 10a1209a0001: Downloading webpage
32    12/03/2022 21:46:02 - Starting queued download URL https://www.itv.com/hub/the-ipcress-file/10a1209a0001 Options  QID 1654

Changing the itv.py to add in the fakehttp reference as suggested and adding in the fakehttp.py file into /mnt/hd2/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor

didn't seem to make any difference.

Any thoughts? Does the download work for anyone else?

Thanks

Rodp
 
You need the new ITV extractor which according to @prpr above is in the latest package version 2021.12.17a. According to @MET above, the fakehttp work-around isn't required for ITV. Your diagnostic is characteristic of the unpatched 2021.12.17. Today, the user agent hack isn't needed.
 
/df said:
You need the new ITV extractor which according to @prpr above is in the latest package version 2021.12.17a.
Click to expand...
2021.12.17b is the latest and as far as I know that just added the 'l' from #484, so I guess itv.py ought to be the same. I'm a bit hampered currently for checking stuff though.

However, the link mentioned in #486 doesn't work for me either:
Code: 
humax# youtube -v https://www.itv.com/hub/the-ipcress-file/10a1209a0001
[debug] System config: [u'--restrict-filenames', u'--prefer-ffmpeg', u'-f', u'best[height<=?1080][fps<=?60]', u'-o', u'/mnt/hd2/My Video/%(title)s.%(ext)s']                                                   [debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.itv.com/hub/the-ipcress-file/10a1209a0001']
[debug] Encodings: locale ASCII, fs ASCII, out ASCII, pref ASCII
[debug] youtube-dl version 2021.12.17
[debug] Python version 2.7.1 (CPython) - Linux-2.6.18-7.1-7405b0-smp-with-libc0
[debug] exe versions: ffmpeg 4.1, ffprobe 4.1
[debug] Proxy map: {}
[debug] Using fake IP 25.182.17.172 (GB) as X-Forwarded-For.
[ITV] 10a1209a0001: Downloading webpage
ERROR: Unable to download webpage: <urlopen error The read operation timed out> (caused by URLError(SSLError('The read operation timed out',),))                                                         File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/mod/lib/python2.7/urllib2.py", line 392, in open
    response = self._open(req, data)
  File "/mod/lib/python2.7/urllib2.py", line 410, in _open
    '_open', req)
  File "/mod/lib/python2.7/urllib2.py", line 370, in _call_chain
    result = func(*args)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/utils.py", line 2737, in https_open
    req, **kwargs)
  File "/mod/lib/python2.7/urllib2.py", line 1161, in do_open
    raise URLError(err)
 
Probably --user-agent 'Mozilla/5.0' will stop the timeout. I didn't need it when I tested earlier today.
 
Is there a maximum size for the output file? I tried to dowmload The Deer Hunter and It's a Mad, Mad, Mad, Mad World from BBC. Both progress until the xxx.mp4.temp file gets to about 5.2gb, before the box freezes or crashes. Process restarts on reboot, until it gets to same point.

Had to delete queue.db to break cycle.
 
Last edited:
Are you sure you're not pulling a much higher resolution than the Humax can play?

Even a Hootenany in HD isn't that big.
 
/df said:
Probably --user-agent 'Mozilla/5.0' will stop the timeout.
Click to expand...
It does, and is currently downloading on a remote machine. Apols, I now remember you said that before. Should I presume that including that for everything won't kill anything else?
 
Unfortunately, until the user-agent plague is finally neutralisedreplaced by something worse in the next few years, we have to set the value on a case-by-case basis.

In this case it might be safe to add code to the extractor to enforce this trivial UA string. A pending decision on this is one reason I haven't yet merged PR #30266 in the upstream repo.
 
Your diagnostic is characteristic of the unpatched 2021.12.17
Click to expand...

I checked the package manage page on webif and it says I'm on 2021.12.17b. I've had trouble in the past updating youtube-dl and so i normally have to uninstall it and qtube and then reinstall from scratch. I then reboot.
I did a Telnet session just now and did youtube-dl --version and it came back with 2012.12.17. If it was version b would the telnet version display b or would it just show 2021.12.17?

Thanks

Rodp
 
rodp said:
If it was version b would the telnet version display b or would it just show 2021.12.17?
Click to expand...
The latter, because this is just my way of showing which patch level we're at. The base code doesn't know about our mods. so doesn't display them. I'd guess it's easy to tweak for the future though, but I haven't looked where to.
 
ok, so it's still failing then for me and I'm confident I have the latest patch and by the sounds of it, no one else is having any luck downloading it too. oh hum :(
 
This is the error I get, even with the --user-agent 'Mozilla/5.0' option

Code: 
57    13/03/2022 21:42:29 - -code 1 -level 0 -errorinfo {{} /mod/webif/plugin/qtube/queue.hook 29} -errorcode {CHILDSTATUS 18946 1}
56    13/03/2022 21:42:29 - Caught error: ERROR: Unable to download webpage: <urlopen error The read operation timed out> (caused by URLError(SSLError('The read operation timed out',),))
55    13/03/2022 21:32:27 - [ITV] 10a1209a0001: Downloading webpage
54    13/03/2022 21:32:02 - Starting queued download URL https://www.itv.com/hub/the-ipcress-file/10a1209a0001 Options --user-agent 'Mozilla/5.0' QID 1654

I'm going to remove both yourtube-dl and qtube again but then remove the whole youtube-dl folder under /mnt/hd2/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl and then add the package again to see if that helps.
 
You must log in or register to reply here.
Back
Top