This timeout is a denial of service amounting to a protocol violation by whatever fronts the ITV Hub site. Instead of looking at the web request and replying according to the HTTP specification, either Yes, here's the requested resource or No, because (eg 404 - not found, 403 - not permitted, ...), it just ignores the request, not because it has too much to do, which might be acceptable, but because it doesn't like the look of the completely protocol-conformant request. Cloudflare and Akamai are two chief suspects in this behaviour, which is now affecting a lot of sites.
By changing the request in some way the virtual bouncer may be bypassed at least temporarily.
I found that setting a UA of just
Mozilla/5.0 worked today, with the current PR version:
Code:
$ python -m youtube_dl --user-agent 'Mozilla/5.0' -F -v --no-geo-bypass 'https://www.itv.com/hub/endeavour/2a1229a0009'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--user-agent', u'Mozilla/5.0', u'-F', u'-v', u'--no-geo-bypass', u'https://www.itv.com/hub/endeavour/2a1229a0009']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Git HEAD: 0945174fa
[debug] Python version 2.7.17 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[ITV] 2a1229a0009: Downloading webpage
[ITV] 2a1229a0009: Downloading JSON metadata
[ITV] 2a1229a0009: Downloading m3u8 information
[info] Available formats for 2a1229a0009:
format code extension resolution note
hls-68 mp4 audio only 68k , mp4a.40.2@ 64k
hls-102 mp4 audio only 102k , mp4a.40.2@ 96k
hls-285 mp4 512x288 285k , avc1.4D401F@ 203k, 25.0fps, mp4a.40.2@ 64k
hls-636 mp4 512x288 636k , avc1.4D401F@ 504k, 25.0fps, mp4a.40.2@ 96k
hls-848 mp4 512x288 848k , avc1.4D401F@ 703k, 25.0fps, mp4a.40.2@ 96k
hls-1272 mp4 896x504 1272k , avc1.4D401F@1103k, 25.0fps, mp4a.40.2@ 96k
hls-1908 mp4 896x504 1908k , avc1.4D401F@1703k, 25.0fps, mp4a.40.2@ 96k (best)
$
I also set
--no-geo-bypass because I (and presumably most members of the forum) don't need to use a fake UK IP, and that might also trigger a virtual "no trainers" or "where's your jacket".
And this is the same thing (abbreviated) with the latest
youtube package on a HD-Fox T2, updated with the latest PR version of
itv.py patched to use
fakehttp.py:
Code:
# youtube-dl --user-agent 'Mozilla/5.0' -F -v --no-geo-bypass "https://www.itv.com/hub/endeavour/2a1229a0009"
[debug] System config: [u'--restrict-filenames', u'--prefer-ffmpeg', u'-f', u'best[height<=?1080][fps<=?60]', u'-o', u'/media/drive1/Video/%(title)s.%(ext)s']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--user-agent', u'Mozilla/5.0', u'-F', u'-v', u'--no-geo-bypass', u'https://www.itv.com/hub/endeavour/2a1229a0009']
[debug] Encodings: locale ASCII, fs ASCII, out ASCII, pref ASCII
[debug] youtube-dl version 2020.09.20
[debug] Python version 2.7.1 (CPython) - Linux-2.6.18-7.1-7405b0-smp-with-libc0
[debug] exe versions: ffmpeg 4.1, ffprobe 4.1
[debug] Proxy map: {}
[ITV] 2a1229a0009: Downloading webpage
[ITV] 2a1229a0009: Downloading JSON metadata
[ITV] 2a1229a0009: Downloading m3u8 information
[info] Available formats for 2a1229a0009:
format code extension resolution note
hls-68 mp4 audio only 68k , mp4a.40.2@ 64k
...
hls-1908 mp4 896x504 1908k , avc1.4D401F@1703k, 25.0fps, mp4a.40.2@ 96k (best)
#
