• The forum software that supports hummy.tv has been upgraded to XenForo 2.3!

    Please bear with us as we continue to tweak things, and feel free to post any questions, issues or suggestions in the upgrade thread.

[youtube-dl] Download files from youtube.com or other video platforms

I guess this now means it's downloaded the JSON, but it seems to need something similar for the video:
Code: 
...
[debug] Proxy map: {}
[ITV] 6js5d0f: Downloading webpage
[ITV] 6js5d0f: Downloading JSON metadata
[debug] Invoking downloader on u'https://playback.brightcovecdn.com/playback/v1/accounts/2821697655001/videos/6318056099
112/high.mp4?bcov_auth=ewoJInR5cGUiOiAiSldUIiwKCSJhbGciOiAiUlMyNTYiCn0.ewoJImFjY2lkIjogIjI4MjE2OTc2NTUwMDEiCn0.XU9GEvV2N
RG__OVbLn9kSt-MX5XHD6MCofzeuD2B89IQOfNYDfPPYk0ZZrQT5Lfj_PNIQh-1UsSGRjpeFu_3IJRtAf5HabobscEcBiAluTw2Vjr5WRTaNeI572h-o2zQo
soQ4aNS67hl0LDGWQfAqgjf1H4EPCrQc-GvlGvoXxgpnnPHQfwLIACoN5TYOREKq26sa4wPEu1v-vWGgQBqssk9IUX2bY7ovfzY9gtec4o7pAXosZvMEiPWx
3PX7pOb4s13Q6zUReHnCwofnnFewCX2QzgAAUMSqkd73iVA0VWg7Atus6Iag40mJWVoVAOgP_EXFOS76c7GJTb-rOELZw'
ERROR: unable to download video data: <urlopen error [Errno 1] _ssl.c:499: error:1407742E:SSL routines:SSL23_GET_SERVER_
HELLO:tlsv1 alert protocol version>
Traceback (most recent call last):
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/YoutubeDL.py", line 1985, in process_info
    success = dl(filename, info_dict)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/YoutubeDL.py", line 1924, in dl
    return fd.download(name, info)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/downloader/http.py", line 351, in real_download
    establish_connection()
  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/downloader/http.py", line 116, in establish_connection
    raise err
URLError: <urlopen error [Errno 1] _ssl.c:499: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version>
 
--external-downloader wget should work for this page.

--external-downloader ffmpeg should also work and would also be able to download some manifest links (.m3u8) as with the Vera test episode https://www.itv.com/watch/vera/1a7314/1a7314a0014 9id from the extractor.

Non-downloadable manifests would be those that use obfuscation not handled by the CF build of ffmpeg.

In fact, the option combination --external-downloader wget --hls-prefer-ffmpeg should cover most TLS download requirements. DASH, ISM and F4M manifests use the built-in HTTP downloader, so videos from a site that serves, eg, DASH segments with a TLS version too high for CF Python won't be downloadable. Other download types would use a different external downloader (eg rtmp).
 
Last edited:
/df said:
--external-downloader wget should work for this page.

--external-downloader ffmpeg should also work
Click to expand...
Both of those work.
In fact, the option combination --external-downloader wget --hls-prefer-ffmpeg should cover most TLS download requirements.
Click to expand...
What does that cover? BBC, ITV and Youtube would seem to be mainly our use. Can those options be used for all of those, or do they just need to be used for ITV?
 
It should be possible to make those options a default. Although BBC doesn't make a fuss about TLS versions; YT may well do, even if it doesn't now, and ITV definitely does (as above). The only negative is that an additional wget process may be used instead of the internal downloader when the latter would have been able to cope (eg, BBC MP4s).

As updated above, some sites may serve DASH manifests whose segments yt-dl can't download, without rebuilding CF Python.

Some builds of ffmpeg supposedly support DASH, but no CF versions, and yt-dl assumes that ffmpeg can't handle a DASH manifest.
 
I have been trying to get ITVX working using the method described in the preceding posts but am having no luck. Here is the output:
Code: 
Lounge# youtube -F https://www.itv.com/watch/vera/1a7314/1a7314a0014         

Traceback (most recent call last):                                           

  File "/mod/lib/python2.7/runpy.py", line 162, in _run_module_as_main       

    "__main__", fname, loader, pkg_name)                                     

  File "/mod/lib/python2.7/runpy.py", line 72, in _run_code                   

    exec code in run_globals                                                 

  File "/mnt/hd2/mod/lib/python2.7/dist-packages/youtube-dl/__main__.py", line 1

6, in <module>                                                               

    import youtube_dl                                                         

  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/__init__.py", lin

e 15, in <module>                                                             

    from .options import (                                                   

  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/options.py", line

 8, in <module>                                                               

    from .downloader.external import list_external_downloaders               

  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/downloader/__init

__.py", line 23, in <module>                                                 

    from .niconico import NiconicoDmcFD                                       

  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/downloader/niconi

co.py", line 11, in <module>

              from ..extractor.niconico import NiconicoIE                                 

  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/__init_ 

_.py", line 9, in <module>                                                       

    from .extractors import *                                                   

  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/extract 

ors.py", line 526, in <module>                                                   

    from .itv import (                                                           

  File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/itv.py" 

, line 9, in <module>                                                           

    from ..utils import (                                                       

ImportError: cannot import name traverse_obj
Any ideas?
 

[youtube-dl] Download files from youtube.com or other video platforms

... Is this some kind of browser timeout rather than anything else? Quite possibly. The lighttpd server can time out a request if it takes too long to complete. For instance, an immediate transcoding operation can show some amount of progress and then refresh as if it had finished. This can...
hummy.tv hummy.tv

Also, a new build is recommended as we/YouTube broke YouTube extraction a couple of weeks ago. The latest git master is good.
 
/df said:
In fact, the option combination --external-downloader wget --hls-prefer-ffmpeg should cover most TLS download requirements.
Click to expand...
Using that with iPlayer results in absolutely endless whining messages about timestamps and a download rate about 40% of what you get if you omit the --hls-prefer-ffmpeg option (12:53 vs. 32:11 on the 1800 news).
 
I'm getting failures trying to download from YouTube. One example (three failed the same):

Code: 
5000    14/03/2023 07:23:36 - -code 1 -level 0 -errorinfo {{} /mod/webif/plugin/qtube/queue.hook 29} -errorcode {CHILDSTATUS 28846 1}
4999    ERROR: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
4998    )
4997    ExtractorError: Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
4996        raise ExtractorError('Unhandled exception in decode')
4995      File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/extractor/youtube.py", line 1524, in _n_descramble
4994    WARNING: [youtube] Unable to decode n-parameter: download likely to be throttled (Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last):
4993  
4992    OSError: [Errno 30] Read-only file system: '/.cache'
4991        mkdir(name, mode)
4990      File "/mod/lib/python2.7/os.py", line 157, in makedirs
4989        makedirs(head, mode)
4988      File "/mod/lib/python2.7/os.py", line 150, in makedirs
4987        makedirs(head, mode)
4986      File "/mod/lib/python2.7/os.py", line 150, in makedirs
4985        os.makedirs(os.path.dirname(fn))
4984      File "/mod/lib/python2.7/dist-packages/youtube-dl/youtube_dl/cache.py", line 58, in store
4983    14/03/2023 07:23:36 - Caught error: WARNING: Writing cache to u'/.cache/youtube-dl/youtube-nsig/21246a91.json' failed: Traceback (most recent call last):

The cause of the fault appears to be the attempt to write to /.cache.
 
No, YT started sending a new uploader URL format that causes the crash (ERROR: ...), owing to a latent bug. Fixed in upstream repo. Also fixed in the latest extractor version, YT started a new rate throttling scheme and a bug in the existing unthrottling code (WARNING: ...) was exposed. I'll shortly push a couple of related fixes, but the current upstream code is fine, especially if you get a combined format (-f best, eg).

In this log there is a Humax packaging issue too. yt-dl is trying to cache data extracted from the YT player JS (2MB of unwanted bloat every time, Invidious FTW). But there's no cache directory:
Code: 
       --cache-dir DIR
              Location in the filesystem where youtube-dl can store some down‐
              loaded      information      permanently.       By       default
              $XDG_CACHE_HOME/youtube-dl  or ~/.cache/youtube-dl .  ...
So, depending on how ~ resolves, the default locations in Humax CF are /youtube-dl, /mod/.cache/youtube-dl or ./.cache/youtube-dl.

The default config needs this line, which can be added in the qtube options field for now (my spies say that the directory is created when needed):
Code: 
--cache-dir /mod/var/cache/youtube-dl
Or export this setting system-wide: XDG_CACHE_HOME=/mod/var/cache
 
Last edited:
Confirmed. The troublesome download now succeeds on both HDR-FOX and HD-FOX after an auto-update (without any intervention from me other than resubmitting the failed queue item). Thanks both.
 
Last edited:
If one wanted to abort a background download, how would one do it? It's not possible to delete a queue item if it's already running.
 
Black Hole said:
If one wanted to abort a background download, how would one do it?
Click to expand...
Manual process killing. Or reboot and delete/hold the item in the queue before it restarts. I think you have a few minutes (4 or 5).
It's not possible to delete a queue item if it's already running.
Click to expand...
This is somewhat irritating. Sometimes it says something's running when it isn't, and there's no easy way out.
 
prpr said:
This is somewhat irritating. Sometimes it says something's running when it isn't, and there's no easy way out.
Click to expand...
It should probably save the PID when starting a queued task and use that to allow killing of a running task and to detect whether a queued item is really still running
 
Are there any up-to-date instructions for how to install the youtube-dl package? It's not listed in the "Available" list in the web IF, and the instructions from Github give this error:
Code: 
humax# wget https://yt-dl.org/downloads/latest/youtube-dl -O /usr/local/bin/youtube-dl
wget: not an http or ftp url: https://yt-dl.org/downloads/latest/youtube-dl
 
You must log in or register to reply here.
Back
Top