youtube-dl: no formats available for some BBC programmes

[prpr]

Anyway, we can only hope for a fix to be accepted into the original package.

Shows as "Travis CI build failed". Don't know what/why, but it doesn't seem like a good thing.

 

[Thaddeus]

Thanks everyone. Like Christmas, I woke up to a message that said "07/10/2020 01:28:35 - Automatically upgraded package youtube-dl from 2020.09.20 to 2020.09.20-1", which I've seen with other packages before but never really paid attention, so I didn't have to use Black Hole or MontysEvilTwin's solutions (but it's good to know these things if I ever dig myself into a hole like this again).

I think there might still be a flaw in it though, as I've tested the first and second episodes of Atlanta and it's only returning audio only or video only options:

07:19:36 - --------------------------------------------------------------
07:19:36 - Starting immediate download of https://www.bbc.co.uk/iplayer/episode/b0b3px4s/ Options -F
07:20:36 - [bbc.co.uk] b0b3px4s: Downloading video page
07:20:38 - [bbc.co.uk] b0b3px4s: Downloading playlist JSON
07:20:38 - [bbc.co.uk] b0b3pwqh: Downloading media selection XML
07:20:39 - [bbc.co.uk] b0b3pwqh: Downloading media selection XML
07:20:39 - [bbc.co.uk] b0b3pwqh: Downloading MPD manifest
07:20:40 - [bbc.co.uk] b0b3pwqh: Downloading MPD manifest
07:20:41 - [bbc.co.uk] b0b3pwqh: Downloading MPD manifest
07:20:41 - [bbc.co.uk] b0b3pwqh: Downloading MPD manifest
07:20:42 - [info] Available formats for b0b3pwqh:
07:20:42 - format code                                                                             extension  resolution note
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash-audio_eng=96000           m4a        audio only [en] DASH audio   96k , m4a_dash container, mp4a.40.5 (48000Hz)
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-audio_eng=96000     m4a        audio only [en] DASH audio   96k , m4a_dash container, mp4a.40.5 (48000Hz)
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash-audio_eng=96000        m4a        audio only [en] DASH audio   96k , m4a_dash container, mp4a.40.5 (48000Hz)
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-audio_eng=96000  m4a        audio only [en] DASH audio   96k , m4a_dash container, mp4a.40.5 (48000Hz)
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=281000              mp4        384x216    DASH video  281k , mp4_dash container, avc3.42C015, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=281000        mp4        384x216    DASH video  281k , mp4_dash container, avc3.42C015, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=281000           mp4        384x216    DASH video  281k , mp4_dash container, avc3.42C015, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=281000     mp4        384x216    DASH video  281k , mp4_dash container, avc3.42C015, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=437000              mp4        512x288    DASH video  437k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=437000        mp4        512x288    DASH video  437k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=437000           mp4        512x288    DASH video  437k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=437000     mp4        512x288    DASH video  437k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=827000              mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=827000        mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=827000           mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=827000     mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=1604000             mp4        960x540    DASH video 1604k , mp4_dash container, avc3.4D401F, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=1604000       mp4        960x540    DASH video 1604k , mp4_dash container, avc3.4D401F, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=1604000          mp4        960x540    DASH video 1604k , mp4_dash container, avc3.4D401F, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=1604000    mp4        960x540    DASH video 1604k , mp4_dash container, avc3.4D401F, 25fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=2812000             mp4        960x540    DASH video 2812k , mp4_dash container, avc3.64001F, 50fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=2812000       mp4        960x540    DASH video 2812k , mp4_dash container, avc3.64001F, 50fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=2812000          mp4        960x540    DASH video 2812k , mp4_dash container, avc3.64001F, 50fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=2812000    mp4        960x540    DASH video 2812k , mp4_dash container, avc3.64001F, 50fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=5070000             mp4        1280x720   DASH video 5070k , mp4_dash container, avc3.640020, 50fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=5070000       mp4        1280x720   DASH video 5070k , mp4_dash container, avc3.640020, 50fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=5070000          mp4        1280x720   DASH video 5070k , mp4_dash container, avc3.640020, 50fps, video only
07:20:42 - stream-uk-pc_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=5070000    mp4        1280x720   DASH video 5070k , mp4_dash container, avc3.640020, 50fps, video only (best)
07:20:43 - Completed immediate download of https://www.bbc.co.uk/iplayer/episode/b0b3px4s/ Options -F

I've tried other iPlayer programmes and they're downloading fine, so I think it's still just a localised issue.

And without wanting insult to injury and turn this thread into a TV review, on this:

Is this series any good?

I'd say for me it's more interesting than something I've truly loved, with various experiments in story telling, some which work better than others. It's definitely got something that I've not seen before though and I think it will be worth revisiting over the years, which is why I'd quite like to have it saved but as you say, "it's only telly" so I'll survive if it's not to be.

Thanks again everyone (especially /df). Even though my attempts last night were a bit stressful at times, I did have moments where it clicked, and I understood more about each step, so I've learnt a few more things about the packages, which is no bad thing.

 

[Black Hole]

I think there might still be a flaw in it though, as I've tested the first and second episodes of Atlanta and it's only returning audio only or video only options

Don't quote me, but I suspect audio only and video only streams get downloaded and combined in the ffmpeg post-process.

 

[/df]

Mostly, combined streams are not offered; if you want to offer M types of audio and N types of video, that would be M*N streams instead of M+N.

Each DASH or HLS stream comes as a series of chunks. A player has to extract the content from each audio and video chunk covering the current playback offset to render them, but doesn't have to wait for intervening chunks if the viewer has moved the playback offset or the network is too slow to keep up with a live stream. A downloader has to (or at least, does) extract and append the content of each type and then merge the results into the output container. In the log quoted above you can see the merge step [ffmpeg] Merging formats into "Atlanta, Series 1, The Big Bang-b0b3ptgc.mp4". Coincidentally, DASH/HLS defeat the ability of the naive user to do "Save Video As ..." from the browser's HTML5 player. youtube-dl FTW, as I think the young folk used to say.

Shows as "Travis CI build failed". Don't know what/why, but it doesn't seem like a good thing.

So good that the build test checks for trailing spaces on comment lines -- I wouldn't want the guilt of having introduced that kind of bloat to the codebase.

Looks like you implemented the proposal in the CF repo package - many thanks. The linked PR is slightly different from the proposal quoted previously, but the change probably won't affect UK users: and not formats added to the test for raising the last_exception.

 

[Thaddeus]

Mostly, combined streams are not offered; if you want to offer M types of audio and N types of video, that would be M*N streams instead of M+N.

Each DASH or HLS stream comes as a series of chunks. A player has to extract the content from each audio and video chunk covering the current playback offset to render them, but doesn't have to wait for intervening chunks if the viewer has moved the playback offset or the network is too slow to keep up with a live stream. A downloader has to (or at least, does) extract and append the content of each type and then merge the results into the output container. In the log quoted above you can see the merge step [ffmpeg] Merging formats into "Atlanta, Series 1, The Big Bang-b0b3ptgc.mp4". Coincidentally, DASH/HLS defeat the ability of the naive user to do "Save Video As ..." from the browser's HTML5 player. youtube-dl FTW, as I think the young folk used to say.

So good that the build test checks for trailing spaces on comment lines -- I wouldn't want the guilt of having introduced that kind of bloat to the codebase.

Looks like you implemented the proposal in the CF repo package - many thanks. The linked PR is slightly different from the proposal quoted previously, but the change probably won't affect UK users: and not formats added to the test for raising the last_exception.

Thanks. I've looked at this with fresh eyes and I now have episode 1 with sound and vision. Next stop, Hollywood.

Thanks again.

 

[prpr]

Looks like you implemented the proposal in the CF repo package - many thanks. The linked PR is slightly different from the proposal quoted previously, but the change probably won't affect UK users: and not formats added to the test for raising the last_exception.

I've updated it again to reflect that. The comments weren't there in your original, so didn't need fixing.