Commit graph

286 commits

Author SHA1 Message Date
Yen Chi Hsuan
1b405bb47d
[downloader/f4m] Tolerate truncate segments when testing
Replaces #9216

Fixes #9214 and test_Bloomberg partially
2016-05-12 20:02:36 +08:00
Yen Chi Hsuan
edaa23f822
[compat] Rename struct_(un)pack to compat_struct_(un)pack 2016-05-10 14:51:38 +08:00
Yen Chi Hsuan
dab0daeeb0
[utils,compat] Move struct_pack and struct_unpack to compat.py 2016-05-10 14:51:38 +08:00
Sergey M․
2937590e8b
[downloader/hls] PEP 8 2016-05-09 22:16:33 +06:00
Sergey M․
e62d9c5caa
[downloader/external] Call ffmpeg with with HTTP_PROXY env variable set (#9437) 2016-05-09 22:05:12 +06:00
Sergey M․
6104cc2985
[downloader/hls] Add event media playlists to unsupported features of hlsnative 2016-05-09 20:55:37 +06:00
Sergey M․
c15c47d19b
[downloader/hls] Remove EXT-X-MEDIA-SEQUENCE from unsupported features for hlsnative 2016-05-09 20:45:03 +06:00
Sergey M
eb785b856f Merge pull request #9358 from dstftw/hls-native-to-ffmpeg-delegation
[downloader/hls] Delegate extraction to ffmpeg when unsupported features detected
2016-05-08 22:07:55 +00:00
Sergey M․
f5436c5d9e
[downloader/external] Add temp fix ffmpeg m3u8 downloads (Closes #9394) 2016-05-08 02:29:26 +06:00
Sergey M․
0d66bd0eab
[downloader/hls] Delegate extraction to ffmpeg when unsupported features detected 2016-05-01 13:56:51 +06:00
Sergey M․
bf09af3acb
Add --hls-prefer-ffmpeg 2016-04-21 23:02:17 +06:00
remitamine
d868f43c58 [ffmpeg] check for - file name in _ffmpeg_filename_argument 2016-04-16 19:45:56 +01:00
remitamine
ce599d5a7e [downloader/external] enable piping for FFmpegFD(closes #2124) 2016-04-14 18:49:02 +01:00
Yen Chi Hsuan
f5d8743e0a
[downloader/rtsp] Print the command 2016-04-14 15:07:31 +08:00
Sergey M․
f7df343b4a [downloader/f4m] Extract routine for removing unsupported encrypted media 2016-03-27 07:41:19 +06:00
Sergey M․
16a8b7986b [downloader/fragment] Document fragment_retries 2016-03-19 20:54:21 +06:00
Sergey M․
617e58d850 [downloader/{common,fragment}] Fix total retries reporting on python 2.6 2016-03-19 20:51:30 +06:00
Sergey M․
e33baba0dd [downloader/dash] Add fragment retry capability
YouTube may often return 404 HTTP error for a fragment causing the
whole download to fail. However if the same fragment is immediately
retried with the same request data this usually succeeds (1-2 attemps
is usually enough) thus allowing to download the whole file successfully.
So, we will retry all fragments that fail with 404 HTTP error for now.
2016-03-19 20:42:23 +06:00
Sergey M․
721f26b821 [downloader/fragment] Add report_retry_fragment 2016-03-19 20:41:24 +06:00
remitamine
4230c4894d [external/downloader] fix rtmp downloading using FFmpegFD 2016-03-14 16:51:01 +01:00
remitamine
2e7e561c1d Merge pull request #8611 from remitamine/ffmpegfd
[downloader/external] Add FFmpegFD
2016-03-13 21:30:27 +01:00
remitamine
d8515fd41c [downloader/external] pass configuration args to ffmpeg 2016-03-13 21:28:26 +01:00
remitamine
694c47b261 [external/downloader] don't pass -t and -ss to ffmpeg 2016-03-13 21:28:16 +01:00
remitamine
77dea16ac8 [downloader/external] check for ffmpeg availablity when it used for m3u8 download 2016-03-13 20:34:51 +01:00
remitamine
6ae27bed01 [download/external] move the check for multiple selected formats to get_suitable_downloader 2016-03-13 20:34:38 +01:00
remitamine
da1973a038 [extractor/__init__] disable time range downloading 2016-03-13 16:16:26 +01:00
remitamine
be24916a7f [downloader/rtsp] Add rtsp and mms downloader 2016-03-13 15:24:02 +01:00
remitamine
2cb99ebbd0 [downloader/external] add can_download mathod for checking downloader availibilty and support 2016-03-13 15:18:51 +01:00
remitamine
91ee320bfa [downloader/external] wrap available_opt in a list 2016-03-13 14:37:45 +01:00
remitamine
634415ca17 [downloader/external] skip FFmpegFD when requesting multiple formats 2016-03-13 12:23:10 +01:00
remitamine
a755f82549 [ffmpeg] convert format ext to ffmpeg output formats codes 2016-03-13 12:15:29 +01:00
Sergey M․
1b5284b13f [downloader/fragment] Make speed more smooth
At the beginning of every segment there was a drop to Unknown speed due to timeslice being too small to calculate speed.
Now last speed from the previous fragment is used.
2016-03-06 05:36:52 +06:00
Yen Chi Hsuan
e9c0cdd389 [jython] Introduce compat_os_name
os.name is always 'java' on Jython
2016-03-03 19:24:24 +08:00
remitamine
f34294fa0c [downloader/external:ffmpegfd] check for None value of start_time 2016-02-20 08:06:12 +01:00
remitamine
99cbe98ce8 [downloader/external] check for external downloaders availability 2016-02-20 07:58:25 +01:00
remitamine
12b84ac8c1 [downloader/external] Add FFmpegFD(fixes #622)
- replace HlsFD and RtspFD
- add basic support for downloading part of the video or audio
2016-02-19 19:29:24 +01:00
Sergey M․
611c1dd96e [refactor] Single quotes consistency 2016-02-14 15:37:17 +06:00
Sergey M․
d800609c62 [refactor] Do not specify redundant None as second argument in dict.get() 2016-02-14 14:25:04 +06:00
Sergey M․
c78c9cd10d [downloader/dash] PEP 8 2016-02-14 14:13:09 +06:00
remitamine
c43fe0268c [downloader/dash] Implement dashsegments fd in terms of fragment fd 2016-02-09 17:25:44 +01:00
Sergey M․
56401e1e5f [downloader/hls] Do not send 'q' to ffmpeg on Windows (Closes #8300) 2016-02-06 23:24:22 +06:00
remitamine
a1a22572fb [downloader/dash] make initialization_url optional 2016-02-02 18:12:32 +01:00
Sergey M․
b8c9926c0a [downloader/f4m] Do not update fragment list while test 2016-01-30 19:43:25 +06:00
Sergey M․
2c2f1efdcd [downloader/fragment] Remove superfluous whitespace 2016-01-30 19:30:31 +06:00
Sergey M․
09104e9930 [downloader/f4m] Add live stream flag to context
Now download progress for f4m livestreams is reported correctly
2016-01-30 19:22:15 +06:00
Sergey M․
5fa1702ca6 [downloader/fragment] Do not report total bytes estimation and eta for live streams 2016-01-30 19:20:52 +06:00
Sergey M․
8ab3fe81d8 [downloader/f4m] Prefer bootstrap url attribute over inline bootstrap info 2016-01-30 18:28:38 +06:00
Jaime Marquínez Ferrándiz
7d106a65ca Add --hls-use-mpegts option
When using the mpegts container hls vidoes can be played while being downloaded (useful if you are recording a live stream).
VLC and mpv play them file, but QuickTime doesn't.
2016-01-30 12:26:40 +01:00
Jaime Marquínez Ferrándiz
32d77eeb04 [downloader/common] report_retry: Don't crash when retries is infinite (fixes #8299) 2016-01-22 14:49:17 +01:00
Jaime Marquínez Ferrándiz
9d90e7de03 [downloader/hls] Ask ffmpeg to quit when interrupting youtube-dl with 'Ctrl+C' (#8252)
Otherwise the mp4 file can't be played.
2016-01-19 22:07:14 +01:00
Sergey M․
b83b782dc4 [downloader/fragment] Move helper data to context dict 2016-01-13 00:00:31 +06:00
Sergey M․
709185a264 [downloader/fragment] More smooth calculations
`downloaded_bytes` is now updated on each fragment progress hook invocation
2016-01-12 23:18:38 +06:00
Sergey M․
9cb1a06b6c [downloader/fragment] Remove unused code and fix zero division error 2016-01-12 22:09:38 +06:00
Jaime Marquínez Ferrándiz
3c91e41614 [downloader/fragment] Don't fail if the 'Content-Length' header is missing
In some dailymotion videos (like http://www.dailymotion.com/video/x3k0dtv from #8156) the segments URLs don't have the 'Content-Length' header and HttpFD sets the 'totat_bytes' field to None, so we also use '0' in that case (since we do different math operations with it).
2016-01-10 14:41:38 +01:00
Jaime Marquínez Ferrándiz
3f17c357d9 [downloader/hls] Don't let ffmpeg read from stdin (#8139)
If you run 'while read aurl ; do youtube-dl "${aurl}"; done < path_to_batch_file'  (batch_file contains one url per line that uses the hls downloader) each call to youtube-dl consumed some characters and 'read' would assing to 'aurl' a non valid url

(This is the same problem that was fixed for the ffmpeg postprocessors in cffcbc02de)
2016-01-04 18:35:31 +01:00
Sergey M․
9b9c5355e4 Rename error_to_str to error_to_compat_str 2015-12-20 07:00:39 +06:00
Sergey M․
7f8b271465 Properly convert errors to strings 2015-12-20 05:27:38 +06:00
Sergey M․
4db43567e8 [downloader/f4m] Decode manifest before fixing 2015-12-11 20:28:44 +06:00
Sergey M․
17b786ae73 [downloader/f4m] Fix malformed manifests (Closes #7823) 2015-12-10 22:59:50 +06:00
Yen Chi Hsuan
94e8c80473 [downloader/hls] Respect Youtubedl-* headers 2015-11-29 12:43:59 +08:00
Yen Chi Hsuan
9d0e366880 [downloader/hls] Remove Accept-encoding from headers passed to ffmpeg
Fails for Youtube Gaming live streams (#7671)
2015-11-27 21:37:45 +08:00
Sergey M?
5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
2015-11-23 21:56:23 +06:00
ashutosh-mishra
17cc153435 Typo fix, found while going through the code. 2015-11-20 22:51:46 +05:30
Jaime Marquínez Ferrándiz
36e6f62cd0 Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x (#7178)
Attributes aren't unicode objects, so they couldn't be directly used in info_dict fields (for example '--write-description' doesn't work with bytes).
2015-10-25 20:13:16 +01:00
Sergey M․
8cd9614abf [downloader/f4m] More accurate fragment URL construction 2015-10-24 21:02:31 +06:00
remitamine
324ac0a243 [downloader/f4m] get the redirected f4m_url and handle url query string properly 2015-10-24 20:05:46 +06:00
Yen Chi Hsuan
985e4fdc07 [downloader/hls] Add headers only for http(s) URLs
ffmpeg 2.8.1 raises an error with -headers and non-http input files.
2015-10-18 13:44:21 +08:00
Sergey M․
964e7b2dd0 [downloader/common] Always skip "already downloaded" check when outputting to stdout 2015-10-12 00:43:54 +06:00
AndroKev
f101079ae0 [downloader/rtmp] Respect --no-continue option
now when downloading a "rtmp-file" the --no-continue option works!
2015-10-12 00:40:37 +06:00
Sergey M․
f102819463 [downloader/hls] Pass http headers to downloader 2015-09-23 02:46:24 +06:00
Jaime Marquínez Ferrándiz
8a7bbd1606 [postprocessor/ffmpeg] Always use the 'file:' protocol for filenames (fixes #6874)
If the filename contains ':' it is interpreted as a protocol.
It also handles filenames starting with '-'.
2015-09-18 14:38:35 +02:00
Sergey M․
133a2b4ac2 [downloader/hls] Properly close stream 2015-09-10 02:41:30 +06:00
Sergey M․
1195a38f46 [downloader/external] Use generalized cli option converters 2015-09-05 03:06:28 +06:00
remitamine
dc534b674f [downloader/external] change _argless_option function to _valueless_option 2015-09-04 21:12:13 +01:00
remitamine
f30c2e8e98 [downloader/external] add _argless_option for option without arguments 2015-09-04 20:57:19 +01:00
remitamine
266b0ad676 [downloader/external] add _bool_option to pass value to bool option 2015-09-04 20:07:36 +01:00
remitamine
e7a8c3032d [downloader/external] Respect --no-check-certificate for curl and aria2c and --proxy for curl 2015-09-03 22:25:33 +01:00
Sergey M․
fcd9e423ec [hls] Properly encode fragment filename 2015-08-31 02:28:36 +06:00
Sergey M․
db8f2bfd99 [f4m] Add missing imports 2015-08-31 02:20:29 +06:00
Sergey M.
55801fc76e Merge pull request #5588 from aajanki/encode_frag_filenames
[f4m] Fix encode error by sanitizing fragment filenames
2015-08-31 02:18:15 +06:00
Sergey M․
7393746da2 [downloader/hls] Add _debug_cmd 2015-08-13 21:10:11 +06:00
Sergey M․
6828c809e4 [downloader/fragment] Respect --retries for fragment based downloaders (Closes #6549) 2015-08-13 21:07:14 +06:00
Sergey M․
9f3da13860 [downloader/external] Use generic _option 2015-08-11 23:05:04 +06:00
Sergey M․
bf812ef714 [downloader/external] Forward --proxy to wget and aria2c 2015-08-11 23:00:45 +06:00
Sergey M․
154655a85a [downloader/external] Respect --no-check-certificate for wget 2015-08-08 19:21:05 +06:00
vijayanand nandam
e0ac521438 adding support for axel download manager 2015-08-08 02:39:05 +06:00
Sergey M․
84bc4dcb0f [downloader/http] Clarify rationale for Content-Range check (#6426) 2015-08-03 02:27:47 +06:00
Sergey M․
10eaa8ef1d [downloader/http] Report unable to resume 2015-08-03 02:25:40 +06:00
Sergey M․
c3124c3085 [downloader/http] Simplify 2015-08-03 02:25:08 +06:00
remitamine
8d5b8b477e [utils] import re 2015-08-03 02:23:35 +06:00
remitamine
d7d2a9a3db [utils] restart download if server does not support byte ranges 2015-08-03 02:23:31 +06:00
Sergey M․
f9a5affad9 [hls] Implement hlsnative fd in terms of fragment fd 2015-07-29 02:28:30 +06:00
Sergey M․
ab81ef8fa7 [f4m] Implement f4m fd in terms of fragment fd 2015-07-29 02:27:50 +06:00
Sergey M․
95d8f7ea12 [fragment] Generalize fragmented media file downloader 2015-07-29 02:26:16 +06:00
Jaime Marquínez Ferrándiz
59db9f8018 [downloader/dash] Improve 'combine_url' (fixes #6341)
In some videos the base_url already ends with '/'.
2015-07-23 12:09:30 +02:00
Jaime Marquínez Ferrándiz
2ee8f5d80f Merge remote-tracking branch 'yan12125/download-dash-segments' (#5886) 2015-07-20 19:34:24 +02:00
Sergey M․
6c4d20cd6f [downloader/external] Fix externals downloaders specified with extension on Windows 2015-06-28 00:08:52 +06:00
Jaime Marquínez Ferrándiz
906e2f0eac [downloader/external] Add downloader for httpie (closes #6079) 2015-06-25 15:48:04 +02:00
Yen Chi Hsuan
5bf3276e8d [downloader/dash] Add testing facility 2015-06-10 14:45:54 +08:00
Yen Chi Hsuan
93dfcb9357 [downloader/dash] Do not pollute ``self`` 2015-06-10 13:44:54 +08:00