Commit graph

11853 commits

Author SHA1 Message Date
Yen Chi Hsuan
d32fa0f12c
[huajiao] Coding style 2016-10-15 14:53:53 +08:00
Yen Chi Hsuan
a347a0d088
Merge branch 'huajiao' of https://github.com/pyx/youtube-dl into pyx-huajiao 2016-10-15 14:53:05 +08:00
Yen Chi Hsuan
77c5b98dcd
[crunchyroll] Skip an invalid _TEST 2016-10-15 14:36:07 +08:00
Yen Chi Hsuan
88ebefc054
[cmt] Fix mgid extraction (closes #10813)
The example in #10813 requires TV provider authentication in Firefox,
while youtube-dl can download it directly with an US proxy.

I'm not sure whether the mgid fix is cmt-specific or it applies to all
mtv-based sites. I keep it in cmt.py until similar patterns are found in
other websites.
2016-10-15 14:27:15 +08:00
Philip Xu
2e638d7bca Made optional fields optional 2016-10-14 14:12:06 -04:00
Sergey M․
a26b174c61
[safari:course] Add support for techbus.safaribooksonline.com 2016-10-15 00:29:33 +07:00
Sergey M․
73c801d660
[orf:tvthek] Fix extraction and modernize (closes #10898) 2016-10-14 23:43:09 +07:00
Yen Chi Hsuan
8c3e448e80
[clipfish] Update _TEST; the old one is gone 2016-10-15 00:12:21 +08:00
Yen Chi Hsuan
2ecbd2ad6f
[chirbit:profile] Fix extraction 2016-10-15 00:01:46 +08:00
Yen Chi Hsuan
62a0b86e4f
[carambatv] Fix extraction
The video requested in #9815 now has videomore embeds.
2016-10-14 23:43:18 +08:00
Yen Chi Hsuan
146969e05b
[videomore] Support <iframe> embed videos
Seen in CarambaTVPage
2016-10-14 23:42:11 +08:00
Yen Chi Hsuan
e2004ccaf7
[canalplus] Fix video_id and update _TESTS
Some tests are gone, and some redirect to different videos
2016-10-14 20:26:12 +08:00
Yen Chi Hsuan
a5f8473145
[cbsinteractive] Fix extraction for cnet.com 2016-10-14 18:20:01 +08:00
Philip Xu
b7f59a3bf6 [huajiao] Add new extractor 2016-10-13 21:51:26 -04:00
Yen Chi Hsuan
580d411931
[parliamentliveuk] Recognize lower case URLs
Closes #10912

Seems parliamentliveuk matches URLs case-insentive. For example this URL
also works:
http://parliamentlive.tv/EvEnt/Index/3F24936f-130f-40bf-9a5d-b3d6479da6a4
2016-10-14 00:44:28 +08:00
Sergey M․
5c4bfd4da5
release 2016.10.12 2016-10-12 21:30:05 +07:00
Sergey M․
bcd6276520
[downloader/common] Remove debug output 2016-10-12 21:22:33 +07:00
Sergey M․
591e384552
[streamable] Remove debug output 2016-10-12 21:22:12 +07:00
Yen Chi Hsuan
9feb1c9731
[dailymotion] Fix extraction and update _TESTS
Closes #10901

Seems all videos use player V5 syntax now
2016-10-12 21:45:49 +08:00
Yen Chi Hsuan
a093cfc78b
[vimeo:review] Fix extraction (#10900)
Now Vimeo Review videos uses React. Thanks @davekaro for analyzing the
problem!
2016-10-12 01:48:06 +08:00
Yen Chi Hsuan
cea364f70c
[extractor/common] Support HTML media elements without child nodes 2016-10-12 01:40:28 +08:00
Yen Chi Hsuan
55642487f0
[nhl] Skip invalid m3u8 formats (closes #10713) 2016-10-11 20:50:52 +08:00
Yen Chi Hsuan
3d643f4cec
[hbo] Add HBOEpisodeIE (#10892) 2016-10-11 17:46:52 +08:00
Yen Chi Hsuan
c452e69d3d
[footyroom] Fix extraction and update _TESTS (closes #10810) 2016-10-11 17:46:13 +08:00
Yen Chi Hsuan
555787d717
[streamable] Add helper for extracting embedded videos 2016-10-11 17:44:35 +08:00
Yen Chi Hsuan
f165ca70eb
[abc.net.au:iview] Fix for non-series videos (closes #10895) 2016-10-11 12:53:27 +08:00
Yen Chi Hsuan
27b8d2ee95
[hbo] Add display_id and another test (#10892) 2016-10-11 12:41:44 +08:00
Yen Chi Hsuan
71cdcb2331
[hbo] Support episode pages (closes #10892) 2016-10-11 12:30:35 +08:00
Yen Chi Hsuan
176006a120
[allocine] Fix for /video/ videos (closes #10860) 2016-10-09 19:42:42 +08:00
Yen Chi Hsuan
65f4c1de3d
[allocine] Fix extraction (closes #10860)
I change the URL of the third test case, because now the original URL
does not contain a video anymore, and there's no easy to get the real
URL from the /film/ one.
2016-10-09 18:58:15 +08:00
Yen Chi Hsuan
b0082629a9
[nextmedia] Support action news (動新聞) on Apple Daily 2016-10-09 18:42:15 +08:00
Déstin Reed
2b51dac1f9
[slutload] Fix test and simplify 2016-10-09 01:17:38 +07:00
Sergey M․
f68901e50a
[reverbnation] Eliminate code duplication in thumbnails extraction 2016-10-09 01:02:35 +07:00
Déstin Reed
3adb9d119e
[reverbnation] Modernize 2016-10-09 01:00:38 +07:00
Remita Amine
1dd58e14d8 [lego] improve info extraction and bypass geo restriction(closes #10872) 2016-10-08 08:33:18 +01:00
Sergey M․
dd4291f729
release 2016.10.07 2016-10-07 22:25:30 +07:00
Sergey M․
f475e88121
[vimeo] PEP 8
[ci skip]
2016-10-07 22:15:26 +07:00
Remita Amine
3c6b3bf221 [iprima] detect geo restriction 2016-10-07 15:53:16 +01:00
Yen Chi Hsuan
38588ab977
[facebook] Fix for new handleServerJS syntax (closes #10846)
According to the dump file in #10846, handleServerJS() now accepts
an optional second argument. It's a string from available dump files.
2016-10-07 20:04:49 +08:00
Yen Chi Hsuan
85bcdd081c
[extractors] Add MmsIE 2016-10-07 19:31:26 +08:00
Yen Chi Hsuan
9dcd6fd3aa
[generic,commonprotocols] Move mms suuport from GenericIE
And use _generic_* helpers in those extractors
2016-10-07 19:24:22 +08:00
Yen Chi Hsuan
98763ee354
[extractor/common] Add id and title helpers for generic IEs 2016-10-07 19:20:53 +08:00
Yen Chi Hsuan
3d83a1ae92
[generic] Support direct MMS links (closes #10838) 2016-10-07 17:50:45 +08:00
Sergey M․
09b9c45e24
[generic] Add support for multiple vimeo embeds (Closes #10862) 2016-10-06 23:22:52 +07:00
Remita Amine
33898fb19c [nzz] Add new extractor(#4407) 2016-10-06 10:45:57 +01:00
Remita Amine
017eb82934 [npo] detect geo restriction 2016-10-05 18:27:02 +01:00
Sergey M․
b1d798887e
[npo] Add support for 2doc.nl (Closes #10842) 2016-10-05 23:43:08 +07:00
Remita Amine
185744f92f [lego] Add new extractor(closes #10369) 2016-10-04 10:30:57 +01:00
Remita Amine
7232e54813 [tonline] Add new extractor(#10376) 2016-10-04 08:00:25 +01:00
Sergey M․
6eb5503b12
[techtalks] Relax _VALID_URL 2016-10-04 02:54:36 +07:00
Aleksander Nitecki
539c881bfc
[techtalks] Allow URL-s with name part omitted. 2016-10-04 02:52:33 +07:00
Sergey M․
c1b2a0858c
[youtube:live] Extend _VALID_URL (Closes #10839) 2016-10-04 02:10:23 +07:00
Remita Amine
215ff6e0f3 [theweatherchannel] Add new extractor(closes #7188) 2016-10-03 18:20:34 +01:00
Déstin Reed
dcdb292fdd
Unify coding cookie 2016-10-03 23:44:29 +07:00
Remita Amine
c1084ddb0c [thisoldhouse] Add new extractor(closes #10837) 2016-10-03 15:27:09 +01:00
Sergey M․
ee5de4e38e
[nhl] Add support for wch2016.com (Closes #10833) 2016-10-03 00:54:02 +07:00
Yen Chi Hsuan
25291b979a Merge pull request #10829 from TRox1972/pornoxo_improve
[pornoxo] Use JWPlatform to improve metadata extraction
2016-10-02 20:19:34 +08:00
Déstin Reed
567a5996ca [pornoxo] Use JWPlatform to improve metadata extraction 2016-10-02 13:07:02 +02:00
Sergey M․
6c152ce20f
release 2016.10.02 2016-10-02 15:58:00 +07:00
Yen Chi Hsuan
703b3afa93
[amcnetworks] Skip a restricted _TEST 2016-10-02 14:25:06 +08:00
Yen Chi Hsuan
99ed78c79e
[jwplatform] Support DASH streams 2016-10-02 14:07:49 +08:00
Yen Chi Hsuan
fd15264172
[jwplatform] Support old-style jwplayer playlists 2016-10-02 13:47:06 +08:00
Yen Chi Hsuan
bd26441205
[utils] Fix xattr error handling 2016-10-02 03:03:41 +08:00
Yen Chi Hsuan
b19e275d99
[__init__] Fix lost xattr if --embed-thumbnail used
Reported at
https://github.com/rg3/youtube-dl/issues/9054#issuecomment-250451823
2016-10-02 02:12:14 +08:00
Sergey M․
f6ba581f89
[byutv:event] Add extractor 2016-10-02 00:50:07 +07:00
Sergey M․
6d2549fb4f
[byutv] Fix id and display id 2016-10-02 00:44:54 +07:00
Déstin Reed
4da4516973
[byutv] Rely on _match_id and _parse_json 2016-10-02 00:41:18 +07:00
Sergey M․
e1e97c2446
[periscope:user] Fix extraction (Closes #10820) 2016-10-01 22:50:47 +07:00
Yen Chi Hsuan
53a7e3d287
[utils] Support xattr as well as pyxattr
Closes #9054

There are two xattr packages in Python, pyxattr [1] and xattr [2]. They
have different APIs.

In old days pyxattr supports Linux only and xattr supports Linux, Mac,
FreeBSD and Solaris, and pyxattr supports Linux only. Recently pyxattr
adds support for Mac OS X. [3]

An old version of [2] is shipped with Mac OS X. However, some Linux
distributions have pyxattr only, for example PLD-Linux [4] and old Arch
Linux. [5] As a result, supporting both is the way to go.

[1] https://github.com/iustin/pyxattr
[2] https://github.com/xattr/xattr
[3] https://github.com/iustin/pyxattr/pull/9
[4] https://github.com/rg3/youtube-dl/issues/5498
[5] https://git.archlinux.org/svntogit/community.git/commit/?id=427c4c76401e386d865ccddea4fbfdc74df80492
    https://git.archlinux.org/svntogit/community.git/commit/?id=59b40da7b69622a6761d364a8b07909e9cccaa56
    python-xattr is added on 2016/06/29 while pyxattr is there for more
    than 6 years
2016-10-01 20:13:04 +08:00
Yen Chi Hsuan
d54739a2e6
[downloader/http] xattr values should be bytes 2016-10-01 19:58:13 +08:00
Yen Chi Hsuan
63e0fd5bcc Merge pull request #10818 from TRox1972/criterion_match_id
[criterion] Rely on _match_id, improve regex and add thumbnail to test
2016-10-01 19:49:18 +08:00
Déstin Reed
9c51a24642 [criterion] Rely on _match_id, improve regex and add thumbnail to test 2016-10-01 13:46:48 +02:00
Yen Chi Hsuan
9bd7bd0b80
[twitch] Skip a 404 test 2016-10-01 16:38:47 +08:00
Yen Chi Hsuan
4a76b73c6c Merge pull request #10817 from TRox1972/clubic_match_id
[clubic] Rely on _match_id and _parse_json
2016-10-01 16:20:12 +08:00
Yen Chi Hsuan
e295618f9e
[dctp] Fix extraction (closes #10734) 2016-10-01 15:22:48 +08:00
Yen Chi Hsuan
d7753d1948
[downloader/http] Use write_xattr function for --xattr-set-filesize 2016-10-01 14:47:20 +08:00
Déstin Reed
eaf9b22f94 [clubic] Rely on _match_id and _parse_json 2016-09-30 20:03:25 +02:00
Sergey M․
a1001f47fc
[instagram] PEP 8 2016-10-01 00:16:08 +07:00
Déstin Reed
1609782258
[Instagram] Extract video dimensions 2016-10-01 00:13:34 +07:00
Sergey M․
de6babf922
[tvland] Extend _VALID_URL (Closes #10812) 2016-09-30 22:30:34 +07:00
Sergey M․
b0582fc806
[vgtv] Add support for tv.aftonbladet.se (Closes #10800) 2016-09-30 00:15:09 +07:00
Sergey M․
af33dd8ee7
[aftonbladet] Remove extractor 2016-09-30 00:13:03 +07:00
Sergey M․
70d7b323b6
[vk] Improve view count extraction 2016-09-29 23:52:29 +07:00
Sergey M․
a7ee8a00f4
[vk] Extract timestamp (Closes #10760) 2016-09-29 23:52:29 +07:00
Sergey M․
c6eed6b8c0
[utils] Lower priority for rare date formats and add tests 2016-09-29 23:52:29 +07:00
Kacper Michajłow
3aa3953d28
[vk] Fix date and view count extraction. 2016-09-29 23:52:29 +07:00
Yen Chi Hsuan
efa97bdcf1
Move write_xattr to utils.py
There are some other places that use xattr functions. It's better to
move it to a common place so that others can use it.
2016-09-30 00:28:32 +08:00
Sergey M․
475f8a4580
[vk] Add support for running live streams (Closes #10799) 2016-09-29 23:21:39 +07:00
Sergey M․
93aa0b6318
[vk] Add support for finished live streams (#10799) 2016-09-29 23:04:10 +07:00
Yen Chi Hsuan
0ce26ef228 Merge pull request #10788 from TRox1972/instagram_comments
[Instagram] Extract comments
2016-09-29 21:54:39 +08:00
Yen Chi Hsuan
0d72ff9c51
[leeco] Recognize more Le Sports URLs (#10794) 2016-09-29 21:39:35 +08:00
Déstin Reed
a56e74e271 [Instagram] Extract comments 2016-09-28 19:32:40 +02:00
Sergey M․
f533490bb7
[ketnet] Extract mzsource formats (#10770) 2016-09-28 22:58:25 +07:00
Remita Amine
8bfda726c2 [limelight:media] improve http formats extraction 2016-09-28 16:34:27 +01:00
Sergey M․
8f0cf20ab9
release 2016.09.27 2016-09-27 23:09:46 +07:00
Sergey M․
cdfcc4ce95
[mtv] Improve _VALID_URL 2016-09-27 22:27:10 +07:00
Kacper Michajłow
e384552590
[vk] Add support for dailymotion embeds
Fixes #10661
2016-09-27 21:58:14 +07:00
Sergey M․
1a2fbe322e
[periscope] Treat timed_out state as finished stream 2016-09-27 21:55:51 +07:00
Sergey M․
f9dd86a112
[npo] Clarify IE_NAMEs (Closes #10775) 2016-09-27 21:37:33 +07:00
Remita Amine
2342733f85 fix tests related to 1978540a5122c53012e17a78841f3da0df77fd34(closes #10774) 2016-09-27 15:31:25 +01:00
Remita Amine
93933c9819 [awaan:video] fix test(closes #10773) 2016-09-27 15:31:25 +01:00
Yen Chi Hsuan
d75d9e343e
[einthusan] Fix extraction (closes #10714) 2016-09-27 14:38:41 +08:00
Sergey M․
72c3d02d29
[promptfile] Improve and modernize 2016-09-26 23:39:54 +07:00
Ondřej Bárta
d3dbb46330
[promptfile] Fix extraction (Closes #10634) 2016-09-26 23:20:58 +07:00
Sergey M․
fffb9cff94
[kaltura] Speed up embed regexes (#10764) 2016-09-26 22:15:58 +07:00
Sergey M․
2d5b4af007
[extractors] Add import for anderetijden extractor 2016-09-25 23:30:57 +07:00
Sergey M․
5742c18bc1
[npo] Add support for anderetijden.nl (Closes #10754) 2016-09-25 22:26:14 +07:00
Sergey M․
ddb19772d5
[vpro] Fix playlist title extraction and update tests 2016-09-25 22:26:06 +07:00
Sergey M․
a3d8b38168
[npo] Generalize playlist extractors 2016-09-25 22:26:00 +07:00
stepshal
2d3d29976b
[youtube] Change test URLs from http to https 2016-09-25 17:45:24 +07:00
Sergey M․
493353c7fd
[prosiebensat1] Add support for advopedia 2016-09-25 06:25:57 +07:00
Sergey M․
0a078550b9
[prosiebensat1] Improve _VALID_URL 2016-09-25 06:19:17 +07:00
Sergey M․
f92bb612c6
[mwave] Relax _VALID_URLs (Closes #10735, closes #10748) 2016-09-25 06:14:32 +07:00
Sergey M․
ddde91952f
[prosiebensat1] Fix playlist support (Closes #10745) 2016-09-25 05:36:18 +07:00
Sergey M․
63c583eb2c
[prosiebensat1] Add support for sat1gold (#10745) 2016-09-25 04:43:10 +07:00
Remita Amine
7fd57de6fb [cbsnews:livevideo] fix extraction and extract m3u8 formats 2016-09-24 22:01:33 +01:00
Remita Amine
e71a450956 [common] add hdcore sign to akamai f4m formats 2016-09-24 21:55:53 +01:00
Remita Amine
27e99078d3 [brightcove:new] add support for live streams 2016-09-24 15:39:48 +01:00
Remita Amine
6f126d903f [download/hls] Delegate downloading to ffmpeg for live streams 2016-09-24 15:39:47 +01:00
Sergey M․
7518a61d41
[soundcloud] Fix typo in playlist base class name 2016-09-24 19:29:49 +07:00
Sergey M․
8e45e1cc4d
[soundcloud] Generalize playlist entries extraction (#10733) 2016-09-24 19:18:01 +07:00
Yen Chi Hsuan
f0bc5a8609
[twitter] Support Periscope embeds (closes #10737)
Also update _TESTS
2016-09-24 20:00:29 +08:00
Remita Amine
a54ffb8aa7 [mtv] add common IE_NAME prefix for MTVIE and MTVVideoIE 2016-09-24 10:50:14 +01:00
Remita Amine
8add4bfecb [mtv] add support for new website urls(closes #8169)(closes #9808) 2016-09-24 10:42:20 +01:00
Yen Chi Hsuan
0711995bca
[openload] Support subtitles (closes #10625) 2016-09-24 14:27:08 +08:00
Yen Chi Hsuan
5968d7d2fe
[extractor/common] Improved support for HTML5 subtitles
Ref: #10625

In a strict sense, <track>s with kind=captions are not subtitles. [1]
openload misuses this attribute, and I guess there will be more
examples, so I add it to common.py.

Also allow extracting information for subtitles-only <video> or <audio>
tags, which is the case of openload.

[1] https://www.w3.org/TR/html5/embedded-content-0.html#attr-track-kind
2016-09-24 14:20:42 +08:00
Sergey M․
e6332059ac
release 2016.09.24 2016-09-24 02:16:47 +07:00
Sergey M․
24628cf7db
[soundcloud:playlist] Provide video id for playlist entries (Closes #10733) 2016-09-24 02:01:01 +07:00
Sergey M․
71ad00c09f
[prosiebensat1] Add support for kabeleinsdoku (Closes #10732) 2016-09-23 21:08:16 +07:00
Remita Amine
45cae3b021 [cbs] extract info from thunder videoPlayerService(closes #10728) 2016-09-22 19:28:22 +01:00
Yen Chi Hsuan
4ddcb5999d
[openload] Fix extraction (closes #10408, closes #10727)
Thanks to @daniel100097 for providing a working version
2016-09-23 01:47:51 +08:00
Yen Chi Hsuan
e3d6bdc8fc
[ustream] Support HLS streams (closes #10698) 2016-09-23 01:11:13 +08:00
Sergey M․
0a439c5c4c
[udemy] Stringify video id 2016-09-22 21:48:53 +07:00
Remita Amine
1978540a51 [ooyala] extract all hls formats 2016-09-21 21:49:52 +01:00
Sergey M․
12f211d0cb
[videomore] Fix embed regex 2016-09-21 22:51:36 +07:00
Remita Amine
3a5a18705f [adobepass] add support MSO that depend on watchTVeverywhere(closes #10709) 2016-09-21 15:57:27 +01:00
Remita Amine
1ae0ae5db0 [cartoonnetwork] add support Adobe Pass auth 2016-09-20 18:52:00 +01:00
Sergey M․
f62a77b99a
[soundcloud] Modernize 2016-09-20 21:56:57 +07:00
coolsa
4bfd294e2f
[soundcloud] Extract license metadata 2016-09-20 21:56:57 +07:00
Remita Amine
e33a7253b2 [fox] add support for Adobe Pass auth(closes #8584) 2016-09-20 15:52:23 +01:00
Remita Amine
c38f06818d add support for Adobe Pass auth in tbs,tnt and trutv extractors(fixes #10642)(closes #10222)(closes #10519) 2016-09-20 11:55:30 +01:00
Sergey M․
cb57386873
release 2016.09.19 2016-09-19 02:58:32 +07:00
Sergey M․
70b4cf9b1b
[crunchyroll] Check if already logged in (Closes #10700) 2016-09-19 02:50:06 +07:00
Sergey M․
cc764a6da8
[twitch:stream] Remove fallback to profile extraction when stream is offline
Main page does not contain profile videos anymore
2016-09-18 19:10:18 +07:00
Yen Chi Hsuan
d8dbf8707d
[thisav] Improve title extraction (closes #10682)
I didn't add a test case as the one in #10682 looks like a copyrighted
product.
2016-09-18 18:35:38 +08:00
Sergey M․
a1da888d0c
[vyborymos] Improve station info extraction 2016-09-18 17:30:55 +07:00
Sergey M․
3acff9423d
release 2016.09.18 2016-09-18 17:16:55 +07:00
Sergey M․
14ae11efab
[vyborymos] Add extractor (Closes #10692) 2016-09-18 16:56:40 +07:00
Sergey M․
190d2027d0
[xfileshare] Add title regex for streamin.to and fallback to video id (Closes #10646) 2016-09-18 07:22:06 +07:00
Sergey M․
26394d021d
[globo:article] Add support for multiple videos (Closes #10653) 2016-09-17 23:34:10 +07:00