Commit graph

396 commits

Author SHA1 Message Date
pukkandan
337734d4a8
[cleanup] Misc 2023-07-06 21:39:55 +05:30
coletdjnz
90db9a3c00
[extractor/youtube:stories] Remove (#7459)
YouTube killed them

https://web.archive.org/web/20230630153050/https://support.google.com/youtube/thread/217640760
2023-07-06 19:02:41 +05:30
coletdjnz
4dc4d8473c
[extractor/youtube] Ignore incomplete data for comment threads by default (#7475)
For both `--ignore-errors` and `--ignore-errors only_download`. Pass `--no-ignore-errors` to not ignore.

Closes https://github.com/yt-dlp/yt-dlp/issues/7474

Authored by: coletdjnz
2023-07-03 10:47:10 +00:00
coletdjnz
fcbc9ed760
[extractor/youtube:tab] Support shorts-only playlists (#7425)
Fixes https://github.com/yt-dlp/yt-dlp/issues/7424

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-06-29 23:26:27 +00:00
pukkandan
d949c10c45
[extractor/youtube] Process post_live over 2 hours 2023-06-26 07:25:52 +05:30
pukkandan
58786a10f2
[extractor/youtube] Add extractor-arg formats
Closes #7417
2023-06-25 20:14:37 +05:30
Simon
89bed01374
[extractor/youtube] Fix comments' is_favorited (#7390)
Authored by: bbilly1
Closes #7389
2023-06-22 23:38:42 +05:30
pukkandan
cd810afe2a
[extractor/youtube] Improve nsig function name extraction 2023-06-22 13:27:18 +05:30
Berkan Teber
71dc18fa29
[extractor/youtube] Improve description parsing performance (#7315)
* The parsing is skipped when not needed
* The regex is improved by simulating atomic groups with lookaheads

Authored by: pukkandan, berkanteber
2023-06-22 12:57:54 +05:30
pukkandan
1e75d97db2
[extractor/youtube] Add ios to default clients used
* IOS is affected neither by 403 or by nsig so helps mitigate them preemptively
* IOS also has higher bit-rate "premium" formats though they are not labeled as such
2023-06-22 01:36:06 +05:30
pukkandan
81ca451480
[extractor/youtube] Workaround 403 for android formats
Ref: https://github.com/TeamNewPipe/NewPipe/issues/9038#issuecomment-1289756816
2023-06-22 00:15:22 +05:30
pukkandan
ad54c9130e
[cleanup] Misc
Closes #6288, Closes #7197, Closes #7265, Closes #7353, Closes #5773
Authored by: mikf, freezboltz, pukkandan
2023-06-21 09:21:20 +05:30
pukkandan
97afb093d4
[extractor/youtube] Ignore wrong fps of some formats 2023-06-21 06:10:39 +05:30
pukkandan
51a07b0dca
[extractor/youtube] Prioritize premium formats
Closes #7283
2023-06-21 06:10:38 +05:30
pukkandan
ff9b0e071f
[extractor/youtube] Determine audio language using automatic captions 2023-06-17 01:43:03 +05:30
coletdjnz
8213ce28a4
[extractor/youtube] Extract channel_is_verified (#7213)
Authored by: coletdjnz
2023-06-08 19:15:39 +05:30
pukkandan
14a14335b2
[extractor/youtube] Misc cleanup
Authored by: coletdjnz
2023-06-08 19:14:57 +05:30
coletdjnz
2fb35f6004
[extractor/youtube] Support shorter relative time format (#7191)
See: https://github.com/TeamNewPipe/NewPipeExtractor/issues/1067

Authored by: coletdjnz
2023-06-03 06:33:51 +00:00
coletdjnz
c35448b7b1
[extractor/youtube] Extract more metadata for comments (#7179)
Adds new comment fields:
* `author_url` - The url to the comment author's page
* `author_is_verified` - Whether the author is verified on the platform
* `is_pinned` - Whether the comment is pinned to the top of the comments

Closes https://github.com/yt-dlp/yt-dlp/issues/5411

Authored by: coletdjnz
2023-06-01 08:43:32 +00:00
coletdjnz
18f8fba7c8
[extractor/youtube] Fix continuation loop with no comments (#7148)
Deep check the response for incomplete data.

Authored by: coletdjnz
2023-05-31 07:08:28 +00:00
coletdjnz
93e12ed76e
[extractor/youtube] Extract uploader metadata for feed/playlist items
Fixes https://github.com/yt-dlp/yt-dlp/issues/7104

Authored by: coletdjnz
2023-05-28 11:31:45 +12:00
Audrey
5caf30dbc3
[extractor/youtube] Extract heatmap data (#7100)
Closes #3888
Authored by: tntmod54321
2023-05-26 17:54:39 +05:30
pukkandan
4823ec9f46
Update to ytdl-commit-d1c6c5
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6

Except:
    * 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
        * There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
    * e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
        * We plan to implement this differently
2023-05-24 23:30:43 +05:30
kangalio
69a40e4a7f
[extractor/youtube:music:search_url] Extract title (#7102)
Authored by: kangalio
Closes #7095
2023-05-22 17:17:06 +05:30
coletdjnz
447afb9eaa
[extractor/youtube] Support podcasts and releases tabs
Closes https://github.com/yt-dlp/yt-dlp/issues/6893

Authored by: coletdjnz
2023-05-20 19:11:03 +12:00
coletdjnz
7666b93604
[extractor/youtube] Define strict uploader metadata mapping (#6384)
New mapping:
```
channel -> channel name
channel_id -> UCID
channel_url -> UCID channel url

uploader -> channel name (same as channel field)
uploader_id -> @handle
uploader_url -> @handle channel url 
```

Authored by: coletdjnz
2023-04-14 07:58:36 +00:00
pukkandan
c6786ff3ba
[extractor/youtube] Revert default formats to https 2023-04-11 21:43:31 +05:30
coletdjnz
141a8dff98
[extractor/youtube] Fix comment loop detection for pinned comments (#6714)
Pinned comments may repeat a second time - this is expected.

Fixes https://github.com/yt-dlp/yt-dlp/issues/6712

Authored by: coletdjnz
2023-04-06 07:44:22 +00:00
Nicholas Defranco
071670cbea
[extractor/youtube] Fix parsing comment_count (#6523)
Closes #5849
Authored by: nick-cd
2023-03-15 04:51:14 +05:30
coletdjnz
607510b9f2
[extractor/youtube] Handle incomplete initial data from watch page (#6510)
Authored by: coletdjnz
2023-03-13 01:43:37 +00:00
pukkandan
e389d172b6
Fix 2a23d92d9e
Closes #6517
2023-03-12 14:47:05 +05:30
pukkandan
2a23d92d9e
[extractor/youtube] Construct fragment list lazily
Building fragment list for all formats take significant time for large videos
2023-03-11 22:46:47 +05:30
pukkandan
86cb922118
[extractor/youtube] Add extractor-arg include_duplicate_formats 2023-03-11 22:34:13 +05:30
Lesmiscore
c795c39f27
[extractor/youtube] Add client name to format_note when -v (#6254)
Authored by: Lesmiscore, pukkandan
2023-03-11 22:33:23 +05:30
pukkandan
c9abebb851
[extractor/youtube] Bypass throttling for -f17
and related cleanup

Thanks @AudricV for the finding
2023-03-09 22:13:03 +05:30
pukkandan
392389b7df
[cleanup] Misc 2023-03-05 03:34:55 +05:30
mushbite
22ccd5420b
[extractor/rutube] Extract chapters from description (#6345)
Authored by: mushbite
2023-03-04 19:03:17 +05:30
coletdjnz
7f51861b18
[extractor/youtube] Detect and break on looping comments (#6301)
Fixes https://github.com/yt-dlp/yt-dlp/issues/6290

Authored by: coletdjnz
2023-03-01 07:56:53 +00:00
pukkandan
5b28cef72d
[cleanup] Misc 2023-02-28 23:51:06 +05:30
pukkandan
31e183557f
[extractor/youtube] Extract channel view_count when /about tab is passed 2023-02-28 23:51:03 +05:30
pukkandan
f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
2023-02-28 23:34:43 +05:30
pukkandan
5038f6d713
[extractor/youtube] Construct dash formats with range query
Closes #6369
2023-02-28 23:14:37 +05:30
pukkandan
a538772969
[cleanup] Misc
Closes #5897
2023-02-17 17:52:22 +05:30
bashonly
c61cf091a5
[extractor/youtube] uploader_id includes @ with handle
Authored by: bashonly
2023-02-17 02:14:45 -06:00
bashonly
149eb0bbf3
[extractor/youtube] Fix uploader_id extraction
Closes #6247
Authored by: bashonly
2023-02-16 08:51:45 -06:00
Bruno Guerreiro
78a78fa74d
[extractor/youtube] Add hyperpipe instances (#6020)
Authored by: Generator
2023-02-12 14:03:45 +05:30
Roland Hieber
05799a48c7
[extractor/youtube] Update invidious and piped instances (#6030)
Authored by: rohieb
2023-02-12 13:22:07 +05:30
Simon Sawicki
6839ae1f6d
[utils] traverse_obj: Fix more bugs
and cleanup uses of `default=[]`

Continued from b1bde57bef
2023-02-10 19:36:55 +05:30
pukkandan
b032ff0f03
[extractor/youtube] Handle consent.youtube 2023-02-03 23:53:42 +05:30
pukkandan
dad2210c0c
[extractor/youtube] Support /live/ URL 2023-02-03 23:53:41 +05:30