Commit Graph

19 Commits

Author SHA1 Message Date
litetex 349ba8db7f
Improve tests and randomness
- Use the existing RNG inside YoutubeParsingHelper
- Deduplicated test-setup for YouTube tests
- Minor improvements
2022-03-27 20:52:38 +02:00
Stypox 0c37c75981
Make getDownloader static & extract getDownloaderType 2022-03-16 17:22:42 +01:00
litetex a2cbdf0991 Updated to JUnit 5 2021-12-27 21:08:08 +01:00
FireMasterK 2eeb0a3403
Rebase + some code improvements + fix extraction of age-restricted videos + update clients version
Here is now the requests which will be made by the `onFetchPage` method of `YoutubeStreamExtractor`:

- the desktop API is fetched.

If there is no streaming data, the desktop player API with the embed client screen will be fetched (and also the player code), then the Android mobile API.
- if there is no streaming data, a `ContentNotAvailableException` will be thrown by using the message provided in playability status

If the video is age restricted, a request to the next endpoint of the desktop player with the embed client screen will be sent.
Otherwise, the next endpoint will be fetched normally, if the content is available.

If the video is not age-restricted, a request to the player endpoint of the Android mobile API will be made.

We can get more streams by using the Android mobile API but some streams may be not available on this API, so the streaming data of the Android mobile API will be first used to get itags and then the streaming data of the desktop internal API will be used.
If the parsing of the Android mobile API went wrong, only the streams of the desktop API will be used.

Other code changes:

- `prepareJsonBuilder` in `YoutubeParsingHelper` was renamed to `prepareDesktopJsonBuilder`
- `prepareMobileJsonBuilder` in `YoutubeParsingHelper` was renamed to `prepareAndroidMobileJsonBuilder`
- two new methods in `YoutubeParsingHelper` were added: `prepareDesktopEmbedVideoJsonBuilder` and `prepareAndroidMobileEmbedVideoJsonBuilder`
- `createPlayerBodyWithSts` is now public and was moved to `YoutubeParsingHelper`
- a new method in `YoutubeJavaScriptExtractor` was added: `resetJavaScriptCode`, which was needed for the method `resetDebofuscationCode` of `YoutubeStreamExtractor`
- `areHardcodedClientVersionAndKeyValid` in `YoutubeParsingHelper` returns now a `boolean` instead of an `Optional<Boolean>`
- the `fetchVideoInfoPage` method of `YoutubeStreamExtractor` was removed because YouTube returns now 404 for every client with the `get_video_info` page
- some unused objects and some warnings in `YoutubeStreamExtractor` were removed and fixed

Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>
2021-08-01 12:39:03 +02:00
TiA4f8R 7753556e66
Adress the last requested changes + update YoutubeCommentsExtractor mocks 2021-08-01 12:39:03 +02:00
TiA4f8R 632772d17f
Adress requested changes in YoutubeParsingHelper 2021-08-01 12:38:58 +02:00
TiA4f8R 70927ddade
Update client version and mocks 2021-08-01 12:38:40 +02:00
TiA4f8R 3017dde67e
Fix some typos 2021-08-01 12:38:36 +02:00
TiA4f8R 991b2c7d73
Use lightweight requests when getting and checking YouTube API key and client version 2021-08-01 12:38:01 +02:00
TobiGr 4b8d4a84b6 Set the NumberGenerator for th remaining tests 2021-04-09 11:59:17 +02:00
XiangRongLin 84eb4b30c9 Use DownloaderFactory in YouTube tests 2021-02-13 20:00:51 +01:00
XiangRongLin 4dad3d60d6 Move DownloaderTestImpl into downloader package 2021-01-10 19:36:21 +01:00
TobiGr bc6de14952 Extract stream and search meta info for YouTube
Add method to extract Google webcache URLs.
2020-12-22 18:19:26 +01:00
wb9688 33b1121fc7 Add tests for parseDurationString() 2020-05-30 17:20:54 +02:00
bopol 636c430743 refactor: create extractors and linkHandler packages for SoundCloud, move YoutubeParsingHelper to youtube package (not linkhandler anymore) 2020-04-10 10:51:05 +02:00
wb9688 eb48524411 Add tests for YouTube Music search 2020-04-01 10:30:51 +02:00
Mauricio Colli 342bdbb852 [YouTube] Avoid crashing by letting exceptions bubble up 2020-03-01 12:49:13 +01:00
wb9688 bfe3eb1409 Use suggested try-if code style 2020-02-29 17:18:50 +01:00
wb9688 beb07a4852 Add test to check whether hardcoded client version is still valid 2020-02-28 16:36:14 +01:00