Commit Graph

763 Commits

Author SHA1 Message Date
TiA4f8R 448b68700c
Use final in exceptions and add AgeRestrictedException
AgeRestrictedException will be thrown only if the reason message equals to "Sign in to confirm your age" and if the age limit is 18.
2021-03-05 16:38:49 +01:00
TiA4f8R 59d6d3f04e
Fix YouTube tests 2021-03-05 16:38:42 +01:00
TiA4f8R 3925204658
Add tests and rename SoundcloudGoPlusException to SoundcloudGoPlusContentException 2021-03-05 16:38:40 +01:00
TiA4f8R 771bb1a2cb
Add three new exceptions for unavailable YouTube contents
These exceptions are thrown on a test with the error messages text, because YouTube returns only "UNPLAYABLE" status in most error cases.
Tests are based with English strings, so changing the lang used by
extractor will throw the generic exception (ContentNotAvailableException).
2021-03-05 16:38:39 +01:00
TiA4f8R 35325d980d
Add GeographicRestrictionException and SoundCloudGoPlusException in NewPipe Extractor to be able to display different error messages
This commit adds two new exceptions in NewPipe Extractor: GeographicRestrictionException and SoundCloudGoPlusException (which extend to ContentNotAvailableException). These exceptions allow showing different error messages to user when a content isn't available in his/her/its country (only used for now by SoundCloudStreamExtractor) or when the content is a SoundCloud Go+ track.
2021-03-05 16:38:37 +01:00
Tobi bc0cda68d1
Merge pull request #570 from TiA4f8R/fix-yt-channels-playlists
Fix YouTube channels contination
2021-03-05 15:26:49 +01:00
Tobi a3c6fceef5
Merge pull request #573 from B0pol/comments-performance
[youtube] improve comments extraction performance
2021-03-05 14:46:27 +01:00
bopol ff5273b882
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeCommentsExtractor.java
Co-authored-by: Tobi <TobiGr@users.noreply.github.com>
2021-03-05 14:39:01 +01:00
Tobi ec1127dd6a
Merge pull request #571 from TeamNewPipe/soundcloud_client_id
[SoundCloud] Update client_id and fix extraction
2021-03-05 14:21:14 +01:00
bopol ed850d0688 [youtube] improve comments extraction performance
- do not parse responseBody twice for continuation
instead try to get commentsTokenInside with the new pattern ("sectionListRenderer")
and try again with the old pattern ("commentSectionRenderer") on failure
- do not unescape responseBody multiple times
   -> parse responseBody less times
2021-03-05 13:37:08 +01:00
golfinq 05213175cd
minor edit, made variable final 2021-03-05 02:49:15 -05:00
golfinq 8bceb57a6c
formatting fix 2021-03-05 02:46:06 -05:00
golfinq ae48bdea4c
Updated to reflect variable name change 2021-03-05 02:40:36 -05:00
TiA4f8R b27efdc9a4
Remove clickTrackingParams in getNextPage method of YoutubeChannelExtractor
Because it's unused with the new request.

Co-authored-by: bopol <bopol@e.email>
2021-03-04 22:50:20 +01:00
TobiGr 2b671b15ce [YouTube] Use new continuation API for channels
Co-authored-by: TiA4f8R <74829229+tia4f8r@users.noreply.github.com>
2021-03-04 21:47:16 +01:00
TobiGr 33594d4aff [SoundCloud] Fix client id extraction
SoundCloud changed their Javascript structure. That caused the client_id to be in the middle of the file (at byte ~43000 ). To have a little buffer, we now fetch the first 50000 bytes.
2021-03-04 21:32:34 +01:00
TobiGr fb2f37d223 [SoundCloud] Update hardcoded client id 2021-03-04 21:25:41 +01:00
XiangRongLin 03b00ff1d6 Move toJsonArray and toJsonObject to JsonUtils 2021-03-04 18:58:51 +01:00
FireMasterK 668b080b3a Use dynamic key. 2021-03-04 08:45:14 +01:00
XiangRongLin 43b46bd408 Use new youtube continuations api for playlists
Requires sending a POST request instead of GET.
clientName and clientVersion, which were required as headers previously now need to be part of the request payload.
continuation id also needs to be part of request body.

quick and dirty solution.
2021-03-03 19:49:26 +01:00
FireMasterK 5333d8a98b
Add more data. 2021-02-19 00:55:03 +05:30
bopol 1a322ad8ed Add uploader verified by service extraction 2021-02-18 16:36:40 +01:00
FireMasterK c24afa2cbb Permalink URL. 2021-02-18 20:16:53 +05:30
FireMasterK 8d54401233 Add fps field as well. 2021-02-18 20:16:53 +05:30
FireMasterK 5b70645d85 Add data to muxed streams aswell. 2021-02-18 20:16:53 +05:30
FireMasterK e975d33fbe Change jdoc / comments. 2021-02-18 20:16:53 +05:30
FireMasterK f31b2a68fd use getters/setters. 2021-02-18 20:16:53 +05:30
FireMasterK d4945ac55c Constructor cleanup. 2021-02-18 20:16:52 +05:30
FireMasterK 3f6a601be8 Fix jdoc / comments. 2021-02-18 20:16:52 +05:30
FireMasterK a7c9905183 Fix requested changes. 2021-02-18 20:16:52 +05:30
FireMasterK 0c0f2d74bc Make fields private
Co-authored-by: bopol <bopol@e.email>
2021-02-18 20:16:52 +05:30
FireMasterK d1054338d3 Fix error with livestreams. 2021-02-18 20:16:52 +05:30
FireMasterK 11eb4932f4 Add data to respective classes. 2021-02-18 20:16:52 +05:30
FireMasterK 525e345ed8 remove duplicate field. 2021-02-18 20:16:52 +05:30
FireMasterK e844d2aed3 Add dash fields to the ItagItem. 2021-02-18 20:16:52 +05:30
bopol 64f6b0478f
Merge pull request #545 from XiangRongLin/comment_reset
Add comment explaining YoutubeParsingHelper.resetClientVersionAndKey
2021-02-18 15:14:03 +01:00
XiangRongLin 26d50aca05 Add comment explaining YoutubeParsingHelper.resetClientVersionAndKey 2021-02-17 19:21:39 +01:00
bopol 84380e4c3a
Merge pull request #536 from B0pol/watchendpoints
support /watch/, /v/ and /w/ subpaths
2021-02-16 20:23:27 +01:00
bopol 41f689b099 Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/linkHandler/YoutubeStreamLinkHandlerFactory.java
Co-authored-by: Stypox <stypox@pm.me>
2021-02-16 19:17:10 +01:00
bopol 7a3d9bdb7d Description: add .equals() and .hashCode() 2021-02-15 18:51:47 +01:00
bopol 557934cb17 use EMPTY_STRING 2021-02-15 18:51:46 +01:00
bopol 9ca52ca68a use only one constant UTF-8 2021-02-15 18:47:54 +01:00
XiangRongLin 50e5718bd1
Merge pull request #518 from XiangRongLin/generate_mocks
Use and generate mocks for youtube tests
2021-02-15 06:39:41 +01:00
bopol 8d89c82caa Soundcloud: fix charts extraction when georestricted 2021-02-13 22:28:31 +01:00
bopol ee2012c948 rename folder to subpath 2021-02-13 22:25:46 +01:00
bopol 8db40db4de use substring instead of split 2021-02-13 22:20:05 +01:00
bopol 81317cf6bc youtube: support /w/ folder 2021-02-13 22:20:05 +01:00
bopol f15c0fcfed support /watch/ and /v/ folders 2021-02-13 22:20:05 +01:00
XiangRongLin b43f3474f5 Make resetClientVersionAndKey public 2021-02-13 20:00:49 +01:00
bopol 2859c7ba5c add comment explaining why .substring(23) 2021-02-13 12:10:41 +01:00