f4404b5cc8
Add tests for more reasons
2021-05-01 17:50:19 +02:00
fc998589dc
Detect channels which have been terminated due to copyright infringement
2021-05-01 17:50:19 +02:00
bb3861ddce
Add AccountTerminatedException for better error handling of terminated channels
2021-05-01 17:50:19 +02:00
9c12dc5609
[SoundCloud] Fix SoundCloud ID extraction
...
resolveIdWithEmbedPlayer() does not work anymore because the JSON data has been extracted to an API call. For this reason, replace resolveIdWithEmbedPlayer() with resolveIdWithWidgetApi)( which performs the API call.
2021-04-26 17:58:34 +02:00
9dc7a1d5ee
[Bandcamp] Add comments (first page)
2021-04-26 17:25:04 +02:00
8f023c1ec7
[SoundCloud] Update client id
2021-04-26 15:14:11 +02:00
7f202db8b1
Fix Google URL detection ( #617 )
2021-04-26 11:49:47 +02:00
6db4bea8ca
Rework Bandcamp comments ( #613 )
2021-04-13 21:10:59 +02:00
3671876721
Fix bandcamp radio stream extractor
2021-04-13 19:30:11 +02:00
90b5c00599
Extract related items #593
2021-04-13 17:56:19 +02:00
318cec7625
[Bandcamp] Add Radio stream segments
2021-04-11 21:19:44 +02:00
55e0f8e725
Merge branch 'master' into dev
2021-04-10 11:33:34 +02:00
33173eb3e6
Make CONSENT COOKIE constants private
2021-04-09 11:59:17 +02:00
4833df60e5
Add method to inject Random into YoutubeParsingHelper and use in tests
2021-04-08 16:36:55 +02:00
080d1e318d
Code improvement
2021-04-07 14:11:47 +02:00
883f16e0ad
[YouTube] Set CONSENT cookie
2021-04-07 14:11:37 +02:00
7b06c696e2
Use Page.body for YoutubeChannelExtractor
2021-04-06 18:14:38 +02:00
eda1b6e199
Use Page.body for YoutubePlaylistExtractor
2021-04-06 18:14:38 +02:00
3af26a2821
Add body field to Page
2021-04-04 11:39:58 +02:00
1925dcf4dc
Merge pull request #597 from fynngodau/related-items
...
Rename getRelatedStreams to getRelatedItems and change return type
2021-04-02 19:54:22 +02:00
c877712647
Add deprecated old method calls to avoid breaking API
2021-04-01 22:32:36 +02:00
14f6f1b7c3
Generify related streams calls and rename method
2021-03-31 20:24:40 +02:00
1498e1905e
Reduce number of type conversions in YouTubeSearchExtractor.collectStreamsFrom by introducing new variable
2021-03-31 13:30:38 +02:00
7ea2cd73d3
Merge pull request #589 from fynngodau/bandcamp-more-recent-pages
...
Load more featured pages and fix featured cover arts not loading
2021-03-31 12:15:12 +02:00
705f6c6e33
Apply review
2021-03-31 11:02:30 +02:00
657b00ca11
Smaller code improvements
2021-03-30 10:10:53 +02:00
8fbd1a2bcf
FIx potential NullPointerException
2021-03-30 10:07:20 +02:00
f2e7c7ebd6
Replace Vector with ArrayList for performance improvements
...
I could not figure out, why these two methods are necessary.
2021-03-30 10:07:20 +02:00
48fd8ca7b3
Remove redundant type declaration
2021-03-30 10:07:20 +02:00
22bd16fca3
Make vars private, final and static
2021-03-30 10:07:20 +02:00
6a0f6e846a
Fix possible bug when language.id field is not present and this the languageCode an empty String.
2021-03-30 10:07:20 +02:00
b9282bbe94
Remove checks which are always true
2021-03-30 10:07:20 +02:00
46eab1ec17
Remove declarations of exceptions which are not thrown from method signatures
2021-03-30 10:07:20 +02:00
c3e23559d7
Remove unused imports
2021-03-30 10:07:20 +02:00
e4c40ae6d1
Use List.isEmpty()
2021-03-30 10:07:20 +02:00
d663be5a78
[Bandcamp] Fix featured albums not showing thumbnails
2021-03-28 13:12:22 +02:00
dbcf61c6f7
[Bandcamp] Load more featured pages
2021-03-28 12:53:05 +02:00
b4dee6d08f
Merge pull request #306 from B0pol/metadata
...
Extract metadata for youtube, soundcloud & mediaccc
2021-03-27 08:45:47 +01:00
070a40e181
[YouTube] Mix Playlist - Fix getting name
2021-03-25 21:47:16 +01:00
bbee15474d
[PeerTube] Fix playlist extraction
2021-03-25 19:19:19 +01:00
ae283314da
Add a comment explaining subscribers count calculation
2021-03-25 17:54:46 +01:00
abb790f465
Remove println in getSubscriberCount from PeertubeAccountExtractor class
2021-03-25 17:49:22 +01:00
efe2b964f0
Extract real subscribers of a Peertube account
...
Apply fix provided in the PR and move all accounts/ strings to a constant, ACCOUNTS.
2021-03-25 17:47:44 +01:00
12835bfae1
Merge pull request #551 from FireMasterK/ff-privacy
...
Change UA to privacy.resistFingerprinting.
2021-03-24 17:58:34 +01:00
ce7cbbc9a0
[YouTube] Fix trending getName()
2021-03-24 09:14:09 +01:00
8806fb4e6b
address reviews
2021-03-19 10:14:43 +01:00
c47cc54908
Extract metadata for YouTube, SoundCloud & MediaCCC
2021-03-19 10:14:43 +01:00
379d7312fa
Don't use a regular expression to find the last segment URL and do code improvements
...
Apply suggestions provided in the PR and remove a redundant import.
2021-03-14 17:54:22 +01:00
0e3e420a25
Fix tests
2021-03-14 17:54:21 +01:00
d61d9d116d
Refactor getAudioStreams method of SoundcloudStreamExtractor
...
Split the method into private methods, in order to have a better reading.
2021-03-14 17:54:20 +01:00
a7b15b51e6
Change t to transcodingJsonObject in SoundcloudStreamExtractor
2021-03-14 17:54:19 +01:00
3bd08a2880
Adress requested changes and use final where possible in SoundcloudStreamExtractor
...
This commit moved the HLS parsing task to a separate method, did little performance improvements and used final where possible in the SoundcloudStreamExtractor file.
2021-03-14 17:54:17 +01:00
cbacd3c0a5
Add a check to don't show MP3 128kbps stream twice and catch IOException when fetching the HLS Manifest
...
If a progressive stream is present in the transcodings, it's unnecessary to show twice an MP3 128kbps stream so if this is the case, the MP3 HLS stream will be not added to the audioStreams, else it will.
This commit also catch fetching errors in HLS manifests parsing and don't add the corresponding stream if an error occurs.
2021-03-14 17:54:16 +01:00
26f1b4e7dc
Support SoundCloud HLS by using a workaround
...
This commit tries to support SoundCloud HLS streams by parsing M3U manifests, get the last segment URL (in order to get track length) and request a segment URL equals to track's duration so it's a single URL.
2021-03-14 17:54:14 +01:00
2e57a8f24f
[Bandcamp] Fix link handler acceptance behaviour
...
* Test for bandcamp footer instead of meta tag (which is not present on
all pages)
* Accept links to /music, not just /releases
* Correctly handle uppercase URLs
2021-03-14 09:51:30 +01:00
7e6f464407
Merge pull request #509 from TiA4f8R/soundcloud-improvements
...
Add new exceptions to be able to display different error messages in apps
2021-03-07 13:51:15 +01:00
22fa131922
Merge branch 'dev' into bandcamp
2021-03-05 19:45:37 +01:00
e55284bb8f
Code style improvements
...
Use final where possible in YoutubeStreamExtractor and do some other code style improvements
2021-03-05 16:38:53 +01:00
d0a1041afd
Prevent NullPointerException and use equalsIgnorecase
2021-03-05 16:38:52 +01:00
890cbba625
Update PaidContent exception
2021-03-05 16:38:51 +01:00
da3cfa967d
Handle age-restricted videos
2021-03-05 16:38:50 +01:00
448b68700c
Use final in exceptions and add AgeRestrictedException
...
AgeRestrictedException will be thrown only if the reason message equals to "Sign in to confirm your age" and if the age limit is 18.
2021-03-05 16:38:49 +01:00
59d6d3f04e
Fix YouTube tests
2021-03-05 16:38:42 +01:00
3925204658
Add tests and rename SoundcloudGoPlusException to SoundcloudGoPlusContentException
2021-03-05 16:38:40 +01:00
771bb1a2cb
Add three new exceptions for unavailable YouTube contents
...
These exceptions are thrown on a test with the error messages text, because YouTube returns only "UNPLAYABLE" status in most error cases.
Tests are based with English strings, so changing the lang used by
extractor will throw the generic exception (ContentNotAvailableException).
2021-03-05 16:38:39 +01:00
35325d980d
Add GeographicRestrictionException and SoundCloudGoPlusException in NewPipe Extractor to be able to display different error messages
...
This commit adds two new exceptions in NewPipe Extractor: GeographicRestrictionException and SoundCloudGoPlusException (which extend to ContentNotAvailableException). These exceptions allow showing different error messages to user when a content isn't available in his/her/its country (only used for now by SoundCloudStreamExtractor) or when the content is a SoundCloud Go+ track.
2021-03-05 16:38:37 +01:00
bc0cda68d1
Merge pull request #570 from TiA4f8R/fix-yt-channels-playlists
...
Fix YouTube channels contination
2021-03-05 15:26:49 +01:00
a3c6fceef5
Merge pull request #573 from B0pol/comments-performance
...
[youtube] improve comments extraction performance
2021-03-05 14:46:27 +01:00
ff5273b882
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeCommentsExtractor.java
...
Co-authored-by: Tobi <TobiGr@users.noreply.github.com>
2021-03-05 14:39:01 +01:00
ec1127dd6a
Merge pull request #571 from TeamNewPipe/soundcloud_client_id
...
[SoundCloud] Update client_id and fix extraction
2021-03-05 14:21:14 +01:00
ed850d0688
[youtube] improve comments extraction performance
...
- do not parse responseBody twice for continuation
instead try to get commentsTokenInside with the new pattern ("sectionListRenderer")
and try again with the old pattern ("commentSectionRenderer") on failure
- do not unescape responseBody multiple times
-> parse responseBody less times
2021-03-05 13:37:08 +01:00
05213175cd
minor edit, made variable final
2021-03-05 02:49:15 -05:00
8bceb57a6c
formatting fix
2021-03-05 02:46:06 -05:00
ae48bdea4c
Updated to reflect variable name change
2021-03-05 02:40:36 -05:00
b27efdc9a4
Remove clickTrackingParams in getNextPage method of YoutubeChannelExtractor
...
Because it's unused with the new request.
Co-authored-by: bopol <bopol@e.email>
2021-03-04 22:50:20 +01:00
2b671b15ce
[YouTube] Use new continuation API for channels
...
Co-authored-by: TiA4f8R <74829229+tia4f8r@users.noreply.github.com>
2021-03-04 21:47:16 +01:00
33594d4aff
[SoundCloud] Fix client id extraction
...
SoundCloud changed their Javascript structure. That caused the client_id to be in the middle of the file (at byte ~43000 ). To have a little buffer, we now fetch the first 50000 bytes.
2021-03-04 21:32:34 +01:00
fb2f37d223
[SoundCloud] Update hardcoded client id
2021-03-04 21:25:41 +01:00
03b00ff1d6
Move toJsonArray and toJsonObject to JsonUtils
2021-03-04 18:58:51 +01:00
668b080b3a
Use dynamic key.
2021-03-04 08:45:14 +01:00
43b46bd408
Use new youtube continuations api for playlists
...
Requires sending a POST request instead of GET.
clientName and clientVersion, which were required as headers previously now need to be part of the request payload.
continuation id also needs to be part of request body.
quick and dirty solution.
2021-03-03 19:49:26 +01:00
a1688fe953
Move BandcampExtractorHelper.getJsonData(String, String) to JsonUtils
2021-02-21 13:51:12 +01:00
70814dcfef
Fix Utils.nonEmptyAndNullJoin
...
When using the index here, it the index needs to be decremented once an element is removed. To cirecumvent this, the native Collections.removeIf() method is used.
2021-02-21 12:51:47 +01:00
b9e8ee8450
Rename BandcampExtractorHelper.smartConcatenate(String[], String) to Utils.nonEmptyAndNullJoin(String, String[])
2021-02-21 12:35:14 +01:00
c07db80ef0
Add BASE_URL and BASE_API_URL to BandcampExtractorHelper
2021-02-21 12:15:45 +01:00
adde4332d1
Code improvements
2021-02-20 23:58:54 +01:00
e8bccfaf5d
Change UA to privacy.resistFingerprinting.
2021-02-20 17:14:41 +05:30
ea120a4637
remove print stacktrace
...
use interface
2021-02-20 00:23:45 +01:00
02920fafa8
Add isUploaderVerified()
2021-02-19 15:51:23 +01:00
91e9309486
Merge remote-tracking branch 'origin/dev' into bandcamp
2021-02-19 13:37:45 +01:00
fa61b864f2
Code improvements
2021-02-19 13:37:24 +01:00
98268e351c
Move radio URL check into a function
2021-02-19 13:35:44 +01:00
5333d8a98b
Add more data.
2021-02-19 00:55:03 +05:30
1a322ad8ed
Add uploader verified by service extraction
2021-02-18 16:36:40 +01:00
c24afa2cbb
Permalink URL.
2021-02-18 20:16:53 +05:30
8d54401233
Add fps field as well.
2021-02-18 20:16:53 +05:30
5b70645d85
Add data to muxed streams aswell.
2021-02-18 20:16:53 +05:30
e975d33fbe
Change jdoc / comments.
2021-02-18 20:16:53 +05:30
f31b2a68fd
use getters/setters.
2021-02-18 20:16:53 +05:30
d4945ac55c
Constructor cleanup.
2021-02-18 20:16:52 +05:30
3f6a601be8
Fix jdoc / comments.
2021-02-18 20:16:52 +05:30
a7c9905183
Fix requested changes.
2021-02-18 20:16:52 +05:30
0c0f2d74bc
Make fields private
...
Co-authored-by: bopol <bopol@e.email>
2021-02-18 20:16:52 +05:30
d1054338d3
Fix error with livestreams.
2021-02-18 20:16:52 +05:30
11eb4932f4
Add data to respective classes.
2021-02-18 20:16:52 +05:30
525e345ed8
remove duplicate field.
2021-02-18 20:16:52 +05:30
e844d2aed3
Add dash fields to the ItagItem.
2021-02-18 20:16:52 +05:30
64f6b0478f
Merge pull request #545 from XiangRongLin/comment_reset
...
Add comment explaining YoutubeParsingHelper.resetClientVersionAndKey
2021-02-18 15:14:03 +01:00
26d50aca05
Add comment explaining YoutubeParsingHelper.resetClientVersionAndKey
2021-02-17 19:21:39 +01:00
54aa5b3042
Use propper structure in KioskExtractors
...
Made BandCampRadioExtractor a Kiosk which holds StreamInfoItems and not InfoItems.
2021-02-16 21:09:42 +01:00
54b8e54f80
Fix potential NPE
2021-02-16 20:59:13 +01:00
16973126a0
Use Collections.emptyList() instead of Collections.EMPTY_LIST or new ArrayList()
2021-02-16 20:25:48 +01:00
84380e4c3a
Merge pull request #536 from B0pol/watchendpoints
...
support /watch/, /v/ and /w/ subpaths
2021-02-16 20:23:27 +01:00
41f689b099
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/linkHandler/YoutubeStreamLinkHandlerFactory.java
...
Co-authored-by: Stypox <stypox@pm.me>
2021-02-16 19:17:10 +01:00
5bf9fddba9
Code improvements
2021-02-15 22:45:17 +01:00
e062c8cb0d
Merge branch 'dev' into bandcamp
2021-02-15 22:09:41 +01:00
7a3d9bdb7d
Description: add .equals() and .hashCode()
2021-02-15 18:51:47 +01:00
557934cb17
use EMPTY_STRING
2021-02-15 18:51:46 +01:00
9ca52ca68a
use only one constant UTF-8
2021-02-15 18:47:54 +01:00
50e5718bd1
Merge pull request #518 from XiangRongLin/generate_mocks
...
Use and generate mocks for youtube tests
2021-02-15 06:39:41 +01:00
8d89c82caa
Soundcloud: fix charts extraction when georestricted
2021-02-13 22:28:31 +01:00
ee2012c948
rename folder to subpath
2021-02-13 22:25:46 +01:00
8db40db4de
use substring instead of split
2021-02-13 22:20:05 +01:00
81317cf6bc
youtube: support /w/ folder
2021-02-13 22:20:05 +01:00
f15c0fcfed
support /watch/ and /v/ folders
2021-02-13 22:20:05 +01:00
b43f3474f5
Make resetClientVersionAndKey public
2021-02-13 20:00:49 +01:00
2859c7ba5c
add comment explaining why .substring(23)
2021-02-13 12:10:41 +01:00
73353996d7
fix more tests
2021-02-12 23:18:38 +01:00
a931e31239
fix tests
2021-02-12 22:36:18 +01:00
d116680ea3
Merge branch 'master' into dev
2021-02-12 22:26:35 +01:00
b0f356dd76
YT channels: fix continuation
2021-02-12 14:39:09 +01:00
fcdb50b825
make getInfo from Extractor public
...
In StreamInfo and CommentsInfo
2021-02-09 14:58:43 +01:00
44c54d403a
fix Invidious channels being accepted by YoutubeStreamLinkHandlerFactory ( #527 )
...
fixes https://github.com/TeamNewPipe/NewPipeExtractor/issues/524 , see it for more details
2021-02-05 18:57:58 +01:00
1414a6f178
Fix extraction of YouTube chapters ( #531 )
...
YouTube changed the name of the chapters in the JSON from engagement-panel-macro-markers to engagement-panel-macro-markers-description-chapters, so extracting chapters doesn't work.
2021-02-03 22:07:34 +01:00
bfa639950e
Merge pull request #528 from Stypox/ytmusic-0
...
[YouTube Music] Fix extracting search item view/subscriber count when = 0
2021-01-29 23:23:23 +01:00
d728c4fad0
Remove unused imports
...
I searched for "StandardCharsets" to prevent possible bugs with Android KitKat. The import is unused and is therefor removed.
There are other usages of the StandardCharsets class in the test implementation, but those are not relevant.
2021-01-29 23:16:40 +01:00
a64dfd7343
[YouTube Music] Fix extracting search item view/subscriber count when = 0
2021-01-29 16:44:24 +01:00
2662ceca22
rename getHeartedByCreator and getPinned to isPinned and isHeartedByCreator
2021-01-23 13:06:07 +01:00
66e4eb2f96
extract whether the comment is pinned
2021-01-22 23:59:22 +01:00
67ddfefdc6
keep invidious instances list in the same order
...
as suggested by @TobiGr
2021-01-22 20:58:14 +01:00
53d3f7989b
update invidious instances list
2021-01-22 19:30:25 +01:00
07570554b6
Fix build and optimize imports
2021-01-15 21:55:40 +01:00
78c2113094
Merge remote-tracking branch 'origin/dev' into bandcamp
2021-01-15 21:49:58 +01:00
7c774c84ce
Merge pull request #505 from TeamNewPipe/meida.ccc.de_search_fix
...
[media.ccc.de] Fix NPE in search results if they contain a future talk
2021-01-15 14:16:59 +01:00
c2c4d97efd
Merge pull request #482 from XiangRongLin/mock_mix_pl_test
...
Mock mix pl tests
2021-01-15 12:54:48 +01:00
951159f75f
Merge pull request #504 from TeamNewPipe/media.ccc.de_recent_order
...
[media.ccc.de] Recent kiosk: order streams by upload date
2021-01-14 23:31:09 +01:00
cc51c5f6a3
Merge pull request #507 from vkay94/add-interval-to-stream-frames
...
Add durationPerFrame to Frameset
2021-01-14 20:27:40 +01:00
6ca7123d5d
Add comment for usage of YoutubeParsingHelper.resetClientVersionAndKey
2021-01-14 20:01:52 +01:00
11dcfe638b
Add durationPerFrame to Frameset and getFrameBoundsAt method
2021-01-14 20:01:06 +01:00
6773ec8699
Merge pull request #510 from B0pol/heartedbycreator
...
comments: add heartedByUploader and extract it for YouTube
2021-01-12 19:00:04 +01:00
0c4836c800
[media.ccc.de] Recent kiosk: Add comment clarifying "length" and "duration" fields have the same value
2021-01-12 18:48:42 +01:00
f3c22da618
comments: add heartedByCreator and extract it for youtube
2021-01-11 18:48:08 +01:00
35e299759e
Add method to allow resetting youtube client version and key
...
This is needed so that a request is made for each test class when running multiple at once. This way RecordingDownloader records all necessary requests.
This works as long as tests are run sequentially and not in parallel.
2021-01-10 20:24:50 +01:00
1bcb9c76a7
Generate equals and hashCode for Request
2021-01-10 19:36:24 +01:00
fe3902c669
Un-comment out 360p webm
2021-01-08 19:43:03 +01:00
2d93b23723
[media.ccc.de] Show conference title in search results instead of slug
2021-01-08 14:15:25 +01:00
52f3bd15f2
[media.ccc.de] Add conference title to "recent" kiosk
...
The "conference_title" field was added upstream recently (see https://github.com/voc/voctoweb/issues/495 )
2021-01-08 12:04:39 +01:00
ab62464e86
Merge pull request #495 from B0pol/sepiasearch
...
[PeerTube] Implement sepia search
2021-01-06 11:06:36 +01:00
0bcea9c2cd
add JDoc
2021-01-01 14:37:32 +01:00
0e45c25ea9
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/peertube/extractors/PeertubeStreamInfoItemExtractor.java
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2021-01-01 14:20:47 +01:00
fae67fbd45
Merge remote-tracking branch 'origin/master' into dev
2021-01-01 13:06:11 +01:00
d63d20f943
Fix JDoc
2020-12-31 18:05:40 +01:00
a896ec41b9
[media.ccc.de] Use Pattern for getting the ive stream id
2020-12-31 17:03:23 +01:00
5dc9a76e3c
[media.ccc.de] Recent kiosk: order streams by upload date
2020-12-30 18:38:09 +01:00
edf8dd0e92
[media.ccc.de] Fix NPE in search results if they contain a future talk
2020-12-30 18:34:28 +01:00
3c8c8e7307
Merge pull request #497 from TiA4f8R/fix-player-js-url-redirect
...
Fix YouTube player JavaScript URL redirection
2020-12-27 18:04:40 +01:00
d30caeb091
Fix YouTube player JavaScript URL redirection
2020-12-27 15:55:03 +01:00
deb9af7bf5
[media.ccc.de] live kiosk - display info of the current talk
2020-12-27 15:52:09 +01:00
b4e1913971
[media.ccc.de] Play live streams
2020-12-27 13:25:48 +01:00
80f4d42226
[Draft] Add live extractor
2020-12-27 11:01:22 +01:00
ed9402c002
[media.ccc.de] Escape "." in id pattern
2020-12-27 11:01:10 +01:00
674f2227f3
make url pattern static
2020-12-27 01:27:37 +01:00
0cfefe222a
Revert "[media.ccc.de] Get conference name from API URL"
...
This reverts commit d1a0686d2f
2020-12-26 16:55:44 +01:00
d1a0686d2f
[media.ccc.de] Get conference name from API URL
2020-12-26 13:15:03 +01:00
41699798fe
[media.ccc.de] Use web URL for uploaderUrl instead of API URL in recent
2020-12-26 12:47:32 +01:00
ed6ae329b3
[media.ccc.de] Add "recent" kiosk
2020-12-25 22:26:58 +01:00
b5e50cc9fb
Merge pull request #484 from TeamNewPipe/yt_,music_search
...
Fix YouTube Music search
2020-12-25 18:30:25 +01:00
d9e2da53c3
Replace explicit string checks whether a playlist íd is a certain YouTube Mix type with calling the dedicated methods
2020-12-25 15:00:34 +01:00
abee0a8df1
Fix typos in YoutubePlaylistLinkHandlerFactory
2020-12-24 10:34:21 +01:00
3033c0b993
[YouTube] Music Mix: Fix playlist not being accepted
...
Regression introduced by YouTube Mix support (#280 )
2020-12-23 21:08:36 +01:00
b62144b49d
[YouTube] Music: Fix search extraction caused by changed JSON response
2020-12-23 21:08:36 +01:00
c9d9bd1e24
Fix typo in ListLinkHandlerFactory#getUrl
2020-12-23 21:05:58 +01:00
3ae924a7f1
Implement sepia search
2020-12-23 15:07:55 +01:00
942d840624
Fix typo in AffiliateService: rename "NO_AFILIATE" to "NO_AFFILIATE"
2020-12-23 12:50:45 +01:00
865c42e273
Fix line break
2020-12-23 12:49:50 +01:00
c9e9953bb0
[Bandcamp] Fix channel link handler factory
2020-12-22 20:02:53 +01:00
41a8ed625d
Make some vars final and add annotations to methods
2020-12-22 18:19:26 +01:00
bc6de14952
Extract stream and search meta info for YouTube
...
Add method to extract Google webcache URLs.
2020-12-22 18:19:26 +01:00
50903730b1
[Bandcamp] Fix accepting HTTP URLs
2020-12-22 08:53:10 +01:00
8d3436565f
[peertube] fix related streams api url when no tags
...
fixes #489
2020-12-19 23:25:37 +01:00
22a415156f
Merge pull request #479 from vkay94/stream-segments
...
Extract stream segments for YouTube
2020-12-19 11:12:22 +01:00
74b46fed2d
[Bandcamp] Fix deprecation in parseDate helper function
2020-12-19 10:40:21 +01:00
116e921d6c
Merge remote-tracking branch 'origin/dev' into bandcamp
2020-12-19 00:26:11 +01:00
cdcb66b93e
Clean up stuff from mix pl
...
Coming from unclean merge and missing hints from IDE
2020-12-16 08:35:28 +01:00
ba8782a9ed
Use lowercase string for extracting cookies
...
The map in a real response happens to ignore upper-/lowercase differences. Other maps used in unit test may not have that behaviour.
2020-12-16 08:32:04 +01:00
85fa006214
Merge pull request #280 from XiangRongLin/mixPL
...
Extractor for youtube mix (auto-generated playlist)
2020-12-14 18:11:07 +01:00
f90f6fcf92
[YouTube] Don't escape & in getUrlFromNavigationEndpoint for playlists
2020-12-12 20:40:13 +01:00
a338e4e08e
[Youtube] Apply review suggestions and avoid channel mix edge case
2020-12-12 20:32:43 +01:00
22d2f7e400
[Youtube] Add cookies to youtube mix request
...
This way youtube wont return duplicates when getting more items of the mix (but youtube can also track us)
2020-12-12 20:32:40 +01:00
421935401f
[Youtube] Add subchannel functions to mix and fix imports
2020-12-12 20:30:57 +01:00
3ff8619bcc
[Youtube] apply wb9688 suggestion (mix)
...
Channel mix adjusments and test
Don't accept youtube music mix urls as playlist
Don't override playlistData to keep getInitialPage()
Remove json constants
Indentation
2020-12-12 20:30:57 +01:00
822cf307f7
[Youtube] Add _ITEMS constants and improve code style
...
Move thumbnail id exctraction code to getThumbnailUrlFromId
Add test for "My mix" detection to service tests
Use ITEM_COUNT_UNKNOWN everywhere instead of -1 and add some tests
2020-12-12 20:30:57 +01:00
df38b1926c
[Youtube] Add tests and take thumbnail image always from first video of mix
...
Also fix getThumbnailUrl for "My Mix"
2020-12-12 20:30:57 +01:00
68a3948af6
[Youtube] Fix get banner url
2020-12-12 20:30:57 +01:00
ec6b99c082
[Youtube] Adjust mix extractor to new user agent
...
Also extract continuation now
2020-12-12 20:30:57 +01:00
0ff054acb4
[Youtube] Extract initial playlist info
2020-12-12 20:30:57 +01:00
a376792a5d
[Youtube] Handle case where url is in "youtube.com/playlist?list=listID" format.
...
This occurs when sharing a mix from the official youtube app.
2020-12-12 20:30:57 +01:00
327a5730a8
[Youtube] Add some comments to mix
2020-12-12 20:30:57 +01:00
d74265c846
[Youtube] Extract getThumbnailUrl into method and change getUploaderName
2020-12-12 20:30:57 +01:00
0efb854d27
[Youtube] Implement mix extractor for auto-generated playlists.
...
-New YoutubeMixPlaylistExtractor, that extracts from a mix (auto-generated playlist).
-The url has the format of "youtube.com/watch?v=videoID&playlistID",
where playlistID always starts with "RD" and usually followed by the videoID.
-Change YoutubePlaylistLinkHandlerFactory to create a linkhandler with the given url if it is a mix.
-Change YoutubeService to return YoutubeMixPlaylistExtractor if the url is a mix.
2020-12-12 20:30:57 +01:00
2ba27b39af
Stream segments: Remove replaceAll from setPreviewUrl
2020-12-12 15:00:45 +01:00
d3eea4f4be
Add stream segments to StreamInfo
2020-12-12 10:24:29 +01:00
57be1f1123
Use Objects.requireNonNull().
2020-12-11 06:09:57 +05:30
4c19a88612
Use Objects.toString().
2020-12-11 06:09:50 +05:30
b8f64595a2
Use Objects' static equals() and hashCode() methods.
2020-12-11 06:09:44 +05:30
9dbacbc618
Fix comments parsing
2020-12-09 14:57:38 +01:00
04dd3d4d32
Rework link handlers to correctly accept external websites
2020-12-05 15:08:26 +01:00
ba3e2302bc
Fix name of media.ccc.de service
2020-12-01 23:54:27 +01:00
b7a995187f
Merge pull request #463 from opusforlife2/update_invidious_URLs
...
Update Invidious URL List
2020-11-28 13:30:32 +01:00
99e7ef013e
[Bandcamp] Apply small changes to code formatting and style
...
Make variables final when possible
Remove unused imports
Improve code formatting
2020-11-24 14:13:31 +01:00
8c369b0f79
Rephrase link in javadoc
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-11-24 14:01:31 +01:00
186936d041
Various changes regarding tests
2020-11-19 22:33:52 +01:00
6bc7e3420e
Merge TNP/dev into fynngodau/dev
2020-11-19 21:32:08 +01:00
5bceff0083
[YouTube] Fix extraction of next page url for the last page of playlist
2020-11-18 19:03:12 +01:00
8347e14952
[YouTube] Fix playlist continuations extraction
2020-11-17 21:04:53 +01:00
2174055ef8
Update Invidious URL list in Parsing Helper
2020-11-11 15:12:31 +00:00
c0afd5213a
Update Invidious URL list in Link Handler Factory
2020-11-11 14:54:16 +00:00
89a77ae74a
[YouTube] Fix detection of ended livestreams and parse livestream upload date
2020-11-04 16:03:08 +01:00
827f7bd137
[YouTube] Cache deobfuscation and improve requests made
...
Fix age restriction extraction
Automatically fixes more things
2020-11-04 16:02:12 +01:00
4fe28d7e3a
Fix YouTube parse error when only a date is present.
2020-11-04 05:49:42 +05:30
9cf9e7e980
Call existing constructor in DateWrapper.
2020-11-04 05:49:41 +05:30
fe31a90cb3
Remove DateTimeFormatter.ISO_OFFSET_DATE_TIME usage.
2020-11-04 05:49:41 +05:30
b13c7e1c1e
Merge pull request #452 from Stypox/yt-import
...
Implement YouTube subscription import from Google takeout
2020-11-03 20:32:17 +01:00
501ec30152
Implement youtube subscription import from Google takeout
2020-11-02 11:04:48 +01:00
82746d172f
Fix typo in DonationLinkHelper and rewrote swtich statement
2020-11-01 17:34:34 -05:00
be9e160333
Fix build
2020-11-01 17:04:02 +01:00
4f04cfccca
Switch from Calendar to OffsetDateTime in DateWrapper.
2020-11-01 06:28:54 +05:30
ee3af63c04
Switch to ChronoUnit.
2020-11-01 06:28:53 +05:30
6cc50b57e3
Merge pull request #410 from Scrxtchy/sets-track-title
...
Fix parsing Soundcloud tracks that contain the term 'sets'
2020-10-31 21:43:21 +01:00
b242e1d113
Merge pull request #362 from Stypox/google-search-urls
...
Add support for Google search redirect url
2020-10-31 21:03:48 +01:00
9e53cf0b56
Fix parameter reassignment and other style issues
...
Also remove left-behind debug statement
2020-10-31 20:58:28 +01:00
3fe55b30ba
Add support for Google search redirect url
2020-10-31 20:58:28 +01:00
c190a3029b
Consider protocol as base url when it is a custom one (e.g. vnd.youtube)
2020-10-31 20:58:28 +01:00
b21e59925d
[PeerTube] fix account and channel extractors
2020-10-29 19:52:29 +01:00
30ed4f2d63
Remove any reference to decrypt and improve error message
2020-10-28 20:26:00 +01:00
62912ee834
Merge branch 'master' into dev
2020-10-28 14:28:06 +01:00
947ce3ee10
Fix parsing Soundcloud tracks that contain the term 'sets'
2020-10-28 07:55:02 +11:00
6dc5ab4015
find playerUrl in another place when assetsPattern regex fails
2020-10-27 13:48:58 +01:00
db0ef83d6b
fix youtube decryption and three attemps bug
...
fixes teamnewpipe/newpipe#4572 fixes #439
2020-10-27 13:40:24 +01:00
c1e9857960
fix subscriber count when subscribe is disabled
...
fixes #305
2020-10-26 19:06:01 +01:00
a39a2cca82
fix redirect channels
2020-10-26 19:05:19 +01:00
01f49e8f66
polish strings
2020-10-26 16:32:39 +01:00
14c0c37d64
Merge pull request #309 from Stypox/improve-stream-tests
...
Improve stream extractor tests & various fixes
2020-10-25 18:49:53 +01:00
366ed71523
[YouTube] Ignore leading characters in video id
2020-10-25 14:34:20 +01:00
8026304a0a
Merge pull request #409 from Bartoshr/bugfix/short-channel-urls
...
Support short custom YouTube channel URLs
2020-10-25 11:43:50 +01:00
57e7994c9e
Add some missing finals, nullables and comments
2020-10-25 08:12:41 +01:00
d0b14644bb
[YouTube/MediaCCC] Consider dates as GMT and not as local
2020-10-24 18:41:41 +02:00
a4097d8d01
[MediaCCC] Return empty list of video-only streams instead of null
2020-10-24 18:39:55 +02:00
68d23defba
[YouTube] Do not catch every exception on getErrorMessage
...
@B0pol suggestion
2020-10-24 18:39:55 +02:00
3191bd6c70
[YouTube] Use final when possible
2020-10-24 18:39:55 +02:00
55bc01d1ce
[SoundCloud] Use final when possible, ide refactorings
2020-10-24 18:39:55 +02:00
06430c4749
[PeerTube] Use final when possible, ide refactorings
2020-10-24 18:39:06 +02:00
fcb9b6f855
[MediaCCC] Use final when possible, ide refactorings
...
Refactorings on `throws` clause
2020-10-24 18:36:22 +02:00
07a90d116a
[MediaCCC] Use regex to parse stream and conference urls
2020-10-24 18:36:21 +02:00
d130fd79c3
[PeerTube] Prepend "accounts/" to channel id for backward compatibility
2020-10-24 18:36:21 +02:00
492db83ccf
[MediaCCC] Return null instead of empty items collector
...
As per the documentation in the base getRelatedStreams()
2020-10-24 18:34:57 +02:00
3b2a1829d4
[MediaCCC] Extract tags
2020-10-24 18:34:57 +02:00
b461da792f
[MediaCCC] Fix link handler inconsistency providing API links
2020-10-24 18:34:34 +02:00
aeeae87641
[PeerTube] Parse timestamp from url (previously unimplemented)
2020-10-24 18:33:42 +02:00
7ae3cb6d07
[PeerTube] Fix link handler inconsistency providing API links
2020-10-24 18:33:11 +02:00
4349be13af
[PeerTube] Return empty audio stream list instead of null
2020-10-24 18:25:56 +02:00
3b2cfb4ca2
[SoundCloud] Return empty video stream list instead of null
...
Also replace every instance of `return new ArrayList<>();` with `return Collections.emptyList();`
2020-10-24 18:23:46 +02:00
072bae321f
[YouTube] Fix frame extraction for livestreams
...
Use saved playerResponse instead of parsing json every time
2020-10-24 18:23:46 +02:00
7cd410f3fc
[YouTube] Return 0 when there is no timestamp, not -2, as per javadoc
2020-10-24 18:23:45 +02:00
7fb867c166
[YouTube] Fix error message obtaining when there is none
2020-10-24 18:22:44 +02:00
29695aed0a
Small field refactor
2020-10-23 16:42:30 +02:00
0e67d820bc
Use static regex pattern for excluded path segments
2020-10-22 20:13:29 +02:00
d3f80d1538
Exlude links which are not channels
2020-10-20 20:06:06 +02:00
6887d59570
[YouTube] Handle urls for Shorts
2020-10-18 12:03:01 +02:00
be9a6f931c
Fix parsing new ytInitialData
2020-10-16 20:27:40 +02:00
1a9bc8caf1
Merge pull request #414 from Scrxtchy/soundcloud-slash
...
Remove tailing slash from SoundCloud URLs
2020-10-16 09:48:44 +02:00
6a70cb9d50
Remove tailing slash from SoundCloud URLs
...
Fixes #412
2020-10-16 17:17:40 +11:00
19e862657a
[YouTube] Fix some decryption exceptions by retrying
2020-10-15 20:16:44 +02:00
e3f996e014
Exlude links which are not channels
2020-10-12 20:59:56 +02:00
7abb4b3713
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/linkHandler/YoutubeChannelLinkHandlerFactory.java
...
Fix typos
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 19:57:45 +02:00
9d63211a66
Fix typos
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 19:56:53 +02:00
8c38a5509e
Prevent attribution_link urls to be accepted by channel links handler
2020-10-09 19:07:38 +02:00
4e04991762
Support short custom youtube channel urls
2020-10-09 17:06:59 +02:00
81b5e7cf3d
Fix extractor
2020-10-08 17:56:03 +02:00
ebbfe7f6d4
Skip YouTube's OTF streams
2020-09-29 10:48:02 +02:00
6633f26ec5
Merge pull request #386 from mhmdanas/remove-subtitles-stream-get-url
...
Remove SubtitlesStream#getURL()
2020-08-26 21:32:36 +02:00
7657c2ed1a
Use initSafeStandardsObjects()
2020-08-15 17:08:07 +02:00
1a63dcb355
Remove overriden field `url`
2020-08-12 22:37:36 +03:00
f0f1c009b2
Remove SubtitlesStream#getURL()
2020-08-10 19:28:56 +03:00
cfe88a74c1
Throw ContentNotSupportedException when opening radio uploader channel
2020-08-09 10:07:49 +02:00
0fb73301e3
[YouTube] Fix crash on empty comment
...
Closes #380
2020-08-05 18:25:40 +02:00
932d094d6a
Upgrade featured playlist urls to https
2020-08-02 16:55:18 +02:00
c12ef3a02d
Merge TNP/dev into fynngodau/dev
2020-08-02 16:43:04 +02:00
5ac80624a4
Fix detection of YouTube's decryption function
...
Closes TeamNewPipe/NewPipe#3951
2020-07-28 01:01:10 +02:00
32dff1541f
Merge pull request #370 from wb9688/fix-yt-age-restricted
...
Fix age restricted YouTube videos
2020-07-26 14:17:19 +02:00
TobiGr
Fynn Godau
XiangRongLin
Stypox
TiA4f8R
bopol
golfinq
FireMasterK
vkay94
Isira Seneviratne
TheAssassin
opusforlife2
Bri@n
Scratch
Kiril Isakov
Bartosz Rumiński
wb9688
Bartosz Rumiński
mhmdanas