Commit Graph

1467 Commits

Author SHA1 Message Date
TobiGr 440a808b8a Update mocks for channels 2021-03-05 13:13:36 +01:00
Tobi b608587e4d
Merge pull request #572 from golfinq/fixed-comments
Fixes missing comments under videos
2021-03-05 11:42:40 +01:00
golfinq 05213175cd
minor edit, made variable final 2021-03-05 02:49:15 -05:00
golfinq 8bceb57a6c
formatting fix 2021-03-05 02:46:06 -05:00
golfinq ae48bdea4c
Updated to reflect variable name change 2021-03-05 02:40:36 -05:00
TiA4f8R b27efdc9a4
Remove clickTrackingParams in getNextPage method of YoutubeChannelExtractor
Because it's unused with the new request.

Co-authored-by: bopol <bopol@e.email>
2021-03-04 22:50:20 +01:00
TobiGr 2b671b15ce [YouTube] Use new continuation API for channels
Co-authored-by: TiA4f8R <74829229+tia4f8r@users.noreply.github.com>
2021-03-04 21:47:16 +01:00
TobiGr 33594d4aff [SoundCloud] Fix client id extraction
SoundCloud changed their Javascript structure. That caused the client_id to be in the middle of the file (at byte ~43000 ). To have a little buffer, we now fetch the first 50000 bytes.
2021-03-04 21:32:34 +01:00
TobiGr fb2f37d223 [SoundCloud] Update hardcoded client id 2021-03-04 21:25:41 +01:00
XiangRongLin 9256b3b848
Merge pull request #567 from XiangRongLin/playlist_continuations
Playlist continuations
2021-03-04 19:05:32 +01:00
XiangRongLin 506cc5fbe9 regenerate mocks 2021-03-04 19:01:58 +01:00
XiangRongLin 03b00ff1d6 Move toJsonArray and toJsonObject to JsonUtils 2021-03-04 18:58:51 +01:00
FireMasterK 668b080b3a Use dynamic key. 2021-03-04 08:45:14 +01:00
XiangRongLin 43b46bd408 Use new youtube continuations api for playlists
Requires sending a POST request instead of GET.
clientName and clientVersion, which were required as headers previously now need to be part of the request payload.
continuation id also needs to be part of request body.

quick and dirty solution.
2021-03-03 19:49:26 +01:00
XiangRongLin beb05bd05c Replace test url, because playlist was deleted 2021-03-03 19:46:14 +01:00
XiangRongLin e13e237392 Fix typo and reword some explanations 2021-02-26 17:48:03 +01:00
Tobi 5d594cfded
Merge pull request #549 from FireMasterK/gradle-update
Update gradle wrapper.
2021-02-24 15:59:28 +01:00
Tobi 965a66bc87
Merge pull request #557 from TiA4f8R/fix-encoding-errors-windows
Set extractor encoding to UTF-8
2021-02-24 15:49:58 +01:00
TiA4f8R b2cf41496d
Set extractor encoding to UTF-8
When compiling the extractor on Windows, you get more than 100 errors with the message error: unmappable character for encoding Cp1252, especially in timeago-parser tasks. This commit fixes it by setting the encoding for all compileJava and compileTestJava tasks to UTF-8.
2021-02-23 14:38:20 +01:00
TobiGr a1688fe953 Move BandcampExtractorHelper.getJsonData(String, String) to JsonUtils 2021-02-21 13:51:12 +01:00
TobiGr 70814dcfef Fix Utils.nonEmptyAndNullJoin
When using the index here, it the index needs to be decremented once an element is removed. To cirecumvent this, the native Collections.removeIf() method is used.
2021-02-21 12:51:47 +01:00
TobiGr b9e8ee8450 Rename BandcampExtractorHelper.smartConcatenate(String[], String) to Utils.nonEmptyAndNullJoin(String, String[]) 2021-02-21 12:35:14 +01:00
TobiGr c07db80ef0 Add BASE_URL and BASE_API_URL to BandcampExtractorHelper 2021-02-21 12:15:45 +01:00
TobiGr adde4332d1 Code improvements 2021-02-20 23:58:54 +01:00
FireMasterK 42bdd8adc5
Update to fix failing mock tests. 2021-02-20 17:14:41 +05:30
FireMasterK e8bccfaf5d
Change UA to privacy.resistFingerprinting. 2021-02-20 17:14:41 +05:30
bopol a9aa385bb3
Merge pull request #554 from B0pol/test-verified
use DownloaderFactory instead of DownloaderTestImpl for search Channe…
2021-02-20 12:12:43 +01:00
bopol 0c831afa7b use DownloaderFactory instead of DownloaderTestImpl for search ChannelVerified 2021-02-20 11:41:30 +01:00
TobiGr ea120a4637 remove print stacktrace
use interface
2021-02-20 00:23:45 +01:00
XiangRongLin adf9d7d10f Add reason field to MockOnly
This enforces developers to document why a test is skipped
2021-02-19 18:54:18 +01:00
XiangRongLin ea52030613 Add MockOnlyRule to allow skipping specific tests based on downloader 2021-02-19 18:54:17 +01:00
TobiGr 02920fafa8 Add isUploaderVerified() 2021-02-19 15:51:23 +01:00
TobiGr 91e9309486 Merge remote-tracking branch 'origin/dev' into bandcamp 2021-02-19 13:37:45 +01:00
TobiGr fa61b864f2 Code improvements 2021-02-19 13:37:24 +01:00
TobiGr 98268e351c Move radio URL check into a function 2021-02-19 13:35:44 +01:00
bopol 27a20e41cd
Merge pull request #550 from FireMasterK/add-more-data
Add more data.
2021-02-18 20:28:27 +01:00
FireMasterK 5333d8a98b
Add more data. 2021-02-19 00:55:03 +05:30
FireMasterK 88890d81a5
Update gradle wrapper. 2021-02-18 21:15:12 +05:30
bopol 9c1a7f7df6
Merge pull request #521 from B0pol/verified
Add uploader verified by service extraction
2021-02-18 16:39:53 +01:00
bopol 1a322ad8ed Add uploader verified by service extraction 2021-02-18 16:36:40 +01:00
bopol db253e202b
Merge pull request #535 from TeamPiped/adaptive-parsing
Adaptive Stream parsing
2021-02-18 15:55:27 +01:00
FireMasterK c24afa2cbb Permalink URL. 2021-02-18 20:16:53 +05:30
FireMasterK 8d54401233 Add fps field as well. 2021-02-18 20:16:53 +05:30
FireMasterK 4d096be14a Revert "Update gradle wrapper."
This reverts commit 591856e3a11998c4b8ba87b5e361e2b9556d760f.
2021-02-18 20:16:53 +05:30
FireMasterK 5b70645d85 Add data to muxed streams aswell. 2021-02-18 20:16:53 +05:30
FireMasterK e975d33fbe Change jdoc / comments. 2021-02-18 20:16:53 +05:30
FireMasterK f31b2a68fd use getters/setters. 2021-02-18 20:16:53 +05:30
FireMasterK d4945ac55c Constructor cleanup. 2021-02-18 20:16:52 +05:30
FireMasterK 3f6a601be8 Fix jdoc / comments. 2021-02-18 20:16:52 +05:30
FireMasterK a7c9905183 Fix requested changes. 2021-02-18 20:16:52 +05:30