f4aad8b014
Fix tests again.
2021-07-31 14:40:51 +05:30
852a65ff18
Add tests for StringUtils
2021-07-30 19:56:57 +02:00
48d897e6ad
Add final and adjust utils class name
2021-07-29 21:33:45 +02:00
1c78976900
[YouTube] Expand regex to match n param decrypt function
...
Temporary solution
2021-07-29 20:26:36 +02:00
79f2d74b04
Fix test
...
The decryption code changes over time. Only check whether the n parameter was changed in the URL
2021-07-28 16:45:31 +02:00
32055147e0
Do some code improvements
...
Use final where possible, annotate some methods and parameters as Nonnull and format new code to be in the 100 characters limit per line.
2021-07-27 17:24:27 +02:00
3a3d1d7f2b
Make YoutubeJavaScriptExtractor and JavaScript methods static
...
Also address review and rewrite some comments
2021-07-20 20:48:11 +02:00
a02ee2e952
Rewrite youtube throttling solution and add tests
2021-07-17 19:10:09 +02:00
dbdc962a07
Update mocks
2021-07-03 13:30:26 +02:00
b9549ace25
[Youtube] Fix lofi girl test and ignore mix with id test
2021-06-29 21:12:20 +02:00
5982431ffb
Remove IPv4 addresses from response during mock recording
2021-06-29 21:03:33 +02:00
c5e8bd368d
Update Mocks
2021-06-23 12:21:57 +02:00
e9a992b0a9
[youtube] Fix extraction of age restricted videos
2021-06-23 12:06:02 +02:00
d4186d100b
Merge pull request #580 from TeamNewPipe/accountTerminated
...
Add AccountTerminatedException for terminated channels
2021-06-08 09:55:58 +02:00
fa444c8298
Remove assertations of exception messages
2021-06-06 11:38:43 +02:00
faa503c7dd
Fix PeertubeAccountExtractorTest
2021-06-03 19:54:42 +02:00
ac31f3a883
Specify UTF-8 file encoding in RecordingDownloader and MockDownloader
...
On Windows, mocks are recorded and read with the Cp1252 encoding so it breaks the mocks on non ASCII characters for Linux OS (and so the CI).
The project is in Java 8, so we can't use FileReader(File, Charset) and FileReader(File, Charset) because these methods require Java 11. Instead of changing the Java version of the extractor, use OutputStreamWriter and FileOutputStream instead of FileWriter and InputStreamReader and FileInputStream instead of FileReader.
2021-06-03 18:38:25 +02:00
ff11c2df2a
Merge pull request #628 from litetex/fix-broken-yt-liked-comments
...
Fix broken yt likes in comments
2021-05-29 11:04:11 +02:00
519bba70b3
Merge pull request #630 from XiangRongLin/fix_tests
...
Fix MetaInfoTest.expectedMetaInfo
2021-05-28 14:16:53 +02:00
e81b0e2885
Tweaked code
2021-05-27 19:48:31 +02:00
8c96545e57
Merge branch 'TeamNewPipe:dev' into fix-broken-yt-liked-comments
2021-05-24 18:06:45 +02:00
12fb18c310
Updated mock data (only the affected)
2021-05-24 18:05:26 +02:00
020acfed71
Fixed tests and added new ones
2021-05-24 18:05:26 +02:00
86bb9efb5d
Added likeCount back to tests
2021-05-24 18:05:25 +02:00
636e27333b
Merge pull request #603 from TeamNewPipe/peertube_hls
...
Add PeerTube HLS streams
2021-05-23 11:49:40 +02:00
172394cd52
Update mocks
2021-05-22 20:33:20 +02:00
18c8cac40c
Fix MetaInfoTest.expectedMetaInfo
...
Expected meta changed
2021-05-21 21:00:08 +02:00
10cf081145
Removed likeCount and added textualVoteCount
2021-05-20 20:10:14 +02:00
1acc53b8db
[PeerTube] Add support for HLS streams
2021-05-15 12:39:23 +02:00
3a3ade20f4
Merge pull request #620 from TeamNewPipe/bandcamp-comments
...
Add Bandcamp comments (first page)
2021-05-02 13:43:53 +02:00
85a4270bd9
[Bandcamp] Fix comments test
2021-05-02 13:07:24 +02:00
ec50570053
Add mocks
2021-05-01 17:50:19 +02:00
f4404b5cc8
Add tests for more reasons
2021-05-01 17:50:19 +02:00
fc998589dc
Detect channels which have been terminated due to copyright infringement
2021-05-01 17:50:19 +02:00
bb3861ddce
Add AccountTerminatedException for better error handling of terminated channels
2021-05-01 17:50:19 +02:00
284362fc15
Merge pull request #608 from TeamNewPipe/youtube-fix-segment-test
...
[YouTube] fix tests
2021-05-01 17:24:41 +02:00
9c12dc5609
[SoundCloud] Fix SoundCloud ID extraction
...
resolveIdWithEmbedPlayer() does not work anymore because the JSON data has been extracted to an API call. For this reason, replace resolveIdWithEmbedPlayer() with resolveIdWithWidgetApi)( which performs the API call.
2021-04-26 17:58:34 +02:00
9dc7a1d5ee
[Bandcamp] Add comments (first page)
2021-04-26 17:25:04 +02:00
90b5c00599
Extract related items #593
2021-04-13 17:56:19 +02:00
1fe645704f
[YouTube] FIx testMoreRelatedItems in RandomQueryNoMorePages
2021-04-11 23:21:35 +02:00
9a2b814d8f
Update mocks
2021-04-11 22:59:01 +02:00
50005ce937
[YouTube] Replace unavailable YouTube Stream Segments test
...
The stream is unavailable in many countries.
2021-04-11 22:58:48 +02:00
318cec7625
[Bandcamp] Add Radio stream segments
2021-04-11 21:19:44 +02:00
55e0f8e725
Merge branch 'master' into dev
2021-04-10 11:33:34 +02:00
b94b316558
Update mocks
2021-04-09 11:59:17 +02:00
4b8d4a84b6
Set the NumberGenerator for th remaining tests
2021-04-09 11:59:17 +02:00
db7ad9c27d
Update Mocks
2021-04-08 16:37:29 +02:00
4833df60e5
Add method to inject Random into YoutubeParsingHelper and use in tests
2021-04-08 16:36:55 +02:00
682ec27737
Update mocks
2021-04-08 14:22:31 +02:00
4ee270fe01
Fix LiveStream Test
2021-04-07 14:11:41 +02:00
883f16e0ad
[YouTube] Set CONSENT cookie
2021-04-07 14:11:37 +02:00
aee3838ed5
Ignore unreliable SoundcloudGoPlusTrack.testRelatedItems
2021-04-06 18:26:45 +02:00
7b06c696e2
Use Page.body for YoutubeChannelExtractor
2021-04-06 18:14:38 +02:00
eda1b6e199
Use Page.body for YoutubePlaylistExtractor
2021-04-06 18:14:38 +02:00
7b36469807
Fix SoundcloudGoPlusTrack
...
https://soundcloud.com/martinsolveig/places#t=0 has related streams
2021-04-04 09:39:26 +02:00
3d70b6d46a
Fix a test and ignore rest
2021-04-03 11:03:10 +02:00
14f6f1b7c3
Generify related streams calls and rename method
2021-03-31 20:24:40 +02:00
dbcf61c6f7
[Bandcamp] Load more featured pages
2021-03-28 12:53:05 +02:00
b4dee6d08f
Merge pull request #306 from B0pol/metadata
...
Extract metadata for youtube, soundcloud & mediaccc
2021-03-27 08:45:47 +01:00
b029779217
[media.cc.de] Disable unreliable tests for now
2021-03-25 19:44:44 +01:00
772de53a66
Increase the number of subscribers in PeertubeAccountExtractorTest
2021-03-25 17:49:20 +01:00
efe2b964f0
Extract real subscribers of a Peertube account
...
Apply fix provided in the PR and move all accounts/ strings to a constant, ACCOUNTS.
2021-03-25 17:47:44 +01:00
12835bfae1
Merge pull request #551 from FireMasterK/ff-privacy
...
Change UA to privacy.resistFingerprinting.
2021-03-24 17:58:34 +01:00
9009f1e277
Merge pull request #517 from XiangRongLin/flaky_test
...
Add MockOnlyRule to allow skipping specific tests based on downloader
2021-03-20 18:04:14 +01:00
152221c7fb
ignore a test with encoding problems
2021-03-19 10:14:43 +01:00
8806fb4e6b
address reviews
2021-03-19 10:14:43 +01:00
c47cc54908
Extract metadata for YouTube, SoundCloud & MediaCCC
2021-03-19 10:14:43 +01:00
0e3e420a25
Fix tests
2021-03-14 17:54:21 +01:00
0438828e36
Add a test for the number of audioStreams in CreativeCommonsPlaysWellWithOthers test
...
It should be only two audio streams for track "Plays Well with Others, Ep 2: What Do an Army of Ants and an Online Encyclopedia Have in Common?" by Creative Commons (https://soundcloud.com/wearecc/plays-well-with-others-ep-2-what-do-an-army-of-ants-and-an-online-encyclopedia-have-in-common ):
- one which is a progressive stream, in MP3 format with a bitrate of 128 kbps
- one which is an HLS stream, in OPUS format with a bitrate of 64 kbps.
2021-03-14 17:54:18 +01:00
2e57a8f24f
[Bandcamp] Fix link handler acceptance behaviour
...
* Test for bandcamp footer instead of meta tag (which is not present on
all pages)
* Accept links to /music, not just /releases
* Correctly handle uppercase URLs
2021-03-14 09:51:30 +01:00
70d9e389b9
[Bandcamp] Add tests for external bandcamp artists
...
Necessary, because the external pages tested before were converted to bandcamp.com pages. See f9d06252f2
2021-03-13 20:51:59 +01:00
f9d06252f2
[Bandcamp] Fix tests
2021-03-13 17:42:39 +01:00
7e6f464407
Merge pull request #509 from TiA4f8R/soundcloud-improvements
...
Add new exceptions to be able to display different error messages in apps
2021-03-07 13:51:15 +01:00
ac51134aed
Merge pull request #546 from XiangRongLin/unignore2
...
Unignore more tests
2021-03-07 13:15:24 +01:00
22fa131922
Merge branch 'dev' into bandcamp
2021-03-05 19:45:37 +01:00
22e6f33f0a
Ignore SoundCloudGeoRestrictedTest until HLS stream extraction is added
...
The ContentNotSupportedException is thrown because no supported audio streams where extracted. However, SoundCLoud does not check, whether there are any streams available.
This commit should be reverted in #526
2021-03-05 16:38:47 +01:00
21158744b1
Add expectedDescriptionIsEmpty() to DefaultStreamExtractorTest
...
Also check if related streams are empty if they are expected to be empty.
2021-03-05 16:38:46 +01:00
f15d7837a1
Fix SoundCloud tests
2021-03-05 16:38:45 +01:00
4e45aef2b3
Fix SoundCloud CreativeCommonsPlaysWellWithOthers test
2021-03-05 16:38:43 +01:00
59d6d3f04e
Fix YouTube tests
2021-03-05 16:38:42 +01:00
3925204658
Add tests and rename SoundcloudGoPlusException to SoundcloudGoPlusContentException
2021-03-05 16:38:40 +01:00
440a808b8a
Update mocks for channels
2021-03-05 13:13:36 +01:00
2b671b15ce
[YouTube] Use new continuation API for channels
...
Co-authored-by: TiA4f8R <74829229+tia4f8r@users.noreply.github.com>
2021-03-04 21:47:16 +01:00
506cc5fbe9
regenerate mocks
2021-03-04 19:01:58 +01:00
43b46bd408
Use new youtube continuations api for playlists
...
Requires sending a POST request instead of GET.
clientName and clientVersion, which were required as headers previously now need to be part of the request payload.
continuation id also needs to be part of request body.
quick and dirty solution.
2021-03-03 19:49:26 +01:00
beb05bd05c
Replace test url, because playlist was deleted
2021-03-03 19:46:14 +01:00
e13e237392
Fix typo and reword some explanations
2021-02-26 17:48:03 +01:00
70814dcfef
Fix Utils.nonEmptyAndNullJoin
...
When using the index here, it the index needs to be decremented once an element is removed. To cirecumvent this, the native Collections.removeIf() method is used.
2021-02-21 12:51:47 +01:00
42bdd8adc5
Update to fix failing mock tests.
2021-02-20 17:14:41 +05:30
e8bccfaf5d
Change UA to privacy.resistFingerprinting.
2021-02-20 17:14:41 +05:30
0c831afa7b
use DownloaderFactory instead of DownloaderTestImpl for search ChannelVerified
2021-02-20 11:41:30 +01:00
adf9d7d10f
Add reason field to MockOnly
...
This enforces developers to document why a test is skipped
2021-02-19 18:54:18 +01:00
ea52030613
Add MockOnlyRule to allow skipping specific tests based on downloader
2021-02-19 18:54:17 +01:00
02920fafa8
Add isUploaderVerified()
2021-02-19 15:51:23 +01:00
91e9309486
Merge remote-tracking branch 'origin/dev' into bandcamp
2021-02-19 13:37:45 +01:00
fa61b864f2
Code improvements
2021-02-19 13:37:24 +01:00
1a322ad8ed
Add uploader verified by service extraction
2021-02-18 16:36:40 +01:00
a64579da2f
Unignore some tests that work on Linux
2021-02-17 20:31:26 +01:00
b99cb8624f
Unignore PublicBroadcasterTest
...
Tests only work on linux, mocks have to be generated on linux too
2021-02-17 20:31:26 +01:00
37366c65ca
Remove encoding of '&' to '&' in test and unignore it
2021-02-17 20:31:26 +01:00
c327922cc0
Add explanations to broken tests
2021-02-17 20:31:26 +01:00
54aa5b3042
Use propper structure in KioskExtractors
...
Made BandCampRadioExtractor a Kiosk which holds StreamInfoItems and not InfoItems.
2021-02-16 21:09:42 +01:00
84380e4c3a
Merge pull request #536 from B0pol/watchendpoints
...
support /watch/, /v/ and /w/ subpaths
2021-02-16 20:23:27 +01:00
a9594f531f
Merge pull request #543 from XiangRongLin/unignore1
...
Unignore and fix some tests
2021-02-16 18:00:16 +01:00
a28e0226c5
Merge pull request #519 from XiangRongLin/better_logging_error_handling
...
Better logging, error handling, doc for mock tests
2021-02-16 15:06:06 +01:00
05cb22c654
Unignore Gronkh.testDescription
...
Remove special character ü from assertion
2021-02-16 08:59:35 +01:00
d9b4c2779f
Unignore YoutubeSearchQHTest.testRegularValues
2021-02-16 08:59:35 +01:00
137272b7a4
Unignore YoutubeCommentsExtractorTest.EmptyComment
...
Mock files were missing
2021-02-16 08:59:34 +01:00
5bf9fddba9
Code improvements
2021-02-15 22:45:17 +01:00
e062c8cb0d
Merge branch 'dev' into bandcamp
2021-02-15 22:09:41 +01:00
557934cb17
use EMPTY_STRING
2021-02-15 18:51:46 +01:00
9ca52ca68a
use only one constant UTF-8
2021-02-15 18:47:54 +01:00
a88b2e6af4
Undo formatting of YoutubeStreamExtractorDefaultTest
2021-02-14 14:57:23 +01:00
81317cf6bc
youtube: support /w/ folder
2021-02-13 22:20:05 +01:00
f15c0fcfed
support /watch/ and /v/ folders
2021-02-13 22:20:05 +01:00
971128c464
Ignore failing tests
2021-02-13 20:28:03 +01:00
92dea0806a
Regenerate json files for youtube tests
2021-02-13 20:28:03 +01:00
d6b9930c18
Use DownloaderFactory in YoutubeCommentsExtractorTest.Pinned
2021-02-13 20:12:39 +01:00
b555f38fa5
Use real downloader for YoutubeSubscriptionExtractorTest
...
It doesn't make network requests anyway
2021-02-13 20:12:39 +01:00
f352b0c7af
YoutubeMusicSearchExtractorTest can't be used with mocks
...
Requires further research
2021-02-13 20:00:51 +01:00
f45b5610bf
Unignore YoutubeChannelLocalizationTest
...
With mocks time is not a concern anymore
2021-02-13 20:00:51 +01:00
19737e06ee
Use DownloaderFactory in YoutubeChannelExtractorTest
...
KurzgesagtAdditional needed to be extracted because it ran a duplicated request.
2021-02-13 20:00:51 +01:00
84eb4b30c9
Use DownloaderFactory in YouTube tests
2021-02-13 20:00:51 +01:00
73353996d7
fix more tests
2021-02-12 23:18:38 +01:00
44c54d403a
fix Invidious channels being accepted by YoutubeStreamLinkHandlerFactory ( #527 )
...
fixes https://github.com/TeamNewPipe/NewPipeExtractor/issues/524 , see it for more details
2021-02-05 18:57:58 +01:00
1414a6f178
Fix extraction of YouTube chapters ( #531 )
...
YouTube changed the name of the chapters in the JSON from engagement-panel-macro-markers to engagement-panel-macro-markers-description-chapters, so extracting chapters doesn't work.
2021-02-03 22:07:34 +01:00
2662ceca22
rename getHeartedByCreator and getPinned to isPinned and isHeartedByCreator
2021-01-23 13:06:07 +01:00
66e4eb2f96
extract whether the comment is pinned
2021-01-22 23:59:22 +01:00
88e4c8667b
Add comments to RecordingDownloader about usage
2021-01-17 19:53:20 +01:00
eecfe09f2c
Check for non-existent folder in MockDownloader
2021-01-17 19:52:58 +01:00
07570554b6
Fix build and optimize imports
2021-01-15 21:55:40 +01:00
78c2113094
Merge remote-tracking branch 'origin/dev' into bandcamp
2021-01-15 21:49:58 +01:00
6888e7773f
Ignore all failing tests
2021-01-15 20:11:40 +01:00
7c774c84ce
Merge pull request #505 from TeamNewPipe/meida.ccc.de_search_fix
...
[media.ccc.de] Fix NPE in search results if they contain a future talk
2021-01-15 14:16:59 +01:00
c2c4d97efd
Merge pull request #482 from XiangRongLin/mock_mix_pl_test
...
Mock mix pl tests
2021-01-15 12:54:48 +01:00
4f81d9d689
Regenerate mock files
2021-01-15 09:51:23 +01:00
cc51c5f6a3
Merge pull request #507 from vkay94/add-interval-to-stream-frames
...
Add durationPerFrame to Frameset
2021-01-14 20:27:40 +01:00
11dcfe638b
Add durationPerFrame to Frameset and getFrameBoundsAt method
2021-01-14 20:01:06 +01:00
c2ff6723d7
Merge pull request #511 from B0pol/errors
...
Fix PeerTube tests by changing instance
2021-01-14 19:15:22 +01:00
4af50c6870
Use @link for downloader
2021-01-12 09:41:57 +01:00
3455f0f23c
Add documentation to testing with mocks
2021-01-12 09:37:48 +01:00
5292cc1636
[peertube] test: remove usage of peertube.mastodon.host
...
It's down for months
2021-01-11 19:46:27 +01:00
f3c22da618
comments: add heartedByCreator and extract it for youtube
2021-01-11 18:48:08 +01:00
63c237db41
Only create directories if they do not exist
2021-01-11 07:57:11 +01:00
e82cb1efea
Remove dependency to commons-io
2021-01-10 20:50:38 +01:00
1ea6c6ce54
Prefix mock file from RecordingDownloader with "generated_mock_"
...
Only read those files in MockDownloader
2021-01-10 20:46:42 +01:00
f91916c017
Remove unnecessary file object creation
2021-01-10 20:39:22 +01:00
255c726f20
Add resource path base to DownloaderFactory
2021-01-10 20:27:34 +01:00
35e299759e
Add method to allow resetting youtube client version and key
...
This is needed so that a request is made for each test class when running multiple at once. This way RecordingDownloader records all necessary requests.
This works as long as tests are run sequentially and not in parallel.
2021-01-10 20:24:50 +01:00
f447a7a450
fix import due to downloader package move
2021-01-10 19:49:14 +01:00
FireMasterK
XiangRongLin
TobiGr
TiA4f8R
bopol
litetex
Fynn Godau
Stypox
vkay94