Commit Graph

57 Commits

Author SHA1 Message Date
Björn Sigurbergsson 1b6fe5edd6
[YouTube] Fix ParsingException when comments are unavailable in a video (#1040)
Co-authored-by: bjs <bjs@elect-it.com>
Co-authored-by: Audric V. <74829229+AudricV@users.noreply.github.com>
Co-authored-by: Kavin <20838718+FireMasterK@users.noreply.github.com>
2023-03-30 19:58:06 +02:00
TobiGr f50b7275af [YouTube] Fix getting next comments pages 2023-01-24 22:39:08 +01:00
Kavin 22a47da8c7
Fix requested change and remove outdated comment. 2023-01-02 20:42:32 +00:00
Kavin 98a90fd9c8
Don't cache comments count and return early on page fetch if no token. 2023-01-02 20:40:48 +00:00
Kavin 2974dfaa48
Only store ajaxJson for initial page and eager fetch the initial continuation. 2023-01-02 20:40:48 +00:00
Kavin 67ef4f4c30
Cleanup and remove optional. 2023-01-02 20:40:48 +00:00
FireMasterK 22f71b010c
Fix for requested changes. 2023-01-02 20:40:48 +00:00
FireMasterK 981aee4092
Add support to extract total comment count. 2023-01-02 20:40:48 +00:00
Stypox 740a37a2de [YouTube] Fix checkstyle issues 2022-03-26 19:42:40 +01:00
litetex 1a67ea100a Refactored code
according to review
2022-02-21 19:56:12 +01:00
litetex f79ce1f52a Refactored YoutubeCommentsExtractor
* Use Java Streaming API
* Use StandardCharsets
* Prevented several NPEs/ArrayIndexOutOfBound
* Reformatted some code so that it's easier readable
2022-02-21 19:56:09 +01:00
FireMasterK 6aabdc6d16
Fix for requested changes. 2021-09-12 01:15:19 +05:30
FireMasterK e8e9e6cd00
Fix comment reply continuations. 2021-08-04 22:14:04 +05:30
FireMasterK ed84658055
Get it working. 2021-08-04 18:35:14 +05:30
FireMasterK f3e4c9d689
Use the youtubei API for YouTube comments
Migrate YouTube comments to the desktop version by using the `next` endpoint of the InnerTube internal API.
With the desktop version, we are able to get the exact like count of YouTube comments (by parsing the accessibility data) (the current extraction is used as a fallback). We are also now able to get if the uploader of the comment is verified or not.

Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>
2021-08-01 12:39:05 +02:00
TiA4f8R 7753556e66
Adress the last requested changes + update YoutubeCommentsExtractor mocks 2021-08-01 12:39:03 +02:00
TiA4f8R 609919db59
Adress again reviews, fix some rebase issues 2021-08-01 12:39:00 +02:00
litetex 6860543b07 Added more doc 2021-07-06 21:16:31 +02:00
litetex a59fead0d7 Fixed typos 2021-07-06 21:16:30 +02:00
litetex 688a1c316b Fixed Exception when YT comments are disabled and added ``commentsDisabled`` field
* Fixed code: Added missing finals (according to NewPipes Checkstyle guide)
* Fixed ``findValue`` method in ``YoutubeCommentsExtractor``
2021-07-06 21:16:29 +02:00
Tobi 12835bfae1
Merge pull request #551 from FireMasterK/ff-privacy
Change UA to privacy.resistFingerprinting.
2021-03-24 17:58:34 +01:00
bopol ff5273b882
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeCommentsExtractor.java
Co-authored-by: Tobi <TobiGr@users.noreply.github.com>
2021-03-05 14:39:01 +01:00
bopol ed850d0688 [youtube] improve comments extraction performance
- do not parse responseBody twice for continuation
instead try to get commentsTokenInside with the new pattern ("sectionListRenderer")
and try again with the old pattern ("commentSectionRenderer") on failure
- do not unescape responseBody multiple times
   -> parse responseBody less times
2021-03-05 13:37:08 +01:00
golfinq 05213175cd
minor edit, made variable final 2021-03-05 02:49:15 -05:00
golfinq 8bceb57a6c
formatting fix 2021-03-05 02:46:06 -05:00
golfinq ae48bdea4c
Updated to reflect variable name change 2021-03-05 02:40:36 -05:00
FireMasterK e8bccfaf5d
Change UA to privacy.resistFingerprinting. 2021-02-20 17:14:41 +05:30
bopol 9ca52ca68a use only one constant UTF-8 2021-02-15 18:47:54 +01:00
vkay94 9dbacbc618 Fix comments parsing 2020-12-09 14:57:38 +01:00
TobiGr 0fb73301e3 [YouTube] Fix crash on empty comment
Closes #380
2020-08-05 18:25:40 +02:00
wb9688 9b6fe1dea6 Throw IllegalArgumentException when Page is invalid 2020-07-06 20:19:31 +02:00
wb9688 4cc312086a Introduce Page class 2020-07-06 20:19:31 +02:00
wb9688 e3bfdba135 Remove getNextPageUrl() function from ListExtractor 2020-07-06 20:11:40 +02:00
bopol de26e00079 changed all == null || isEmpty() to isNullOrEmpty() 2020-05-11 11:40:24 +02:00
bopol 2564bcf399 refactor comments
rename methods in CommentsInfoItemExtractor interface to match the other method names across NewPipeExtractor
remove getName in (Youtube|SoundCloud|Peertube)CommentsExtractor and move it up in CommentsExtractor, return "Comments" instead
2020-04-21 20:05:54 +02:00
Tobias Groza a129c65c63
more improvements of comments
Co-Authored-By: Stypox <stypox@pm.me>
2020-02-16 23:59:31 +01:00
TobiGr 030465b5d4 Improve code formatting and optimise imports 2020-02-15 13:36:54 +01:00
Mauricio Colli 3638f0e0ea
Implement time ago parser and improve localization handling
- Handle special cases for languages where the number is not shown
- Rework the Downloader base implementation, allowing for more
advanced things to be done
- Separate the localization from the content country (just like
YouTube let's the user choose both).
2019-11-03 15:46:21 -03:00
Ritvik Saraf 53058802e2 fix comment url 2019-03-14 08:49:37 +01:00
Ritvik Saraf b05fa45080 resolved review comments 2018-12-28 13:02:00 +05:30
Ritvik Saraf c77050dc70 handle empty comments 2018-12-23 08:24:36 +05:30
Ritvik Saraf 9ad102df3f fixed fetching youtube client name 2018-12-23 08:24:36 +05:30
Ritvik Saraf 99a0134b1c removed unchecked cast 2018-10-19 19:33:36 +05:30
Ritvik Saraf e85958b180 merged upstream/master 2018-09-29 13:19:00 +05:30
Ritvik Saraf d1ff1c7589 using mobile website since it is faster 2018-09-28 04:54:57 +05:30
Ritvik Saraf c2ed99b1b7 no comments 2018-09-27 23:01:27 +05:30
Ritvik Saraf ad8066830d getting client version and name from initial page 2018-09-27 22:32:13 +05:30
Ritvik Saraf 6b620914b6 moved cookie logic outside 2018-09-27 02:04:12 +05:30
Ritvik Saraf fb1419608a using getDataString 2018-09-27 00:44:55 +05:30
Ritvik Saraf ce76885553 removed generics 2018-09-26 04:21:58 +05:30