Commit Graph

250 Commits

Author SHA1 Message Date
Mauricio Colli d8280ce0da
[YouTube] Parse watching count in live streams items 2019-11-03 15:46:22 -03:00
Mauricio Colli 3638f0e0ea
Implement time ago parser and improve localization handling
- Handle special cases for languages where the number is not shown
- Rework the Downloader base implementation, allowing for more
advanced things to be done
- Separate the localization from the content country (just like
YouTube let's the user choose both).
2019-11-03 15:46:21 -03:00
wojcik-online 180836c180
Base Implementation: Parse the upload date of StreamInfoItems
In the format '2 days ago' (in English) on a YouTube channel page.
(Parser extensible to other pages.)
2019-11-03 15:46:20 -03:00
Mauricio Colli c4fe2183ce
[YouTube] Improve detection of reCAPTCHA pages 2019-10-29 02:00:41 -03:00
Mauricio Colli 9a325b280d
[YouTube] Make detection of age restricted pages more reliable 2019-10-29 02:00:40 -03:00
Mauricio Colli 91c360df5e
Remove section of dead code 2019-10-29 02:00:39 -03:00
Tobias Groza 06f2144e4d
[YouTube] Remove GEMA exception handling (#202) 2019-10-05 14:59:05 +02:00
Vasiliy d8279f91f6
Merge branch 'dev' of https://github.com/TeamNewPipe/NewPipeExtractor into feature/frames 2019-09-28 14:46:14 +03:00
TobiGr 0710f31a39 Fix TeamNewPipe/NewPipeExtractor#197 2019-09-25 08:37:26 +02:00
Stypox 3f1ba93be5
Fix NPE when extracting itags with non-existing streamingData key 2019-09-12 15:08:17 +02:00
Stypox 0d8fb65003
Fix NPE on determining whether stream is live on Youtube 2019-09-12 15:07:07 +02:00
Stypox 24a37b88a9
Use pre-generated playerResponse field in yt's getHlsUrl()
Also refactored code to always throw exception when the url can't be found
2019-09-11 20:12:30 +02:00
Stypox 9c423a0a40
Use FORMATS to get video+audio streams on yt
Not ADAPTIVE_FORMATS
2019-09-11 20:04:28 +02:00
Stypox e5e8c66686
Readd signature decryption in YtStreamExtractor 2019-09-11 19:56:16 +02:00
Stypox d9570d8634
Use pre-generated playerResponse field everywhere in YtStreamExtractor 2019-09-11 19:35:08 +02:00
Stypox 63a37c48e3
Remove println left behind 2019-09-11 19:31:39 +02:00
Stypox bf017bf5b9
Fix TeamNewPipe/NewPipe#2615 2019-09-11 19:05:41 +02:00
Vasiliy f084cfec24
Refactor frames extraction 2019-09-10 19:38:51 +03:00
Vasiliy a6c94c7a9d
Grub frames preview from youtube 2019-08-26 19:14:09 +03:00
jimbo1qaz e38d906ff9 Fix timestamp links in Youtube video descriptions
For some reason, in NewPipeExtractor,
comments were loaded from JSON by YoutubeCommentsInfoItemExtractor as text,
sent via CommentsInfoItem#getCommentText to NewPipe,
where timestamps are converted to hyperlinks using Linkify:
https://github.com/TeamNewPipe/NewPipe/pull/2168

On the other hand, video descriptions are handled in NewPipeExtractor
by scraping the watch-page HTML.
There, timestamp links were previously mangled (and now properly parsed),
before being sent as HTML via YoutubeStreamExtractor#getDescription
to NewPipe (where HTML gets converted to Spanned).

The logic introduced in this commit is different from the above PR,
since it operates in the extractor, and mutates the HTML DOM
rather than identifying via regex.
2019-08-18 01:05:41 -07:00
Stypox 06689a2f27
Add url to ReCaptchaException
Sometimes YouTube introduces recaptchas only on some pages. By adding an url to the ReCaptchaException the NewPipe app is able to use that url to load the page that originally caused the problem.
Also removed every instance of exception caught and rethrown with a different description: it makes no sense and it removes part of the useful stacktrace.
2019-08-17 09:09:07 +02:00
Christian Schabesberger 5798c8fdf5 fix duration can not be paresd
update gradle to version 5.1

fix sts issue for agegated videos

GOD DAMN FUCKING BULLSHIT

add duratin for controversal/age gated videos

bring back sts

remove ignores

fix ogg test
2019-07-31 16:11:42 +02:00
Stypox c70d28597b
Add fallback for urls not conaining the "sp" tag
If ever YouTube changes thing again (or uses old urls for some unknown reason), this prevents the extractor from crashing.
As suggested here: 133cc032d9 (r283529811)
2019-05-14 13:57:45 +02:00
Stypox 133cc032d9
Fix invalid yt url: signature tag name is not always "signature"
Thanks to @omarroth for the suggestion: see TeamNewPipe/NewPipeExtractor#155
2019-05-13 21:25:35 +02:00
Christian Schabesberger 560c648e92 fix decrypt regex for akamai 2 times in file 2019-03-14 16:49:30 +01:00
Christian Schabesberger dd61d66cf5 speed up finding decrypt function 2019-03-14 09:07:19 +01:00
Christian Schabesberger e072bf6461 fix dercrypt error due to wrong dollar sign detection 2019-03-14 08:49:11 +01:00
Ritvik Saraf c60d6ec0ab removed unused import 2019-01-19 18:20:02 +05:30
Ritvik Saraf 50c4783f71 fixed extracting hls url 2019-01-19 18:07:35 +05:30
Kunal Sareen b6790bbe6b Updating regex 2019-01-18 21:47:34 +11:00
kapodamy e4afb21862 make Subtitle object Stream compliant
* merge SubtitlesFormat into MediaFormat
* implement Stream interface on Subtitle class
* misc fixes: dont show a snackbar error on age-restricted videos, lint fix.
2018-11-08 16:29:05 -03:00
Christian Schabesberger 7129d6db55 add inline documentation for StreamExtractor 2018-11-07 18:28:44 +01:00
Christian Schabesberger ad4b15ac66
Merge pull request #114 from TeamNewPipe/localization
Localization
2018-09-24 14:00:20 +02:00
Christian Schabesberger 49c2eb5185 fix failing watch links 2018-09-20 13:51:06 +02:00
Christian Schabesberger ce2bbee1e9 add propper localization 2018-09-15 22:29:28 +02:00
Christian Schabesberger 96b4c97cee
Merge pull request #109 from karyogamy/sig-fix-update
Decryption Signature Regex Update
2018-09-15 22:10:31 +02:00
John Zhen Mo ed73ae55f1 -Added more decrypt function name matching regex.
-Cleaned up decryption code generation method.
2018-09-07 22:25:07 -07:00
Christian Schabesberger 66c3c3f452 fix channel links in description part 2 2018-09-07 22:18:22 +02:00
Christian Schabesberger 4469d11307 fix channel links in description 2018-09-07 21:40:36 +02:00
John Zhen Mo 119843bfac -Fix signature extraction parse exception. 2018-09-07 11:34:14 -07:00
Christian Schabesberger fef71aeccc roleback using href for description links 2018-08-28 11:55:12 +02:00
Christian Schabesberger 3aa7ff6d0e make ytdescription alter href instead of the text of links 2018-08-27 20:45:43 +02:00
Christian Schabesberger e662c97433 make dash parser ignore segmented streams 2018-08-21 17:23:56 +02:00
Christian Schabesberger c4e16c7337 fix hashtag parsing 2018-08-17 16:22:50 +02:00
Christian Schabesberger 981cb333b5 fix link parsing for yt internal links
blub
2018-08-16 17:44:09 +02:00
Christian Schabesberger 389a87fc89 add link parsing for youtube description 2018-08-16 17:11:18 +02:00
Christian Schabesberger 28788a05db rename uih and remove afiliate link foo 2018-07-13 18:02:40 +02:00
Christian Schabesberger bd5423fe2a make less tests fail 2018-07-01 16:55:48 +02:00
Christian Schabesberger 17f46b8b12 rename UrlIdHandler to UIHFactory 2018-07-01 16:55:48 +02:00
Christian Schabesberger 94d1eb6f32 restructure youtube service 2018-05-08 21:19:03 +02:00