Go to file

AudricV 7de3753a81 [YouTube] Refactor JavaScript player management API This commit is introducing breaking changes. For clients, everything is managed in a new class called YoutubeJavaScriptPlayerManager: - caching JavaScript base player code and its extracted code (functions and variables); - getting player signature timestamp; - getting deobfuscated signatures of streaming URLs; - getting streaming URLs with a throttling parameter deobfuscated, if applicable. The class delegates the extraction parts to external package-private classes: - YoutubeJavaScriptExtractor, to extract and download YouTube's JavaScript base player code: it always already present before and has been edited to mainly remove the previous caching system and made it package-private; - YoutubeSignatureUtils, for player signature timestamp and signature deobfuscation function of streaming URLs, added in a recent commit; - YoutubeThrottlingParameterUtils, which was originally YoutubeThrottlingDecrypter, for throttling parameter of streaming URLs deobfuscation function and checking whether this parameter is in a streaming URL. YoutubeJavaScriptPlayerManager caches and then runs the extracted code if it has been executed successfully. The cache system of throttling parameters deobfuscated values has been kept, its size can be get using the getThrottlingParametersCacheSize method and can be cleared independently using the clearThrottlingParametersCache method. If an exception occurs during the extraction or the parsing of a function property which is not related to JavaScript base player code fetching, it is stored until caches are cleared, making subsequent failing extraction calls of the requested function or property faster and consuming less resources, as the result should be the same until the base player code changes. All caches can be reset using the clearAllCaches method of YoutubeJavaScriptPlayerManager. Classes using JavaScript base player code and utilities directly (in the code and its tests) have been also updated in this commit.		2023-09-21 21:59:32 +02:00
.github	Bump actions/checkout from 3 to 4	2023-09-05 10:00:13 +00:00
checkstyle	Block wrong nullable/nonnull imports in checkstyle	2022-11-28 14:42:18 +01:00
extractor	[YouTube] Refactor JavaScript player management API	2023-09-21 21:59:32 +02:00
gradle/wrapper	Update gradle wrapper to 7.5.1	2022-08-18 15:09:35 -04:00
timeago-parser	[YouTube] Fix parsing short relative date formats (English only) (#1068 )	2023-06-18 21:41:29 +02:00
.gitignore	ignore workspace and folder configs	2020-05-11 21:23:50 +02:00
LICENSE	Update the license to the latest version of https://www.gnu.org/licenses/gpl-3.0.txt	2022-03-19 17:59:27 +01:00
README.md	Add proguard rules to README	2023-07-05 08:25:23 +02:00
build.gradle	Merge pull request #1083 from TeamNewPipe/dependabot/gradle/org.junit-junit-bom-5.10.0	2023-08-06 18:58:27 +02:00
gradlew	Update gradle wrapper to 7.5.1	2022-08-18 15:09:35 -04:00
gradlew.bat	Update gradle wrapper to 7.5.1	2022-08-18 15:09:35 -04:00
jitpack.yml	Use JDK 11 for Jitpack gradle builds.	2022-11-15 18:43:19 +00:00
settings.gradle	Created gradle module and moved existing code to new one	2018-03-14 00:44:02 -03:00

README.md

NewPipe Extractor

JDoc • Documentation

NewPipe Extractor is a library for extracting things from streaming sites. It is a core component of NewPipe, but could be used independently.

Usage

NewPipe Extractor is available at JitPack's Maven repo.

If you're using Gradle, you could add NewPipe Extractor as a dependency with the following steps:

Add maven { url 'https://jitpack.io' } to the repositories in your build.gradle.
Add implementation 'com.github.TeamNewPipe:NewPipeExtractor:INSERT_VERSION_HERE' to the dependencies in your build.gradle. Replace INSERT_VERSION_HERE with the latest release.
If you are using tools to minimize your project, make sure to keep the files below, by e.g. adding the following lines to your proguard file:

## Rules for NewPipeExtractor
-keep class org.schabi.newpipe.extractor.timeago.patterns.** { *; }
-keep class org.mozilla.javascript.** { *; }
-keep class org.mozilla.classfile.ClassFileWriter
-dontwarn org.mozilla.javascript.tools.**

Note: To use NewPipe Extractor in Android projects with a minSdk below 26, API desugaring is required. If the minSdk is below 19, the desugar_jdk_libs_nio artifact is required, which requires Android Gradle Plugin (AGP) version 7.4.0.

Testing changes

To test changes quickly you can build the library locally. A good approach would be to add something like the following to your settings.gradle:

includeBuild('../NewPipeExtractor') {
    dependencySubstitution {
        substitute module('com.github.TeamNewPipe:NewPipeExtractor') with project(':extractor')
    }
}

Another approach would be to use the local Maven repository, here's a gist of how to use it:

Add mavenLocal() in your project repositories list (usually as the first entry to give priority above the others).
It's recommended that you change the version of this library (e.g. LOCAL_SNAPSHOT).
Run gradle's ìnstall task to deploy this library to your local repository (using the wrapper, present in the root of this project: ./gradlew install)
Change the dependency version used in your project to match the one you chose in step 2 (implementation 'com.github.TeamNewPipe:NewPipeExtractor:LOCAL_SNAPSHOT')

Tip for Android Studio users: After you make changes and run the install task, use the menu option File → "Sync with File System" to refresh the library in your project.

Supported sites

The following sites are currently supported:

YouTube
SoundCloud
media.ccc.de
PeerTube (no P2P)
Bandcamp

License

NewPipe Extractor is Free Software: You can use, study share and improve it at your will. Specifically you can redistribute and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.