* Feat: initial support for the ipfs-search engine
* Feat: enable paging for ipfs search
* Make ipfs-search code more readable
* added support for images, music and video to the ipfs search engine
* FIX: redefined some variables that where redefining built-ins
* adjust code so it works on older python versions
* Feat: add support for time ranges
Adds an option to the oscar theme that puts an archive.today link next to the web.archive.org cache link. In preferences change 'Show archive.today links' to 'On'
* [fix] make autofocus configurable (#1984)
"Results page: having text cursor still in search field is annoying #1984"
Allows autofocus on the results page to be configured either in the UI
or in the searx server config.
* fix commented code
Co-authored-by: Ben Collerson <benc@benon.com>
* [fix] google engine: results XPath
* [fix] google & youtube - set EU consent cookie
This change the previous bypass method for Google consent using
``ucbcb=1`` (6face215b8) to accept the consent using ``CONSENT=YES+``.
The youtube_noapi and google have a similar API, at least for the consent[1].
Get CONSENT cookie from google reguest::
curl -i "https://www.google.com/search?q=time&tbm=isch" \
-A "Mozilla/5.0 (X11; Linux i686; rv:102.0) Gecko/20100101 Firefox/102.0" \
| grep -i consent
...
location: https://consent.google.com/m?continue=https://www.google.com/search?q%3Dtime%26tbm%3Disch&gl=DE&m=0&pc=irp&uxe=eomtm&hl=en-US&src=1
set-cookie: CONSENT=PENDING+936; expires=Wed, 24-Jul-2024 11:26:20 GMT; path=/; domain=.google.com; Secure
...
PENDING & YES [2]:
Google change the way for consent about YouTube cookies agreement in EU
countries. Instead of showing a popup in the website, YouTube redirects the
user to a new webpage at consent.youtube.com domain ... Fix for this is to
put a cookie CONSENT with YES+ value for every YouTube request
[1] https://github.com/iv-org/invidious/pull/2207
[2] https://github.com/TeamNewPipe/NewPipeExtractor/issues/592
Closes: https://github.com/searxng/searxng/issues/1432
* [fix] sjp engine - convert enginename to a latin1 compliance name
The engine name is not only a *name* its also a identifier that is used in
logs, HTTP headers and more. Unicode characters in the name of an engine could
cause various issues.
Closes: https://github.com/searxng/searxng/issues/1544
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [fix] engine tineye: handle 422 response of not supported img format
Closes: https://github.com/searxng/searxng/issues/1449
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* bypass google consent with ucbcb=1
* [mod] Adds Lingva translate engine
Add the lingva engine (which grabs data from google translate). Results from
Lingva are added to the infobox results.
* openstreetmap engine: return the localized named.
For example: display "Tokyo" instead of "東京都" when the language is English.
* [fix] engines/openstreetmap.py typo: user_langage --> user_language
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* Wikidata engine: ignore dummy entities
* Wikidata engine: minor change of the SPARQL request
The engine can be slow especially when the query won't return any answer.
See https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual/MWAPI#Find_articles_in_Wikipedia_speaking_about_cheese_and_see_which_Wikibase_items_they_correspond_to
Co-authored-by: Léon Tiekötter <leon@tiekoetter.com>
Co-authored-by: Emilien Devos <contact@emiliendevos.be>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Emilien Devos <github@emiliendevos.be>
Co-authored-by: ta <alt3753.7@gmail.com>
Co-authored-by: Alexandre Flament <alex@al-f.net>
## What does this PR do?
This PR prepares for removing `httpx`, and reverts back to `requests`.
## Why is this change important?
`httpx` hasn't proven itself to be faster or better than `requests`. On the other
hand it has caused issues on Windows.
=============================================
Please update your environment to use requests instead of httpx.
=============================================
* [fix] Rename ccengine engine to openverse
The CC engine was merged with WordPress and renamed to Openverse
Source: https://wordpress.org/news/2021/05/welcome-to-openverse/
* [fix] ccengine engine - avoid unwanted redirects
api.openverse.engineering is a little picky and wants to have a trailing slash
in the path:
/v1/images? -->/ v1/images/?
otherwise it redirects, here is the debug log:
DEBUG searx.network.openverse : HTTP Request: GET https://api.openverse.engineering/v1/images?&page=1&page_size=20&format=json&q=foo "HTTP/2 301 Moved Permanently" (text/html; charset=utf-8)
DEBUG searx.network.openverse : HTTP Request: GET https://api.openverse.engineering/v1/images/?&page=1&page_size=20&format=json&q=foo "HTTP/2 200 OK" (application/json)
WARNING searx.engines.openverse : ErrorContext('searx/search/processors/online.py', 105, 'count_error(', None, '1 redirects, maximum: 0', ('200', 'OK', 'api.openverse.engineering')) True
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [fix] FutureWarning from lxml
Just in case if content is None, the original code will skip extract_text(), and
just append the None value to 'content'. So just add allow_none=True, and this
will return None without raising a ValueError in extract_text().
* [enh] Add pagination to Brave
Also added ```&spellcheck=1``` because now it is disabled by default, not returning any ```suggestion_xpath```.
Co-authored-by: Léon Tiekötter <leon@tiekoetter.com>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: capric98 <42015599+capric98@users.noreply.github.com>
Co-authored-by: Allen <64094914+allendema@users.noreply.github.com>
* [enh] Add Tineye reverse image search
Other optional parametesr:
"&sort=crawl_date" can be appended to search_string to sort results by date.
"&domain=example.org" can be implemented to search_string to get results from just one domain.
Public instances could get relatively fast timed-out for 3600s.
* [enh] Add TIneye to settings.yml
Check if that's the right shortcut.
* [mod] Fix checks
* [mod] Try to fix checks
* [mod] Use Four spaces for indentation
And set paging back to True
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
## What does this PR do?
Gives the user the possibility to search their own prowlarr instances.
Info: https://wiki.servarr.com/en/prowlarr
Github: https://github.com/Prowlarr/Prowlarr
## Why is this change important?
Prowlarr searchs multiple upstream search providers, thus allows to use that functionality through searx.
Add configurable setting to rank search results higher when part of the
domain (e.g. 'en' in 'en.wikipedia.org' or 'de' in 'beispiel.de')
matches the selected search language. Does not apply to e.g. 'be' in
'youtube.com'.
Closes#206
The implementation uses the Qwant API (https://api.qwant.com/v3). The API is
undocumented but can be reverse engineered by reading the network log of
https://www.qwant.com/ queries.
This implementation is used by different qwant engines in the settings.yml::
- name: qwant
categories: general
...
- name: qwant news
categories: news
...
- name: qwant images
categories: images
...
- name: qwant videos
categories: videos
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- Change Libgen provider and use https by default.
- Umcomment Urbandictionary but disable it by default, it is working.
- Uncomment Ebay as it is working correctly.
(For ebay in the future: base_url should be changed from settings.yml just like peertube or invidious)
Suggestions should be added too.
suggestion_xpath: //div[@class="text-gray h6"]/a
You can try it with:
!brave recurzuoin
Suggested-by: @allendema in https://github.com/searx/searx/issues/2857#issuecomment-904837023
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
BTW add about section to the YAML configuration
It now shows descriptions with their correct URLs when there are videos in the
search results, pulling content_xpath from snippet-description instead of
snippet-content.
Suggested-by: @eagle-dogtooth https://github.com/searx/searx/issues/2857#issuecomment-869119968
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
wibyweb
Kian-Meng Ang
Adam Tauber
Rob
Elena Poelman
Ben Collerson
Noémi Ványi
james-still
liimee
Markus Heiser
Noémi Ványi
iko
Allen
Dario Nuevo
Émilien Devos
israelyago
Finn
Paul Alcock
Marc Abonce Seguin