Commit Graph

458 Commits

Author SHA1 Message Date
Allen 0c351ea364
[enh] Add Tineye reverse image search (#3040)
* [enh] Add Tineye reverse image search 

Other optional parametesr:

"&sort=crawl_date" can be appended to search_string to sort results by date.
"&domain=example.org" can be implemented to search_string to get results from just one domain.

Public instances could get relatively fast timed-out for 3600s.

* [enh] Add TIneye to settings.yml 

Check if that's the right shortcut.

* [mod] Fix checks

* [mod] Try to fix checks

* [mod] Use Four spaces for indentation

And set paging back to True

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2022-01-22 12:15:19 +01:00
Dario Nuevo 1a18adcc16
New files engine: Prowlarr (#3118)
## What does this PR do?

Gives the user the possibility to search their own prowlarr instances.

Info: https://wiki.servarr.com/en/prowlarr
Github: https://github.com/Prowlarr/Prowlarr

## Why is this change important?

Prowlarr searchs multiple upstream search providers, thus allows to use that functionality through searx.
2022-01-15 19:18:15 +01:00
Allen 0c2165324d
[Fix] Add suggestions + Fix xpaths (#3082)
* [mod] Add Suggestion to Petalsearch

* [Fix] Changed xpath for Petalsearch
2022-01-07 21:49:08 +01:00
Émilien Devos 8cde08ded2
Disable onesearch by default (#3099)
onesearch is not available everywhere and thus display an error by default in searx
2022-01-07 21:42:51 +01:00
israelyago b90616a25f
Remove categories from onesearch config
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-11-18 08:19:19 -03:00
israelyago 6b3915a2dc
Removed paging from onesearch config
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-11-18 08:18:50 -03:00
israelyago 0d28fd2efe
Merge branch 'master' into onesearch-engine 2021-11-17 15:27:11 -03:00
Israel Yago Pereira 4b785677d8 Onesearch pagination 2021-11-17 15:14:43 -03:00
Israel Yago Pereira 258c6fbd5a Onesearch engine without pagination 2021-11-17 15:14:43 -03:00
Israel Yago Pereira 8e00249633 WIP: onesearch engine 2021-11-17 15:14:43 -03:00
Finn 8c3454fd1b
[enh] Improve ranking based on language (#3053)
Add configurable setting to rank search results higher when part of the
domain (e.g. 'en' in 'en.wikipedia.org' or 'de' in 'beispiel.de')
matches the selected search language. Does not apply to e.g. 'be' in
'youtube.com'.

Closes #206
2021-11-15 20:31:22 +01:00
Markus Heiser 3205785059 [fix] Qwant engines - implement API v3 and add 'quant videos'
The implementation uses the Qwant API (https://api.qwant.com/v3). The API is
undocumented but can be reverse engineered by reading the network log of
https://www.qwant.com/ queries.

This implementation is used by different qwant engines in the settings.yml::

  - name: qwant
    categories: general
    ...
  - name: qwant news
    categories: news
    ...
  - name: qwant images
    categories: images
    ...
  - name: qwant videos
    categories: videos
    ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-10 21:08:48 +02:00
Allen b0888c6ca3
[enh] Add Pagination to Wiby / Fix Kaufland (#3000)
* [fix] Fix Kaufland engine

Changed Xpath expressions

* [enh] Remove tracking params from Kaufland results

* [enh] Add pagination to Wiby

* [fix] Properly select title_xpath

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-10-10 20:52:06 +02:00
Allen d91f1ede61
[mod] Fix Libgen + Uncomment Ebay and Urbandictiorany (#2986)
- Change Libgen provider and use https by default.
- Umcomment Urbandictionary but disable it by default, it is working.
- Uncomment Ebay as it is working correctly.
(For ebay in the future: base_url should be changed from settings.yml  just like peertube or invidious)
2021-10-10 15:42:43 +02:00
Paul Alcock bb34685dfa
Add IMDB support (#2980)
Closes #1145
2021-10-02 13:41:38 +02:00
Markus Heiser 0b3488158b [mod] engines - add superuser.com (Stack Exchange API)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-02 11:48:39 +02:00
Markus Heiser a130c7c7a3 [mod] engines - add askubuntu.com (Stack Exchange API)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-02 11:47:11 +02:00
Markus Heiser 5e84d670a2 [mod] replace old stackoverflow engine by Stack Exchange API v2.3
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-02 11:43:51 +02:00
Markus Heiser e88bea53eb [enh] add suggestions to brave engine
Suggestions should be added too.

    suggestion_xpath: //div[@class="text-gray h6"]/a

You can try it with:

    !brave recurzuoin

Suggested-by: @allendema in https://github.com/searx/searx/issues/2857#issuecomment-904837023
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-13 21:27:04 +02:00
Markus Heiser 7075fc1324 [fix] brave engine: shows descriptions with their correct URLs
BTW add about section to the YAML configuration

It now shows descriptions with their correct URLs when there are videos in the
search results, pulling content_xpath from snippet-description instead of
snippet-content.

Suggested-by: @eagle-dogtooth https://github.com/searx/searx/issues/2857#issuecomment-869119968
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-13 21:27:02 +02:00
Markus Heiser 9afb845a00 [enh] add Brave-Search engine (XPath)
Add https://search.brave.com which was requested by @kaonashi696 [1].
This patch was suggested by @eagle-dogtooth [2].

[1] https://github.com/searx/searx/issues/2857
[2] https://github.com/searx/searx/issues/2857#issuecomment-867199241

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-13 21:26:05 +02:00
Allen 7e3a30940b
[mod] Change shortcut to azl 2021-09-05 20:52:11 +02:00
Allen 8b6d5a0e5b
[enh] Add azlyrics.com
Upstream example query: https://search.azlyrics.com/search.php?q=The+verbal+acupunture&w=lyrics&p=1
2021-08-31 01:46:55 +02:00
Finn b1e5ff3cf7
[fix] Update xpaths for petalsearch.com (#2932) 2021-08-12 18:27:15 +02:00
Allen be401469d2
[mod] Apply HTTPS where possible + fix small typo (#2922) 2021-08-07 14:56:01 +02:00
Allen 76606e7372
[enh] Add whaleslide.com (#2861)
Upstream example: https://whaleslide.com/search/web/runlevels

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-08-03 21:31:08 +02:00
Noémi Ványi dc99fec831 [fix] add new list element to petal so it does not overwrite previous engines 2021-08-03 21:21:28 +02:00
Allen ef246b7661
[enh] Add Petalsearch.com engine (xpath) (#2897)
* [enh] Add Petalsearch.com engine (xpath)

Upstream example query:
https://petalsearch.com/search?query=Felis%20lynx%20Linnaeus&sregion=de&pn=1

* Fix Petalsearch title_xpath

Fixes the title_xpath from Petalsearch which got changed in the meantime.
2021-08-03 21:15:34 +02:00
Allen 36cf794cfa [enh] Add woxikon.de synonyme (xpath)
Upstream example query:
https://synonyme.woxikon.de/synonyme/test.php
2021-08-02 21:58:17 +02:00
Allen e83c5fd0fd [enh] Add gpodder.net (JSON)
Upstream query example:
https://gpodder.net/search.json?q=linux
2021-08-02 21:56:59 +02:00
Allen ec4e48e5df [enh] Add pagination support for Library Genesis
[enh] Add pagination support for Library Genesis and add it to "files" category too.
2021-08-02 21:52:12 +02:00
Noémi Ványi bb724cabf9
Merge pull request #2898 from allendema/ask.com
[enh] Add ask.com engine
2021-08-02 20:12:49 +02:00
Marc Abonce Seguin a5839a66d6
Update onion engines to v3 (#2904)
downgrade httpx:
PR https://github.com/encode/httpx/pull/1522
made some breaking changes in AsyncHTTPTransport that affect
our code in https://github.com/searx/searx/blob/master/searx/network/client.py

remove not_evil which has been down for a while now:
https://old.reddit.com/r/onions/search/?q=not+evil&restrict_sr=on&t=year
2021-08-02 20:03:55 +02:00
Allen 1d60d4253a
[enh] Add Kaufland.de
thumbnail_xpath not working currently.
2021-07-29 13:34:58 +02:00
Allen 573f91143c
[enh] Add ask.com engine 2021-07-19 18:24:39 +02:00
Adam Tauber 198aad43e0 [enh] add mongodb offline engine 2021-07-05 19:47:15 +02:00
Adam Tauber 2af7c60598 [mod] disable wikimini engine by default 2021-07-05 19:40:51 +02:00
Alexandre Flament 8bf216eab6 [mod] google: add "use_mobile_ui" parameter to use mobile endpoint.
disable by default, it has to be enabled in settings.yml

related to  #159
2021-07-03 16:53:31 +02:00
Jordan Webb 60ad4118d6
Add json_engine configuration for ManKier 2021-05-31 13:54:50 -05:00
Noémi Ványi 0267563970
Merge pull request #2830 from jordemort/pypi
Add xpath configuration for PyPI
2021-05-31 20:36:43 +02:00
Adam Tauber c8d2b5eb34 [doc] add info about redis engine dependency 2021-05-30 19:25:26 +02:00
Adam Tauber 97269be680 [enh] add redis offline engine 2021-05-30 19:20:17 +02:00
Jordan Webb 66d06b05fe
Add xpath configuration for PyPI 2021-05-28 16:32:32 -05:00
Allen 28e4ef9173
Adds Dogpile as an engine (#2822)
* Add Dogpile engine

Example Query:
[https://www.dogpile.com/serp?q=streisand+effect](https://www.dogpile.com/serp?q=streisand+effect)

* Remove double "engines: " text
2021-05-26 23:39:32 +02:00
Noémi Ványi c486adf8f7 Minor fixes to wikimini engine 2021-05-26 23:34:25 +02:00
LL Productions FR bed044cc62 Add Wikimini
Fixed a mistake

Update settings.yml

Squashed 3 commits that is for adding Wikimini to Searx
2021-05-22 21:38:00 +02:00
Markus Heiser 0647b34b1d [fix] engine archive is - search_url has been changed
BTW: set soft_max_redirects from *archive is* by 1, to prevent logging::

    DEBUG:httpx._client:HTTP Request: GET https://archive.is/search/?q=www.python.org "HTTP/2 302 Found"
    DEBUG:httpx._client:HTTP Request: GET https://archive.is/www.python.org "HTTP/2 200 OK"
    DEBUG:searx:archive is: ErrorContext('searx/search/processors/online.py', 110, 'count_error(self.engine_name,', None, '1 redirects, maximum: 0', ('200', 'OK', 'archive.is')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-22 20:42:24 +02:00
Adam Tauber 9b5415ea2f [mod] disable /stats page by default to prevent potential data leak 2021-05-13 22:21:46 +02:00
Noémi Ványi 8e90a214ce Add sqlite engine
Closes #2808
2021-05-13 21:40:25 +02:00
Alexandre Flament 14fe1779b7 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-05-03 21:39:54 +02:00