Commit Graph

2911 Commits

Author SHA1 Message Date
Noémi Ványi 179784068f
Bump pylint from 2.10.2 to 2.12.2 (#3124)
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.10.2 to 2.12.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/main/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.10.2...v2.12.2)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-15 20:23:09 +01:00
Dario Nuevo 1a18adcc16
New files engine: Prowlarr (#3118)
## What does this PR do?

Gives the user the possibility to search their own prowlarr instances.

Info: https://wiki.servarr.com/en/prowlarr
Github: https://github.com/Prowlarr/Prowlarr

## Why is this change important?

Prowlarr searchs multiple upstream search providers, thus allows to use that functionality through searx.
2022-01-15 19:18:15 +01:00
Andy Jones 3ddd0f8944
Update httpx and friends to 0.21.3 (#3121) 2022-01-15 19:16:10 +01:00
Allen 321ddc91bc
[enh] Add autocompleter from Brave (#3109)
* [enh] Add autocompleter from Brave

Raw response example: https://search.brave.com/api/suggest?q=how%20to:%20with%20j

Headers are needed in order to get a 200 response, thus Searx user-agent is used.

Other URL param could be  '&rich=false' or  '&rich=true'.
2022-01-15 19:08:53 +01:00
Noémi Ványi 82ac634070 make port configurable in MySQL engine
Closes #3117
2022-01-11 22:49:53 +01:00
Dario Nuevo 8f07442fb6
feature: new engine xpath_flex (#3119) 2022-01-11 22:44:19 +01:00
Dario Nuevo d1f6e0a3b1
products results: add possibility to show if a product is in stock or not.. (#3120) 2022-01-11 22:39:08 +01:00
searx-bot 1b1eaa6630
Update searx.data - update_firefox_version.py (#3079)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-01-07 21:49:50 +01:00
searx-bot bf96bf5ce4
Update searx.data - update_ahmia_blacklist.py (#3080)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-01-07 21:49:29 +01:00
Allen 0c2165324d
[Fix] Add suggestions + Fix xpaths (#3082)
* [mod] Add Suggestion to Petalsearch

* [Fix] Changed xpath for Petalsearch
2022-01-07 21:49:08 +01:00
Émilien Devos 8cde08ded2
Disable onesearch by default (#3099)
onesearch is not available everywhere and thus display an error by default in searx
2022-01-07 21:42:51 +01:00
Finn 5dc886136b
[fix] Qwant: Remove extra q from URL (#3091)
Fixes #3090
2022-01-07 21:41:39 +01:00
dalf c11b0189a8 Update searx.data - update_wikidata_units.py 2022-01-01 06:10:23 +00:00
israelyago b90616a25f
Remove categories from onesearch config
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-11-18 08:19:19 -03:00
israelyago 6b3915a2dc
Removed paging from onesearch config
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-11-18 08:18:50 -03:00
israelyago 0d28fd2efe
Merge branch 'master' into onesearch-engine 2021-11-17 15:27:11 -03:00
Israel Yago Pereira f1f3ad97d9 Remove debug log from onesearch engine 2021-11-17 15:15:17 -03:00
Israel Yago Pereira 4b785677d8 Onesearch pagination 2021-11-17 15:14:43 -03:00
Israel Yago Pereira 51530bc394 Fix code style 2021-11-17 15:14:43 -03:00
Israel Yago Pereira 258c6fbd5a Onesearch engine without pagination 2021-11-17 15:14:43 -03:00
Israel Yago Pereira 8e00249633 WIP: onesearch engine 2021-11-17 15:14:43 -03:00
Markus Heiser 4d36aee57b [fix] engine - yahoo: rewrite and fix issues
Languages are supported by mapping the language to a domain.  If domain is not
found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.

Closes #3020

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16 20:30:10 +01:00
0xhtml ebbb9f60af [fix] Prevent missing setting error in ranking
Prevent error when the prefer_configured_language setting is missing.

Fixes #3063
2021-11-16 16:14:38 +01:00
searx-bot db2e8fd8b2
Update searx.data - update_wikidata_units.py (#3050)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2021-11-15 20:32:34 +01:00
searx-bot 4129233774
Update searx.data - update_ahmia_blacklist.py (#3049)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2021-11-15 20:32:25 +01:00
searx-bot bd9c03b483
Update searx.data - update_currencies.py (#3048)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2021-11-15 20:32:14 +01:00
searx-bot abe43c4702
Update searx.data - update_external_bangs.py (#3047)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2021-11-15 20:32:03 +01:00
Finn 8c3454fd1b
[enh] Improve ranking based on language (#3053)
Add configurable setting to rank search results higher when part of the
domain (e.g. 'en' in 'en.wikipedia.org' or 'de' in 'beispiel.de')
matches the selected search language. Does not apply to e.g. 'be' in
'youtube.com'.

Closes #206
2021-11-15 20:31:22 +01:00
Noémi Ványi 967e20dd1e adjust comment based on previous patch 2021-11-14 17:51:22 +01:00
Noémi Ványi dff7ee91f9 Check for settings under /etc/searx/settings.yml first
The patch introduced earlier broke the behaviour for instance
admins running searx from packages. This fix aims to provide
compatibility for everyone.

Closes #3061
2021-11-14 17:46:01 +01:00
Israel Yago Pereira a5fd30bf4d fix wrong func call 2021-11-12 13:12:50 -03:00
jecarr 3b3fb93074
Add codebase settings.yml to settings_loader.get_user_settings_path() 2021-11-10 16:16:05 +13:00
Alexandre Flament bc60d834c5 [enh] verify that Tor proxy works every time searx starts
based on @MarcAbonce commit on searx
2021-10-13 00:06:37 -07:00
Noémi Ványi 3bcca43abf Fix qwant engine, only get results from categories
Closes #3014
2021-10-12 20:06:37 +02:00
Alexandre Flament ee86a63556 [enh] flask debug mode: reload the app when searx/settings.yml changes 2021-10-10 21:38:35 +02:00
Noémi Ványi a0fb8ebeaf Fix style errors in Qwant engine 2021-10-10 21:13:55 +02:00
Markus Heiser 263db54aa9 [fix] qwant engine - prevent API locale exception on lang 'all'
Has been reported in [1], error message::

    Error
        Error: searx.exceptions.SearxEngineAPIException
        Percentage: 0
        Parameters: ('API error::locale must be a string,locale must be one of
        the following values: en_gb, en_ie, en_us, en_ca, en_in, en_my, en_au,
        en_nz, cy_gb, gd_gb, de_de, de_ch, de_at, fr_fr, br_fr, fr_be, fr_ch,
        fr_ca, fr_ad, fc_ca, ec_ca, co_fr, es_es, es_ar, es_cl, es_co, es_mx,
        es_pe, es_ad, ca_es, ca_ad, ca_fr, eu_es, eu_fr, it_it, it_ch, pt_br,
        pt_pt, pt_ad, nl_be, nl_nl, pl_pl, zh_hk, zh_cn, fi_fi, bg_bg, et_ee,
        hu_hu, da_dk, nb_no, sv_se, ko_kr, th_th, cs_cz, ro_ro, el_gr',)
        File name: searx/engines/qwant.py:114
        Function: response
        Code: raise SearxEngineAPIException('API error::' + msg)

[1] https://github.com/searxng/searxng/issues/222

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-10 21:08:48 +02:00
Markus Heiser b10c1346d7 [fix] qwant engine - prevent exception on date/time value is None
Has been reported in [1], error messages::

  Error
       Error: ValueError
       Percentage: 0
       Parameters: ()
       File name: searx/engines/qwant.py:159
       Function: response
       Code: pub_date = datetime.fromtimestamp(item['date'], None)

    Error
        Error: TypeError
        Percentage: 0
        Parameters: ('an integer is required (got type NoneType)',)
        File name: searx/engines/qwant.py:196
        Function: response
       Code: pub_date = datetime.fromtimestamp(item['date'])

Fix timedelta from seconds to milliseconds [1], error message::

    Error
        Error: TypeError
        Percentage: 0
        Parameters: ('unsupported type for timedelta seconds component: NoneType',)
        File name: searx/engines/qwant.py:195
        Function: response
        Code: length = timedelta(seconds=item['duration'])

[1] https://github.com/searxng/searxng/issues/222

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-10 21:08:48 +02:00
Alexandre Flament 7aa94b7084 [mod] qwant engine: fix typos / minor change
minor modification of commit 628b5703f3aeeed117772696f83efb344d6f337e
(no functionnal change)
2021-10-10 21:08:48 +02:00
Markus Heiser 2b69710aef [mod] improve video results of the qwant engine
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-10 21:08:48 +02:00
Markus Heiser 3205785059 [fix] Qwant engines - implement API v3 and add 'quant videos'
The implementation uses the Qwant API (https://api.qwant.com/v3). The API is
undocumented but can be reverse engineered by reading the network log of
https://www.qwant.com/ queries.

This implementation is used by different qwant engines in the settings.yml::

  - name: qwant
    categories: general
    ...
  - name: qwant news
    categories: news
    ...
  - name: qwant images
    categories: images
    ...
  - name: qwant videos
    categories: videos
    ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-10 21:08:48 +02:00
Allen b0888c6ca3
[enh] Add Pagination to Wiby / Fix Kaufland (#3000)
* [fix] Fix Kaufland engine

Changed Xpath expressions

* [enh] Remove tracking params from Kaufland results

* [enh] Add pagination to Wiby

* [fix] Properly select title_xpath

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-10-10 20:52:06 +02:00
Allen 79dc10e382
[fix] Update about section of Invidious and Rumble + Change filtron error wording (#2959)
* [fix] Update about section of Invidious

Another website and new documentation

* [fix] Correct engine name in for Rumble (#11)

* [fix] Wording for Filtron error message (#12)
2021-10-10 16:44:21 +02:00
Allen d91f1ede61
[mod] Fix Libgen + Uncomment Ebay and Urbandictiorany (#2986)
- Change Libgen provider and use https by default.
- Umcomment Urbandictionary but disable it by default, it is working.
- Uncomment Ebay as it is working correctly.
(For ebay in the future: base_url should be changed from settings.yml  just like peertube or invidious)
2021-10-10 15:42:43 +02:00
Finn c45c87f293
[fix] CSS: improve text overflow of custom select (#2985)
Partly-fixes: #2984
2021-10-10 15:38:18 +02:00
Noémi Ványi c3034bd883
Merge pull request #3002 from noctux/fix-custom-languages-for-engines
[fix] use engine-type when looking up supported_languages from JSON files
2021-10-10 14:47:35 +02:00
Faiazov Dmitrii 5b5d280140 change firefox versions to latest 2021-10-10 09:25:40 +03:00
Simon Schuster 23d6c9c798 [fix] use engine-type when looking up supported_languages from JSON files
searx/data/engines_languages.json stores language information for
several searchengines in a json endoded dict that maps engine-"types" to
their supported languages; for instance there is an entry "google",
mapping to the supported languages of the google engine.

However, the lookup code did not use the engine 'type' (as in: the
filename searx/engines/<enginetype>.py), but instead the manually
configured engine name from settings.yml when querying. This is
problematic as soon as users start to specify additional engine
instances with custom names in the config file, as for instance
suggested as a workaround for multilingual search in the manual[0]:

> engines:
>   - name : google english
>     engine : google
>     language : english

Here, the engine name "google english" will be used for the lookup in
the json file, which does not exist. The empty supported_languages then
lead to a type error later in the processing callchain.

This patch changes the behaviour to use the engine's entry-"type"
("google" in the above example) for the lookup. This should fix bug #2928.

0: https://searx.github.io/searx/user/search_syntax.html#multilingual-search
2021-10-06 19:33:43 +02:00
Igor Rzegocki 54a2cd040e
healthcheck endpoint 2021-10-03 10:25:56 +02:00
Noémi Ványi 53c4031f96 [fix] minor style fixes after picking open street map changes 2021-10-02 15:05:03 +02:00