lasers/searx
1
0
Fork 0
mirror of https://github.com/searx/searx synced 2024-11-29 21:50:11 +01:00
Code Issues Projects Releases Wiki Activity
4,272 Commits 19 Branches 18 Tags 37 MiB
76606e7372
Commit Graph

4272 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Allen
76606e7372
[enh] Add whaleslide.com (#2861) 
Upstream example: https://whaleslide.com/search/web/runlevels

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2021-08-03 21:31:08 +02:00
Noémi Ványi
dc99fec831 [fix] add new list element to petal so it does not overwrite previous engines 2021-08-03 21:21:28 +02:00
searx-bot
291b154a61
Update searx.data - update_currencies.py (#2916) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
 2021-08-03 21:17:05 +02:00
Allen
ef246b7661
[enh] Add Petalsearch.com engine (xpath) (#2897) 
* [enh] Add Petalsearch.com engine (xpath)

Upstream example query:
https://petalsearch.com/search?query=Felis%20lynx%20Linnaeus&sregion=de&pn=1

* Fix Petalsearch title_xpath

Fixes the title_xpath from Petalsearch which got changed in the meantime.
 2021-08-03 21:15:34 +02:00
Allen
36cf794cfa [enh] Add woxikon.de synonyme (xpath) 
Upstream example query:
https://synonyme.woxikon.de/synonyme/test.php
 2021-08-02 21:58:17 +02:00
Allen
e83c5fd0fd [enh] Add gpodder.net (JSON) 
Upstream query example:
https://gpodder.net/search.json?q=linux
 2021-08-02 21:56:59 +02:00
Allen
ec4e48e5df [enh] Add pagination support for Library Genesis 
[enh] Add pagination support for Library Genesis and add it to "files" category too.
 2021-08-02 21:52:12 +02:00
Émilien Devos
ee443d9739 Fix google images 
Proposed fix in https://github.com/searx/searx/pull/2115#issuecomment-876716010

Closes #2914
 2021-08-02 20:14:54 +02:00
Noémi Ványi
bb724cabf9
Merge pull request #2898 from allendema/ask.com 
[enh] Add ask.com engine
 2021-08-02 20:12:49 +02:00
Samuel Dudík
13a608b0b8
Fix Seznam engine (#2905) 
## What does this PR do?

Fixes the Seznam engine by updating XPath strings.

## Why is this change important?

Without this PR Seznam returns no results.

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2021-08-02 20:09:31 +02:00
Marc Abonce Seguin
a5839a66d6
Update onion engines to v3 (#2904) 
downgrade httpx:
PR https://github.com/encode/httpx/pull/1522
made some breaking changes in AsyncHTTPTransport that affect
our code in https://github.com/searx/searx/blob/master/searx/network/client.py

remove not_evil which has been down for a while now:
https://old.reddit.com/r/onions/search/?q=not+evil&restrict_sr=on&t=year
 2021-08-02 20:03:55 +02:00
Noémi Ványi
05b9ceddd5
Merge pull request #2915 from allendema/kaufland 
[enh] Add Kaufland.de
 2021-08-02 20:00:09 +02:00
Allen
1d60d4253a
[enh] Add Kaufland.de 
thumbnail_xpath not working currently.
 2021-07-29 13:34:58 +02:00
Allen
573f91143c
[enh] Add ask.com engine 2021-07-19 18:24:39 +02:00
Noémi Ványi
3b192e6387 Add blog post about NoSQL datastores 2021-07-16 12:08:55 +02:00
Adam Tauber
a8988885a5 [fix] pep8 2021-07-05 19:57:53 +02:00
Adam Tauber
198aad43e0 [enh] add mongodb offline engine 2021-07-05 19:47:15 +02:00
Adam Tauber
b2ed65eb6d [fix] initialize redis engine at the right time 2021-07-05 19:40:51 +02:00
Adam Tauber
2af7c60598 [mod] disable wikimini engine by default 2021-07-05 19:40:51 +02:00
Markus Heiser
cbc50a9bc5 [fix] google-news engine - KeyError: 'hl in request 
Since we added

- 1c67b6aec [enh] google engine: supports "default language"

there is a KeyError: 'hl in request,error pattern::

    ERROR:searx.searx.search.processor.online:engine google news : exception : 'hl'
    Traceback (most recent call last):
      File "searx/search/processors/online.py", line 144, in search
        search_results = self._search_basic(query, params)
      File "searx/search/processors/online.py", line 118, in _search_basic
        self.engine.request(query, params)
      File "searx/engines/google_news.py", line 97, in request
        if lang_info['hl'] == 'en':
      KeyError: 'hl'

Closes: https://github.com/searxng/searxng/issues/154
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-03 16:53:31 +02:00
Markus Heiser
98a63058e5 [fix] google answers: normalize space of the answers. 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-03 16:53:31 +02:00
Markus Heiser
412677d495 [mod] google engine: reduce mobile UI parameters to what is needed 
Reverse engineering shows that not all of the parameters used by google's mobile
UI (aka "more results" button) are needed [1].

[1] https://github.com/searxng/searxng/pull/160#issuecomment-865013625

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-03 16:53:31 +02:00
Alexandre Flament
8bf216eab6 [mod] google: add "use_mobile_ui" parameter to use mobile endpoint. 
disable by default, it has to be enabled in settings.yml

related to  #159
 2021-07-03 16:53:31 +02:00
Alexandre Flament
3863f5a83f [enh] google engine: supports "default language" 
Same behaviour behaviour than Whoogle [1].  Only the google engine with the
"Default language" choice "(all)"" is changed by this patch.

When searching for a locate place, the result are in the expect language,
without missing results [2]:

  > When a language is not specified, the language interpretation is left up to
  > Google to decide how the search results should be delivered.

The query parameters are copied from Whoogle.  With the ``all`` language:

- add parameter ``source=lnt``
- don't use parameter ``lr``
- don't add a ``Accept-Language`` HTTP header.

The new signature of function ``get_lang_info()`` is:

    lang_info = get_lang_info(params, lang_list, custom_aliases, supported_any_language)

Argument ``supported_any_language`` is True for google.py and False for the other
google engines.  With this patch the function now returns:

- query parameters: ``lang_info['params']``
- HTTP headers: ``lang_info['headers']``
- and as before this patch:
  - ``lang_info['subdomain']``
  - ``lang_info['country']``
  - ``lang_info['language']``

[1] https://github.com/benbusby/whoogle-search
[2] https://github.com/benbusby/whoogle-search/releases/tag/v0.5.4
 2021-07-03 16:53:31 +02:00
dependabot[bot]
e628d75727 Bump httpx[http2] from 0.17.1 to 0.18.2 
Bumps [httpx[http2]](https://github.com/encode/httpx) from 0.17.1 to 0.18.2.
- [Release notes](https://github.com/encode/httpx/releases)
- [Changelog](https://github.com/encode/httpx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/httpx/compare/0.17.1...0.18.2)

---
updated-dependencies:
- dependency-name: httpx[http2]
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-07-03 15:45:33 +02:00
dependabot[bot]
75e85f2a38 Bump sphinx-tabs from 2.1.0 to 3.1.0 
Bumps [sphinx-tabs](https://github.com/executablebooks/sphinx-tabs) from 2.1.0 to 3.1.0.
- [Release notes](https://github.com/executablebooks/sphinx-tabs/releases)
- [Changelog](https://github.com/executablebooks/sphinx-tabs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/executablebooks/sphinx-tabs/compare/v2.1.0...v3.1.0)

---
updated-dependencies:
- dependency-name: sphinx-tabs
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-07-03 15:37:10 +02:00
dalf
66206bfb36 Update searx.data - update_ahmia_blacklist.py 2021-07-03 15:31:09 +02:00
dalf
5bfcc120ba Update searx.data - update_wikidata_units.py 2021-07-03 15:21:20 +02:00
dalf
5c57f83ac6 Update searx.data - update_currencies.py 2021-07-03 15:21:01 +02:00
Noémi Ványi
c5d63a5c97
Merge pull request #2837 from searx/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-06-03 22:17:29 +02:00
dalf
9e7a68480c Update searx.data - update_currencies.py 2021-06-03 22:13:13 +02:00
Noémi Ványi
2db2dfa874
Merge pull request #2836 from searx/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-06-03 22:12:38 +02:00
dalf
89acf2462a Update searx.data - update_wikidata_units.py 2021-06-03 22:03:36 +02:00
Noémi Ványi
0d10ad5602
Merge pull request #2835 from searx/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-06-03 21:56:31 +02:00
dalf
3033b3297f Update searx.data - update_firefox_version.py 2021-06-03 21:48:45 +02:00
Noémi Ványi
6b738021f7
Merge pull request #2834 from searx/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-06-03 21:48:08 +02:00
dalf
fc5973cf95 Update searx.data - update_ahmia_blacklist.py 2021-06-03 21:44:22 +02:00
Noémi Ványi
d89b42879b
Merge pull request #2829 from jordemort/mankier 
Add json_engine configuration for ManKier
 2021-06-03 21:40:37 +02:00
Jordan Webb
60ad4118d6
Add json_engine configuration for ManKier 2021-05-31 13:54:50 -05:00
Noémi Ványi
0267563970
Merge pull request #2830 from jordemort/pypi 
Add xpath configuration for PyPI
 2021-05-31 20:36:43 +02:00
Adam Tauber
c8d2b5eb34 [doc] add info about redis engine dependency 2021-05-30 19:25:26 +02:00
Adam Tauber
01a8a5814a [fix] pylint 2021-05-30 19:25:03 +02:00
Adam Tauber
ea7ccf2422 [fix] correct kv template formatting and remove internal data 2021-05-30 19:20:46 +02:00
Adam Tauber
97269be680 [enh] add redis offline engine 2021-05-30 19:20:17 +02:00
Jordan Webb
66d06b05fe
Add xpath configuration for PyPI 2021-05-28 16:32:32 -05:00
Noémi Ványi
22a79a4896 Add blog post about SQL servers 2021-05-27 10:29:07 +02:00
Allen
28e4ef9173
Adds Dogpile as an engine (#2822) 
* Add Dogpile engine

Example Query:
[https://www.dogpile.com/serp?q=streisand+effect](https://www.dogpile.com/serp?q=streisand+effect)

* Remove double "engines: " text
 2021-05-26 23:39:32 +02:00
Noémi Ványi
c486adf8f7 Minor fixes to wikimini engine 2021-05-26 23:34:25 +02:00
LL Productions FR
bed044cc62 Add Wikimini 
Fixed a mistake

Update settings.yml

Squashed 3 commits that is for adding Wikimini to Searx
 2021-05-22 21:38:00 +02:00
Markus Heiser
0647b34b1d [fix] engine archive is - search_url has been changed 
BTW: set soft_max_redirects from *archive is* by 1, to prevent logging::

    DEBUG:httpx._client:HTTP Request: GET https://archive.is/search/?q=www.python.org "HTTP/2 302 Found"
    DEBUG:httpx._client:HTTP Request: GET https://archive.is/www.python.org "HTTP/2 200 OK"
    DEBUG:searx:archive is: ErrorContext('searx/search/processors/online.py', 110, 'count_error(self.engine_name,', None, '1 redirects, maximum: 0', ('200', 'OK', 'archive.is')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-22 20:42:24 +02:00