lasers/searx
1
0
Fork 0
mirror of https://github.com/searx/searx synced 2024-11-29 13:40:23 +01:00
Code Issues Projects Releases Wiki Activity
4,452 Commits 19 Branches 18 Tags 37 MiB
f231d79a5d
Commit Graph

4452 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Markus Heiser
f231d79a5d [fix] engine: Semantic Scholar (Science) // rework & fix 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-03-05 20:59:11 +01:00
Noémi Ványi
c56f2f1d6b Skip result in Semantic Scholar engine if URL is missing 2022-03-03 22:06:04 +01:00
searx-bot
e2ab703f3e
Update searx.data - update_firefox_version.py (#3171) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-03-02 22:29:00 +01:00
searx-bot
c9777de0d5
Update searx.data - update_wikidata_units.py (#3170) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-03-02 22:23:06 +01:00
Marc Abonce Seguin
c9e6d9f5f6
Fix Tor proxy when using httpx 0.21.x (#3165) 
## What does this PR do?

This should fix #3164.
The problem is that `httpx` keeps making breaking changes to their library, so we just have to adjust the code a little bit to make it work with the new version of the library.


## Related issues
Closes  #3164
 2022-03-01 20:21:25 +01:00
Noémi Ványi
0669bfd7a5
Fix issues in network after updating httpx to 0.21.x (#3169) 
* [mod] upgrade httpx 0.21.2

httpx 0.21.2 and httpcore 0.14.4 fix multiple issues:
* https://github.com/encode/httpx/releases/tag/0.21.2
* https://github.com/encode/httpcore/releases/tag/0.14.4

so most of the workarounds in searx.network have been removed.

* pick even more changes from searxng

Co-authored-by: Alexandre Flament <alex@al-f.net>
 2022-02-28 22:05:20 +01:00
searx-bot
0248777f95
Update searx.data - update_ahmia_blacklist.py (#3158) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
 2022-02-11 21:24:50 +01:00
searx-bot
22ecae7d48
Update searx.data - update_currencies.py (#3157) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
 2022-02-11 21:24:43 +01:00
searx-bot
fa2ad3cb03
Update searx.data - update_wikidata_units.py (#3156) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
 2022-02-11 21:24:26 +01:00
searx-bot
bf021c538d
Update searx.data - update_firefox_version.py (#3155) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
 2022-02-11 21:24:12 +01:00
israelyago
3fd18ab51b
Fix digg engine (#3150) 2022-01-30 16:41:53 +01:00
Noémi Ványi
a164585118 Add extra features to Gigablast engine: 
* fast can be enabled to results are returned quicker
* collection can be configured
* search_type can be changed to images or news

Closes #3078
 2022-01-22 19:14:45 +01:00
iko
01e28757d3
Fixed Hoogle engine (#3146) 2022-01-22 18:22:24 +01:00
Noémi Ványi
accba7afb2 Install searx as root in Docker 
Closes #2901
 2022-01-22 18:09:38 +01:00
Noémi Ványi
ea38fea711
Pick image_proxy changes from searxng (#2965) 
* [mod] /image_proxy: don't decompress images

* [fix] image_proxy: always close the httpx respone

previously, when the content type was not an image and some other error,
the httpx response was not closed

* [mod] /image_proxy: use HTTP/1 instead of HTTP/2

httpx: HTTP/2 is slow when a lot data is downloaded.
https://github.com/dalf/pyhttp-benchmark

also, the usage of HTTP/1 decreases the load average

* [mod] searx.utils.dict_subset: rewrite with comprehension

Co-authored-by: Alexandre Flament <alex@al-f.net>
 2022-01-22 13:49:00 +01:00
Alexandre Flament
ad7e00ad03 [fix] startpage autocompletion 2022-01-22 12:18:57 +01:00
Allen
0c351ea364
[enh] Add Tineye reverse image search (#3040) 
* [enh] Add Tineye reverse image search 

Other optional parametesr:

"&sort=crawl_date" can be appended to search_string to sort results by date.
"&domain=example.org" can be implemented to search_string to get results from just one domain.

Public instances could get relatively fast timed-out for 3600s.

* [enh] Add TIneye to settings.yml 

Check if that's the right shortcut.

* [mod] Fix checks

* [mod] Try to fix checks

* [mod] Use Four spaces for indentation

And set paging back to True

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-01-22 12:15:19 +01:00
Noémi Ványi
fd9d6b58d5 Add scheme to img_src and thumbnail_url if missing from URL 
Closes #3092
 2022-01-22 11:59:21 +01:00
Noémi Ványi
148090df12 Minor fixes to satisfy the linter 2022-01-21 17:59:10 +01:00
Alexandre Flament
d592159cc5 [fix] startpage: workaround to use the startpage network 
workaround for the issue #762
 2022-01-21 17:59:10 +01:00
Markus Heiser
036d80ed20 [mod] starpage engine: add comment about Startpage's FFox add-on 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser
a4bc089091 [fix] startpage engine: fetch CAPTCHA & issues related to PR-695 
In case of CAPTCHA raise a SearxEngineCaptchaException and suspend for 7 days.
When get_sc_code() fails raise a SearxEngineResponseException and suspend for 7
days.

[1] https://github.com/searxng/searxng/pull/695

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser
1076d7e52e [fix] Get an actual sc argument from startpage's home page. 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser
a6184ac32c [pylint] Startpage engine 
Fix remarks from pylint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser
4750586fb0 [fix] startpage engine - avoid captcha 
Startpage has introduced new anti-scraping measures that make SearXNG instances
run into captchas:

1. some arguments has been removed and a new `sc` has been added.
2. search path changed from `do/search` to `sp/search`
3. POST request is no longer needed

Closes: https://github.com/searxng/searxng/issues/692
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser
99128537a8 [fix] googel engine - "some results are invalids: invalid content" 
Fix google issues listet in the `/stats?engine=google` and message::

    some results are invalids: invalid content

The log is::

    DEBUG   searx                         : result: invalid content: {'url': 'https://de.wikipedia.org/wiki/Foo', 'title': 'Foo - Wikipedia', 'content': None, 'engine': 'google'}
    WARNING searx.engines.google          : ErrorContext('searx/search/processors/abstract.py', 111, 'result_container.extend(self.engine_name, search_results)', None, 'some results are invalids: invalid content', ()) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
dependabot[bot]
5e45b5abd7
Bump sphinx from 4.3.2 to 4.4.0 (#3143) 
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.3.2 to 4.4.0.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.3.2...v4.4.0)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-21 17:31:05 +01:00
dependabot[bot]
1440cefb8d
Bump flask from 2.0.1 to 2.0.2 (#3142) 
Bumps [flask](https://github.com/pallets/flask) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/pallets/flask/releases)
- [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/flask/compare/2.0.1...2.0.2)

---
updated-dependencies:
- dependency-name: flask
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-21 17:17:38 +01:00
Markus Heiser
26c92d5f50 [fix] google engine: remove adds and fix mobile_ui selector 
1. Fix issue reported in comment [1]
2. Fix XPath selector for the response of google's mobile UI, reported in
   comment [2]

[1] https://github.com/searxng/searxng/pull/777#issuecomment-1015121322
[2] https://github.com/searxng/searxng/pull/777#issuecomment-1015236238

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-20 08:33:53 +01:00
Émilien Devos
a2ec27696c Update XPath for Google engine 2022-01-19 23:03:36 +01:00
dependabot[bot]
6ebe9b15a2
Bump flask from 1.1.2 to 2.0.2 (#2881) 
* Bump flask from 1.1.2 to 2.0.2

Bumps [flask](https://github.com/pallets/flask) from 1.1.2 to 2.0.1.
- [Release notes](https://github.com/pallets/flask/releases)
- [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/flask/compare/1.1.2...2.0.1)

---
updated-dependencies:
- dependency-name: flask
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* rm non existent version

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
Co-authored-by: Noémi Ványi <sitbackandwait@gmail.com>
 2022-01-17 23:02:32 +01:00
Maciej "RooTer" Urbański
acefa65ac5
Run tests under python 3.10 (#3035) 
* fix SC2086 on mkdir $SEARX_SETTINGS_PATH
* run tests under python 3.10
* Update requirements.txt for now to downgrade transifex

Co-authored-by: Noémi Ványi <sitbackandwait@gmail.com>
 2022-01-17 22:45:01 +01:00
dependabot[bot]
f0c77a91d1
Bump pallets-sphinx-themes from 2.0.1 to 2.0.2 (#3134) 
Bumps [pallets-sphinx-themes](https://github.com/pallets/pallets-sphinx-themes) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/pallets/pallets-sphinx-themes/releases)
- [Changelog](https://github.com/pallets/pallets-sphinx-themes/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/pallets-sphinx-themes/compare/2.0.1...2.0.2)

---
updated-dependencies:
- dependency-name: pallets-sphinx-themes
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-01-16 21:43:40 +01:00
dependabot[bot]
2460d614fa
Bump linuxdoc from 20210324 to 20211220 (#3135) 
Bumps [linuxdoc](https://github.com/return42/linuxdoc) from 20210324 to 20211220.
- [Release notes](https://github.com/return42/linuxdoc/releases)
- [Commits](https://github.com/return42/linuxdoc/commits)

---
updated-dependencies:
- dependency-name: linuxdoc
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-01-16 21:40:06 +01:00
Noémi Ványi
2b6a0f8189 Change sudo docker to docker during image building 2022-01-16 21:31:16 +01:00
Alexandre Flament
fb0180719b [mod] GitHub workflow: use cache 2022-01-16 21:03:19 +01:00
Markus Heiser
74ed96b792 [test.robot] update gecko driver / required by selenium 4.1.0 
Update gecko driver to v0.30.0 [1]

[1] https://github.com/mozilla/geckodriver/releases/tag/v0.30.0

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-16 18:29:20 +01:00
Noémi Ványi
dff070abba Update base of Dockerfile to Alpine 3.15 2022-01-16 16:32:03 +01:00
dependabot[bot]
d8aa365473
Bump selenium from 3.141.0 to 4.1.0 (#3125) 
Bumps [selenium](https://github.com/SeleniumHQ/Selenium) from 3.141.0 to 4.1.0.
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-3.141.0...selenium-4.1.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-16 15:54:01 +01:00
Noémi Ványi
f0842c76e5
Drop Python 3.6 support (#3133) 2022-01-16 15:04:32 +01:00
dependabot[bot]
491208832d
Bump twine from 3.4.2 to 3.7.1 (#3132) 
Bumps [twine](https://github.com/pypa/twine) from 3.4.2 to 3.7.1.
- [Release notes](https://github.com/pypa/twine/releases)
- [Changelog](https://github.com/pypa/twine/blob/main/docs/changelog.rst)
- [Commits](https://github.com/pypa/twine/compare/3.4.2...3.7.1)

---
updated-dependencies:
- dependency-name: twine
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-16 14:09:46 +01:00
dependabot[bot]
30b4f6a700
Bump sphinx-jinja from 1.1.1 to 1.4.0 (#3130) 
Bumps [sphinx-jinja](https://github.com/tardyp/sphinx-jinja) from 1.1.1 to 1.4.0.
- [Release notes](https://github.com/tardyp/sphinx-jinja/releases)
- [Changelog](https://github.com/tardyp/sphinx-jinja/blob/master/ChangeLog)
- [Commits](https://github.com/tardyp/sphinx-jinja/commits/1.4.0)

---
updated-dependencies:
- dependency-name: sphinx-jinja
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-16 14:09:30 +01:00
dependabot[bot]
ab3fc67682
Bump jinja2 from 3.0.2 to 3.0.3 (#3131) 
Bumps [jinja2](https://github.com/pallets/jinja) from 3.0.2 to 3.0.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.0.2...3.0.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 23:27:33 +01:00
dependabot[bot]
c7dc3bc030
Bump sphinx from 4.2.0 to 4.3.2 (#3129) 
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.2.0 to 4.3.2.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.2.0...v4.3.2)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 23:09:18 +01:00
dependabot[bot]
3306fff9db
Bump sphinx-issues from 1.2.0 to 3.0.1 (#3128) 
Bumps [sphinx-issues](https://github.com/sloria/sphinx-issues) from 1.2.0 to 3.0.1.
- [Release notes](https://github.com/sloria/sphinx-issues/releases)
- [Commits](https://github.com/sloria/sphinx-issues/compare/1.2.0...3.0.1)

---
updated-dependencies:
- dependency-name: sphinx-issues
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 22:23:58 +01:00
dependabot[bot]
da453cedc2
Bump lxml from 4.6.3 to 4.7.1 (#3127) 
Bumps [lxml](https://github.com/lxml/lxml) from 4.6.3 to 4.7.1.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.6.3...lxml-4.7.1)

---
updated-dependencies:
- dependency-name: lxml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 22:17:28 +01:00
dependabot[bot]
efc83d2b4d
Bump splinter from 0.15.0 to 0.17.0 (#3126) 
Bumps [splinter](https://github.com/cobrateam/splinter) from 0.15.0 to 0.17.0.
- [Release notes](https://github.com/cobrateam/splinter/releases)
- [Changelog](https://github.com/cobrateam/splinter/blob/master/docs/news.rst)
- [Commits](https://github.com/cobrateam/splinter/compare/0.15.0...0.17.0)

---
updated-dependencies:
- dependency-name: splinter
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 22:17:16 +01:00
dependabot[bot]
c7283d841c
Bump pycodestyle from 2.7.0 to 2.8.0 (#3037) 
Bumps [pycodestyle](https://github.com/PyCQA/pycodestyle) from 2.7.0 to 2.8.0.
- [Release notes](https://github.com/PyCQA/pycodestyle/releases)
- [Changelog](https://github.com/PyCQA/pycodestyle/blob/main/CHANGES.txt)
- [Commits](https://github.com/PyCQA/pycodestyle/compare/2.7.0...2.8.0)

---
updated-dependencies:
- dependency-name: pycodestyle
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 20:31:01 +01:00
dependabot[bot]
b8de673a36
Bump aiounittest from 1.4.0 to 1.4.1 (#3036) 
Bumps [aiounittest](https://github.com/kwarunek/aiounittest) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/kwarunek/aiounittest/releases)
- [Commits](https://github.com/kwarunek/aiounittest/compare/1.4.0...1.4.1)

---
updated-dependencies:
- dependency-name: aiounittest
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 20:29:26 +01:00
Noémi Ványi
179784068f
Bump pylint from 2.10.2 to 2.12.2 (#3124) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.10.2 to 2.12.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/main/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.10.2...v2.12.2)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 20:23:09 +01:00