Commit Graph

196 Commits

Author SHA1 Message Date
Alexandre Flament 5fb9a5c7c1
Merge pull request #2411 from dalf/update-secret-key-check
Update secret key check
2020-12-27 18:04:53 +01:00
Markus Heiser b6dcf90816 Revert "[fix] sphinx 3.1 CSS issues"
This reverts commit 0616684baa.

Since PR https://github.com/sphinx-doc/sphinx/pull/7878 has been merged into
Spinx-doc (v3.1.2), this patch is no longer needed:

  See sphinx-doc project, PR 7838 & 7484 with elementary patch to the basic CSS:

  - https://github.com/sphinx-doc/sphinx/issues/7838#issuecomment-646009605
  - https://github.com/sphinx-doc/sphinx/pull/7484#issuecomment-646058972

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-27 11:27:42 +01:00
Alexandre Flament eb1b5289d5 [mod] documentation: change the jinja context doesn't depend on searx.webapp
Before this commit, in the documentation, the jinja context is 'webapp' and contains
the global variable in the searx.webapp module.

This commit changes this to include only the mandatory variables to build the
documentation.
2020-12-27 10:00:35 +01:00
Markus Heiser 3beede7ab7 [doc] describe 'make books/{name}.html' and 'books/{name}.pdf'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-24 13:55:22 +01:00
Alexandre Flament 02fc4147ce [mod] dictzone, translated, currency_convert: use engine_type online_curency and online_dictionnary 2020-12-17 11:39:36 +01:00
Alexandre Flament 7ec8bc3ea7 [mod] split searx.search into different processors
see searx.search.processors.abstract.EngineProcessor

First the method searx call the get_params method.

If the return value is not None, then the searx call the method search.
2020-12-17 11:39:36 +01:00
Alexandre Flament 9bc1856e2b [mod] themes: remove legacy, courgette and pix-art themes 2020-12-17 11:33:28 +01:00
Alexandre Flament 7865566338
Merge pull request #2384 from return42/improve-doc
Improve doc of the installation skripts
2020-12-14 18:22:12 +01:00
Markus Heiser dc94a1d4c0 [doc] add a tip about how to brand and stash a searx instance setup
BTW: drop a trailing whitespace

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-13 15:20:33 +01:00
Markus Heiser c8d2b19c85 [doc] document why to use sudoer to run intsallation scripts
related to:

- https://gist.github.com/dalf/60450e19b042bd39b9ce470c095d89b3#gistcomment-3263827
- https://github.com/searx/searx/issues/2013#issuecomment-647132313
- https://github.com/searx/searx/issues/2017#issuecomment-647700327

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-13 15:19:04 +01:00
Markus Heiser ea62df1dc0 [doc] update developer quickstart guide
See https://github.com/searx/searx/issues/2369#issuecomment-740652878

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-12 20:31:49 +01:00
Alexandre Flament d703119d3a [enh] add raise_for_httperror
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before

the check is done in poolrequests.py (was before in search.py).

update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
2020-12-11 14:37:08 +01:00
Markus Heiser a70b9b9f61 [doc] recommend to use 'use_default_settings=True'
Since #2291 is merged, it is recommend to use::

  use_default_settings=True

1. Add a template file use_default_settings.yml::

    SEARX_SETTINGS_TEMPLATE="${REPO_ROOT}/utils/templates/etc/searx/use_default_settings.yml"

2. In Chapter "Configuration" recommend to make use of
   'use_default_settings=True' and describe it

3. Rewrite of docs/admin/settings.rst
   - move chapter 'settings.yml location' to the top
   - update and split chapter 'Global Settings'

4. Add environment SEARX_SETTINGS_TEMPLATE to .config.sh

5. Use environment $SEARX_SETTINGS_TEMPLATE in the utils/searx.sh script

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-08 20:19:10 +01:00
Alexandre Flament 925bb561a2
Merge pull request #2352 from dalf/no_http
Remove HTTP connections as much as possible
2020-12-06 10:18:49 +01:00
Alexandre Flament f0054d67f1 [fix] wikipedia engine: don't raise an error when the query is not found
Add a new parameter "raise_for_status", set by default to True.
When True, any HTTP status code >= 300 raise an exception ( #2332 )
When False, the engine can manage the HTTP status code by itself.
2020-12-04 20:04:39 +01:00
Alexandre Flament 38d32337e7 [mod] documentations & comments: update http://* URL to https://*.
About http://lesscss.org see https://github.com/less/less-docs/issues/520
2020-12-04 16:52:25 +01:00
Alexandre Flament 1d0c368746 [enh] record details exception per engine
add an new API /stats/errors
2020-12-03 10:22:48 +01:00
Alexandre Flament a1e6bc4cee
Merge pull request #2291 from dalf/settings2
[enh] user settings can relied on the default settings
2020-12-01 14:57:12 +01:00
Noémi Ványi 4a36a3044d
Add recoll engine (#2325)
recoll is a local search engine based on Xapian:
http://www.lesbonscomptes.com/recoll/

By itself recoll does not offer web or API access,
this can be achieved using recoll-webui:
https://framagit.org/medoc92/recollwebui.git

This engine uses a custom 'files' result template

set `base_url` to the location where recoll-webui can be reached
set `dl_prefix` to a location where the file hierarchy as indexed by recoll can be reached
set `search_dir` to the part of the indexed file hierarchy to be searched, use an empty string to search the entire search domain
2020-11-30 08:35:15 +01:00
Alexandre Flament b4b81a5e1a [enh] settings.yml: add use_default_settings option (2nd version) 2020-11-27 19:40:04 +01:00
Alexandre Flament 1cfe7f2a75 [enh] settings.yml: add use_default_settings option
This change is backward compatible with the existing configurations.

If a settings.yml loaded from an user defined location (SEARX_SETTINGS_PATH or /etc/searx/settings.yml),
then this settings can relied on the default settings.yml with this option:
user_default_settings:True
2020-11-26 18:27:27 +01:00
Alexandre Flament 3786920df9 [enh] Add multiple outgoing proxies
credits go to @bauruine see https://github.com/searx/searx/pull/1958
2020-11-20 15:29:21 +01:00
Pierre Chevalier 482da5c6c7 Correct what was obviously a typo in vir(t)ualenv. 2020-11-13 16:06:16 +01:00
rachmadani haryono c03e4c86bc
Feature/standalone searx update (#1591)
* chg: dev: update standalone_searx

parent d8a5df721b33dd8a7cc9e21dba4060f21d629f69
author rachmadaniHaryono <foreturiga@gmail.com> 1603896594 +0800
committer rachmadaniHaryono <foreturiga@gmail.com> 1603896619 +0800

chg: dev: debug engine_shortcuts
chg: dev: only initilize if engine is given
chg: dev: split main
chg: dev: standalone_searx
chg: dev: update standalone_searx
chg: doc: remove unnecessary log
chg: test: differentiate travis
chg: test: disable shortcut
chg: test: use default engine settings
fix: dev: category choices
fix: dev: duplicate engine shortcut
fix: dev: travis python3
fix: test:  use empty string as shortcut
fix: test: apkm
fix: test: engine shortcut
fix: test: mypy
fix: test: parameter
fix: test: pep8
fix: test: py2 compatibilities
fix: test: searx settings
fix: test: travis engines
new: dev: deduplicate engine
new: dev: main receive engines parameter
new: dev: parse_argument accept engines parameter
new: dev: split search query from get_result func
new: test: basic result case
Suggestions: use RawTextQuery to make the suggestions URLs. Update all themes accordingly.

* new: doc: searx import and init

* chg: dev: parse_argument

- doc
- run on __main__
- simple parse_args

* chg: doc: module

* chg: dev: import section

- remove unused python path modification
- new required package

* chg: dev: script run

- parse_argument func return directly parsed results
- main func return dict instead json text
- dump directly on sys.stdout.write

* chg: dev: get_search_query and get_search_query func

* chg: dev: main func

- move inner function outside
- return dict instead of json text

* new: dev: add utils to doc sys path

* new: doc: standalone_searx

* fix: doc: run script

* chg: dev: mypy type hint

* chg: dev: SearchQuery don't have attr engines

* chg: dev: reset engines __init__

* chg: test: unit test update

* chg: dev: pylint and flake8

* new: test: standalone_searx

* chg: dev: main func and doc

* chg: dev: import and type hint

* new: dev: main func

- remove get_result func
- single func which just translate dict

* chg: test: put mypy on dev requirement

* chg: doc: update

* new: doc: add standalone_searx module member

* chg: doc: shell command line

* chg: dev: remove mypy

* chg: doc: module docstring
2020-11-04 12:38:54 +00:00
Markus Heiser 45f58a4a2a [fix] searx configuration - quote bash calls in sed replacements
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-11-02 10:02:59 +01:00
Markus Heiser edef9d2f87 [doc] searx configuration - add 'mkdir /etc/searx' before copy
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-11-02 10:02:59 +01:00
Markus Heiser dd23ad8ba2 [fix] doc - add missing quotation marks in docs/build-templates
Escape the all the quotation marks (" --> \") in docs/build-templates.

The scripts are *bash evaluated* (e.g)::

  eval "echo \"$(< "${REPO_ROOT}/docs/build-templates/searx.rst")\""

Backslash, quotation marks and other need to be escaped.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-11-02 10:02:59 +01:00
Markus Heiser e697d78d5a [mod] make docs-live - remove --port option from sphinx-autobuild
Setting port fixed to 8080 has drawbacks when this port is already in use from
another application [1].  Removing this option from the command line offers the
use of the sphinx-autobuild defaults which are much more flexible [2].

[1] https://github.com/searx/searx/issues/2282
[2] https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-10-30 20:30:20 +01:00
renyhp 3bada084a2
Extract to /usr/local/filtron/local
The /usr/local/filtron/local/go directory will be created by tar.
2020-10-19 17:49:03 +02:00
renyhp aaf9334c79
Fix typo xgd-open > xdg-open 2020-10-19 15:37:45 +02:00
Noémi Ványi f0e6678aac Add blogpost about `command` engine 2020-10-08 13:32:32 +02:00
Alexandre Flament 2cafc5462d [fix] revert PR #2232 and #2230 2020-10-01 16:38:54 +02:00
Alexandre Flament 9a856083df [fix] Sphinx 3.2.1 build: call searx.engines.initialize_engines
Fix admin/engines.html
Fix #2230
2020-09-30 10:20:12 +02:00
Alexandre Flament 6c6985aaf8
Merge pull request #2230 from dalf/fix-sphinx-build
[fix] Sphinx 3.2.1 build: add missing "engines" variable in the jinja context
2020-09-30 09:29:09 +02:00
Alexandre Flament 6d56c754a7 [fix] Sphinx 3.2.1 build: add missing "engines" variable in the jinja context
Fix the admin/engines.html page
2020-09-29 12:30:10 +02:00
Alexandre Flament f204e4903d [fix] migration from github.com/asciimoo/searx to github.com/searx/searx : fix URLs 2020-09-28 16:44:14 +02:00
Markus Heiser 9d60ae1ce6 [fix] missing 'alias' typo in docs/admin/installation-nginx.rst
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-09-28 11:40:45 +02:00
Alexandre Flament 8ffede6140 [fix] docker and k8s: run searx
fix #2181
2020-09-10 10:07:49 +02:00
Adam Tauber 4a913247b2 [enh] add option to configure proxies per engine - closes #1827 2020-09-01 15:59:22 +02:00
Adam Tauber 1f2dc6c647 [enh] add external plugin support 2020-07-28 13:10:40 +02:00
Gardouille b13ccb5ce1 Filtron sample: Add missing comma 2020-07-25 11:34:35 +02:00
Dalf 4a35975b42 [enh] update documentation about docker 2020-07-22 14:58:04 +02:00
Markus Heiser 3fe78c64c6 [fix] minor typos in docs/admin/settings.rst
This comit is a replacement for https://github.com/asciimoo/searx/pull/2005

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-07-13 17:49:07 +02:00
Markus Heiser 72e86be0ce [fix docs] typo in the template to create a new user
closes: https://github.com/asciimoo/searx/issues/2062

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-07-13 17:49:07 +02:00
Lukas van den Berk 4829a76aae
Created new plugin type custom_results. Added new plugin bang_redirect (#2027)
* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Refactored getting search query. Also changed bang operator to ! and is now working.

* Removed prints

* Removed temporary bangs_redirect.js file. Updated plugin documentation

* Added unit test for the bangs plugin

* Fixed a unit test and added 2 more for bangs plugin

* Changed back to default settings.yml

* Added myself to AUTHORS.rst

* Refacored working of custom plugin.

* Refactored _get_bangs_data from list to dict to improve search speed.

* Decoupled bangs plugin from webserver with redirect_url

* Refactored bangs unit tests

* Fixed unit test bangs. Removed dubbel parsing in bangs.py

* Removed a dumb print statement

* Refactored bangs plugin to core engine.

* Removed bangs plugin.

* Refactored external bangs unit tests from plugin to core.

* Removed custom_results/bangs documentation from plugins.rst

* Added newline in settings.yml so the PR stays clean.

* Changed searx/plugins/__init__.py back to the old file

* Removed newline search.py

* Refactored get_external_bang_operator from utils to external_bang.py

* Removed unnecessary import form test_plugins.py

* Removed _parseExternalBang and _isExternalBang from query.py

* Removed get_external_bang_operator since it was not necessary

* Simplified external_bang.py

* Simplified external_bang.py

* Moved external_bangs unit tests to test_webapp.py. Fixed return in search with external_bang

* Refactored query parsing to unicode to support python2

* Refactored query parsing to unicode to support python2

* Refactored bangs plugin to core engine.

* Refactored search parameter to search_query in external_bang.py
2020-07-03 13:25:04 +00:00
Markus Heiser bdd24a6e71
Merge branch 'master' into blog-dev-in-lxc 2020-06-29 06:14:42 +00:00
Noémi Ványi 2addee3893 Use gender neutral pronouns
Closes #2031
2020-06-28 11:40:04 +02:00
Markus Heiser 9c71f620a7 [add blog post] Developing in Linux containers [202006]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-06-24 17:49:38 +02:00
Markus Heiser 18805ee352 [fix] markup of auto generated installation instruction (uWSGI part)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-06-24 15:15:15 +02:00
Markus Heiser 0616684baa [fix] sphinx 3.1 CSS issues
See sphinx-doc project, PR 7838 & 7484 with elementary patch to the basic CSS:

- https://github.com/sphinx-doc/sphinx/issues/7838#issuecomment-646009605
- https://github.com/sphinx-doc/sphinx/pull/7484#issuecomment-646058972

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-06-18 19:02:22 +02:00