mirror of https://github.com/searx/searx
Merge branch 'searx:master' into master
This commit is contained in:
br4nnigan
GitHub
commit
e631313080
|
|
@ -424,7 +424,7 @@ Special thanks to `NLNet <https://nlnet.nl>`__ for sponsoring multiple features
|
|||
- Removed engines: faroo
|
||||
|
||||
Special thanks to `NLNet <https://nlnet.nl>`__ for sponsoring multiple features of this release.
|
||||
Special thanks to https://www.accessibility.nl/english for making accessibilty audit.
|
||||
Special thanks to https://www.accessibility.nl/english for making accessibility audit.
|
||||
|
||||
News
|
||||
~~~~
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@
|
|||
|
||||
## Author's checklist
|
||||
|
||||
<!-- additional notes for reviewiers -->
|
||||
<!-- additional notes for reviewers -->
|
||||
|
||||
## Related issues
|
||||
|
||||
|
|
|
|||
|
|
@ -93,8 +93,8 @@ was created because the majority of the maintainers at the time did not find
|
|||
the new proposed features privacy respecting enough. The most significant issue is with
|
||||
engine metrics.
|
||||
|
||||
Searx is built for privacy conscious users. It comes a unique set of
|
||||
challanges. One of the problems we face is that users rather not report bugs,
|
||||
Searx is built for privacy conscious users. It comes with a unique set of
|
||||
challenges. One of the problems we face is that users rather not report bugs,
|
||||
because they do not want to publicly share what engines they use or what search
|
||||
query triggered a problem. It is a challenge we accepted.
|
||||
|
||||
|
|
@ -124,8 +124,8 @@ instances locally, instead of using public instances.
|
|||
Why should I use SearxNG?
|
||||
#########################
|
||||
|
||||
SearxNG has rolling releases, depencencies updated more frequently, and engines are fixed
|
||||
SearxNG has rolling releases, dependencies updated more frequently, and engines are fixed
|
||||
faster. It is easy to set up your own public instance, and monitor its
|
||||
perfomance and metrics. It is simple to maintain as an instance adminstrator.
|
||||
performance and metrics. It is simple to maintain as an instance administrator.
|
||||
|
||||
As a user, it provides a prettier user interface and nicer experience.
|
||||
|
|
|
|||
|
|
@ -100,7 +100,7 @@ update_conf() {
|
|||
# There is a new version
|
||||
if [ $FORCE_CONF_UPDATE -ne 0 ]; then
|
||||
# Replace the current configuration
|
||||
printf '⚠️ Automaticaly update %s to the new version\n' "${CONF}"
|
||||
printf '⚠️ Automatically update %s to the new version\n' "${CONF}"
|
||||
if [ ! -f "${OLD_CONF}" ]; then
|
||||
printf 'The previous configuration is saved to %s\n' "${OLD_CONF}"
|
||||
mv "${CONF}" "${OLD_CONF}"
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ workers = 4
|
|||
# The right granted on the created socket
|
||||
chmod-socket = 666
|
||||
|
||||
# Plugin to use and interpretor config
|
||||
# Plugin to use and interpreter config
|
||||
single-interpreter = true
|
||||
master = true
|
||||
plugin = python3
|
||||
|
|
|
|||
|
|
@ -0,0 +1,129 @@
|
|||
=====================================
|
||||
Run shell commands from your instance
|
||||
=====================================
|
||||
|
||||
Command line engines are custom engines that run commands in the shell of the
|
||||
host. In this article you can learn how to create a command engine and how to
|
||||
customize the result display.
|
||||
|
||||
The command
|
||||
===========
|
||||
|
||||
When specifyng commands, you must make sure the commands are available on the
|
||||
searx host. Searx will not install anything for you. Also, make sure that the
|
||||
``searx`` user on your host is allowed to run the selected command and has
|
||||
access to the required files.
|
||||
|
||||
Access control
|
||||
==============
|
||||
|
||||
Be careful when creating command engines if you are running a public
|
||||
instance. Do not expose any sensitive information. You can restrict access by
|
||||
configuring a list of access tokens under tokens in your ``settings.yml``.
|
||||
|
||||
Available settings
|
||||
==================
|
||||
|
||||
* ``command``: A comma separated list of the elements of the command. A special
|
||||
token ``{{QUERY}}`` tells searx where to put the search terms of the
|
||||
user. Example: ``['ls', '-l', '-h', '{{QUERY}}']``
|
||||
* ``query_type``: The expected type of user search terms. Possible values:
|
||||
``path`` and ``enum``. ``path`` checks if the uesr provided path is inside the
|
||||
working directory. If not the query is not executed. ``enum`` is a list of
|
||||
allowed search terms. If the user submits something which is not included in
|
||||
the list, the query returns an error.
|
||||
* ``delimiter``: A dict containing a delimiter char and the "titles" of each
|
||||
element in keys.
|
||||
* ``parse_regex``: A dict containing the regular expressions for each result
|
||||
key.
|
||||
* ``query_enum``: A list containing allowed search terms if ``query_type`` is
|
||||
set to ``enum``.
|
||||
* ``working_dir``: The directory where the command has to be executed. Default:
|
||||
``.``
|
||||
* ``result_separator``: The character that separates results. Default: ``\n``
|
||||
|
||||
Customize the result template
|
||||
=============================
|
||||
|
||||
There is a default result template for displaying key-value pairs coming from
|
||||
command engines. If you want something more tailored to your result types, you
|
||||
can design your own template.
|
||||
|
||||
Searx relies on `Jinja2 <https://jinja.palletsprojects.com/>`_ for
|
||||
templating. If you are familiar with Jinja, you will not have any issues
|
||||
creating templates. You can access the result attributes with ``{{
|
||||
result.attribute_name }}``.
|
||||
|
||||
In the example below the result has two attributes: ``header`` and ``content``.
|
||||
To customize their diplay, you need the following template (you must define
|
||||
these classes yourself):
|
||||
|
||||
.. code:: html
|
||||
|
||||
<div class="result">
|
||||
<div class="result-header">
|
||||
{{ result.header }}
|
||||
</div>
|
||||
<div class="result-content">
|
||||
{{ result.content }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Then put your template under ``searx/templates/{theme-name}/result_templates``
|
||||
named ``your-template-name.html``. You can select your custom template with the
|
||||
option ``result_template``.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: your engine name
|
||||
engine: command
|
||||
result_template: your-template-name.html
|
||||
|
||||
Examples
|
||||
========
|
||||
|
||||
Find files by name
|
||||
------------------
|
||||
|
||||
The first example is to find files on your searx host. It uses the command
|
||||
`find` available on most Linux distributions. It expects a path type query. The
|
||||
path in the search request must be inside the ``working_dir``.
|
||||
|
||||
The results are displayed with the default `key-value.html` template. A result
|
||||
is displayed in a single row table with the key "line".
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : find
|
||||
engine : command
|
||||
command : ['find', '.', '-name', '{{QUERY}}']
|
||||
query_type : path
|
||||
shortcut : fnd
|
||||
tokens : []
|
||||
disabled : True
|
||||
delimiter :
|
||||
chars : ' '
|
||||
keys : ['line']
|
||||
|
||||
|
||||
Find files by contents
|
||||
-----------------------
|
||||
|
||||
In the second example, we define an engine that searches in the contents of the
|
||||
files under the ``working_dir``. The search type is not defined, so the user can
|
||||
input any string they want. To restrict the input, you can set the ``query_type``
|
||||
to ``enum`` and only allow a set of search terms to protect
|
||||
yourself. Alternatively, make the engine private, so no one malevolent accesses
|
||||
the engine.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : regex search in files
|
||||
engine : command
|
||||
command : ['grep', '{{QUERY}}']
|
||||
shortcut : gr
|
||||
tokens : []
|
||||
disabled : True
|
||||
delimiter :
|
||||
chars : ' '
|
||||
keys : ['line']
|
||||
|
|
@ -37,7 +37,7 @@ Disabled **D** Engine type **ET**
|
|||
------------- ----------- -------------------- ------------
|
||||
Safe search **SS**
|
||||
------------- ----------- ---------------------------------
|
||||
Weigth **W**
|
||||
Weight **W**
|
||||
------------- ----------- ---------------------------------
|
||||
Disabled **D**
|
||||
------------- ----------- ---------------------------------
|
||||
|
|
@ -86,3 +86,60 @@ Show errors **DE**
|
|||
|
||||
{% endfor %}
|
||||
|
||||
.. flat-table:: Additional engines (commented out in settings.yml)
|
||||
:header-rows: 1
|
||||
:stub-columns: 2
|
||||
|
||||
* - Name
|
||||
- Base URL
|
||||
- Host
|
||||
- Port
|
||||
- Paging
|
||||
|
||||
* - elasticsearch
|
||||
- localhost:9200
|
||||
-
|
||||
-
|
||||
- False
|
||||
|
||||
* - meilicsearch
|
||||
- localhost:7700
|
||||
-
|
||||
-
|
||||
- True
|
||||
|
||||
* - mongodb
|
||||
-
|
||||
- 127.0.0.1
|
||||
- 21017
|
||||
- True
|
||||
|
||||
* - mysql_server
|
||||
-
|
||||
- 127.0.0.1
|
||||
- 3306
|
||||
- True
|
||||
|
||||
* - postgresql
|
||||
-
|
||||
- 127.0.0.1
|
||||
- 5432
|
||||
- True
|
||||
|
||||
* - redis_server
|
||||
-
|
||||
- 127.0.0.1
|
||||
- 6379
|
||||
- False
|
||||
|
||||
* - solr
|
||||
- localhost:8983
|
||||
-
|
||||
-
|
||||
- True
|
||||
|
||||
* - sqlite
|
||||
-
|
||||
-
|
||||
-
|
||||
- True
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ Example
|
|||
Scenario:
|
||||
|
||||
#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
|
||||
#. the Recoll search inteface can be reached at https://recoll.example.org/ and
|
||||
#. the Recoll search interface can be reached at https://recoll.example.org/ and
|
||||
#. the contents of this filesystem can be reached though https://download.example.org/reference
|
||||
|
||||
.. code:: yaml
|
||||
|
|
|
|||
|
|
@ -19,5 +19,9 @@ Administrator documentation
|
|||
filtron
|
||||
morty
|
||||
engines
|
||||
private-engines
|
||||
command-engine
|
||||
indexer-engines
|
||||
no-sql-engines
|
||||
plugins
|
||||
buildhosts
|
||||
|
|
|
|||
|
|
@ -0,0 +1,89 @@
|
|||
==================
|
||||
Search in indexers
|
||||
==================
|
||||
|
||||
Searx supports three popular indexer search engines:
|
||||
|
||||
* Elasticsearch
|
||||
* Meilisearch
|
||||
* Solr
|
||||
|
||||
Elasticsearch
|
||||
=============
|
||||
|
||||
Make sure that the Elasticsearch user has access to the index you are querying.
|
||||
If you are not using TLS during your connection, set ``enable_http`` to ``True``.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : elasticsearch
|
||||
shortcut : es
|
||||
engine : elasticsearch
|
||||
base_url : http://localhost:9200
|
||||
username : elastic
|
||||
password : changeme
|
||||
index : my-index
|
||||
query_type : match
|
||||
enable_http : True
|
||||
|
||||
Available settings
|
||||
------------------
|
||||
|
||||
* ``base_url``: URL of Elasticsearch instance. By default it is set to ``http://localhost:9200``.
|
||||
* ``index``: Name of the index to query. Required.
|
||||
* ``query_type``: Elasticsearch query method to use. Available: ``match``,
|
||||
``simple_query_string``, ``term``, ``terms``, ``custom``.
|
||||
* ``custom_query_json``: If you selected ``custom`` for ``query_type``, you must
|
||||
provide the JSON payload in this option.
|
||||
* ``username``: Username in Elasticsearch
|
||||
* ``password``: Password for the Elasticsearch user
|
||||
|
||||
Meilisearch
|
||||
===========
|
||||
|
||||
If you are not using TLS during connection, set ``enable_http`` to ``True``.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : meilisearch
|
||||
engine : meilisearch
|
||||
shortcut: mes
|
||||
base_url : http://localhost:7700
|
||||
index : my-index
|
||||
enable_http: True
|
||||
|
||||
Available settings
|
||||
------------------
|
||||
|
||||
* ``base_url``: URL of the Meilisearch instance. By default it is set to http://localhost:7700
|
||||
* ``index``: Name of the index to query. Required.
|
||||
* ``auth_key``: Key required for authentication.
|
||||
* ``facet_filters``: List of facets to search in.
|
||||
|
||||
Solr
|
||||
====
|
||||
|
||||
If you are not using TLS during connection, set ``enable_http`` to ``True``.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : solr
|
||||
engine : solr
|
||||
shortcut : slr
|
||||
base_url : http://localhost:8983
|
||||
collection : my-collection
|
||||
sort : asc
|
||||
enable_http : True
|
||||
|
||||
Available settings
|
||||
------------------
|
||||
|
||||
* ``base_url``: URL of the Meilisearch instance. By default it is set to http://localhost:8983
|
||||
* ``collection``: Name of the collection to query. Required.
|
||||
* ``sort``: Sorting of the results. Available: ``asc``, ``desc``.
|
||||
* ``rows``: Maximum number of results from a query. Default value: 10.
|
||||
* ``field_list``: List of fields returned from the query.
|
||||
* ``default_fields``: Default fields to query.
|
||||
* ``query_fields``: List of fields with a boost factor. The bigger the boost
|
||||
factor of a field, the more important the field is in the query. Example:
|
||||
``qf="field1^2.3 field2"``
|
||||
|
|
@ -94,8 +94,8 @@ My experience is, that this command is a bit buggy.
|
|||
|
||||
.. _uwsgi configuration:
|
||||
|
||||
Alltogether
|
||||
===========
|
||||
All together
|
||||
============
|
||||
|
||||
Create the configuration ini-file according to your distribution (see below) and
|
||||
restart the uwsgi application.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,170 @@
|
|||
===========================
|
||||
Query SQL and NoSQL servers
|
||||
===========================
|
||||
|
||||
SQL
|
||||
===
|
||||
|
||||
SQL servers are traditional databases with predefined data schema. Furthermore,
|
||||
modern versions also support BLOB data.
|
||||
|
||||
You can search in the following servers:
|
||||
|
||||
* `PostgreSQL`_
|
||||
* `MySQL`_
|
||||
* `SQLite`_
|
||||
|
||||
The configuration of the new database engines are similar. You must put a valid
|
||||
SELECT SQL query in ``query_str``. At the moment you can only bind at most
|
||||
one parameter in your query.
|
||||
|
||||
Do not include LIMIT or OFFSET in your SQL query as the engines
|
||||
rely on these keywords during paging.
|
||||
|
||||
PostgreSQL
|
||||
----------
|
||||
|
||||
Required PyPi package: ``psychopg2``
|
||||
|
||||
You can find an example configuration below:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : postgresql
|
||||
engine : postgresql
|
||||
database : my_database
|
||||
username : searx
|
||||
password : password
|
||||
query_str : 'SELECT * from my_table WHERE my_column = %(query)s'
|
||||
shortcut : psql
|
||||
|
||||
|
||||
Available options
|
||||
~~~~~~~~~~~~~~~~~
|
||||
* ``host``: IP address of the host running PostgreSQL. By default it is ``127.0.0.1``.
|
||||
* ``port``: Port number PostgreSQL is listening on. By default it is ``5432``.
|
||||
* ``database``: Name of the database you are connecting to.
|
||||
* ``username``: Name of the user connecting to the database.
|
||||
* ``password``: Password of the database user.
|
||||
* ``query_str``: Query string to run. Keywords like ``LIMIT`` and ``OFFSET`` are not allowed. Required.
|
||||
* ``limit``: Number of returned results per page. By default it is 10.
|
||||
|
||||
MySQL
|
||||
-----
|
||||
|
||||
Required PyPi package: ``mysql-connector-python``
|
||||
|
||||
This is an example configuration for quering a MySQL server:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : mysql
|
||||
engine : mysql_server
|
||||
database : my_database
|
||||
username : searx
|
||||
password : password
|
||||
limit : 5
|
||||
query_str : 'SELECT * from my_table WHERE my_column=%(query)s'
|
||||
shortcut : mysql
|
||||
|
||||
|
||||
Available options
|
||||
~~~~~~~~~~~~~~~~~
|
||||
* ``host``: IP address of the host running MySQL. By default it is ``127.0.0.1``.
|
||||
* ``port``: Port number MySQL is listening on. By default it is ``3306``.
|
||||
* ``database``: Name of the database you are connecting to.
|
||||
* ``auth_plugin``: Authentication plugin to use. By default it is ``caching_sha2_password``.
|
||||
* ``username``: Name of the user connecting to the database.
|
||||
* ``password``: Password of the database user.
|
||||
* ``query_str``: Query string to run. Keywords like ``LIMIT`` and ``OFFSET`` are not allowed. Required.
|
||||
* ``limit``: Number of returned results per page. By default it is 10.
|
||||
|
||||
SQLite
|
||||
------
|
||||
|
||||
You can read from your database ``my_database`` using this example configuration:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : sqlite
|
||||
engine : sqlite
|
||||
shortcut: sq
|
||||
database : my_database
|
||||
query_str : 'SELECT * FROM my_table WHERE my_column=:query'
|
||||
|
||||
|
||||
Available options
|
||||
~~~~~~~~~~~~~~~~~
|
||||
* ``database``: Name of the database you are connecting to.
|
||||
* ``query_str``: Query string to run. Keywords like ``LIMIT`` and ``OFFSET`` are not allowed. Required.
|
||||
* ``limit``: Number of returned results per page. By default it is 10.
|
||||
|
||||
NoSQL
|
||||
=====
|
||||
|
||||
NoSQL data stores are used for storing arbitrary data without first defining their
|
||||
structure. To query the supported servers, you must install their drivers using PyPi.
|
||||
|
||||
You can search in the following servers:
|
||||
|
||||
* `Redis`_
|
||||
* `MongoDB`_
|
||||
|
||||
Redis
|
||||
-----
|
||||
|
||||
Reqired PyPi package: ``redis``
|
||||
|
||||
Example configuration:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : mystore
|
||||
engine : redis_server
|
||||
exact_match_only : True
|
||||
host : 127.0.0.1
|
||||
port : 6379
|
||||
password : secret-password
|
||||
db : 0
|
||||
shortcut : rds
|
||||
enable_http : True
|
||||
|
||||
Available options
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
* ``host``: IP address of the host running Redis. By default it is ``127.0.0.1``.
|
||||
* ``port``: Port number Redis is listening on. By default it is ``6379``.
|
||||
* ``password``: Password if required by Redis.
|
||||
* ``db``: Number of the database you are connecting to.
|
||||
* ``exact_match_only``: Enable if you need exact matching. By default it is ``True``.
|
||||
|
||||
|
||||
MongoDB
|
||||
-------
|
||||
|
||||
Required PyPi package: ``pymongo``
|
||||
|
||||
Below is an example configuration for using a MongoDB collection:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name : mymongo
|
||||
engine : mongodb
|
||||
shortcut : icm
|
||||
host : '127.0.0.1'
|
||||
port : 27017
|
||||
database : personal
|
||||
collection : income
|
||||
key : month
|
||||
enable_http: True
|
||||
|
||||
|
||||
Available options
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
* ``host``: IP address of the host running MongoDB. By default it is ``127.0.0.1``.
|
||||
* ``port``: Port number MongoDB is listening on. By default it is ``27017``.
|
||||
* ``password``: Password if required by Redis.
|
||||
* ``database``: Name of the database you are connecting to.
|
||||
* ``collection``: Name of the collection you want to search in.
|
||||
* ``exact_match_only``: Enable if you need exact matching. By default it is ``True``.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 74 KiB |
|
|
@ -0,0 +1,44 @@
|
|||
=============================
|
||||
How to create private engines
|
||||
=============================
|
||||
|
||||
If you are running your public searx instance, you might want to restrict access
|
||||
to some engines. Maybe you are afraid of bots might abusing the engine. Or the
|
||||
engine might return private results you do not want to share with strangers.
|
||||
|
||||
Server side configuration
|
||||
=========================
|
||||
|
||||
You can make any engine private by setting a list of tokens in your settings.yml
|
||||
file. In the following example, we set two different tokens that provide access
|
||||
to the engine.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: my-private-google
|
||||
engine: google
|
||||
shortcut: pgo
|
||||
tokens: ['my-secret-token-1', 'my-secret-token-2']
|
||||
|
||||
|
||||
To access the private engine, you must distribute the tokens to your searx
|
||||
users. It is up to you how you let them know what the access token is you
|
||||
created.
|
||||
|
||||
Client side configuration
|
||||
=========================
|
||||
|
||||
As a searx instance user, you can add any number of access tokens on the
|
||||
Preferences page. You have to set a comma separated lists of strings in "Engine
|
||||
tokens" input, then save your new preferences.
|
||||
|
||||
.. image:: prefernces-private.png
|
||||
:width: 600px
|
||||
:align: center
|
||||
:alt: location of token textarea
|
||||
|
||||
Once the Preferences page is loaded again, you can see the information of the
|
||||
private engines you got access to. If you cannot see the expected engines in the
|
||||
engines list, double check your token. If there is no issue with the token,
|
||||
contact your instance administrator.
|
||||
|
||||
|
|
@ -129,7 +129,7 @@ Global Settings
|
|||
outgoing: # communication with search engines
|
||||
request_timeout : 2.0 # default timeout in seconds, can be override by engine
|
||||
# max_request_timeout: 10.0 # the maximum timeout in seconds
|
||||
useragent_suffix : "" # informations like an email address to the administrator
|
||||
useragent_suffix : "" # information like an email address to the administrator
|
||||
pool_connections : 100 # Number of different hosts
|
||||
pool_maxsize : 10 # Number of simultaneous requests by host
|
||||
# uncomment below section if you want to use a proxy
|
||||
|
|
|
|||
|
|
@ -0,0 +1,48 @@
|
|||
=================================
|
||||
Private searx project is finished
|
||||
=================================
|
||||
|
||||
We are officially finished with the Private searx project. The goal was to
|
||||
extend searx capabilities beyond just searching on the Internet. We added
|
||||
support for offline engines. These engines do not connect to the Internet,
|
||||
they find results locally.
|
||||
|
||||
As some of the offline engines run commands on the searx host, we added an
|
||||
option to protect any engine by making them private. Private engines can only be
|
||||
accessed using a token.
|
||||
|
||||
After searx was prepared to run offline queries we added numerous new engines:
|
||||
|
||||
1. Command line engine
|
||||
2. MySQL
|
||||
3. PostgreSQL
|
||||
4. SQLite
|
||||
5. Redis
|
||||
6. MongoDB
|
||||
|
||||
We also added new engines that communicate over HTTP, but you might want to keep
|
||||
them private:
|
||||
|
||||
1. Elasticsearch
|
||||
2. Meilisearch
|
||||
3. Solr
|
||||
|
||||
The last step was to document this work. We added new tutorials on creating
|
||||
command engines, making engines private and also adding a custom result template
|
||||
to your own engines.
|
||||
|
||||
Acknowledgement
|
||||
===============
|
||||
|
||||
The project was sponsored by `Search and Discovery Fund`_ of `NLnet
|
||||
Foundation`_. We would like to thank the NLnet for not only the funds, but the
|
||||
conversations and their ideas. They were truly invested and passionate about
|
||||
supporting searx.
|
||||
|
||||
.. _Search and Discovery Fund: https://nlnet.nl/discovery
|
||||
.. _NLnet Foundation: https://nlnet.nl/
|
||||
|
||||
|
||||
| Happy hacking.
|
||||
| kvch // 2022.09.30 23:15
|
||||
|
||||
|
|
@ -15,3 +15,4 @@ Blog
|
|||
search-indexer-engines
|
||||
sql-engines
|
||||
search-database-engines
|
||||
documentation-offline-engines
|
||||
|
|
|
|||
|
|
@ -207,7 +207,7 @@ debug services from filtron and morty analogous use:
|
|||
Another point we have to notice is that each service (:ref:`searx <searx.sh>`,
|
||||
:ref:`filtron <filtron.sh>` and :ref:`morty <morty.sh>`) runs under dedicated
|
||||
system user account with the same name (compare :ref:`create searx user`). To
|
||||
get a shell from theses accounts, simply call one of the scripts:
|
||||
get a shell from these accounts, simply call one of the scripts:
|
||||
|
||||
.. tabs::
|
||||
|
||||
|
|
@ -311,7 +311,7 @@ of the container:
|
|||
|
||||
Now we can develop as usual in the working tree of our desktop system. Every
|
||||
time the software was changed, you have to restart the searx service (in the
|
||||
conatiner):
|
||||
container):
|
||||
|
||||
.. tabs::
|
||||
|
||||
|
|
@ -370,7 +370,7 @@ We build up a fully functional searx suite in a archlinux container:
|
|||
$ sudo -H ./utils/lxc.sh install suite searx-archlinux
|
||||
|
||||
To access HTTP from the desktop we installed nginx for the services inside the
|
||||
conatiner:
|
||||
container:
|
||||
|
||||
.. tabs::
|
||||
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ you can use your owm template by placing the template under
|
|||
``searx/templates/{theme_name}/result_templates/{template_name}`` and setting
|
||||
``result_template`` attribute to ``{template_name}``.
|
||||
|
||||
Futhermore, if you do not want to expose these engines on a public instance, you can
|
||||
Furthermore, if you do not want to expose these engines on a public instance, you can
|
||||
still add them and limit the access by setting ``tokens`` as described in the `blog post about
|
||||
private engines`_.
|
||||
|
||||
|
|
@ -29,7 +29,7 @@ structure.
|
|||
Redis
|
||||
-----
|
||||
|
||||
Reqired package: ``redis``
|
||||
Required package: ``redis``
|
||||
|
||||
Redis is a key value based data store usually stored in memory.
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ All of the engines above are added to ``settings.yml`` just commented out, as yo
|
|||
Please note that if you are not using HTTPS to access these engines, you have to enable
|
||||
HTTP requests by setting ``enable_http`` to ``True``.
|
||||
|
||||
Futhermore, if you do not want to expose these engines on a public instance, you can
|
||||
Furthermore, if you do not want to expose these engines on a public instance, you can
|
||||
still add them and limit the access by setting ``tokens`` as described in the `blog post about
|
||||
private engines`_.
|
||||
|
||||
|
|
@ -57,7 +57,7 @@ small-scale (less than 10 million documents) data collections. E.g. it is great
|
|||
web pages you have visited and searching in the contents later.
|
||||
|
||||
The engine supports faceted search, so you can search in a subset of documents of the collection.
|
||||
Futhermore, you can search in Meilisearch instances that require authentication by setting ``auth_token``.
|
||||
Furthermore, you can search in Meilisearch instances that require authentication by setting ``auth_token``.
|
||||
|
||||
Here is a simple example to query a Meilisearch instance:
|
||||
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ Before enabling MySQL engine, you must install the package ``mysql-connector-pyt
|
|||
The authentication plugin is configurable by setting ``auth_plugin`` in the attributes.
|
||||
By default it is set to ``caching_sha2_password``.
|
||||
|
||||
This is an example configuration for quering a MySQL server:
|
||||
This is an example configuration for querying a MySQL server:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@ engine file
|
|||
argument type information
|
||||
======================= =========== ========================================================
|
||||
categories list pages, in which the engine is working
|
||||
paging boolean support multible pages
|
||||
paging boolean support multiple pages
|
||||
time_range_support boolean support search time range
|
||||
engine_type str ``online`` by default, other possibles values are
|
||||
``offline``, ``online_dictionary``, ``online_currency``
|
||||
|
|
@ -159,7 +159,7 @@ parsed arguments
|
|||
----------------
|
||||
|
||||
The function ``def request(query, params):`` always returns the ``params``
|
||||
variable. Inside searx, the following paramters can be used to specify a search
|
||||
variable. Inside searx, the following parameters can be used to specify a search
|
||||
request:
|
||||
|
||||
=================== =========== ==========================================================================
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ generated and deployed at :docs:`github.io <.>`. For build prerequisites read
|
|||
:ref:`docs build`.
|
||||
|
||||
The source files of Searx's documentation are located at :origin:`docs`. Sphinx
|
||||
assumes source files to be encoded in UTF-8 by defaul. Run :ref:`make docs.live
|
||||
assumes source files to be encoded in UTF-8 by default. Run :ref:`make docs.live
|
||||
<make docs.live>` to build HTML while editing.
|
||||
|
||||
.. sidebar:: Further reading
|
||||
|
|
@ -227,13 +227,13 @@ To refer anchors use the `ref role`_ markup:
|
|||
|
||||
.. code:: reST
|
||||
|
||||
Visit chapter :ref:`reST anchor`. Or set hyperlink text manualy :ref:`foo
|
||||
Visit chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo
|
||||
bar <reST anchor>`.
|
||||
|
||||
.. admonition:: ``:ref:`` role
|
||||
:class: rst-example
|
||||
|
||||
Visist chapter :ref:`reST anchor`. Or set hyperlink text manualy :ref:`foo
|
||||
Visist chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo
|
||||
bar <reST anchor>`.
|
||||
|
||||
.. _reST ordinary ref:
|
||||
|
|
@ -494,8 +494,8 @@ Figures & Images
|
|||
is flexible. To get best results in the generated output format, install
|
||||
ImageMagick_ and Graphviz_.
|
||||
|
||||
Searx's sphinx setup includes: :ref:`linuxdoc:kfigure`. Scaleable here means;
|
||||
scaleable in sense of the build process. Normally in absence of a converter
|
||||
Searx's sphinx setup includes: :ref:`linuxdoc:kfigure`. Scalable here means;
|
||||
scalable in sense of the build process. Normally in absence of a converter
|
||||
tool, the build process will break. From the authors POV it’s annoying to care
|
||||
about the build process when handling with images, especially since he has no
|
||||
access to the build process. With :ref:`linuxdoc:kfigure` the build process
|
||||
|
|
@ -503,7 +503,7 @@ continues and scales output quality in dependence of installed image processors.
|
|||
|
||||
If you want to add an image, you should use the ``kernel-figure`` (inheritance
|
||||
of :dudir:`figure`) and ``kernel-image`` (inheritance of :dudir:`image`)
|
||||
directives. E.g. to insert a figure with a scaleable image format use SVG
|
||||
directives. E.g. to insert a figure with a scalable image format use SVG
|
||||
(:ref:`svg image example`):
|
||||
|
||||
.. code:: reST
|
||||
|
|
@ -1185,7 +1185,7 @@ and *targets* (e.g. a ref to :ref:`row 2 of table's body <row body 2>`).
|
|||
- cell 4.4
|
||||
|
||||
* - row 5
|
||||
- cell 5.1 with automatic span to rigth end
|
||||
- cell 5.1 with automatic span to right end
|
||||
|
||||
* - row 6
|
||||
- cell 6.1
|
||||
|
|
@ -1237,7 +1237,7 @@ and *targets* (e.g. a ref to :ref:`row 2 of table's body <row body 2>`).
|
|||
- cell 4.4
|
||||
|
||||
* - row 5
|
||||
- cell 5.1 with automatic span to rigth end
|
||||
- cell 5.1 with automatic span to right end
|
||||
|
||||
* - row 6
|
||||
- cell 6.1
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ Prefix: ``:``
|
|||
Prefix: ``?``
|
||||
to add engines and categories to the currently selected categories
|
||||
|
||||
Abbrevations of the engines and languages are also accepted. Engine/category
|
||||
Abbreviations of the engines and languages are also accepted. Engine/category
|
||||
modifiers are chainable and inclusive (e.g. with :search:`!it !ddg !wp qwer
|
||||
<?q=%21it%20%21ddg%20%21wp%20qwer>` search in IT category **and** duckduckgo
|
||||
**and** wikipedia for ``qwer``).
|
||||
|
|
|
|||
2
manage
2
manage
|
|
@ -192,7 +192,7 @@ docker.build() {
|
|||
# awk to remove the "v" and the "g"
|
||||
SEARX_GIT_VERSION=$(git describe --tags | awk -F'-' '{OFS="-"; $1=substr($1, 2); if ($3) { $3=substr($3, 2); } print}')
|
||||
|
||||
# add the suffix "-dirty" if the repository has uncommited change
|
||||
# add the suffix "-dirty" if the repository has uncommitted change
|
||||
# /!\ HACK for searx/searx: ignore utils/brand.env
|
||||
git update-index -q --refresh
|
||||
if [ ! -z "$(git diff-index --name-only HEAD -- | grep -v 'utils/brand.env')" ]; then
|
||||
|
|
|
|||
|
|
@ -1,19 +1,19 @@
|
|||
mock==4.0.3
|
||||
mock==5.0.1
|
||||
nose2[coverage_plugin]==0.12.0
|
||||
cov-core==1.15.0
|
||||
pycodestyle==2.9.1
|
||||
pylint==2.15.0
|
||||
splinter==0.18.1
|
||||
pycodestyle==2.10.0
|
||||
pylint==2.15.9
|
||||
splinter==0.19.0
|
||||
transifex-client==0.14.3; python_version < '3.10'
|
||||
transifex-client==0.12.5; python_version == '3.10'
|
||||
selenium==4.4.3
|
||||
twine==4.0.1
|
||||
selenium==4.7.2
|
||||
twine==4.0.2
|
||||
Pallets-Sphinx-Themes==2.0.2
|
||||
docutils==0.18
|
||||
Sphinx==5.1.1
|
||||
Sphinx==5.3.0
|
||||
sphinx-issues==3.0.1
|
||||
sphinx-jinja==2.0.2
|
||||
sphinx-tabs==3.4.1
|
||||
sphinxcontrib-programoutput==0.17
|
||||
sphinx-autobuild==2021.3.14
|
||||
linuxdoc==20211220
|
||||
linuxdoc==20221127
|
||||
|
|
|
|||
|
|
@ -1,13 +1,13 @@
|
|||
Brotli==1.0.9
|
||||
babel==2.10.3
|
||||
certifi==2022.9.14
|
||||
babel==2.11.0
|
||||
certifi==2022.12.7
|
||||
flask-babel==2.0.0
|
||||
flask==2.2.2
|
||||
jinja2==3.1.2
|
||||
langdetect==1.0.9
|
||||
lxml==4.9.1
|
||||
lxml==4.9.2
|
||||
pygments==2.12.0
|
||||
python-dateutil==2.8.2
|
||||
pyyaml==6.0
|
||||
requests[socks]==2.28.1
|
||||
requests[socks]==2.28.2
|
||||
setproctitle==1.3.2
|
||||
|
|
|
|||
|
|
@ -41,11 +41,11 @@ if settings['ui']['static_path']:
|
|||
|
||||
'''
|
||||
enable debug if
|
||||
the environnement variable SEARX_DEBUG is 1 or true
|
||||
the environment variable SEARX_DEBUG is 1 or true
|
||||
(whatever the value in settings.yml)
|
||||
or general.debug=True in settings.yml
|
||||
disable debug if
|
||||
the environnement variable SEARX_DEBUG is 0 or false
|
||||
the environment variable SEARX_DEBUG is 0 or false
|
||||
(whatever the value in settings.yml)
|
||||
or general.debug=False in settings.yml
|
||||
'''
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
|
|
@ -1,8 +1,9 @@
|
|||
{
|
||||
"versions": [
|
||||
"102.0",
|
||||
"101.0.1",
|
||||
"101.0"
|
||||
"108.0.1",
|
||||
"108.0",
|
||||
"107.0.1",
|
||||
"107.0"
|
||||
],
|
||||
"os": [
|
||||
"Windows NT 10.0; WOW64",
|
||||
|
|
|
|||
|
|
@ -214,8 +214,23 @@
|
|||
"Q110742003": "dppx",
|
||||
"Q1131660": "st",
|
||||
"Q1137675": "cr",
|
||||
"Q114002440": "𒄀",
|
||||
"Q114002534": "𒃻",
|
||||
"Q114002568": "𒂠",
|
||||
"Q114002639": "𒈨𒊑",
|
||||
"Q114002688": "𒋗𒋛",
|
||||
"Q114002734": "𒊺",
|
||||
"Q114002796": "𒂆",
|
||||
"Q114002897": "𒊬",
|
||||
"Q114002930": "𒀺",
|
||||
"Q114002955": "𒀹𒃷",
|
||||
"Q114002974": "𒃷",
|
||||
"Q114002998": "𒁓",
|
||||
"Q114018694": "𒄥",
|
||||
"Q114018781": "𒁀𒌷𒂵",
|
||||
"Q1140444": "Zb",
|
||||
"Q1140577": "Yb",
|
||||
"Q114589269": "A",
|
||||
"Q1152074": "Pb",
|
||||
"Q1152323": "Tb",
|
||||
"Q1165799": "mil",
|
||||
|
|
@ -236,12 +251,14 @@
|
|||
"Q12257695": "Eb/s",
|
||||
"Q12257696": "EB/s",
|
||||
"Q12261466": "kB/s",
|
||||
"Q12263659": "mgal",
|
||||
"Q12265780": "Pb/s",
|
||||
"Q12265783": "PB/s",
|
||||
"Q12269121": "Yb/s",
|
||||
"Q12269122": "YB/s",
|
||||
"Q12269308": "Zb/s",
|
||||
"Q12269309": "ZB/s",
|
||||
"Q1238720": "vols",
|
||||
"Q1247300": "cm H₂O",
|
||||
"Q12714022": "cwt",
|
||||
"Q12789864": "GeV",
|
||||
|
|
@ -255,6 +272,7 @@
|
|||
"Q1322380": "Ts",
|
||||
"Q1323615": "oz t",
|
||||
"Q132643": "kr",
|
||||
"Q133011": "Ls",
|
||||
"Q13400897": "g",
|
||||
"Q13479685": "mm H2O",
|
||||
"Q1351253": "Eib",
|
||||
|
|
@ -350,6 +368,7 @@
|
|||
"Q194339": "B$",
|
||||
"Q1970718": "mam",
|
||||
"Q1972579": "pdl",
|
||||
"Q19877834": "cd-ft",
|
||||
"Q199462": "LE",
|
||||
"Q199471": "Afs",
|
||||
"Q200323": "dm",
|
||||
|
|
@ -707,7 +726,6 @@
|
|||
"Q56157048": "pmol",
|
||||
"Q56160603": "fmol",
|
||||
"Q56302633": "UM",
|
||||
"Q56317116": "mgal",
|
||||
"Q56317622": "Q_P",
|
||||
"Q56318907": "kbar",
|
||||
"Q56349362": "Bs.S",
|
||||
|
|
|
|||
|
|
@ -142,7 +142,7 @@ def load_engine(engine_data):
|
|||
|
||||
engine.stats = {
|
||||
'sent_search_count': 0, # sent search
|
||||
'search_count': 0, # succesful search
|
||||
'search_count': 0, # successful search
|
||||
'result_count': 0,
|
||||
'engine_time': 0,
|
||||
'engine_time_count': 0,
|
||||
|
|
@ -171,7 +171,7 @@ def load_engine(engine_data):
|
|||
categories.setdefault(category_name, []).append(engine)
|
||||
|
||||
if engine.shortcut in engine_shortcuts:
|
||||
logger.error('Engine config error: ambigious shortcut: {0}'.format(engine.shortcut))
|
||||
logger.error('Engine config error: ambiguous shortcut: {0}'.format(engine.shortcut))
|
||||
sys.exit(1)
|
||||
|
||||
engine_shortcuts[engine.shortcut] = engine.name
|
||||
|
|
|
|||
|
|
@ -70,7 +70,7 @@ def request(query, params):
|
|||
if params['time_range'] in time_range_dict:
|
||||
params['url'] += time_range_string.format(interval=time_range_dict[params['time_range']])
|
||||
|
||||
# bing videos did not like "older" versions < 70.0.1 when selectin other
|
||||
# bing videos did not like "older" versions < 70.0.1 when selecting other
|
||||
# languages then 'en' .. very strange ?!?!
|
||||
params['headers']['User-Agent'] = 'Mozilla/5.0 (X11; Linux x86_64; rv:73.0.1) Gecko/20100101 Firefox/73.0.1'
|
||||
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ from searx.poolrequests import get
|
|||
|
||||
# about
|
||||
about = {
|
||||
"website": 'https://lite.duckduckgo.com/lite',
|
||||
"website": 'https://lite.duckduckgo.com/lite/',
|
||||
"wikidata_id": 'Q12805',
|
||||
"official_api_documentation": 'https://duckduckgo.com/api',
|
||||
"use_official_api": False,
|
||||
|
|
@ -45,7 +45,7 @@ language_aliases = {
|
|||
time_range_dict = {'day': 'd', 'week': 'w', 'month': 'm', 'year': 'y'}
|
||||
|
||||
# search-url
|
||||
url = 'https://lite.duckduckgo.com/lite'
|
||||
url = 'https://lite.duckduckgo.com/lite/'
|
||||
url_ping = 'https://duckduckgo.com/t/sl_l'
|
||||
|
||||
|
||||
|
|
@ -73,6 +73,7 @@ def request(query, params):
|
|||
# link again and again ..
|
||||
|
||||
params['headers']['Content-Type'] = 'application/x-www-form-urlencoded'
|
||||
params['headers']['Referer'] = 'https://lite.duckduckgo.com/'
|
||||
|
||||
# initial page does not have an offset
|
||||
if params['pageno'] == 2:
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ def response(resp):
|
|||
# * book / performing art / film / television / media franchise / concert tour / playwright
|
||||
# * prepared food
|
||||
# * website / software / os / programming language / file format / software engineer
|
||||
# * compagny
|
||||
# * company
|
||||
|
||||
content = ''
|
||||
heading = search_res.get('Heading', '')
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ def response(resp):
|
|||
|
||||
search_res = loads(resp.text)
|
||||
|
||||
# check if items are recieved
|
||||
# check if items are received
|
||||
if 'items' not in search_res:
|
||||
return []
|
||||
|
||||
|
|
|
|||
|
|
@ -282,7 +282,7 @@ def response(resp):
|
|||
|
||||
# google *sections*
|
||||
if extract_text(eval_xpath(result, g_section_with_header)):
|
||||
logger.debug("ingoring <g-section-with-header>")
|
||||
logger.debug("ignoring <g-section-with-header>")
|
||||
continue
|
||||
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
"""Google (News)
|
||||
|
||||
For detailed description of the *REST-full* API see: `Query Parameter
|
||||
Definitions`_. Not all parameters can be appied:
|
||||
Definitions`_. Not all parameters can be applied:
|
||||
|
||||
- num_ : the number of search results is ignored
|
||||
- save_ : is ignored / Google-News results are always *SafeSearch*
|
||||
|
|
@ -155,7 +155,7 @@ def response(resp):
|
|||
padding = (4 -(len(jslog) % 4)) * "="
|
||||
jslog = b64decode(jslog + padding)
|
||||
except binascii.Error:
|
||||
# URL cant be read, skip this result
|
||||
# URL can't be read, skip this result
|
||||
continue
|
||||
|
||||
# now we have : b'[null, ... null,"https://www.cnn.com/.../index.html"]'
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
"""Google (Video)
|
||||
|
||||
For detailed description of the *REST-full* API see: `Query Parameter
|
||||
Definitions`_. Not all parameters can be appied.
|
||||
Definitions`_. Not all parameters can be applied.
|
||||
|
||||
.. _admonition:: Content-Security-Policy (CSP)
|
||||
|
||||
|
|
@ -163,7 +163,7 @@ def response(resp):
|
|||
|
||||
# google *sections*
|
||||
if extract_text(eval_xpath(result, g_section_with_header)):
|
||||
logger.debug("ingoring <g-section-with-header>")
|
||||
logger.debug("ignoring <g-section-with-header>")
|
||||
continue
|
||||
|
||||
title = extract_text(eval_xpath_getindex(result, title_xpath, 0))
|
||||
|
|
|
|||
|
|
@ -72,7 +72,7 @@ def response(resp):
|
|||
elif properties.get('osm_type') == 'R':
|
||||
osm_type = 'relation'
|
||||
else:
|
||||
# continue if invalide osm-type
|
||||
# continue if invalid osm-type
|
||||
continue
|
||||
|
||||
url = result_base_url.format(osm_type=osm_type,
|
||||
|
|
|
|||
|
|
@ -71,7 +71,7 @@ def response(resp):
|
|||
if 'downloadUrl' in result:
|
||||
new_result['torrentfile'] = result['downloadUrl']
|
||||
|
||||
# magnet link *may* be in guid, but it may be also idential to infoUrl
|
||||
# magnet link *may* be in guid, but it may be also identical to infoUrl
|
||||
if 'guid' in result and isinstance(result['guid'], str) and result['guid'].startswith('magnet'):
|
||||
new_result['magnetlink'] = result['guid']
|
||||
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ search_url = base_url + 'sp/search?'
|
|||
|
||||
# specific xpath variables
|
||||
# ads xpath //div[@id="results"]/div[@id="sponsored"]//div[@class="result"]
|
||||
# not ads: div[@class="result"] are the direct childs of div[@id="results"]
|
||||
# not ads: div[@class="result"] are the direct children of div[@id="results"]
|
||||
results_xpath = '//div[@class="w-gl__result__main"]'
|
||||
link_xpath = './/a[@class="w-gl__result-title result-link"]'
|
||||
content_xpath = './/p[@class="w-gl__description"]'
|
||||
|
|
@ -91,8 +91,7 @@ def get_sc_code(headers):
|
|||
dom = html.fromstring(resp.text)
|
||||
|
||||
try:
|
||||
# href --> '/?sc=adrKJMgF8xwp20'
|
||||
href = eval_xpath(dom, '//a[@class="footer-home__logo"]')[0].get('href')
|
||||
href = eval_xpath(dom, '//input[@name="sc"]')[0].get('value')
|
||||
except IndexError as exc:
|
||||
# suspend startpage API --> https://github.com/searxng/searxng/pull/695
|
||||
raise SearxEngineResponseException(
|
||||
|
|
@ -216,7 +215,7 @@ def _fetch_supported_languages(resp):
|
|||
# native name, the English name of the writing script used by the language,
|
||||
# or occasionally something else entirely.
|
||||
|
||||
# this cases are so special they need to be hardcoded, a couple of them are mispellings
|
||||
# this cases are so special they need to be hardcoded, a couple of them are misspellings
|
||||
language_names = {
|
||||
'english_uk': 'en-GB',
|
||||
'fantizhengwen': ['zh-TW', 'zh-HK'],
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ WIKIDATA_PROPERTIES = {
|
|||
# SERVICE wikibase:label: https://en.wikibooks.org/wiki/SPARQL/SERVICE_-_Label#Manual_Label_SERVICE
|
||||
# https://en.wikibooks.org/wiki/SPARQL/WIKIDATA_Precision,_Units_and_Coordinates
|
||||
# https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format#Data_model
|
||||
# optmization:
|
||||
# optimization:
|
||||
# * https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/query_optimization
|
||||
# * https://github.com/blazegraph/database/wiki/QueryHints
|
||||
QUERY_TEMPLATE = """
|
||||
|
|
@ -335,7 +335,7 @@ def get_attributes(language):
|
|||
add_amount('P2046') # area
|
||||
add_amount('P281') # postal code
|
||||
add_label('P38') # currency
|
||||
add_amount('P2048') # heigth (building)
|
||||
add_amount('P2048') # height (building)
|
||||
|
||||
# Media
|
||||
for p in ['P400', # platform (videogames, computing)
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ def request(query, params):
|
|||
|
||||
# replace private user area characters to make text legible
|
||||
def replace_pua_chars(text):
|
||||
pua_chars = {'\uf522': '\u2192', # rigth arrow
|
||||
pua_chars = {'\uf522': '\u2192', # right arrow
|
||||
'\uf7b1': '\u2115', # set of natural numbers
|
||||
'\uf7b4': '\u211a', # set of rational numbers
|
||||
'\uf7b5': '\u211d', # set of real numbers
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ time_range_support = False
|
|||
|
||||
time_range_url = '&hours={time_range_val}'
|
||||
'''Time range URL parameter in the in :py:obj:`search_url`. If no time range is
|
||||
requested by the user, the URL paramter is an empty string. The
|
||||
requested by the user, the URL parameter is an empty string. The
|
||||
``{time_range_val}`` replacement is taken from the :py:obj:`time_range_map`.
|
||||
|
||||
.. code:: yaml
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ def get_external_url(url_id, item_id, alternative="default"):
|
|||
"""Return an external URL or None if url_id is not found.
|
||||
|
||||
url_id can take value from data/external_urls.json
|
||||
The "imdb_id" value is automaticaly converted according to the item_id value.
|
||||
The "imdb_id" value is automatically converted according to the item_id value.
|
||||
|
||||
If item_id is None, the raw URL with the $1 is returned.
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ def load_single_https_ruleset(rules_path):
|
|||
rules = []
|
||||
exclusions = []
|
||||
|
||||
# parse childs from ruleset
|
||||
# parse children from ruleset
|
||||
for ruleset in root:
|
||||
# this child define a target
|
||||
if ruleset.tag == 'target':
|
||||
|
|
|
|||
|
|
@ -2435,7 +2435,7 @@
|
|||
<rule from="^http://widgets\.yahoo\.com/[^?]*"
|
||||
to="https://www.yahoo.com/" />
|
||||
|
||||
<rule from="^http://((?:\w\w|fr-ca\.actualites|address|\w\w\.address|admanager|(?:\w\w|global)\.adserver|adspecs|\w+\.adspecs|\w+\.adspecs-new|advertising|\w\w\.advertising|beap\.adx|c5a?\.ah|(?:s-)?cookex\.amp|(?:[aosz]|apac|y3?)\.analytics|anc|answers|(?:\w\w|espanol|malaysia)\.answers|antispam|\w\w\.antispam|vn\.antoan|au\.apps|global\.ard|astrology|\w\w\.astrology|hk\.(?:(?:info|f1\.master|f1\.page|search|store|edit\.store|user)\.)?auctions|autos|\w\w\.autos|ar\.ayuda|(?:clicks\.beap|csc\.beap|pn1|row|us)\.bc|tw\.bid|tw\.(?:campaign|master|mb|page|search|store|user)\.bid|(?:m\.)?tw\.bigdeals|tw\.billing|biz|boss|(?:tw\.partner|tw)\.buy|(?:\w\w\.)?calendar|careers|\w\w\.cars|(?:\w\w|es-us)\.celebridades|(?:\w\w\.)?celebrity|tw\.charity|i?chart|(?:\w\w|es-us)\.cine|\w\w\.cinema|(?:\w\w|es-us)\.clima|migration\.cn|(?:deveopers\.)?commercecentral|br\.contribuidores|(?:uk\.)?contributor|au\.dating|(?:\w\w|es-us)\.deportes|developer|tw\.dictionary|dir|downloads|s-b\.dp|(?:eu\.|na\.|sa\.|tw\.)?edit|tw\.(?:ysm\.)?emarketing|en-maktoob|\w\w\.entertainment|espanol|edit\.europe|eurosport|(?:de|es|it|uk)\.eurosport|everything|\w\w\.everything|\w+\.fantasysports|au\.fango|tw\.fashion|br\.financas|finance|(?:\w\w|tw\.chart|espanol|tw\.futures|streamerapi)\.finance|(?:\w\w|es-us)\.finanzas|nz\.rss\.food|nz\.forums|games|(?:au|ca|uk)\.games|geo|gma|groups|(?:\w\w|asia|espanol|es-us|fr-ca|moderators)\.groups|health|help|(?:\w\w|secure)\.help|homes|(?:tw|tw\.v2)\.house|info|\w\w\.info|tw\.tool\.ks|au\.launch|legalredirect|(?:\w\w)\.lifestyle|(?:gh\.bouncer\.)?login|us\.l?rd|local|\w\w\.local|m|r\.m|\w\w\.m|mail|(?:\w\w\.overview|[\w-]+(?:\.c\.yom)?)\.mail|maktoob|malaysia|tw\.(?:user\.)?mall|maps|(?:\w\w|espanol|sgws2)\.maps|messenger|(?:\w\w|malaysia)\.messenger|\w\w\.meteo|mlogin|mobile|(?:\w\w|espanol|malaysia)\.mobile|tw\.(?:campaign\.)?money|tw\.movie|movies|(?:au|ca|nz|au\.rss|nz\.rss|tw|uk)\.movies|[\w.-]+\.msg|(?:\w\w|es-us)\.mujer|music|ca\.music|[\w-]+\.musica|my|us\.my|de\.nachrichten|ucs\.netsvs|news|(?:au|ca|fr|gr|hk|in|nz|ph|nz\.rss|sg|tw|uk)\.news|cookiex\.ngd|(?:\w\w|es-us)\.noticias|omg|(?:\w\w|es-us)\.omg|au\.oztips|rtb\.pclick|pilotx1|pipes|play|playerio|privacy|profile|tw\.promo|(?:au|hk|nz)\.promotions|publishing|(?:analytics|mailapps|media|ucs|us-locdrop|video)\.query|hk\.rd|(?:\w\w\.|fr-ca\.)?safely|screen|(?:\w\w|es-us)\.screen|scribe|search|(?:\w\w|w\w\.blog|\w\w\.dictionary|finance|\w\w\.finance|images|\w\w\.images|\w\w\.knowledge|\w\w\.lifestyle|\w\w\.local|malaysia|movies|\w\w\.movies|news|\w\w\.news|malaysia\.news|r|recipes|\w\w\.recipes|shine|shopping|\w\w\.shopping|sports|\w\w\.sports|tools|au\.tv|video|\w\w\.video|malaysia\.video)\.search|sec|rtb\.pclick\.secure|security|tw\.security|\w\w\.seguranca|\w\w\.seguridad|es-us\.seguridad|\w\w\.seguro|tw\.serviceplus|settings|shine|ca\.shine|shopping|ca\.shopping|\w+\.sitios|dashboard\.slingstone|(?:au\.|order\.)?smallbusiness|smarttv|rd\.software|de\.spiele|sports|(?:au|ca|fr|hk|nz|ph|profiles|au\.rss|nz\.rss|tw)\.sports|tw\.stock|au\.thehype|\w\w\.tiempo|es\.todo|toolbar|(?:\w\w|data|malaysia)\.toolbar|(?:au|nz)\.totaltravel|transparency|travel|tw\.travel||tv|(?:ar|au|de|fr|es|es-us|it|mx|nz|au\.rss|uk)\.tv|tw\.uwant|(?:mh|nz|qos|yep)\.video|weather|(?:au|ca|hk|in|nz|sg|ph|uk|us)\.weather|de\.wetter|www|au\.yel|video\.media\.yql|dmros\.ysm)\.)?yahoo\.com/"
|
||||
<rule from="^http://((?:\w\w|fr-ca\.actualites|address|\w\w\.address|admanager|(?:\w\w|global)\.adserver|adspecs|\w+\.adspecs|\w+\.adspecs-new|advertising|\w\w\.advertising|beap\.adx|c5a?\.ah|(?:s-)?cookex\.amp|(?:[aosz]|apac|y3?)\.analytics|anc|answers|(?:\w\w|espanol|malaysia)\.answers|antispam|\w\w\.antispam|vn\.antoan|au\.apps|global\.ard|astrology|\w\w\.astrology|hk\.(?:(?:info|f1\.master|f1\.page|search|store|edit\.store|user)\.)?auctions|autos|\w\w\.autos|ar\.ayuda|(?:clicks\.beap|csc\.beap|pn1|row|us)\.bc|tw\.bid|tw\.(?:campaign|master|mb|page|search|store|user)\.bid|(?:m\.)?tw\.bigdeals|tw\.billing|biz|boss|(?:tw\.partner|tw)\.buy|(?:\w\w\.)?calendar|careers|\w\w\.cars|(?:\w\w|es-us)\.celebridades|(?:\w\w\.)?celebrity|tw\.charity|i?chart|(?:\w\w|es-us)\.cine|\w\w\.cinema|(?:\w\w|es-us)\.clima|migration\.cn|(?:developers\.)?commercecentral|br\.contribuidores|(?:uk\.)?contributor|au\.dating|(?:\w\w|es-us)\.deportes|developer|tw\.dictionary|dir|downloads|s-b\.dp|(?:eu\.|na\.|sa\.|tw\.)?edit|tw\.(?:ysm\.)?emarketing|en-maktoob|\w\w\.entertainment|espanol|edit\.europe|eurosport|(?:de|es|it|uk)\.eurosport|everything|\w\w\.everything|\w+\.fantasysports|au\.fango|tw\.fashion|br\.financas|finance|(?:\w\w|tw\.chart|espanol|tw\.futures|streamerapi)\.finance|(?:\w\w|es-us)\.finanzas|nz\.rss\.food|nz\.forums|games|(?:au|ca|uk)\.games|geo|gma|groups|(?:\w\w|asia|espanol|es-us|fr-ca|moderators)\.groups|health|help|(?:\w\w|secure)\.help|homes|(?:tw|tw\.v2)\.house|info|\w\w\.info|tw\.tool\.ks|au\.launch|legalredirect|(?:\w\w)\.lifestyle|(?:gh\.bouncer\.)?login|us\.l?rd|local|\w\w\.local|m|r\.m|\w\w\.m|mail|(?:\w\w\.overview|[\w-]+(?:\.c\.yom)?)\.mail|maktoob|malaysia|tw\.(?:user\.)?mall|maps|(?:\w\w|espanol|sgws2)\.maps|messenger|(?:\w\w|malaysia)\.messenger|\w\w\.meteo|mlogin|mobile|(?:\w\w|espanol|malaysia)\.mobile|tw\.(?:campaign\.)?money|tw\.movie|movies|(?:au|ca|nz|au\.rss|nz\.rss|tw|uk)\.movies|[\w.-]+\.msg|(?:\w\w|es-us)\.mujer|music|ca\.music|[\w-]+\.musica|my|us\.my|de\.nachrichten|ucs\.netsvs|news|(?:au|ca|fr|gr|hk|in|nz|ph|nz\.rss|sg|tw|uk)\.news|cookiex\.ngd|(?:\w\w|es-us)\.noticias|omg|(?:\w\w|es-us)\.omg|au\.oztips|rtb\.pclick|pilotx1|pipes|play|playerio|privacy|profile|tw\.promo|(?:au|hk|nz)\.promotions|publishing|(?:analytics|mailapps|media|ucs|us-locdrop|video)\.query|hk\.rd|(?:\w\w\.|fr-ca\.)?safely|screen|(?:\w\w|es-us)\.screen|scribe|search|(?:\w\w|w\w\.blog|\w\w\.dictionary|finance|\w\w\.finance|images|\w\w\.images|\w\w\.knowledge|\w\w\.lifestyle|\w\w\.local|malaysia|movies|\w\w\.movies|news|\w\w\.news|malaysia\.news|r|recipes|\w\w\.recipes|shine|shopping|\w\w\.shopping|sports|\w\w\.sports|tools|au\.tv|video|\w\w\.video|malaysia\.video)\.search|sec|rtb\.pclick\.secure|security|tw\.security|\w\w\.seguranca|\w\w\.seguridad|es-us\.seguridad|\w\w\.seguro|tw\.serviceplus|settings|shine|ca\.shine|shopping|ca\.shopping|\w+\.sitios|dashboard\.slingstone|(?:au\.|order\.)?smallbusiness|smarttv|rd\.software|de\.spiele|sports|(?:au|ca|fr|hk|nz|ph|profiles|au\.rss|nz\.rss|tw)\.sports|tw\.stock|au\.thehype|\w\w\.tiempo|es\.todo|toolbar|(?:\w\w|data|malaysia)\.toolbar|(?:au|nz)\.totaltravel|transparency|travel|tw\.travel||tv|(?:ar|au|de|fr|es|es-us|it|mx|nz|au\.rss|uk)\.tv|tw\.uwant|(?:mh|nz|qos|yep)\.video|weather|(?:au|ca|hk|in|nz|sg|ph|uk|us)\.weather|de\.wetter|www|au\.yel|video\.media\.yql|dmros\.ysm)\.)?yahoo\.com/"
|
||||
to="https://$1yahoo.com/" />
|
||||
|
||||
<rule from="^http://([\w-]+)\.yahoofs\.com/"
|
||||
|
|
|
|||
|
|
@ -97,7 +97,7 @@ class SessionSinglePool(requests.Session):
|
|||
self.mount('http://', http_adapter)
|
||||
|
||||
def close(self):
|
||||
"""Call super, but clear adapters since there are managed globaly"""
|
||||
"""Call super, but clear adapters since there are managed globally"""
|
||||
self.adapters.clear()
|
||||
super().close()
|
||||
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ class Setting:
|
|||
return self.value
|
||||
|
||||
def save(self, name, resp):
|
||||
"""Save cookie ``name`` in the HTTP reponse obect
|
||||
"""Save cookie ``name`` in the HTTP response object
|
||||
|
||||
If needed, its overwritten in the inheritance."""
|
||||
resp.set_cookie(name, self.value, max_age=COOKIE_MAX_AGE)
|
||||
|
|
@ -125,7 +125,7 @@ class MultipleChoiceSetting(EnumStringSetting):
|
|||
self.value.append(choice)
|
||||
|
||||
def save(self, name, resp):
|
||||
"""Save cookie ``name`` in the HTTP reponse obect
|
||||
"""Save cookie ``name`` in the HTTP response object
|
||||
"""
|
||||
resp.set_cookie(name, ','.join(self.value), max_age=COOKIE_MAX_AGE)
|
||||
|
||||
|
|
@ -160,7 +160,7 @@ class SetSetting(Setting):
|
|||
self.values = set(elements) # pylint: disable=attribute-defined-outside-init
|
||||
|
||||
def save(self, name, resp):
|
||||
"""Save cookie ``name`` in the HTTP reponse obect
|
||||
"""Save cookie ``name`` in the HTTP response object
|
||||
"""
|
||||
resp.set_cookie(name, ','.join(self.values), max_age=COOKIE_MAX_AGE)
|
||||
|
||||
|
|
@ -209,7 +209,7 @@ class MapSetting(Setting):
|
|||
self.key = data # pylint: disable=attribute-defined-outside-init
|
||||
|
||||
def save(self, name, resp):
|
||||
"""Save cookie ``name`` in the HTTP reponse obect
|
||||
"""Save cookie ``name`` in the HTTP response object
|
||||
"""
|
||||
if hasattr(self, 'key'):
|
||||
resp.set_cookie(name, self.key, max_age=COOKIE_MAX_AGE)
|
||||
|
|
@ -253,7 +253,7 @@ class SwitchableSetting(Setting):
|
|||
self.enabled.add(choice['id'])
|
||||
|
||||
def save(self, resp): # pylint: disable=arguments-differ
|
||||
"""Save cookie in the HTTP reponse obect
|
||||
"""Save cookie in the HTTP response object
|
||||
"""
|
||||
resp.set_cookie('disabled_{0}'.format(self.value), ','.join(self.disabled), max_age=COOKIE_MAX_AGE)
|
||||
resp.set_cookie('enabled_{0}'.format(self.value), ','.join(self.enabled), max_age=COOKIE_MAX_AGE)
|
||||
|
|
@ -517,7 +517,7 @@ class Preferences:
|
|||
return ret_val
|
||||
|
||||
def save(self, resp):
|
||||
"""Save cookie in the HTTP reponse obect
|
||||
"""Save cookie in the HTTP response object
|
||||
"""
|
||||
for user_setting_name, user_setting in self.key_value_settings.items():
|
||||
if user_setting.locked:
|
||||
|
|
|
|||
|
|
@ -197,10 +197,10 @@ class BangParser(QueryPartParser):
|
|||
self.raw_text_query.enginerefs.append(EngineRef(value, 'none'))
|
||||
return True
|
||||
|
||||
# check if prefix is equal with categorie name
|
||||
# check if prefix is equal with category name
|
||||
if value in categories:
|
||||
# using all engines for that search, which
|
||||
# are declared under that categorie name
|
||||
# are declared under that category name
|
||||
self.raw_text_query.enginerefs.extend(EngineRef(engine.name, value)
|
||||
for engine in categories[value]
|
||||
if (engine.name, value) not in self.raw_text_query.disabled_engines)
|
||||
|
|
@ -216,7 +216,7 @@ class BangParser(QueryPartParser):
|
|||
self._add_autocomplete(first_char + suggestion)
|
||||
return
|
||||
|
||||
# check if query starts with categorie name
|
||||
# check if query starts with category name
|
||||
for category in categories:
|
||||
if category.startswith(value):
|
||||
self._add_autocomplete(first_char + category)
|
||||
|
|
@ -309,7 +309,7 @@ class RawTextQuery:
|
|||
|
||||
def getFullQuery(self):
|
||||
"""
|
||||
get full querry including whitespaces
|
||||
get full query including whitespaces
|
||||
"""
|
||||
return '{0} {1}'.format(' '.join(self.query_parts), self.getQuery()).strip()
|
||||
|
||||
|
|
|
|||
|
|
@ -143,9 +143,9 @@ def result_score(result, language):
|
|||
if language in domain_parts:
|
||||
weight *= 1.1
|
||||
|
||||
occurences = len(result['positions'])
|
||||
occurrences = len(result['positions'])
|
||||
|
||||
return sum((occurences * weight) / position for position in result['positions'])
|
||||
return sum((occurrences * weight) / position for position in result['positions'])
|
||||
|
||||
|
||||
class ResultContainer:
|
||||
|
|
@ -252,7 +252,7 @@ class ResultContainer:
|
|||
|
||||
result['engines'] = set([result['engine']])
|
||||
|
||||
# strip multiple spaces and cariage returns from content
|
||||
# strip multiple spaces and carriage returns from content
|
||||
if result.get('content'):
|
||||
result['content'] = WHITESPACE_REGEX.sub(' ', result['content'])
|
||||
|
||||
|
|
@ -278,7 +278,7 @@ class ResultContainer:
|
|||
return merged_result
|
||||
else:
|
||||
# it's an image
|
||||
# it's a duplicate if the parsed_url, template and img_src are differents
|
||||
# it's a duplicate if the parsed_url, template and img_src are different
|
||||
if result.get('img_src', '') == merged_result.get('img_src', ''):
|
||||
return merged_result
|
||||
return None
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@ def run(engine_name_list, verbose):
|
|||
stderr.write(f'{BOLD_SEQ}Engine {name:30}{RESET_SEQ}Checking\n')
|
||||
checker = searx.search.checker.Checker(processor)
|
||||
checker.run()
|
||||
if checker.test_results.succesfull:
|
||||
if checker.test_results.successful:
|
||||
stdout.write(f'{BOLD_SEQ}Engine {name:30}{RESET_SEQ}{GREEN}OK{RESET_SEQ}\n')
|
||||
if verbose:
|
||||
stdout.write(f' {"found languages":15}: {" ".join(sorted(list(checker.test_results.languages)))}\n')
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@ def run():
|
|||
logger.debug('Checking %s engine', name)
|
||||
checker = Checker(processor)
|
||||
checker.run()
|
||||
if checker.test_results.succesfull:
|
||||
if checker.test_results.successful:
|
||||
result['engines'][name] = {'success': True}
|
||||
else:
|
||||
result['engines'][name] = {'success': False, 'errors': checker.test_results.errors}
|
||||
|
|
|
|||
|
|
@ -146,7 +146,7 @@ class TestResults:
|
|||
self.languages.add(language)
|
||||
|
||||
@property
|
||||
def succesfull(self):
|
||||
def successful(self):
|
||||
return len(self.errors) == 0
|
||||
|
||||
def __iter__(self):
|
||||
|
|
@ -291,7 +291,7 @@ class ResultContainerTests:
|
|||
self._record_error('No result')
|
||||
|
||||
def one_title_contains(self, title: str):
|
||||
"""Check one of the title contains `title` (case insensitive comparaison)"""
|
||||
"""Check one of the title contains `title` (case insensitive comparison)"""
|
||||
title = title.lower()
|
||||
for result in self.result_container.get_ordered_results():
|
||||
if title in result['title'].lower():
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@ class OnlineProcessor(EngineProcessor):
|
|||
|
||||
def _send_http_request(self, params):
|
||||
# create dictionary which contain all
|
||||
# informations about the request
|
||||
# information about the request
|
||||
request_args = dict(
|
||||
headers=params['headers'],
|
||||
cookies=params['cookies'],
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ search:
|
|||
default_lang : "" # Default search language - leave blank to detect from browser information or use codes from 'languages.py'
|
||||
ban_time_on_fail : 5 # ban time in seconds after engine errors
|
||||
max_ban_time_on_fail : 120 # max ban time in seconds after engine errors
|
||||
prefer_configured_language: False # increase weight of results in confiugred language in ranking
|
||||
prefer_configured_language: False # increase weight of results in configured language in ranking
|
||||
|
||||
server:
|
||||
port : 8888
|
||||
|
|
@ -73,7 +73,7 @@ ui:
|
|||
outgoing: # communication with search engines
|
||||
request_timeout : 2.0 # default timeout in seconds, can be override by engine
|
||||
# max_request_timeout: 10.0 # the maximum timeout in seconds
|
||||
useragent_suffix : "" # suffix of searx_useragent, could contain informations like an email address to the administrator
|
||||
useragent_suffix : "" # suffix of searx_useragent, could contain information like an email address to the administrator
|
||||
pool_connections : 100 # Number of different hosts
|
||||
pool_maxsize : 10 # Number of simultaneous requests by host
|
||||
# uncomment below section if you want to use a proxy
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ def get_user_settings_path():
|
|||
# find location of settings.yml
|
||||
if 'SEARX_SETTINGS_PATH' in environ:
|
||||
# if possible set path to settings using the
|
||||
# enviroment variable SEARX_SETTINGS_PATH
|
||||
# environment variable SEARX_SETTINGS_PATH
|
||||
return check_settings_yml(environ['SEARX_SETTINGS_PATH'])
|
||||
|
||||
# if not, get it from /etc/searx, or last resort the codebase
|
||||
|
|
@ -132,7 +132,7 @@ def load_settings(load_user_setttings=True):
|
|||
default_settings = load_yaml(default_settings_path)
|
||||
update_settings(default_settings, user_settings)
|
||||
return (default_settings,
|
||||
'merge the default settings ( {} ) and the user setttings ( {} )'
|
||||
'merge the default settings ( {} ) and the user settings ( {} )'
|
||||
.format(default_settings_path, user_settings_path))
|
||||
|
||||
# the user settings, fully replace the default configuration
|
||||
|
|
|
|||
|
|
@ -96,7 +96,7 @@
|
|||
|
||||
{% if 'method' not in locked_preferences %}
|
||||
{% set method_label = _('Method') %}
|
||||
{% set method_info = _('Change how forms are submited, <a href="http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_methods" rel="external">learn more about request methods</a>') %}
|
||||
{% set method_info = _('Change how forms are submitted, <a href="http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_methods" rel="external">learn more about request methods</a>') %}
|
||||
{{ preferences_item_header(method_info, method_label, rtl, 'method') }}
|
||||
<select class="form-control {{ custom_select_class(rtl) }}" name="method" id="method">
|
||||
<option value="POST" {% if method == 'POST' %}selected="selected"{% endif %}>POST</option>
|
||||
|
|
|
|||
|
|
@ -114,6 +114,6 @@ if __name__ == '__main__':
|
|||
run_robot_tests([getattr(robot, x) for x in dir(robot) if x.startswith('test_')])
|
||||
except Exception: # pylint: disable=broad-except
|
||||
errors = True
|
||||
print('Error occured: {0}'.format(traceback.format_exc()))
|
||||
print('Error occurred: {0}'.format(traceback.format_exc()))
|
||||
test_layer.tearDown()
|
||||
sys.exit(1 if errors else 0)
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ def parse_lang(preferences: Preferences, form: Dict[str, str], raw_text_query: R
|
|||
return preferences.get_value('language')
|
||||
# get language
|
||||
# set specific language if set on request, query or preferences
|
||||
# TODO support search with multible languages
|
||||
# TODO support search with multiple languages
|
||||
if len(raw_text_query.languages):
|
||||
query_lang = raw_text_query.languages[-1]
|
||||
elif 'language' in form:
|
||||
|
|
@ -216,7 +216,7 @@ def get_search_query_from_webapp(preferences: Preferences, form: Dict[str, str])
|
|||
disabled_engines = preferences.engines.get_disabled()
|
||||
|
||||
# parse query, if tags are set, which change
|
||||
# the serch engine or search-language
|
||||
# the search engine or search-language
|
||||
raw_text_query = RawTextQuery(form['q'], disabled_engines)
|
||||
|
||||
# set query
|
||||
|
|
@ -231,7 +231,7 @@ def get_search_query_from_webapp(preferences: Preferences, form: Dict[str, str])
|
|||
|
||||
if not is_locked('categories') and raw_text_query.enginerefs and raw_text_query.specific:
|
||||
# if engines are calculated from query,
|
||||
# set categories by using that informations
|
||||
# set categories by using that information
|
||||
query_engineref_list = raw_text_query.enginerefs
|
||||
else:
|
||||
# otherwise, using defined categories to
|
||||
|
|
|
|||
|
|
@ -225,7 +225,7 @@ def code_highlighter(codelines, language=None):
|
|||
language = 'text'
|
||||
|
||||
try:
|
||||
# find lexer by programing language
|
||||
# find lexer by programming language
|
||||
lexer = get_lexer_by_name(language, stripall=True)
|
||||
except:
|
||||
# if lexer is not found, using default one
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ class UnicodeWriter:
|
|||
# Fetch UTF-8 output from the queue ...
|
||||
data = self.queue.getvalue()
|
||||
data = data.strip('\x00')
|
||||
# ... and reencode it into the target encoding
|
||||
# ... and re-encode it into the target encoding
|
||||
data = self.encoder.encode(data)
|
||||
# write to the target stream
|
||||
self.stream.write(data.decode())
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ from searx.engines.wikidata import send_wikidata_query
|
|||
|
||||
|
||||
# ORDER BY (with all the query fields) is important to keep a deterministic result order
|
||||
# so multiple invokation of this script doesn't change currencies.json
|
||||
# so multiple invocation of this script doesn't change currencies.json
|
||||
SARQL_REQUEST = """
|
||||
SELECT DISTINCT ?iso4217 ?unit ?unicode ?label ?alias WHERE {
|
||||
?item wdt:P498 ?iso4217; rdfs:label ?label.
|
||||
|
|
@ -29,7 +29,7 @@ ORDER BY ?iso4217 ?unit ?unicode ?label ?alias
|
|||
"""
|
||||
|
||||
# ORDER BY (with all the query fields) is important to keep a deterministic result order
|
||||
# so multiple invokation of this script doesn't change currencies.json
|
||||
# so multiple invocation of this script doesn't change currencies.json
|
||||
SPARQL_WIKIPEDIA_NAMES_REQUEST = """
|
||||
SELECT DISTINCT ?iso4217 ?article_name WHERE {
|
||||
?item wdt:P498 ?iso4217 .
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ engines_languages_file = Path(searx_dir) / 'data' / 'engines_languages.json'
|
|||
languages_file = Path(searx_dir) / 'languages.py'
|
||||
|
||||
|
||||
# Fetchs supported languages for each engine and writes json file with those.
|
||||
# Fetches supported languages for each engine and writes json file with those.
|
||||
def fetch_supported_languages():
|
||||
|
||||
engines_languages = dict()
|
||||
|
|
|
|||
20
utils/lib.sh
20
utils/lib.sh
|
|
@ -451,17 +451,17 @@ install_template() {
|
|||
fi
|
||||
|
||||
if [[ -f "${dst}" ]] && cmp --silent "${template_file}" "${dst}" ; then
|
||||
info_msg "file ${dst} allready installed"
|
||||
info_msg "file ${dst} already installed"
|
||||
return 0
|
||||
fi
|
||||
|
||||
info_msg "diffrent file ${dst} allready exists on this host"
|
||||
info_msg "different file ${dst} already exists on this host"
|
||||
|
||||
while true; do
|
||||
choose_one _reply "choose next step with file $dst" \
|
||||
"replace file" \
|
||||
"leave file unchanged" \
|
||||
"interactiv shell" \
|
||||
"interactive shell" \
|
||||
"diff files"
|
||||
|
||||
case $_reply in
|
||||
|
|
@ -474,7 +474,7 @@ install_template() {
|
|||
"leave file unchanged")
|
||||
break
|
||||
;;
|
||||
"interactiv shell")
|
||||
"interactive shell")
|
||||
echo -e "// edit ${_Red}${dst}${_creset} to your needs"
|
||||
echo -e "// exit with [${_BCyan}CTRL-D${_creset}]"
|
||||
sudo -H -u "${owner}" -i
|
||||
|
|
@ -1018,8 +1018,8 @@ nginx_install_app() {
|
|||
|
||||
nginx_include_apps_enabled() {
|
||||
|
||||
# Add the *NGINX_APPS_ENABLED* infrastruture to a nginx server block. Such
|
||||
# infrastruture is already known from fedora and centos, including apps (location
|
||||
# Add the *NGINX_APPS_ENABLED* infrastructure to a nginx server block. Such
|
||||
# infrastructure is already known from fedora and centos, including apps (location
|
||||
# directives) from the /etc/nginx/default.d folder into the *default* nginx
|
||||
# server.
|
||||
|
||||
|
|
@ -1521,7 +1521,7 @@ _apt_pkg_info_is_updated=0
|
|||
|
||||
pkg_install() {
|
||||
|
||||
# usage: TITEL='install foobar' pkg_install foopkg barpkg
|
||||
# usage: TITLE='install foobar' pkg_install foopkg barpkg
|
||||
|
||||
rst_title "${TITLE:-installation of packages}" section
|
||||
echo -e "\npackage(s)::\n"
|
||||
|
|
@ -1557,7 +1557,7 @@ pkg_install() {
|
|||
|
||||
pkg_remove() {
|
||||
|
||||
# usage: TITEL='remove foobar' pkg_remove foopkg barpkg
|
||||
# usage: TITLE='remove foobar' pkg_remove foopkg barpkg
|
||||
|
||||
rst_title "${TITLE:-remove packages}" section
|
||||
echo -e "\npackage(s)::\n"
|
||||
|
|
@ -1623,7 +1623,7 @@ git_clone() {
|
|||
# git_clone <url> <path> [<branch> [<user>]]
|
||||
#
|
||||
# First form uses $CACHE/<name> as destination folder, second form clones
|
||||
# into <path>. If repository is allready cloned, pull from <branch> and
|
||||
# into <path>. If repository is already cloned, pull from <branch> and
|
||||
# update working tree (if needed, the caller has to stash local changes).
|
||||
#
|
||||
# git clone https://github.com/searx/searx searx-src origin/master searxlogin
|
||||
|
|
@ -1696,7 +1696,7 @@ lxc_init_container_env() {
|
|||
# usage: lxc_init_container_env <name>
|
||||
|
||||
# Create a /.lxcenv file in the root folder. Call this once after the
|
||||
# container is inital started and before installing any boilerplate stuff.
|
||||
# container is initial started and before installing any boilerplate stuff.
|
||||
|
||||
info_msg "create /.lxcenv in container $1"
|
||||
cat <<EOF | lxc exec "${1}" -- bash | prefix_stdout "[${_BBlue}${1}${_creset}] "
|
||||
|
|
|
|||
|
|
@ -107,7 +107,7 @@ show
|
|||
:suite: show services of all (or <name>) containers from the LXC suite
|
||||
:images: show information of local images
|
||||
cmd
|
||||
use single qoutes to evaluate in container's bash, e.g.: 'echo \$(hostname)'
|
||||
use single quotes to evaluate in container's bash, e.g.: 'echo \$(hostname)'
|
||||
-- run command '...' in all containers of the LXC suite
|
||||
:<name>: run command '...' in container <name>
|
||||
install
|
||||
|
|
@ -178,7 +178,7 @@ main() {
|
|||
lxc_delete_container "$2"
|
||||
fi
|
||||
;;
|
||||
*) usage "uknown or missing container <name> $2"; exit 42;;
|
||||
*) usage "unknown or missing container <name> $2"; exit 42;;
|
||||
esac
|
||||
;;
|
||||
start|stop)
|
||||
|
|
@ -190,7 +190,7 @@ main() {
|
|||
info_msg "lxc $1 $2"
|
||||
lxc "$1" "$2" | prefix_stdout "[${_BBlue}${i}${_creset}] "
|
||||
;;
|
||||
*) usage "uknown or missing container <name> $2"; exit 42;;
|
||||
*) usage "unknown or missing container <name> $2"; exit 42;;
|
||||
esac
|
||||
;;
|
||||
show)
|
||||
|
|
|
|||
|
|
@ -402,7 +402,7 @@ EOF
|
|||
}
|
||||
|
||||
enable_debug() {
|
||||
warn_msg "Do not enable debug in production enviroments!!"
|
||||
warn_msg "Do not enable debug in production environments!!"
|
||||
info_msg "Enabling debug option needs to reinstall systemd service!"
|
||||
set_service_env_debug true
|
||||
}
|
||||
|
|
|
|||
|
|
@ -436,7 +436,7 @@ install_settings() {
|
|||
choose_one action "What should happen to the settings file? " \
|
||||
"keep configuration unchanged" \
|
||||
"use origin settings" \
|
||||
"start interactiv shell"
|
||||
"start interactive shell"
|
||||
case $action in
|
||||
"keep configuration unchanged")
|
||||
info_msg "leave settings file unchanged"
|
||||
|
|
@ -446,7 +446,7 @@ install_settings() {
|
|||
info_msg "install origin settings"
|
||||
cp "${SEARX_SETTINGS_TEMPLATE}" "${SEARX_SETTINGS_PATH}"
|
||||
;;
|
||||
"start interactiv shell")
|
||||
"start interactive shell")
|
||||
backup_file "${SEARX_SETTINGS_PATH}"
|
||||
echo -e "// exit with [${_BCyan}CTRL-D${_creset}]"
|
||||
sudo -H -i
|
||||
|
|
@ -600,7 +600,7 @@ EOF
|
|||
}
|
||||
|
||||
enable_debug() {
|
||||
warn_msg "Do not enable debug in production enviroments!!"
|
||||
warn_msg "Do not enable debug in production environments!!"
|
||||
info_msg "try to enable debug mode ..."
|
||||
tee_stderr 0.1 <<EOF | sudo -H -i 2>&1 | prefix_stdout "$_service_prefix"
|
||||
cd ${SEARX_SRC}
|
||||
|
|
@ -833,8 +833,8 @@ rst-doc() {
|
|||
|
||||
eval "echo \"$(< "${REPO_ROOT}/docs/build-templates/searx.rst")\""
|
||||
|
||||
# I use ubuntu-20.04 here to demonstrate that versions are also suported,
|
||||
# normaly debian-* and ubuntu-* are most the same.
|
||||
# I use ubuntu-20.04 here to demonstrate that versions are also supported,
|
||||
# normally debian-* and ubuntu-* are most the same.
|
||||
|
||||
for DIST_NAME in ubuntu-20.04 arch fedora; do
|
||||
(
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ disable-logging = true
|
|||
# The right granted on the created socket
|
||||
chmod-socket = 666
|
||||
|
||||
# Plugin to use and interpretor config
|
||||
# Plugin to use and interpreter config
|
||||
single-interpreter = true
|
||||
|
||||
# enable master process
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ disable-logging = true
|
|||
# The right granted on the created socket
|
||||
chmod-socket = 666
|
||||
|
||||
# Plugin to use and interpretor config
|
||||
# Plugin to use and interpreter config
|
||||
single-interpreter = true
|
||||
|
||||
# enable master process
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ disable-logging = true
|
|||
# The right granted on the created socket
|
||||
chmod-socket = 666
|
||||
|
||||
# Plugin to use and interpretor config
|
||||
# Plugin to use and interpreter config
|
||||
single-interpreter = true
|
||||
|
||||
# enable master process
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ disable-logging = true
|
|||
# The right granted on the created socket
|
||||
chmod-socket = 666
|
||||
|
||||
# Plugin to use and interpretor config
|
||||
# Plugin to use and interpreter config
|
||||
single-interpreter = true
|
||||
|
||||
# enable master process
|
||||
|
|
|
|||
Loading…
Reference in New Issue