lasers
/
searx
mirror of https://github.com/searx/searx
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
4,439 Commits 19 Branches 18 Tags 37 MiB
Commit Graph

91 Commits

Author SHA1 Message Date
Noémi Ványi ea38fea711
Pick image_proxy changes from searxng (#2965) 
* [mod] /image_proxy: don't decompress images

* [fix] image_proxy: always close the httpx respone

previously, when the content type was not an image and some other error,
the httpx response was not closed

* [mod] /image_proxy: use HTTP/1 instead of HTTP/2

httpx: HTTP/2 is slow when a lot data is downloaded.
https://github.com/dalf/pyhttp-benchmark

also, the usage of HTTP/1 decreases the load average

* [mod] searx.utils.dict_subset: rewrite with comprehension

Co-authored-by: Alexandre Flament <alex@al-f.net>
 2022-01-22 13:49:00 +01:00
Noémi Ványi fd9d6b58d5 Add scheme to img_src and thumbnail_url if missing from URL 
Closes #3092
 2022-01-22 11:59:21 +01:00
Alexandre Flament 88a96baedc [enh] replace requests by httpx 2021-05-03 21:39:37 +02:00
Alexandre Flament 3f8ebf70b1 [fix] pylint: use "raise ... from ..." 2020-12-20 09:46:53 +01:00
Alexandre Flament de887c6347 [mod] bing_news: use eval_xpath_getindex 
remove unused function searx.utils.list_get
 2020-12-03 10:22:48 +01:00
Alexandre Flament 1d0c368746 [enh] record details exception per engine 
add an new API /stats/errors
 2020-12-03 10:22:48 +01:00
Alexandre Flament b00d108673 [mod] pylint: numerous minor code fixes 2020-12-01 15:21:19 +01:00
Alexandre Flament 3038052c79 [mod] remove unused import 
use
from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url  # NOQA
so it is possible to easily remove all unused import using autoflake:
autoflake --in-place --recursive --remove-all-unused-imports searx tests
 2020-11-14 14:11:02 +01:00
Alexandre Flament ca593728af [mod] duckduckgo_definitions: display only user friendly attributes / URL 
various bug fixes
 2020-10-28 08:09:25 +01:00
Alexandre Flament a9dc54bebc [mod] Add searx.data module 
Instead of loading the data/*.json in different location,
load these files in the new searx.data module.
 2020-10-07 10:29:34 +02:00
Alexandre Flament 15013e64d8 [fix] drop Python 2: use importlib instead of imp.load_source 
imp.load_source is not documented in Python 3
see documentation : https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly

partial fix of https://github.com/searx/searx/issues/1674
 2020-10-06 09:42:11 +02:00
Alexandre Flament 8f914a28fa [mod] searx.utils.normalize_url: remove Yahoo hack 
* The hack for Yahoo URLs is not necessary anymore. (see searx.engines.yahoo.parse_url)
* move the URL normalization in extract_url to normalize_url
 2020-10-03 10:02:50 +02:00
Alexandre Flament c1d10bde02 [mod] searx/utils.py: add docstring 2020-10-02 18:17:01 +02:00
Alexandre Flament 2006eb4680 [mod] move extract_text, extract_url to searx.utils 2020-10-02 18:13:56 +02:00
Alexandre Flament ad0758e52a [mod] add searx/webutils.py 
contains utility functions and classes used only by webapp.py
 2020-09-22 11:57:06 +02:00
Alexandre Flament 6deb85072a [fix] searx.utils.HTMLTextExtractor: invalid HTML don't raise an Exception 
Close #2188
 2020-09-13 10:28:11 +02:00
Alexandre Flament bdac99d4f0 Drop Python 2 (5/n): searx.utils.is_valid_lang, input parameter is a str instead of bytes 
Fix bug in translated.py and dictzone.py
 2020-09-10 10:49:42 +02:00
Dalf c225db45c8 Drop Python 2 (4/n): SearchQuery.query is a str instead of bytes 2020-09-10 10:49:42 +02:00
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 2020-09-10 10:39:04 +02:00
Dalf 85b3723345 [mod] speed optimization 
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
 2019-11-15 09:33:15 +01:00
Noémi Ványi 5796dc60c9 fix pep 8 check 2019-10-16 15:52:48 +02:00
Noémi Ványi a6f20caf32 add initial support for offline engines && command engine 2019-10-16 15:52:48 +02:00
Adam Tauber 72459b246b [fix] convert bytes type to string in language detection (fixes dictzone) 2019-10-16 14:52:57 +02:00
Alexandre Flament 2179079a91
[fix] fix flickr_noapi decoding (#1655) 
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
 2019-08-02 13:37:13 +02:00
Dalf 7e201cbf65 [mod] use cache in _match_language function to speed up searx start time significantly 2019-07-19 08:58:08 +02:00
rachmadani haryono ec88fb8a0f [fix] secret_key can be bytes instead of a string (#1602) 
Fix #1600
In settings.yml, the secret_key can be written as string or as base64 encoded data using !!binary notation.
 2019-07-17 10:09:09 +02:00
Alex 50c836864a fetch_firefox_version.py : compatible with Python 3 and minor fixes. 2018-08-05 10:55:42 +02:00
Alexandre Flament 066bd916bf [mod] fetch firefox versions in a standalone script 2018-08-05 10:10:15 +02:00
Adam Tauber d51732c0e5
Merge pull request #1303 from MarcAbonce/bing 
Fix bing "garbage" results
 2018-07-09 11:00:37 +02:00
Marc Abonce Seguin c7000cd1df [fix] update user agent versions 
this fixes duckduckgo error response
 2018-06-23 16:24:06 -05:00
Adam Tauber aef2b07969 [fix] add basestring for py3 2018-06-14 11:48:31 +02:00
Marc Abonce Seguin 75b276f408 fix bing "garbage" results (issue #1275) 2018-05-20 18:13:32 -05:00
Marc Abonce Seguin 772c048d01 refactor engine's search language handling 
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
 2018-03-27 00:08:03 -06:00
Adam Tauber 0969e50c5b [fix] convert json engine result attributes to string - closes #1006 2017-12-01 20:54:12 +01:00
Adam Tauber b5071fea6a [fix] remove trailing 0x00 from csv output 2017-11-21 16:58:51 +01:00
Adam Tauber 3d6c67951a [fix] resurrect csv output in py2 2017-11-21 16:51:45 +01:00
Noémi Ványi e73cb14889 fix hmac python3 compatibility 2017-09-08 21:33:11 +02:00
misnyo 33fd938016 [mod] int_or_zero refactored to searx_utils 2017-09-04 20:05:04 +02:00
potato 9b82cb1908 [fix] is_valid_lang fixed for new languages.py + dictzone engine encoding 2017-06-25 18:29:19 +02:00
Alexandre Flament 9c91ab33f8 [mod] settings.yml can be /etc/searx/settings.yml 
The exact order is
* first from SEARX_SETTINGS_PATH,
* if not found then from searx code base,
* if not found then from /etc/searx/settings.yml
* if not found an exception stops searx loading
 2017-05-15 22:19:42 +02:00
Alexandre Flament ee080feaed [mod] the static and templates directories can be defined in the settings.yml 2017-05-15 21:23:13 +02:00
Adam Tauber 52e615dede [enh] py3 compatibility 2017-05-15 12:02:30 +02:00
Alexandre Flament 7fdfeca3a4 [mod] add a __common__ template that can't be selected but that provides a common place for shared templates. 
What has been moved into this template :
* opensearch*.xml is always the same whatever the themes.
* the text inside */about.html
 2017-01-20 15:40:38 +01:00
Adam Tauber 55dc538398 [mod] move load_module function to utils 2016-11-19 17:51:19 +01:00
Noémi Ványi 53c9fde992 fix kickass torrents engine 2016-10-22 05:27:18 +02:00
potato 983415bc38 [enh] is_valid_lang moved to utils 2016-09-06 16:43:48 +02:00
Noemi Vanyi 3a1c5876b1 add digbt engine 
Unfortunately, it is quite slow so it is disabled.
Furthermore, the display of number of files is wrong
on digbt.org, so it is not displayed on searx.
 2016-08-13 16:19:00 +02:00
marc a4c77f88d0 [fix] exception if locale doesn't have a date format 
occitan, for example
 2016-08-05 23:51:04 -05:00
stepshal b3ab221b98 Fix anomalous backslash in string 2016-07-11 23:53:13 +07:00
stepshal cd9b494cb5 Fix quantity of blank lines after code object. 2016-07-10 21:44:27 +07:00