Commit Graph

23 Commits

Author SHA1 Message Date
Kian-Meng Ang 629ebb426f
Fix typos (#3366)
Found via `codespell -S ./searx/translations,./searx/data,./searx/static -L ans,te,fo,doubleclick,tthe,dum`
2022-09-29 23:06:59 +02:00
Noémi Ványi 85034b49ef
Remove `httpx` and use `requests` instead (#3305)
## What does this PR do?

This PR prepares for removing `httpx`, and reverts back to `requests`.

## Why is this change important?

`httpx` hasn't proven itself to be faster or better than `requests`. On the other
hand it has caused issues on Windows.

=============================================
Please update your environment to use requests instead of httpx.
=============================================
2022-07-30 20:56:56 +02:00
Alexandre Flament 14fe1779b7 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-05-03 21:39:54 +02:00
Alexandre Flament 88a96baedc [enh] replace requests by httpx 2021-05-03 21:39:37 +02:00
Alexandre Flament 99e0651cea [mod] by default allow only HTTPS, not HTTP
Related to https://github.com/searx/searx/pull/2373
2021-03-08 11:35:08 +01:00
Alexandre Flament 74d56f6cfb [mod] poolrequests: for one (user request, engine) always use the same HTTPAdapter
The duckduckgo engine requires an additional request after the results have been sent.
This commit makes sure that the second request uses the same HTTPAdapter
= the same IP address, and the same proxy.
2021-02-09 14:33:36 +01:00
Alexandre Flament d703119d3a [enh] add raise_for_httperror
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before

the check is done in poolrequests.py (was before in search.py).

update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
2020-12-11 14:37:08 +01:00
Alexandre Flament 3786920df9 [enh] Add multiple outgoing proxies
credits go to @bauruine see https://github.com/searx/searx/pull/1958
2020-11-20 15:29:21 +01:00
Alexandre Flament 93f7f7eee2 [mod] upgrade requests to version 2.24.0. use ssl instead of pyopenssl.
requests 2.24.0 uses the ssl module except if it doesn't support SNI, in this case searx fallbacks to pyopenssl.
searx logs a critical message and exit if the ssl modules doesn't support SNI and pyOpenSSL is not installed.
searx logs a critical message and exit if the ssl version is older than 1.0.2.
in requirements.txt, pyopenssl is still required to install searx as a fallback.
2020-09-26 19:30:27 +02:00
Dalf 7888377743 Drop Python 2 (3/n): objects 2020-09-10 10:39:04 +02:00
Adam Tauber 4a913247b2 [enh] add option to configure proxies per engine - closes #1827 2020-09-01 15:59:22 +02:00
Adam Tauber 27e95846ea [fix] measure request duration only if total_time attribute is set - fixes autocompleter 2017-11-23 01:52:54 +01:00
Alexandre Flament c1cfe97851 [enh] timeout and total HTTP time are managed by searx.poolrequests 2017-11-22 18:33:35 +01:00
Adam Tauber f2b13a7de0 [enh] allow default empty values for normal and result proxies 2017-07-20 13:32:20 +02:00
Guilhem Bonnefille 3f395304f9 Fix pep8 2016-05-02 19:25:09 +02:00
Guilhem Bonnefille f18a3a77ef Use new settings to fine tune the connection pool
In some circumstances, it is necessary to increase size of pool.
2016-04-28 14:05:01 +02:00
Adam Tauber bd22e9a336 [fix] pep8 compatibilty 2016-01-18 12:47:31 +01:00
Adam Tauber 357fc47811 [fix] lock request pool generator 2015-10-31 14:40:59 +01:00
Adam Tauber 1fcf066a81 [mod] change settings file structure according to #314 2015-08-02 20:32:22 +02:00
Alexandre Flament cde37be4f5 [enh] basic support for http proxy (see #236) 2015-04-25 11:44:53 +02:00
Adam Tauber 03579c225f [enh] support of multiple outgoing IPs 2015-02-22 23:24:49 +01:00
Adam Tauber 06186e72a9 [fix] poolrequest post method parameters 2015-01-22 17:20:44 +01:00
dalf d07cfd9089 [enh] use one single http connection pool : improve response time. close #100 2015-01-21 11:33:16 +01:00