Commit Graph

6 Commits

Author SHA1 Message Date
Marc Abonce Seguin c9e6d9f5f6
Fix Tor proxy when using httpx 0.21.x (#3165)
## What does this PR do?

This should fix #3164.
The problem is that `httpx` keeps making breaking changes to their library, so we just have to adjust the code a little bit to make it work with the new version of the library.


## Related issues
Closes  #3164
2022-03-01 20:21:25 +01:00
Noémi Ványi 0669bfd7a5
Fix issues in network after updating httpx to 0.21.x (#3169)
* [mod] upgrade httpx 0.21.2

httpx 0.21.2 and httpcore 0.14.4 fix multiple issues:
* https://github.com/encode/httpx/releases/tag/0.21.2
* https://github.com/encode/httpcore/releases/tag/0.14.4

so most of the workarounds in searx.network have been removed.

* pick even more changes from searxng

Co-authored-by: Alexandre Flament <alex@al-f.net>
2022-02-28 22:05:20 +01:00
Noémi Ványi ea38fea711
Pick image_proxy changes from searxng (#2965)
* [mod] /image_proxy: don't decompress images

* [fix] image_proxy: always close the httpx respone

previously, when the content type was not an image and some other error,
the httpx response was not closed

* [mod] /image_proxy: use HTTP/1 instead of HTTP/2

httpx: HTTP/2 is slow when a lot data is downloaded.
https://github.com/dalf/pyhttp-benchmark

also, the usage of HTTP/1 decreases the load average

* [mod] searx.utils.dict_subset: rewrite with comprehension

Co-authored-by: Alexandre Flament <alex@al-f.net>
2022-01-22 13:49:00 +01:00
Alexandre Flament bc60d834c5 [enh] verify that Tor proxy works every time searx starts
based on @MarcAbonce commit on searx
2021-10-13 00:06:37 -07:00
Alexandre Flament 8d2ea790de [fix] searx.network: fix rare cases where LOOP is None
* searx.network.client.LOOP is initialized in a thread
* searx.network.__init__ imports LOOP which may happen
  before the thread has initialized LOOP

This commit adds a new function "searx.network.client.get_loop()"
to fix this issue
2021-05-03 21:47:04 +02:00
Alexandre Flament 14fe1779b7 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-05-03 21:39:54 +02:00