Commit Graph

288 Commits

Author SHA1 Message Date
Frederic Cambus b6570e10a9 Do not send the HSTS header if tls is not enabled in the server context. 2021-04-08 09:17:06 +02:00
Joris Vink cef5ac4003 bump copyright year. 2021-01-11 23:46:08 +01:00
Joris Vink aaf8be40c2 Add casts for isxdigit and isspace.
The argument must be representable as an unsigned char or EOF.
2020-09-08 19:19:56 +02:00
Joris Vink c83f34c938 Rework http_pretty_error a tiny bit. 2020-03-04 08:36:10 +01:00
Frederic Cambus fe43ed09ac Add the http_pretty_error configuration option.
When enabled, Kore returns HTML error pages for status codes 4xx and
5xx instead of empty content.
2020-03-04 08:22:51 +01:00
Joris Vink db31f37ab0 Add a "return" configuration option.
This hooks into the existing redirection framework but allows you
to quickly deny certain paths with a 403 or other status code.

The snippet below would for example declare a filemap served from 'www'
directory but denying all access to the files under the 'data' directory:

filemap		/	www
deny		/data	403
2020-03-03 11:28:16 +01:00
Joris Vink 82d7b58405 Add trailing byte in http_token and http_field_content. 2020-02-19 10:38:41 +01:00
Frederic Cambus b4ebee5913 Add the http_server_version configuration option.
This allows setting a custom server header from the config file, for
example to mask the version number.
2020-02-19 08:07:02 +01:00
Joris Vink 9d0aef0079 bump copyright 2020-02-10 14:47:33 +01:00
Joris Vink fa2e8ef0b6 Add support for config based redirection.
Inside the domain contexts a 'redirect' rule will allow you to redirect
a request to another URI.

Ex:

Redirect all requests with a 301 to example.com

	redirect ^/.*$ 301 https://example.com

Using capture groups

	redirect ^/account/(.*)$ 301 https://example.com/account/$1

Adding the query string in the mix

	redirect ^/(.*)$ 301 https://example.com/$1?$qs
2020-02-07 06:42:33 +01:00
Joris Vink ca70f9d726 TLS improvements.
These changes improve the constraint kore had with client authentication and
multiple domains.

- Add kore_x509_subject_name() which will return a C string containing
  the x509 subject name in full (in utf8).

- Log TLS errors if client authentication was turned on, will help debug
  issues with client authentication in the future.

- If SNI was present in the TLS handshake, check it against the host specified
  in the HTTP request and send a 421 in case they mismatch.

- Throw a 403 if client authentication was enabled but no client certificate
  was specified.
2019-11-19 11:09:24 +01:00
Joris Vink 7cf0006f52 fix potential NULL dereferences.
found by clang --analyze, reminded by fahlgren@
2019-11-13 11:23:02 +01:00
Joris Vink 7350131232 Allow listening of tls/notls ports at the same time.
Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.

It is time to allow non TLS listeners without having to rebuild Kore.

This commit changes your configuration format and will break existing
applications their config.

Configurations now get listener {} contexts:

listen default {
	bind 127.0.0.1 8888
}

The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).

If you want to turn off TLS on that listener, specify "tls no" in that
context.

Domains now need to be attached to a listener:

Eg:
	domain * {
		attach	default
	}

For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:

	kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:27:04 +02:00
Joris Vink 937c39f041 Many many Python improvements.
- Kore can now fully be configured via Python code if one wants nothing to
  do with configuration files.

- Kore can now start single python files and no longer requires them to be
  inside a module directory.

- Pass all regex capture groups to the handler methods, allowing you to
  get access to them immediately.

- Change python websocket_handshake to take callable objects directly.

- Added a new deployment configuration option. If set to "dev" or
  "development" Kore will automatically foreground, no chroot / etc.
  If set to "production" Kore *will* chroot, drop privs, etc.

- Many more..

These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Joris Vink c32880e12b http_argument_decode() can fail. 2019-08-02 11:34:45 +02:00
Joris Vink 0b9e174af4 http_argument_urldecode() could fail. 2019-07-20 20:35:58 +02:00
Joris Vink 0f58de36f4 Be a little less strict on url decoding. 2019-07-16 21:31:32 +02:00
Joris Vink 3cc7d6e238 Allow kore.prerequests to be async. 2019-06-07 21:06:54 +02:00
Joris Vink a4d18ca276 Add HTTP runlocks.
A way to serialize access to HTTP page handlers in case you are
using some asynchronous api such as pgsql or libcurl stuff.
2019-06-02 16:29:54 +02:00
Joris Vink 88553cd2dd Immediately remove completed HTTP requests.
No need to wait until the next time http_process() is called, which
could result in HTTP requests backing up even though we are processing
them at a fast pace.
2019-05-29 20:30:43 +02:00
Joris Vink a10dfe03fe make sure user-defined headers are set for > 500.
a commit done in 2018 prevented http responses with error codes
> 500 to include any user-set headers, preventing a developer
to include things like content-type etc.

reported by Arun Babu via users@
2019-05-12 20:53:27 +02:00
Joris Vink 3c9a141cd0 allow an iterator to be passed to req.response().
if an iterator is passed kore will send the response with
transfer-encoding: chunked and call the iterator for every
chunk that was sent.

The iterator must return a utf-8 string.

Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
Joris Vink d0a6958747 Let http_state_create() take an "onfree" callback.
This function is called when an HTTP request is being free'd,
allowing you to perform any sort of state cleanup attached
to the HTTP request.
2019-04-28 21:48:16 +02:00
Joris Vink 2c88bc6120 Add asynchronous libcurl support.
This commit adds the CURL=1 build option. When enabled allows
you to schedule CURL easy handles onto the Kore event loop.

It also adds an easy to use HTTP client API that abstracts away the
settings required from libcurl to make HTTP requests.

Tied together with HTTP request state machines this means you can
write fully asynchronous HTTP client requests in an easy way.

Additionally this exposes that API to the Python code as well
allowing you do to things like:

	client = kore.httpclient("https://kore.io")
	status, body = await client.get()

Introduces 2 configuration options:
	- curl_recv_max
		Max incoming bytes for a response.

	- curl_timeout
		Timeout in seconds before a transfer is cancelled.

This API also allows you to take the CURL easy handle and send emails
with it, run FTP, etc. All asynchronously.
2019-04-24 00:15:17 +02:00
Joris Vink c89ba3daa3 check http timeouts better 2019-04-12 14:26:47 +02:00
Joris Vink aa49e181b6 Add http_[header|body]_timeout.
If the HTTP request headers or the HTTP body have not arrived before
these timeouts expire, Kore will send a 408 back to the client.
2019-04-11 20:51:49 +02:00
Joris Vink a191445f76 set body length+offset to 0 when populating data.
otherwise this isn't properly picked up by http_body_read() later
if dealing with in-memory HTTP bodies and you get inconsistent behaviour.
2019-04-02 22:26:44 +02:00
Joris Vink eb9b7f7b14 explicitly include sys/types.h
some smaller libc variants do not include this from sys/param.h.
2019-03-06 09:29:46 +01:00
Joris Vink bf1e8e5ffb bump copyright to 2019 2019-02-22 16:57:28 +01:00
Joris Vink 9aa0e95643 Rework accesslog handling.
Move away from the parent constantly hitting the disk for every
accesslog the workers are sending.

The workers will now write their own accesslogs to shared
memory before the parent will pick those up. The parent
will flush them to disk once every second or if they grow
larger then 1MB.

This removes the heavy penalty for having access logs
turned on when you are dealing with a large volume
of requests.
2018-12-22 09:25:00 +01:00
Joris Vink 61b385ae11 do not set CONN_CLOSE_EMPTY for 1.0 until we reply. 2018-11-30 22:12:43 +01:00
Joris Vink 2dd66586ff several python improvements.
- add kore.time() as equivalent for kore_time_ms().
- call waitpid() until no more children are available for reaping otherwise
  we risk missing a process if several die at the same time and only one
  SIGCHLD is delivered to us.
- drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
Joris Vink dda2e1fb2c Some things still talk http/1.0. 2018-10-26 21:24:51 +02:00
Joris Vink 20a0103f1e Add async/await support for socket i/o in python.
This means you can now do things like:

	resp = await koresock.recv(1024)
	await koresock.send(resp)

directly from page handlers if they are defined as async.

Adds lots more to the python goo such as fatalx(), bind_unix(),
task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Joris Vink 442bdef79b allow kore to bind to unix sockets via bind_unix. 2018-10-07 20:49:16 +02:00
Joris Vink 566fefd031 do not http_argument_urldecode for multipart data. 2018-08-16 14:11:28 +02:00
Joris Vink cf1f624367 let filerefs to operate on ms precision for mtime. 2018-07-24 19:56:36 +02:00
Joris Vink 821c1df8ec use method not allowed when required 2018-07-18 16:24:28 +02:00
Joris Vink 916ce222b4 better fix for 5a5d9fd0.
Don't let net_recv_flush() do things as long as the HTTP layer
owns the buffer. When we have sent a response kick the read end
back into gear ourselves by calling net_recv_flush().
2018-07-18 16:10:41 +02:00
Joris Vink 5a5d9fd0c2 alloc space for nb->buf after taking ownership. 2018-07-18 14:36:13 +02:00
Joris Vink 1447f6573f better http header validation. 2018-07-17 20:17:05 +02:00
Joris Vink 616af063e3 Calculate an md over the incoming HTTP body.
This is calculated while the HTTP body is incoming over the wire, once
the body is fully received the digest will be available for the page
handlers to obtain.

You can obtain a hex string for this md via http_body_digest() or
dereferences the http_request and look at http_body_digest manually
for the bytes.
2018-07-17 14:53:55 +02:00
Joris Vink 0726a26c0c Allow restriction of methods for paths.
Now Kore will automatically send a 400 bad request in case the
method was not allowed on the path.
2018-07-17 14:23:57 +02:00
Joris Vink f02f88295c revert b5e122 for now. 2018-07-06 11:21:46 +02:00
Joris Vink 47c1a1d195 set referer to NULL in http_request_new(). 2018-07-05 05:02:49 +00:00
Joris Vink b5e122419b Let http_populate_post() listen to content-type 2018-07-03 08:25:06 +02:00
Joris Vink 4a8d8ab7f8 log referer in accesslog if present. 2018-06-29 22:37:48 +02:00
Joris Vink 72073701b0 Add last-modified and if-modified-since for filemaps. 2018-06-29 09:56:04 +02:00
Joris Vink 521ff6a11d catch more bad ranges in http_argument_urldecode() 2018-06-28 15:39:03 +02:00
Joris Vink 70e945afb7 limit http_argument_urldecode() to sane characters 2018-06-28 15:27:55 +02:00