This commit introduces the ability to add authenticators to filemaps.
Just like in normal routes, the authenticators will be resolved first
before allowing access to the filemap entries.
Configuration wise, the authenticator is an optional value after the
filemap config directive:
filemap / webroot myauth
In the Python API you can now pass the authenticator for a filemap entry
but turning the value of the filemap into a tuple with the first entry
being the path and the second being the auth dict:
AUTH AUTH={
"type": "cookie",
"value": "cookiename",
"redirect": "/auth/",
"verify": verify_cookie
}
domain.filemaps({
"/css/": "webroot/css",
"/secret/": ("webroot/secret", AUTH)
})
The parent process never differentiated between a worker process
asking for a shutdown or a worker process calling fatalx() when
it came to its exit code.
I made some changes here so the parent process will exit with
an exit code 1 if anything worker related went wrong (fatalx/death policy).
use it in places explicit_bzero() used to be called.
The kore_mem_zero() is a best effort to try and let the compiler
not optimize the code away. Highly platform dependent.
This work moves all TLS / crypto related code into a tls_openssl.c
file and adds a tls_none.c which contains just stubs.
Allows compilation of Kore with TLS_BACKEND=none to remove building
against OpenSSL.
Also adds code for SHA1/SHA2 taken from openssh-portable so we don't
depend on those being present anymore in libcrypto.
It wasn't possible for the parent process to send messages
directly via kore_msg_send() to other worker processes.
This is now rectified to from the parent process one can call
kore_msg_send() with a worker destination and it'll work.
This commit adds improved hooks for Python and a new signal delivery hook.
For the Python API kore_worker_configure() and kore_worker_teardown() had
to be implemented before this commit. Now one can create a workerstart
and workerend method in their koreapp as those will be called when
they exist.
The new signal hook is either kore_worker_signal() or koreapp.signal.
This new hook is called after the worker event code handles the received
signal itself first.
With this commit there is also a new kore_signal_trap() API call allowing
you to more easily trap new signals. This API also also exported to the
Python part of the code under kore.sigtrap()
Adding the hooks keyword with a dictionary attached to specify
the relevant hooks will hook them for the given route.
Eg:
domain.route("/", self.index, methods=["get"],
hooks={
"on_free": self.request_free
}
)
These are the same hooks available via a normal Kore route configuration.
- Make sure we drain the worker log channel if it dies
so we can flush out any lingering log messages.
- Get rid of the raise() in the parent to signal ourselves
we should terminate. Instead depend on the new kore_quit.
- Always attempt to reap children one way or the other.
If set, will call a given handler with the prototype of
`void body_chunk(struct http_request *req, const void *data, size_t len);`
for each chunk of the received HTTP body, allowing a developer to handle
it in their own way.
The incoming body is still being handled and retained in the same way
as before (in a kore_buf or temporary file).
While here, allow HTTP_STATUS_CONTINUE to work via http_response() and
make the handling of incoming HTTP header data a bit better.
Introduce an on_headers callback for routes, allowing one to inspect
the headers before the request is processed further.
Additionall,
Add a new way of obtaining HTTP headers. Much like http_argument_get_*()
functions, these new APIs allow you to fetch the data of an HTTP header
as a specified C type.
The new APIs are:
* http_request_header_int16()
* http_request_header_uint16()
* http_request_header_int32()
* http_request_header_uint32()
* http_request_header_int64()
* http_request_header_uint64()
* http_request_header_float()
* http_request_header_double()
Should make it easier to operate in HTTP header data in a safe way.
No need to always roll your own string to int conversion functions.
Routes are now configured in a context per route:
route /path {
handler handler_name
methods get post head
validate qs:get id v_id
}
All route related configurations are per-route, allowing multiple
routes for the same path (for different methods).
The param context is removed and merged into the route context now
so that you use the validate keyword to specify what needs validating.
Starting with the privsep config, this commit changes the following:
- Removes the root, runas, keymgr_root, keymgr_runas, acme_root and
acme_runas configuration options.
Instead these are now configured via a privsep configuration context:
privsep worker {
root /tmp
runas nobody
}
This is also configurable via Python using the new kore.privsep() method:
kore.privsep("worker", root="/tmp", runas="nobody", skip=["chroot"])
Tied into this we also better handle worker startup:
- Per worker process, wait until it signalled it is ready.
- If a worker fails at startup, display its last log lines more clearly.
- Don't start acme process if no domain requires acme.
- Remove each process its individual startup log message in favour
of a generalized one that displays its PID, root and user.
- At startup, log the kore version and built-ins in a nicer way.
- The worker processes now check things they need to start running
before signaling they are ready (such as access to CA certs for
TLS client authentication).
Before each worker process would either directly print to stdout if
Kore was running in foreground mode, or syslog otherwise.
With this commit the workers will submit their log messages to the
parent process who will either put it onto stdout or syslog.
This change in completely under the hood and users shouldn't care about it.
1) Add @kore.route as a decorator for Python.
This decorator can be used on non-class methods to automatically
declare their route and parameters.
Takes the same arguments as the kore.domain.route function that
exists today.
Provides a nice clean way of setting up Kore if you dont want
a whole class based approach.
2) Remove the requirement for the name for kore.server() and the
kore.domain(attach=) keywords.
Instead of no name was given, the name "default" is used in both
places resulting in less boilerplating.
3) Allow multiple routes to be defined for the same URI as long
as the methods are different. So you can have one method for GET /
and another for POST /.
All changes combined condense the initial experience of getting
a Kore Python app up and running:
eg:
import kore
kore.server(ip="127.0.0.1", port="8888", tls=False)
kore.domain("*")
@kore.route("/", methods=["get"])
async def index(req):
req.response(200, b'get method')
@kore.route("/", methods=["post"])
async def index_post(req)
req.response(200, b'post method')
- Kore now only supports OpenSSL 1.1.1 and LibreSSL 3.x.
- Revise the default TLS ciphersuites.
- Kore now carries ffdhe4096.pem and installs it under PREFIX/share/kore.
- Kore its tls_dhparam config setting defaults to the path mentioned above
so you no longer have to set it.
- Try harder to mark integers as KORE_JSON_TYPE_INTEGER, especially if
they fit in the internal representation of one (int64_t).
- Move error codes into the JSON code itself, rather then requiring
a kore_json data structure. This allows the JSON API to relay errors
such as "item not found" or "type mismatch" properly when looking at items.
- When asking for a KORE_JSON_TYPE_INTEGER_U64 and a KORE_JSON_TYPE_INTEGER
was found with the same name, check if it could be returned properly and do
so if possible.
- Make sure tls-alpn01 works even if the underlying SSL library ends up
calling the ALPN callback *before* the SNI extension was parsed and
the correct domain was selected.
LibreSSL still does this, and older OpenSSL did too I believe, however
OpenSSL grew a clue and always makes sure SNI is called first.
Yes, TLS extensions have no fixed order but it still makes sense to
notify applications using your library of the SNI extension first
before anything else almost.
Oh well.
Add 2 new types:
KORE_JSON_TYPE_INTEGER
signed integer type, internally stored as s64.
KORE_JSON_TYPE_INTEGER_U64
unsigned integer type, internally stored as u64.
Kore JSON parser will prefer marking integers as INTEGER_U64 if it
was unsigned and did not have fractions.
This handles the default option parsing in Kore and should be called
by single_binary=yes builds in kore_parent_configure() unless they
want to handle their own argument parsing.
- Remove the edge trigger io hacks we had in place.
- Use level triggered io for the libcurl fds instead.
- Batch all curl events together and process them at the end
of our worker event loop.
Inside the domain contexts a 'redirect' rule will allow you to redirect
a request to another URI.
Ex:
Redirect all requests with a 301 to example.com
redirect ^/.*$ 301 https://example.com
Using capture groups
redirect ^/account/(.*)$ 301 https://example.com/account/$1
Adding the query string in the mix
redirect ^/(.*)$ 301 https://example.com/$1?$qs
These changes improve the constraint kore had with client authentication and
multiple domains.
- Add kore_x509_subject_name() which will return a C string containing
the x509 subject name in full (in utf8).
- Log TLS errors if client authentication was turned on, will help debug
issues with client authentication in the future.
- If SNI was present in the TLS handshake, check it against the host specified
in the HTTP request and send a 421 in case they mismatch.
- Throw a 403 if client authentication was enabled but no client certificate
was specified.
A new acme process is created that communicates with the acme servers.
This process does not hold any of your private keys (no account keys,
no domain keys etc).
Whenever the acme process requires a signed payload it will ask the keymgr
process to do the signing with the relevant keys.
This process is also sandboxed with pledge+unveil on OpenBSD and seccomp
syscall filtering on Linux.
The implementation only supports the tls-alpn-01 challenge. This means that
you do not need to open additional ports on your machine.
http-01 and dns-01 are currently not supported (no wildcard support).
A new configuration option "acme_provider" is available and can be set
to the acme server its directory. By default this will point to the
live letsencrypt environment:
https://acme-v02.api.letsencrypt.org/directory
The acme process can be controlled via the following config options:
- acme_root (where the acme process will chroot/chdir into).
- acme_runas (the user the acme process will run as).
If none are set, the values from 'root' and 'runas' are taken.
If you want to turn on acme for domains you do it as follows:
domain kore.io {
acme yes
}
You do not need to specify certkey/certfile anymore, if they are present
still
they will be overwritten by the acme system.
The keymgr will store all certificates and keys under its root
(keymgr_root), the account key is stored as "/account-key.pem" and all
obtained certificates go under "certificates/<domain>/fullchain.pem" while
keys go under "certificates/<domain>/key.pem".
Kore will automatically renew certificates if they will expire in 7 days
or less.
If set to "yes" then Kore will trace its child processes and properly
notify you of seccomp violations while still allowing the syscalls.
This can be very useful when running Kore on new platforms that have
not been properly tested with seccomp, allowing me to adjust the default
policies as we move further.
Now everything that has the "newer" OpenSSL API (1.1.x) is hidden
behind a KORE_OPENSSL_NEWER_API define. Tone down minimum libressl
version again to 2.7.5.
Joris Vink
Daniel Fahlgren