This work moves all TLS / crypto related code into a tls_openssl.c
file and adds a tls_none.c which contains just stubs.
Allows compilation of Kore with TLS_BACKEND=none to remove building
against OpenSSL.
Also adds code for SHA1/SHA2 taken from openssh-portable so we don't
depend on those being present anymore in libcrypto.
Routes are now configured in a context per route:
route /path {
handler handler_name
methods get post head
validate qs:get id v_id
}
All route related configurations are per-route, allowing multiple
routes for the same path (for different methods).
The param context is removed and merged into the route context now
so that you use the validate keyword to specify what needs validating.
- Kore now only supports OpenSSL 1.1.1 and LibreSSL 3.x.
- Revise the default TLS ciphersuites.
- Kore now carries ffdhe4096.pem and installs it under PREFIX/share/kore.
- Kore its tls_dhparam config setting defaults to the path mentioned above
so you no longer have to set it.
Inside the domain contexts a 'redirect' rule will allow you to redirect
a request to another URI.
Ex:
Redirect all requests with a 301 to example.com
redirect ^/.*$ 301 https://example.com
Using capture groups
redirect ^/account/(.*)$ 301 https://example.com/account/$1
Adding the query string in the mix
redirect ^/(.*)$ 301 https://example.com/$1?$qs
A new acme process is created that communicates with the acme servers.
This process does not hold any of your private keys (no account keys,
no domain keys etc).
Whenever the acme process requires a signed payload it will ask the keymgr
process to do the signing with the relevant keys.
This process is also sandboxed with pledge+unveil on OpenBSD and seccomp
syscall filtering on Linux.
The implementation only supports the tls-alpn-01 challenge. This means that
you do not need to open additional ports on your machine.
http-01 and dns-01 are currently not supported (no wildcard support).
A new configuration option "acme_provider" is available and can be set
to the acme server its directory. By default this will point to the
live letsencrypt environment:
https://acme-v02.api.letsencrypt.org/directory
The acme process can be controlled via the following config options:
- acme_root (where the acme process will chroot/chdir into).
- acme_runas (the user the acme process will run as).
If none are set, the values from 'root' and 'runas' are taken.
If you want to turn on acme for domains you do it as follows:
domain kore.io {
acme yes
}
You do not need to specify certkey/certfile anymore, if they are present
still
they will be overwritten by the acme system.
The keymgr will store all certificates and keys under its root
(keymgr_root), the account key is stored as "/account-key.pem" and all
obtained certificates go under "certificates/<domain>/fullchain.pem" while
keys go under "certificates/<domain>/key.pem".
Kore will automatically renew certificates if they will expire in 7 days
or less.
Now everything that has the "newer" OpenSSL API (1.1.x) is hidden
behind a KORE_OPENSSL_NEWER_API define. Tone down minimum libressl
version again to 2.7.5.
Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.
It is time to allow non TLS listeners without having to rebuild Kore.
This commit changes your configuration format and will break existing
applications their config.
Configurations now get listener {} contexts:
listen default {
bind 127.0.0.1 8888
}
The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).
If you want to turn off TLS on that listener, specify "tls no" in that
context.
Domains now need to be attached to a listener:
Eg:
domain * {
attach default
}
For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:
kore.listen("name", ip=None, port=None, path=None, tls=True).
- Kore can now fully be configured via Python code if one wants nothing to
do with configuration files.
- Kore can now start single python files and no longer requires them to be
inside a module directory.
- Pass all regex capture groups to the handler methods, allowing you to
get access to them immediately.
- Change python websocket_handshake to take callable objects directly.
- Added a new deployment configuration option. If set to "dev" or
"development" Kore will automatically foreground, no chroot / etc.
If set to "production" Kore *will* chroot, drop privs, etc.
- Many more..
These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
With these changes CRLs can be reloaded like certificates
by sending a SIGUSR1 to the parent process.
Track mtime on both certificate files and CRL files as well
and only submit them to the workers if this has changed.
Move away from the parent constantly hitting the disk for every
accesslog the workers are sending.
The workers will now write their own accesslogs to shared
memory before the parent will pick those up. The parent
will flush them to disk once every second or if they grow
larger then 1MB.
This removes the heavy penalty for having access logs
turned on when you are dealing with a large volume
of requests.
This commit removes TLS 1.0 support no matter what OpenSSL
you are linking against.
Changes the value of tls_version from 1.2 to both. Meaning if
you link with OpenSSL 1.1.1 you will get 1.2 + 1.3.
Now anyone can schedule events and get a callback to work as long
as the user data structure that is added for the event begins
with a kore_event data structure.
All event state is now kept in that kore_event structure and renamed
CONN_[READ|WRITE]_POSSIBLE to KORE_EVENT_[READ|WRITE].
This commit introduces the ability for the keymgr process
to reload the certificates/keys for domains when receiving
a SIGUSR1 signal.
The keymgr receives 2 new configuration options:
- keymgr_root_path
The root path where the keymgr will live.
If -n is not specified when the application starts the
keymgr process will chroot into here.
- keymgr_runas_user
The user the keymgr will drop privileges towards if
-r was not specified.
All certfile and certkey configuration options are now relative to the
keymgr_root_path configuration setting.
The keymgr process will now also load the certificate for the domain
(rather then the workers) and submit these to the worker processes so
they can be reloaded when required.
Worker processes will refuse connections until the TLS configuration
for a given domain is completed (aka: the workers receive the certificate
for that domain).
Other changes:
- client_certificates renamed to client_verify.
- the chroot configuration option is now called root.
- kore is a little more verbose if privsep options are missing.
- filemaps are now relative to the root configuration option.
You can now per domain configure the depth for x509 chain validation:
client_verify_depth 1
By default this is 1.
While here change around some log messages and properly set
the callback for x509 verification rather then via hoops and loops.
The HTTP layer used to make a copy of each incoming header and its
value for a request. Stop doing that and make HTTP headers zero-copy
all across the board.
This change comes with some api function changes, notably the
http_request_header() function which now takes a const char ** rather
than a char ** out pointer.
This commit also constifies several members of http_request, beware.
Additional rework how the worker processes deal with the accept lock.
Before:
if a worker held the accept lock and it accepted a new connection
it would release the lock for others and back off for 500ms before
attempting to grab the lock again.
This approach worked but under high load this starts becoming obvious.
Now:
- workers not holding the accept lock and not having any connections
will wait less long before returning from kore_platform_event_wait().
- workers not holding the accept lock will no longer blindly wait
an arbitrary amount in kore_platform_event_wait() but will look
at how long until the next lock grab is and base their timeout
on that.
- if a worker its next_lock timeout is up and failed to grab the
lock it will try again in half the time again.
- the worker process holding the lock will when releasing the lock
double check if it still has space for newer connections, if it does
it will keep the lock until it is full. This prevents the lock from
bouncing between several non busy worker processes all the time.
Additional fixes:
- Reduce the number of times we check the timeout list, only do it twice
per second rather then every event tick.
- Fix solo worker count for TLS (we actually hold two processes, not one).
- Make sure we don't accidentally miscalculate the idle time causing new
connections under heavy load to instantly drop.
- Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
LibreSSL defines OPENSSL_VERSION_NUMBER as 0x20000000L but does not have
the 1.1.0 API so we have to carefully check for LIBRESSL_VERSION_TEXT as
well before using that new API.
Eventually I will phase out 1.0.2 down the line to get rid of the
nightmare that is the 2 different APIs.
This commit adds full support for building kore with 1.1.0e while
retaining the privsep keymanager support.
based on excellent work done by @hiwk.
Before this function would block client I/O and existing HTTP requests
until the keymgr process responsed with a result.
This commit changes that behaviour and makes this function call
the http_process() function if we end up waiting for the keymgr.
This means that while waiting for a response we at least start
making headway with existing HTTP requests if the response is
not immediate.
- Change pools to use mmap() for allocating regions.
- Change kore_malloc() to use pools for commonly sized objects.
(split into multiple of 2 buckets, starting at 8 bytes up to 8192).
- Rename kore_mem_free() to kore_free().
The preallocated pools will hold up to 128K of elements per block size.
In case a larger object is to be allocated kore_malloc() will use
malloc() instead.
Kore will now isolate RSA private keys to a separate process (keymgr).
Worker processes that require RSA signing for TLS connections will
communicate with this keymgr process in order to do so.
This behaviour cannot be disabled and is always turned on.
Joris Vink