This work moves all TLS / crypto related code into a tls_openssl.c
file and adds a tls_none.c which contains just stubs.
Allows compilation of Kore with TLS_BACKEND=none to remove building
against OpenSSL.
Also adds code for SHA1/SHA2 taken from openssh-portable so we don't
depend on those being present anymore in libcrypto.
These will spew out the required CFLAGS and LDFLAGS respectively
when compiling source code for use in Kore applications.
This should make it easier to integrate this into existing
build systems where using kodev may be a bit annoying.
Eg: gcc -Wall -std=c99 `kodev cflags` koreapp.c `kodev ldflags` -o koreapp.so
Routes are now configured in a context per route:
route /path {
handler handler_name
methods get post head
validate qs:get id v_id
}
All route related configurations are per-route, allowing multiple
routes for the same path (for different methods).
The param context is removed and merged into the route context now
so that you use the validate keyword to specify what needs validating.
Before each worker process would either directly print to stdout if
Kore was running in foreground mode, or syslog otherwise.
With this commit the workers will submit their log messages to the
parent process who will either put it onto stdout or syslog.
This change in completely under the hood and users shouldn't care about it.
1) Add @kore.route as a decorator for Python.
This decorator can be used on non-class methods to automatically
declare their route and parameters.
Takes the same arguments as the kore.domain.route function that
exists today.
Provides a nice clean way of setting up Kore if you dont want
a whole class based approach.
2) Remove the requirement for the name for kore.server() and the
kore.domain(attach=) keywords.
Instead of no name was given, the name "default" is used in both
places resulting in less boilerplating.
3) Allow multiple routes to be defined for the same URI as long
as the methods are different. So you can have one method for GET /
and another for POST /.
All changes combined condense the initial experience of getting
a Kore Python app up and running:
eg:
import kore
kore.server(ip="127.0.0.1", port="8888", tls=False)
kore.domain("*")
@kore.route("/", methods=["get"])
async def index(req):
req.response(200, b'get method')
@kore.route("/", methods=["post"])
async def index_post(req)
req.response(200, b'post method')
- Kore now only supports OpenSSL 1.1.1 and LibreSSL 3.x.
- Revise the default TLS ciphersuites.
- Kore now carries ffdhe4096.pem and installs it under PREFIX/share/kore.
- Kore its tls_dhparam config setting defaults to the path mentioned above
so you no longer have to set it.
Includes the kore-serve utility that spins up a Kore instance
on the local directory and serves the contents via a filemap
on localhost port 8888.
Used by myself when hacking on the kore website.
This will place the required sources for building
single binary builds under $PREFIX/share/kore.
The kodev utility will now pickup this KORE_SOURCE path automatically
unless another one is given via the conf/build.conf file or the KORE_SOURCE
environment path.
Without it python_curlopt.h might not be available at the right
time when using something like make -j4:
src/python.c:50:10: fatal error: 'python_curlopt.h' file not found
^~~~~~~~~~~~~~~~~~
1 error generated.
Signed-off-by: Tobias Kortkamp <t@tobik.me>
Kore already exposed parts of this via the kore.httpclient() method but
this commit takes it a bit further and exposes the libcurl interface
completely (including the setopt options).
tldr:
handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README")
handle.setopt(kore.CURLOPT_TIMEOUT, 5)
data = await handle.run()
print("%s" % data.decode())
A new acme process is created that communicates with the acme servers.
This process does not hold any of your private keys (no account keys,
no domain keys etc).
Whenever the acme process requires a signed payload it will ask the keymgr
process to do the signing with the relevant keys.
This process is also sandboxed with pledge+unveil on OpenBSD and seccomp
syscall filtering on Linux.
The implementation only supports the tls-alpn-01 challenge. This means that
you do not need to open additional ports on your machine.
http-01 and dns-01 are currently not supported (no wildcard support).
A new configuration option "acme_provider" is available and can be set
to the acme server its directory. By default this will point to the
live letsencrypt environment:
https://acme-v02.api.letsencrypt.org/directory
The acme process can be controlled via the following config options:
- acme_root (where the acme process will chroot/chdir into).
- acme_runas (the user the acme process will run as).
If none are set, the values from 'root' and 'runas' are taken.
If you want to turn on acme for domains you do it as follows:
domain kore.io {
acme yes
}
You do not need to specify certkey/certfile anymore, if they are present
still
they will be overwritten by the acme system.
The keymgr will store all certificates and keys under its root
(keymgr_root), the account key is stored as "/account-key.pem" and all
obtained certificates go under "certificates/<domain>/fullchain.pem" while
keys go under "certificates/<domain>/key.pem".
Kore will automatically renew certificates if they will expire in 7 days
or less.
Mostly compliant, ignores \uXXXX in strings for now.
New API functions:
void kore_json_init(struct kore_json *json, const u_int8_t *data, size_t len);
- Prepares JSON data for parsing.
int kore_json_parse(struct kore_json *json)
- Parses the JSON data prepared via kore_json_init. Returns KORE_RESULT_ERROR
if parsing failed or KORE_RESULT_OK if it succeeded.
struct kore_json_item *kore_json_get(struct kore_json *json, const char *path,
int type);
- Try to find the object matching a given search patch and type.
eg, given a JSON structure of:
{
"reasons": {
"strings": [
"first reason",
"second"
]
}
}
one can obtain the second element in the reasons.strings array via:
item = kore_json_get(json, "reasons/strings[0]", KORE_JSON_TYPE_STRING);
Returns NULL if the item was not found or a type mismatch was hit,
otherwise will return the item of that type.
The kore_json_item data structure has a data member that contains the
relevant bits depending on the type:
KORE_JSON_TYPE_ARRAY, KORE_JSON_TYPE_OBJECT:
the data.items member is valid.
KORE_JSON_TYPE_STRING:
the data.string member is valid.
KORE_JSON_TYPE_NUMBER:
the data.number member is valid.
KORE_JSON_TYPE_LITERAL:
the data.literal member is valid.
void kore_json_cleanup(struct kore_json *json);
- Cleanup any resources
const char *kore_json_strerror(struct kore_json *json);
- Return pointer to human readable error string.
Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.
It is time to allow non TLS listeners without having to rebuild Kore.
This commit changes your configuration format and will break existing
applications their config.
Configurations now get listener {} contexts:
listen default {
bind 127.0.0.1 8888
}
The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).
If you want to turn off TLS on that listener, specify "tls no" in that
context.
Domains now need to be attached to a listener:
Eg:
domain * {
attach default
}
For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:
kore.listen("name", ip=None, port=None, path=None, tls=True).
With this commit all Kore processes (minus the parent) are running
under seccomp.
The worker processes get the bare minimum allowed syscalls while each module
like curl, pgsql, etc will add their own filters to allow what they require.
New API functions:
int kore_seccomp_filter(const char *name, void *filter, size_t len);
Adds a filter into the seccomp system (must be called before
seccomp is enabled).
New helpful macro:
define KORE_SYSCALL_ALLOW(name)
Allow the syscall with a given name, should be used in
a sock_filter data structure.
New hooks:
void kore_seccomp_hook(void);
Called before seccomp is enabled, allows developers to add their
own BPF filters into seccomp.
This commit adds the CURL=1 build option. When enabled allows
you to schedule CURL easy handles onto the Kore event loop.
It also adds an easy to use HTTP client API that abstracts away the
settings required from libcurl to make HTTP requests.
Tied together with HTTP request state machines this means you can
write fully asynchronous HTTP client requests in an easy way.
Additionally this exposes that API to the Python code as well
allowing you do to things like:
client = kore.httpclient("https://kore.io")
status, body = await client.get()
Introduces 2 configuration options:
- curl_recv_max
Max incoming bytes for a response.
- curl_timeout
Timeout in seconds before a transfer is cancelled.
This API also allows you to take the CURL easy handle and send emails
with it, run FTP, etc. All asynchronously.
Add KORE_PYTHON_LIB and KORE_PYTHON_INC which can be set
by a caller in case the libraries exist somewhere else.
Add KORE_CRYPTO to be able to override the name of the default
crypto library Kore would link with.
This means you can now do things like:
resp = await koresock.recv(1024)
await koresock.send(resp)
directly from page handlers if they are defined as async.
Adds lots more to the python goo such as fatalx(), bind_unix(),
task_create() and socket_wrap().
A filemap is a way of telling Kore to serve files from a directory
much like a traditional webserver can do.
Kore filemaps only handles files. Kore does not generate directory
indexes or deal with non-regular files.
The way files are sent to a client differs a bit per platform and
build options:
default:
- mmap() backed file transfer due to TLS.
NOTLS=1
- sendfile() under FreeBSD, macOS and Linux.
- mmap() backed file for OpenBSD.
The opened file descriptors/mmap'd regions are cached and reused when
appropriate. If a file is no longer in use it will be closed and evicted
from the cache after 30 seconds.
New API's are available allowing developers to use these facilities via:
void net_send_fileref(struct connection *, struct kore_fileref *);
void http_response_fileref(struct http_request *, struct kore_fileref *);
Kore will attempt to match media types based on file extensions. A few
default types are built-in. Others can be added via the new "http_media_type"
configuration directive.
