If a coroutine is killed from another coroutine and the killed coroutine
was waiting on a kore.lock() object, it would have been incorrectly
woken up again once said lock was released.
This would cause a Python exception that a generator was already
running and a crash due to the pool element already being freed.
Track the active locking operation per coroutine so we can remove
the coroutine if it is killed, fixing the problem.
This method allows you to set a Python object and obtain it
by calling the method again without any arguments.
eg:
foo = SomeClass()
kore.app(foo)
foo = kore.app()
- Fix the curl-extract-opt.sh generation script to work on newer
curl releases as the header changed slightly.
- Use the correct handles when calling curl_easy_setopt() inside
of our setopt functions exported via Python.
- Add a curl.setbody() method, allowing a body to be sent to be set.
(eg when sending mail via SMTP).
- Regen of our python_curlopt.h from 7.71.1
This hooks into the existing redirection framework but allows you
to quickly deny certain paths with a 403 or other status code.
The snippet below would for example declare a filemap served from 'www'
directory but denying all access to the files under the 'data' directory:
filemap / www
deny /data 403
Inside the domain contexts a 'redirect' rule will allow you to redirect
a request to another URI.
Ex:
Redirect all requests with a 301 to example.com
redirect ^/.*$ 301 https://example.com
Using capture groups
redirect ^/account/(.*)$ 301 https://example.com/account/$1
Adding the query string in the mix
redirect ^/(.*)$ 301 https://example.com/$1?$qs
Instead of adding all listening sockets into the kqueue at platform init,
do it in the first call to kore_platform_enable_accept().
This way a worker process can still call kore_server_create() in its
kore_worker_configure() hook.
Kore already exposed parts of this via the kore.httpclient() method but
this commit takes it a bit further and exposes the libcurl interface
completely (including the setopt options).
tldr:
handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README")
handle.setopt(kore.CURLOPT_TIMEOUT, 5)
data = await handle.run()
print("%s" % data.decode())
If waitpid() returns -1 check if errno is ECHILD, just mark the worker
process as exited.
This could happen if Kore starts without keymgr/acme but those would still
be accounted for.
These changes improve the constraint kore had with client authentication and
multiple domains.
- Add kore_x509_subject_name() which will return a C string containing
the x509 subject name in full (in utf8).
- Log TLS errors if client authentication was turned on, will help debug
issues with client authentication in the future.
- If SNI was present in the TLS handshake, check it against the host specified
in the HTTP request and send a 421 in case they mismatch.
- Throw a 403 if client authentication was enabled but no client certificate
was specified.
A new acme process is created that communicates with the acme servers.
This process does not hold any of your private keys (no account keys,
no domain keys etc).
Whenever the acme process requires a signed payload it will ask the keymgr
process to do the signing with the relevant keys.
This process is also sandboxed with pledge+unveil on OpenBSD and seccomp
syscall filtering on Linux.
The implementation only supports the tls-alpn-01 challenge. This means that
you do not need to open additional ports on your machine.
http-01 and dns-01 are currently not supported (no wildcard support).
A new configuration option "acme_provider" is available and can be set
to the acme server its directory. By default this will point to the
live letsencrypt environment:
https://acme-v02.api.letsencrypt.org/directory
The acme process can be controlled via the following config options:
- acme_root (where the acme process will chroot/chdir into).
- acme_runas (the user the acme process will run as).
If none are set, the values from 'root' and 'runas' are taken.
If you want to turn on acme for domains you do it as follows:
domain kore.io {
acme yes
}
You do not need to specify certkey/certfile anymore, if they are present
still
they will be overwritten by the acme system.
The keymgr will store all certificates and keys under its root
(keymgr_root), the account key is stored as "/account-key.pem" and all
obtained certificates go under "certificates/<domain>/fullchain.pem" while
keys go under "certificates/<domain>/key.pem".
Kore will automatically renew certificates if they will expire in 7 days
or less.
If set to "yes" then Kore will trace its child processes and properly
notify you of seccomp violations while still allowing the syscalls.
This can be very useful when running Kore on new platforms that have
not been properly tested with seccomp, allowing me to adjust the default
policies as we move further.
If the kodev tool is built with MINIMAL=1 it will not compile in
support for creating application skeletons, only to build apps, etc.
Building with MINIMAL=1 drops the openssl linkage.
We actually woke up the coroutine that originally spawned the process
when we reap it, but another coroutine may have taken over the object.
This mimics how we do things for the pysock_op things.
Now everything that has the "newer" OpenSSL API (1.1.x) is hidden
behind a KORE_OPENSSL_NEWER_API define. Tone down minimum libressl
version again to 2.7.5.
Allow JSON to be constructed via kore_json_create_item and its
handy macro family:
- kore_json_create_object()
- kore_json_create_array()
- kore_json_create_string()
- kore_json_create_number()
- kore_json_create_literal().
Adds kore_json_item_tobuf() to convert a JSON item into a string
representation in a kore_buf data structure.
Renames the kore_json_get* family to kore_json_find* instead.
Allows for quite clean code:
struct kore_buf buf;
struct kore_json_item *root;
root = kore_json_create_object(NULL, NULL);
kore_json_create_string(root, "hello", "world");
kore_json_create_number(root, "value", 2.241);
kore_buf_init(&buf, 128);
kore_json_item_tobuf(root, &buf);
kore_json_item_free(root);
kore_buf_cleanup(&buf);
In cases where a request is immediately completed in libcurl its multi
handle and no additional i/o is happening a coro can get stuck waiting
to be run.
Prevent this by lowering netwait from KORE_WAIT_INFINITE if there
are pending python coroutines.
Changes kore_curl_init() to take a flag parameter, much like pgsql api
in which you specify KORE_CURL_ASYNC or KORE_CURL_SYNC.
If KORE_CURL_ASYNC is specified, Kore will behave as before.
If Kore_CURL_SYNC is specified, Kore will execute the libcurl immediately
and return once it has been completed.
This allows you to send Python objects that can be run through pickle
to other worker processes.
If your application implements koreapp.onmsg() you will be able to receive
these objects.
A new hook in the koreapp class is called right before seccomp
is enabled. This hook receives a Kore seccomp object which has
the following methods:
seccomp.allow("syscall")
seccomp.allow_arg("syscall", arg, value)
seccomp.allow_flag("syscall", arg, flag)
seccomp.allow_mask("syscall", arg, mask)
seccomp.deny("syscall")
seccomp.deny_arg("syscall", arg, value, errno=EACCES)
seccomp.deny_flag("syscall", arg, flag, errno=EACCES)
seccomp.deny_mask("syscall", arg, mask, errno=EACCES)
This allows you to finetune the seccomp filters for your application
from inside your koreapp.
Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.
It is time to allow non TLS listeners without having to rebuild Kore.
This commit changes your configuration format and will break existing
applications their config.
Configurations now get listener {} contexts:
listen default {
bind 127.0.0.1 8888
}
The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).
If you want to turn off TLS on that listener, specify "tls no" in that
context.
Domains now need to be attached to a listener:
Eg:
domain * {
attach default
}
For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:
kore.listen("name", ip=None, port=None, path=None, tls=True).
- Kore can now fully be configured via Python code if one wants nothing to
do with configuration files.
- Kore can now start single python files and no longer requires them to be
inside a module directory.
- Pass all regex capture groups to the handler methods, allowing you to
get access to them immediately.
- Change python websocket_handshake to take callable objects directly.
- Added a new deployment configuration option. If set to "dev" or
"development" Kore will automatically foreground, no chroot / etc.
If set to "production" Kore *will* chroot, drop privs, etc.
- Many more..
These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
More BPF helper macros, more helper for granular syscall checking.
Use these throughout kore where it makes sense.
The new helpers:
- KORE_SYSCALL_DENY_ARG(name, arg, value, errno):
Deny the system call with errno if the argument matches value.
- KORE_SYSCALL_DENY_MASK(name, arg, mask, errno):
Deny the system call with errno if the mask argument does not match
the exact mask given.
- KORE_SYSCALL_DENY_WITH_FLAG(name, arg, flag, errno):
Deny the system call with errno if the argument contains the
given flag.
The reverse also exists:
- KORE_SYSCALL_ALLOW_ARG()
- KORE_SYSCALL_ALLOW_MASK()
- KORE_SYSCALL_ALLOW_WITH_FLAG()
With this commit all Kore processes (minus the parent) are running
under seccomp.
The worker processes get the bare minimum allowed syscalls while each module
like curl, pgsql, etc will add their own filters to allow what they require.
New API functions:
int kore_seccomp_filter(const char *name, void *filter, size_t len);
Adds a filter into the seccomp system (must be called before
seccomp is enabled).
New helpful macro:
define KORE_SYSCALL_ALLOW(name)
Allow the syscall with a given name, should be used in
a sock_filter data structure.
New hooks:
void kore_seccomp_hook(void);
Called before seccomp is enabled, allows developers to add their
own BPF filters into seccomp.
Allows killing of coroutines, given their task id.
The kore.task_create() method now returns the task id for a newly
created task to the caller.
While here, change the coroutine task id to a uint32 from uint64.
There is no need for it to be 64bit. (famous last words)
If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine
traces while running. These traces include the filename, function and line
number where the coroutines are waking up, running and suspended.
- decouple pgsql from the HTTP request allowing it to be used in other
contexts as well (such as a task, etc).
- change names to dbsetup() and dbquery().
eg:
result = kore.dbquery("db", "select foo from bar")
Prevents a stall in case there is still data in the read end of the socket
but PQisBusy() told us to not fetch a result yet. In that case we end up
stalling due to epoll not giving us another EPOLLIN event due to EPOLLET.
In case libcurl instructs us to call the timeout function as soon
as possible (timeout == 0 in curl_timeout), don't try to be clever
with a timeout value of 10ms.
Instead call the timeout function once we get back in the worker
event loop. This makes things a lot snappier as we don't depend
on epoll/kqueue waiting for io for 10ms (which actually isn't 10ms...).
- If Kore is built with PYTHON=1 you can now specify the module that
should be loaded on the command-line.
eg: $ kore -frn myapp
- Add skeleton generation for python applications to kodev.
eg: $ kodev create -p myapp
This should make it a whole lot easier to get started with kore python.
1) Add @kore.prerequest python decorator.
Using this decorator on a function will cause that function
to always be executed *before* any page handler is run.
eg:
@kore.prerequest
def _check(req):
if req.method == kore.HTTP_METHOD_POST:
req.populate_post()
2) Allow attributes to be set on the pyhttp object.
If the connection on which we are about to send the response was
marked as disconnecting, do not go ahead and hook into the disconnect
callback (it will never be called, it is already disconnecting).
Instead just return, the connection will be removed anyway.
We grab a reference to the pyhttp_client for the client_op data structure
but never removed it. This caused the pyhttp_client object to never
be released when out of scope.
python_coro_create() puts the HTTP request to sleep, but if they
finish immediately they will be removed from the list and should
properly be woken up or they are removed from the wrong list.
Introduce kore_curl_strerror(), use this in kore_curl_logerror()
instead of assuming our errbuf has been populated.
Also use it in the python httpclient when throwing an exception rather
then looking at the errbuf member which may or may not be empty.
No need to wait until the next time http_process() is called, which
could result in HTTP requests backing up even though we are processing
them at a fast pace.
1) If a coroutine was waken up by another releasing a lock it was waiting
on we would incorrectly remove the pylock_op when the newly awoken coroutine
hits the iternext for pylock.
2) If a coroutine attempts to grab a lock it was woken up for only to fail
we did not properly rearm the coroutine to be woken up later to try again.
a commit done in 2018 prevented http responses with error codes
> 500 to include any user-set headers, preventing a developer
to include things like content-type etc.
reported by Arun Babu via users@
if an iterator is passed kore will send the response with
transfer-encoding: chunked and call the iterator for every
chunk that was sent.
The iterator must return a utf-8 string.
Works wonderful with TemplateStream from jinja2.
- Add kore_pgsql_query_param_fields() which allows you to pass in the
arrays for values, lengths and formats yourself.
- Add kore_pgsql_column_binary() which will return 1 if the given column
index contains a binary result or 0 if it contains a text result.
- Change the query call in req.pgsql() for Python to always use the
parameterized queries.
This adds the 'params' and 'binary' keywords to the req.pgsql method.
Eg:
result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"),
params=["this is my value"])
This commit adds the CURL=1 build option. When enabled allows
you to schedule CURL easy handles onto the Kore event loop.
It also adds an easy to use HTTP client API that abstracts away the
settings required from libcurl to make HTTP requests.
Tied together with HTTP request state machines this means you can
write fully asynchronous HTTP client requests in an easy way.
Additionally this exposes that API to the Python code as well
allowing you do to things like:
client = kore.httpclient("https://kore.io")
status, body = await client.get()
Introduces 2 configuration options:
- curl_recv_max
Max incoming bytes for a response.
- curl_timeout
Timeout in seconds before a transfer is cancelled.
This API also allows you to take the CURL easy handle and send emails
with it, run FTP, etc. All asynchronously.
By default kore will restart worker processes if they terminate
unexpected. However in certain scenarios you may want to bring down
an entire kore instance if a worker process fails.
By setting worker_death_policy to "terminate" the Kore server will
completely stop if a worker exits unexpected.
this change also stops python coroutines from waking up very
late after their timeout has expired.
in filerefs, don't prime the timer until we actually have something
to expire, and kill the timer when the last ref drops.
Attach the events directly to the pysocket data structure instead of
one event per pysocket_op.
Makes the code easier, gives us a good performance boost and reduces
the number of system calls required when doing an await on a socket.
if a crl is expired or not-yet-valid SSL_get_verify_result()
will return these errors too so check for them explicitly
instead of depending on X509_V_OK.
found by @dacechavez
With these changes CRLs can be reloaded like certificates
by sending a SIGUSR1 to the parent process.
Track mtime on both certificate files and CRL files as well
and only submit them to the workers if this has changed.
Move away from the parent constantly hitting the disk for every
accesslog the workers are sending.
The workers will now write their own accesslogs to shared
memory before the parent will pick those up. The parent
will flush them to disk once every second or if they grow
larger then 1MB.
This removes the heavy penalty for having access logs
turned on when you are dealing with a large volume
of requests.
- add kore.time() as equivalent for kore_time_ms().
- call waitpid() until no more children are available for reaping otherwise
we risk missing a process if several die at the same time and only one
SIGCHLD is delivered to us.
- drain a RECV socket operation if eof is set but no exception was given.
Before we just doubled in size the second we exhausted a pool instead
of doing a more controlled expansion.
Now we will expand at 25% of the initial elm count whenever we need to.
Will help with memory pressure in certain scenarios.
Allows one to run coroutines concurrently and gather all their
results in a single returned list.
If any of the coroutines throw an exception the exception is
returned as the value of that coroutine in the returned list.
This commit removes TLS 1.0 support no matter what OpenSSL
you are linking against.
Changes the value of tls_version from 1.2 to both. Meaning if
you link with OpenSSL 1.1.1 you will get 1.2 + 1.3.
This adds kore.proc to the python runtime allowing async processing
handling:
The kore.proc method takes the command to run and an optional timeout
parameter in milliseconds. If the process did not exit normally after
that amount of time a TimeoutError exception is raised.
For instance:
async def run(cmd):
proc = kore.proc(cmd, 1000)
try:
await proc.send("hello")
proc.close_stdin()
except TimeoutError:
proc.kill()
retcode = await proc.reap()
return retcode
If exists these functions are called when the worker is exiting
and when right before the parent exists.
Allows for cleanup code for applications if need to do cleanup on exit.
- Take CFLAGS, CXXFLAGS and LDFLAGS from environment
if present and append them to the build.conf cflags
- Allow overriding of the OBJDIR via KORE_OBJDIR.
- Allow overriding of build flavor via KORE_BUILD_FLAVOR.
This means you can now do things like:
resp = await koresock.recv(1024)
await koresock.send(resp)
directly from page handlers if they are defined as async.
Adds lots more to the python goo such as fatalx(), bind_unix(),
task_create() and socket_wrap().
Now anyone can schedule events and get a callback to work as long
as the user data structure that is added for the event begins
with a kore_event data structure.
All event state is now kept in that kore_event structure and renamed
CONN_[READ|WRITE]_POSSIBLE to KORE_EVENT_[READ|WRITE].
If a worker process dies it automatically gets respawned by the
parent process, but sometimes you want the entire server to go down
if a worker encounters an error. This is what fatalx() does.
Calling fatalx() from a worker process will initiate a full shutdown
of the kore server you are running under.
Joris Vink
Frederic Cambus
Raymond Pasco
Erik Karlsson