Now everything that has the "newer" OpenSSL API (1.1.x) is hidden
behind a KORE_OPENSSL_NEWER_API define. Tone down minimum libressl
version again to 2.7.5.
Allow JSON to be constructed via kore_json_create_item and its
handy macro family:
- kore_json_create_object()
- kore_json_create_array()
- kore_json_create_string()
- kore_json_create_number()
- kore_json_create_literal().
Adds kore_json_item_tobuf() to convert a JSON item into a string
representation in a kore_buf data structure.
Renames the kore_json_get* family to kore_json_find* instead.
Allows for quite clean code:
struct kore_buf buf;
struct kore_json_item *root;
root = kore_json_create_object(NULL, NULL);
kore_json_create_string(root, "hello", "world");
kore_json_create_number(root, "value", 2.241);
kore_buf_init(&buf, 128);
kore_json_item_tobuf(root, &buf);
kore_json_item_free(root);
kore_buf_cleanup(&buf);
In cases where a request is immediately completed in libcurl its multi
handle and no additional i/o is happening a coro can get stuck waiting
to be run.
Prevent this by lowering netwait from KORE_WAIT_INFINITE if there
are pending python coroutines.
Changes kore_curl_init() to take a flag parameter, much like pgsql api
in which you specify KORE_CURL_ASYNC or KORE_CURL_SYNC.
If KORE_CURL_ASYNC is specified, Kore will behave as before.
If Kore_CURL_SYNC is specified, Kore will execute the libcurl immediately
and return once it has been completed.
This allows you to send Python objects that can be run through pickle
to other worker processes.
If your application implements koreapp.onmsg() you will be able to receive
these objects.
A new hook in the koreapp class is called right before seccomp
is enabled. This hook receives a Kore seccomp object which has
the following methods:
seccomp.allow("syscall")
seccomp.allow_arg("syscall", arg, value)
seccomp.allow_flag("syscall", arg, flag)
seccomp.allow_mask("syscall", arg, mask)
seccomp.deny("syscall")
seccomp.deny_arg("syscall", arg, value, errno=EACCES)
seccomp.deny_flag("syscall", arg, flag, errno=EACCES)
seccomp.deny_mask("syscall", arg, mask, errno=EACCES)
This allows you to finetune the seccomp filters for your application
from inside your koreapp.
Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.
It is time to allow non TLS listeners without having to rebuild Kore.
This commit changes your configuration format and will break existing
applications their config.
Configurations now get listener {} contexts:
listen default {
bind 127.0.0.1 8888
}
The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).
If you want to turn off TLS on that listener, specify "tls no" in that
context.
Domains now need to be attached to a listener:
Eg:
domain * {
attach default
}
For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:
kore.listen("name", ip=None, port=None, path=None, tls=True).
- Kore can now fully be configured via Python code if one wants nothing to
do with configuration files.
- Kore can now start single python files and no longer requires them to be
inside a module directory.
- Pass all regex capture groups to the handler methods, allowing you to
get access to them immediately.
- Change python websocket_handshake to take callable objects directly.
- Added a new deployment configuration option. If set to "dev" or
"development" Kore will automatically foreground, no chroot / etc.
If set to "production" Kore *will* chroot, drop privs, etc.
- Many more..
These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
More BPF helper macros, more helper for granular syscall checking.
Use these throughout kore where it makes sense.
The new helpers:
- KORE_SYSCALL_DENY_ARG(name, arg, value, errno):
Deny the system call with errno if the argument matches value.
- KORE_SYSCALL_DENY_MASK(name, arg, mask, errno):
Deny the system call with errno if the mask argument does not match
the exact mask given.
- KORE_SYSCALL_DENY_WITH_FLAG(name, arg, flag, errno):
Deny the system call with errno if the argument contains the
given flag.
The reverse also exists:
- KORE_SYSCALL_ALLOW_ARG()
- KORE_SYSCALL_ALLOW_MASK()
- KORE_SYSCALL_ALLOW_WITH_FLAG()
With this commit all Kore processes (minus the parent) are running
under seccomp.
The worker processes get the bare minimum allowed syscalls while each module
like curl, pgsql, etc will add their own filters to allow what they require.
New API functions:
int kore_seccomp_filter(const char *name, void *filter, size_t len);
Adds a filter into the seccomp system (must be called before
seccomp is enabled).
New helpful macro:
define KORE_SYSCALL_ALLOW(name)
Allow the syscall with a given name, should be used in
a sock_filter data structure.
New hooks:
void kore_seccomp_hook(void);
Called before seccomp is enabled, allows developers to add their
own BPF filters into seccomp.
Allows killing of coroutines, given their task id.
The kore.task_create() method now returns the task id for a newly
created task to the caller.
While here, change the coroutine task id to a uint32 from uint64.
There is no need for it to be 64bit. (famous last words)
If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine
traces while running. These traces include the filename, function and line
number where the coroutines are waking up, running and suspended.
- decouple pgsql from the HTTP request allowing it to be used in other
contexts as well (such as a task, etc).
- change names to dbsetup() and dbquery().
eg:
result = kore.dbquery("db", "select foo from bar")
Prevents a stall in case there is still data in the read end of the socket
but PQisBusy() told us to not fetch a result yet. In that case we end up
stalling due to epoll not giving us another EPOLLIN event due to EPOLLET.
In case libcurl instructs us to call the timeout function as soon
as possible (timeout == 0 in curl_timeout), don't try to be clever
with a timeout value of 10ms.
Instead call the timeout function once we get back in the worker
event loop. This makes things a lot snappier as we don't depend
on epoll/kqueue waiting for io for 10ms (which actually isn't 10ms...).
- If Kore is built with PYTHON=1 you can now specify the module that
should be loaded on the command-line.
eg: $ kore -frn myapp
- Add skeleton generation for python applications to kodev.
eg: $ kodev create -p myapp
This should make it a whole lot easier to get started with kore python.
1) Add @kore.prerequest python decorator.
Using this decorator on a function will cause that function
to always be executed *before* any page handler is run.
eg:
@kore.prerequest
def _check(req):
if req.method == kore.HTTP_METHOD_POST:
req.populate_post()
2) Allow attributes to be set on the pyhttp object.
If the connection on which we are about to send the response was
marked as disconnecting, do not go ahead and hook into the disconnect
callback (it will never be called, it is already disconnecting).
Instead just return, the connection will be removed anyway.
We grab a reference to the pyhttp_client for the client_op data structure
but never removed it. This caused the pyhttp_client object to never
be released when out of scope.
python_coro_create() puts the HTTP request to sleep, but if they
finish immediately they will be removed from the list and should
properly be woken up or they are removed from the wrong list.
Introduce kore_curl_strerror(), use this in kore_curl_logerror()
instead of assuming our errbuf has been populated.
Also use it in the python httpclient when throwing an exception rather
then looking at the errbuf member which may or may not be empty.
No need to wait until the next time http_process() is called, which
could result in HTTP requests backing up even though we are processing
them at a fast pace.
Joris Vink
Frederic Cambus