It was hardcoded that if KORE_USE_PYTHON was defined we would
look at the passed argument on the command-line as the python
script or module to be run.
This won't work when adding more runtimes.
So instead call a kore_runtime_resolve() function that in
turn calls each available runtime its resolve function.
That resolve function will check if its a script / module
that it can load, and if so will load it.
This way we can remove all those KORE_USE_PYTHON blocks in the
Kore startup path and we pave the way for lua.
This commit adds improved hooks for Python and a new signal delivery hook.
For the Python API kore_worker_configure() and kore_worker_teardown() had
to be implemented before this commit. Now one can create a workerstart
and workerend method in their koreapp as those will be called when
they exist.
The new signal hook is either kore_worker_signal() or koreapp.signal.
This new hook is called after the worker event code handles the received
signal itself first.
With this commit there is also a new kore_signal_trap() API call allowing
you to more easily trap new signals. This API also also exported to the
Python part of the code under kore.sigtrap()
Adding the hooks keyword with a dictionary attached to specify
the relevant hooks will hook them for the given route.
Eg:
domain.route("/", self.index, methods=["get"],
hooks={
"on_free": self.request_free
}
)
These are the same hooks available via a normal Kore route configuration.
If set, will call a given handler with the prototype of
`void body_chunk(struct http_request *req, const void *data, size_t len);`
for each chunk of the received HTTP body, allowing a developer to handle
it in their own way.
The incoming body is still being handled and retained in the same way
as before (in a kore_buf or temporary file).
While here, allow HTTP_STATUS_CONTINUE to work via http_response() and
make the handling of incoming HTTP header data a bit better.
This function now takes any remaining arguments passed on the command line
after kore parsed its own.
For C the new prototype looks like this:
void kore_parent_configure(int argc, char **argv);
For python code, kore will pass each argument to the function so you
can do things like:
def kore_parent_configure(arg1, arg2):
The HTTP layer used to make a copy of each incoming header and its
value for a request. Stop doing that and make HTTP headers zero-copy
all across the board.
This change comes with some api function changes, notably the
http_request_header() function which now takes a const char ** rather
than a char ** out pointer.
This commit also constifies several members of http_request, beware.
Additional rework how the worker processes deal with the accept lock.
Before:
if a worker held the accept lock and it accepted a new connection
it would release the lock for others and back off for 500ms before
attempting to grab the lock again.
This approach worked but under high load this starts becoming obvious.
Now:
- workers not holding the accept lock and not having any connections
will wait less long before returning from kore_platform_event_wait().
- workers not holding the accept lock will no longer blindly wait
an arbitrary amount in kore_platform_event_wait() but will look
at how long until the next lock grab is and base their timeout
on that.
- if a worker its next_lock timeout is up and failed to grab the
lock it will try again in half the time again.
- the worker process holding the lock will when releasing the lock
double check if it still has space for newer connections, if it does
it will keep the lock until it is full. This prevents the lock from
bouncing between several non busy worker processes all the time.
Additional fixes:
- Reduce the number of times we check the timeout list, only do it twice
per second rather then every event tick.
- Fix solo worker count for TLS (we actually hold two processes, not one).
- Make sure we don't accidentally miscalculate the idle time causing new
connections under heavy load to instantly drop.
- Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
doing this allows us to get rid of the validator reload
and handler reload as well as fixing websocket runtime
callbacks which were never being resolved upon module reloads.
Joris Vink
Joris Vink