mirrors
/
kore
mirror of https://git.kore.io/kore.git
1
0
Fork 1
Code Issues Projects Releases Wiki Activity
kore/src/runtime.c

296 lines
6.6 KiB
C
Raw Normal View History

Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
/*
i forgot, it's 2022.
2022-01-31 22:02:06 +01:00
* Copyright (c) 2017-2022 Joris Vink <joris@coders.se>
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <sys/param.h>
explicitly include sys/types.h some smaller libc variants do not include this from sys/param.h.
2019-03-06 09:29:46 +01:00
#include <sys/types.h>
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#include "kore.h"
#if !defined(KORE_NO_HTTP)
#include "http.h"
#endif
#if defined(KORE_USE_PYTHON)
#include "python_api.h"
#endif
Initial lua runtime. Works enough so one can do basic configuration and handle HTTP.
2023-01-21 23:41:35 +01:00
#if defined(KORE_USE_LUA)
#include "lua_api.h"
#endif
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
static void native_runtime_execute(void *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static int native_runtime_onload(void *, int);
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
static void native_runtime_signal(void *, int);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void native_runtime_connect(void *, struct connection *);
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
static void native_runtime_configure(void *, int, char **);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#if !defined(KORE_NO_HTTP)
static int native_runtime_http_request(void *, struct http_request *);
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
static void native_runtime_http_request_free(void *, struct http_request *);
Add the on_body_chunk handler for routes. If set, will call a given handler with the prototype of `void body_chunk(struct http_request *req, const void *data, size_t len);` for each chunk of the received HTTP body, allowing a developer to handle it in their own way. The incoming body is still being handled and retained in the same way as before (in a kore_buf or temporary file). While here, allow HTTP_STATUS_CONTINUE to work via http_response() and make the handling of incoming HTTP header data a bit better.
2021-09-17 19:28:06 +02:00
static void native_runtime_http_body_chunk(void *, struct http_request *,
const void *, size_t);
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
static int native_runtime_validator(void *, struct http_request *,
const void *);
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
static void native_runtime_wsmessage(void *, struct connection *, u_int8_t,
const void *, size_t);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#endif
struct kore_runtime kore_native_runtime = {
KORE_RUNTIME_NATIVE,
#if !defined(KORE_NO_HTTP)
.http_request = native_runtime_http_request,
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
.http_request_free = native_runtime_http_request_free,
Fix indentation
2021-12-13 10:48:29 +01:00
.http_body_chunk = native_runtime_http_body_chunk,
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
.validator = native_runtime_validator,
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
.wsconnect = native_runtime_connect,
.wsmessage = native_runtime_wsmessage,
.wsdisconnect = native_runtime_connect,
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#endif
.onload = native_runtime_onload,
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
.signal = native_runtime_signal,
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
.connect = native_runtime_connect,
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
.execute = native_runtime_execute,
.configure = native_runtime_configure
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
};
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
static struct kore_runtime *runtimes[] = {
#if defined(KORE_USE_PYTHON)
&kore_python_runtime,
Initial lua runtime. Works enough so one can do basic configuration and handle HTTP.
2023-01-21 23:41:35 +01:00
#endif
#if defined(KORE_USE_LUA)
&kore_lua_runtime,
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
#endif
NULL
};
remove const from kore_runtime_count(). via some github PR, please don't make me interact with github too much.
2023-11-03 12:25:39 +01:00
size_t
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
kore_runtime_count(void)
{
return ((sizeof(runtimes) / sizeof(runtimes[0])) - 1);
}
void
kore_runtime_resolve(const char *module, const struct stat *st)
{
int i;
if (runtimes[0] == NULL)
return;
for (i = 0; runtimes[i] != NULL; i++) {
if (runtimes[i]->resolve == NULL)
continue;
if (runtimes[i]->resolve(module, st))
break;
}
if (runtimes[i] == NULL)
fatal("No runtime available to run '%s'", module);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
struct kore_runtime_call *
kore_runtime_getcall(const char *symbol)
{
void *ptr;
struct kore_runtime_call *rcall;
struct kore_runtime *runtime;
ptr = kore_module_getsym(symbol, &runtime);
if (ptr == NULL)
return (NULL);
rcall = kore_malloc(sizeof(*rcall));
rcall->addr = ptr;
rcall->runtime = runtime;
return (rcall);
}
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
void
kore_runtime_execute(struct kore_runtime_call *rcall)
{
rcall->runtime->execute(rcall->addr);
}
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
void
kore_runtime_configure(struct kore_runtime_call *rcall, int argc, char **argv)
{
rcall->runtime->configure(rcall->addr, argc, argv);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
int
kore_runtime_onload(struct kore_runtime_call *rcall, int action)
{
return (rcall->runtime->onload(rcall->addr, action));
}
void
kore_runtime_connect(struct kore_runtime_call *rcall, struct connection *c)
{
rcall->runtime->connect(rcall->addr, c);
}
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
void
kore_runtime_signal(struct kore_runtime_call *rcall, int sig)
{
rcall->runtime->signal(rcall->addr, sig);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#if !defined(KORE_NO_HTTP)
int
kore_runtime_http_request(struct kore_runtime_call *rcall,
struct http_request *req)
{
return (rcall->runtime->http_request(rcall->addr, req));
}
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
void
kore_runtime_http_request_free(struct kore_runtime_call *rcall,
struct http_request *req)
{
rcall->runtime->http_request_free(rcall->addr, req);
}
Add the on_body_chunk handler for routes. If set, will call a given handler with the prototype of `void body_chunk(struct http_request *req, const void *data, size_t len);` for each chunk of the received HTTP body, allowing a developer to handle it in their own way. The incoming body is still being handled and retained in the same way as before (in a kore_buf or temporary file). While here, allow HTTP_STATUS_CONTINUE to work via http_response() and make the handling of incoming HTTP header data a bit better.
2021-09-17 19:28:06 +02:00
void
kore_runtime_http_body_chunk(struct kore_runtime_call *rcall,
struct http_request *req, const void *data, size_t len)
{
rcall->runtime->http_body_chunk(rcall->addr, req, data, len);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
int
kore_runtime_validator(struct kore_runtime_call *rcall,
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
struct http_request *req, const void *data)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
return (rcall->runtime->validator(rcall->addr, req, data));
}
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
void
kore_runtime_wsconnect(struct kore_runtime_call *rcall, struct connection *c)
{
rcall->runtime->wsconnect(rcall->addr, c);
}
void
kore_runtime_wsmessage(struct kore_runtime_call *rcall, struct connection *c,
u_int8_t op, const void *data, size_t len)
{
rcall->runtime->wsmessage(rcall->addr, c, op, data, len);
}
void
kore_runtime_wsdisconnect(struct kore_runtime_call *rcall, struct connection *c)
{
rcall->runtime->wsdisconnect(rcall->addr, c);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#endif
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
static void
native_runtime_execute(void *addr)
{
void (*cb)(void);
*(void **)&(cb) = addr;
cb();
}
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
static void
native_runtime_configure(void *addr, int argc, char **argv)
{
void (*cb)(int, char **);
*(void **)&(cb) = addr;
cb(argc, argv);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void
native_runtime_connect(void *addr, struct connection *c)
{
void (*cb)(struct connection *);
*(void **)&(cb) = addr;
cb(c);
}
static int
native_runtime_onload(void *addr, int action)
{
int (*cb)(int);
*(void **)&(cb) = addr;
return (cb(action));
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
}
static void
native_runtime_signal(void *addr, int sig)
{
void (*cb)(int);
*(void **)&(cb) = addr;
cb(sig);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
#if !defined(KORE_NO_HTTP)
static int
native_runtime_http_request(void *addr, struct http_request *req)
{
int (*cb)(struct http_request *);
*(void **)&(cb) = addr;
return (cb(req));
}
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
static void
native_runtime_http_request_free(void *addr, struct http_request *req)
{
int (*cb)(struct http_request *);
*(void **)&(cb) = addr;
cb(req);
}
Add the on_body_chunk handler for routes. If set, will call a given handler with the prototype of `void body_chunk(struct http_request *req, const void *data, size_t len);` for each chunk of the received HTTP body, allowing a developer to handle it in their own way. The incoming body is still being handled and retained in the same way as before (in a kore_buf or temporary file). While here, allow HTTP_STATUS_CONTINUE to work via http_response() and make the handling of incoming HTTP header data a bit better.
2021-09-17 19:28:06 +02:00
static void
native_runtime_http_body_chunk(void *addr, struct http_request *req,
const void *data, size_t len)
{
void (*cb)(struct http_request *, const void *, size_t);
*(void **)&(cb) = addr;
cb(req, data, len);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static int
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
native_runtime_validator(void *addr, struct http_request *req, const void *data)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
int (*cb)(struct http_request *, const void *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
*(void **)&(cb) = addr;
return (cb(req, data));
}
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
static void
native_runtime_wsmessage(void *addr, struct connection *c, u_int8_t op,
const void *data, size_t len)
{
void (*cb)(struct connection *, u_int8_t, const void *, size_t);
*(void **)&(cb) = addr;
cb(c, op, data, len);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#endif