mirrors
/
kore
mirror of https://git.kore.io/kore.git
1
0
Fork 1
Code Issues Projects Releases Wiki Activity
kore/src/python.c

7034 lines
152 KiB
C
Raw Permalink Normal View History

Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
/*
* Copyright (c) 2016 Stanislav Yudin <stan@endlessinsomnia.com>
i forgot, it's 2022.
2022-01-31 22:02:06 +01:00
* Copyright (c) 2017-2022 Joris Vink <joris@coders.se>
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
include sys/param.h in python.c
2018-11-29 09:11:31 +01:00
#include <sys/param.h>
correct includes
2018-07-07 13:23:43 +02:00
#include <sys/types.h>
Append full module path if it is a directory.
2019-03-16 16:13:52 +01:00
#include <sys/stat.h>
correct includes
2018-07-07 13:23:43 +02:00
#include <sys/socket.h>
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
#include <sys/wait.h>
cleanup.
2019-03-01 20:56:03 +01:00
#include <sys/un.h>
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
allow double quotes in kore.proc argument lists.
2019-02-22 16:56:34 +01:00
#include <ctype.h>
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#include <libgen.h>
Python coro under-the-hood improvements. - Change python coroutine id to a uint64_t. - Add kore.task_id() to return active coro its id.
2021-12-02 21:58:13 +01:00
#include <inttypes.h>
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
#include <signal.h>
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
#include <fcntl.h>
#include <unistd.h>
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
#include <stdarg.h>
Add kore_tls_x509_data(). Use it in the Python code, which requires it.
2022-02-18 09:14:30 +01:00
#include <stddef.h>
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#include "kore.h"
#include "http.h"
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
#if defined(KORE_USE_PGSQL)
#include "pgsql.h"
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#endif
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
#if defined(KORE_USE_CURL)
#include "curl.h"
#endif
Allow acme config via python api
2019-11-13 23:01:24 +01:00
#if defined(KORE_USE_ACME)
#include "acme.h"
#endif
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#include "python_api.h"
#include "python_methods.h"
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
#if defined(KORE_USE_CURL)
#include "python_curlopt.h"
#endif
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
#include <frameobject.h>
Python improvements: Rework corotracing for 3.11. In the upcoming Python 3.11 release the PyCoroObject no longer has a full PyFrameObject, but instead their internal frame struct _PyInterpreterFrame. Use that when we are building against 3.11 or higher so we can still provide useful tracing functionality (and so that it builds).
2022-09-25 00:29:08 +02:00
#if PY_VERSION_HEX >= 0x030b0000
#include <internal/pycore_frame.h>
#endif
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
#if PY_VERSION_HEX < 0x030A0000
typedef enum {
PYGEN_RETURN = 0,
PYGEN_ERROR = -1,
PYGEN_NEXT = 1,
} PySendResult;
#endif
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
struct reqcall {
PyObject *f;
make sure prereqs are in order
2019-06-07 21:33:00 +02:00
TAILQ_ENTRY(reqcall) list;
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
};
Improve kore.proc in the Python API. Allow passing of an env keyword, allowing you to set environment variables that may be required by the subprocess. The env keyword must be a list with correctly formed environment variables. eg: proc = kore.proc("/bin/myproc", env=[ "LD_LIBRARY_PATH=/my/path" ] )
2022-10-19 08:04:16 +02:00
union deconst {
char *p;
const char *cp;
};
make sure prereqs are in order
2019-06-07 21:33:00 +02:00
TAILQ_HEAD(reqcall_list, reqcall);
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
make python_module_init() non static
2021-07-10 10:02:46 +02:00
PyMODINIT_FUNC python_module_init(void);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *python_import(const char *);
Hack around some hidden Python symbols. The _PyInterpreterFrame_GetLine() is hidden in dynamic libs so roll our own variant of it. Shuffle the old code so we always end up calling python_resolve_frame_line() no matter the Python version.
2022-09-26 08:48:29 +02:00
static int python_resolve_frame_line(void *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *pyconnection_alloc(struct connection *);
static PyObject *python_callable(PyObject *, const char *);
allow double quotes in kore.proc argument lists.
2019-02-22 16:56:34 +01:00
static void python_split_arguments(char *, char **, size_t);
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
static void python_kore_recvobj(struct kore_msg *, const void *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
static PyObject *python_cmsg_to_list(struct msghdr *);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
static const char *python_string_from_dict(PyObject *, const char *);
static int python_bool_from_dict(PyObject *, const char *, int *);
static int python_long_from_dict(PyObject *, const char *, long *);
Always use http_response_stream() in req.response. Keep around the python bytes object while we stream the contents of it out over the wire. This avoids an entire copy of the data.
2018-11-14 10:03:32 +01:00
static int pyhttp_response_sent(struct netbuf *);
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
static PyObject *pyhttp_file_alloc(struct http_file *);
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
static PyObject *pyhttp_request_alloc(const struct http_request *);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
static struct python_coro *python_coro_create(PyObject *,
struct http_request *);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
static struct kore_domain *python_route_domain_resolve(struct pyroute *);
static int python_route_install(struct pyroute *);
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
static int python_route_params(PyObject *, struct kore_route *,
const char *, int, int);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
static int python_route_methods(PyObject *, PyObject *,
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
struct kore_route *);
static int python_route_auth(PyObject *, struct kore_route *);
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
static int python_route_hooks(PyObject *, struct kore_route *);
static int python_route_hook_set(PyObject *, const char *,
struct kore_runtime_call **);
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
static int python_coro_run(struct python_coro *);
static void python_coro_wakeup(struct python_coro *);
static void python_coro_suspend(struct python_coro *);
static void python_coro_trace(const char *, struct python_coro *);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static void pysocket_evt_handle(void *, int);
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
static void pysocket_op_timeout(void *, u_int64_t);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static PyObject *pysocket_op_create(struct pysocket *,
int, const void *, size_t);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
static struct pysocket *pysocket_alloc(void);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static PyObject *pysocket_async_recv(struct pysocket_op *);
static PyObject *pysocket_async_send(struct pysocket_op *);
static PyObject *pysocket_async_accept(struct pysocket_op *);
static PyObject *pysocket_async_connect(struct pysocket_op *);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
static void pylock_do_release(struct pylock *);
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
static void pytimer_run(void *, u_int64_t);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
static void pyproc_timeout(void *, u_int64_t);
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
static void pysuspend_wakeup(void *, u_int64_t);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
static void pygather_reap_coro(struct pygather_op *,
struct python_coro *);
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
static int pyhttp_preprocess(struct http_request *);
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
static int pyhttp_iterobj_chunk_sent(struct netbuf *);
static int pyhttp_iterobj_next(struct pyhttp_iterobj *);
static void pyhttp_iterobj_disconnect(struct connection *);
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
static int pyconnection_x509_cb(void *, int, int, const char *,
const void *, size_t, int);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
#if defined(KORE_USE_PGSQL)
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
static int pykore_pgsql_result(struct pykore_pgsql *);
static void pykore_pgsql_callback(struct kore_pgsql *, void *);
static int pykore_pgsql_params(struct pykore_pgsql *, PyObject *);
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
static int pykore_pgsql_params(struct pykore_pgsql *, PyObject *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#endif
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
#if defined(KORE_USE_CURL)
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
static void python_curl_http_callback(struct kore_curl *, void *);
static void python_curl_handle_callback(struct kore_curl *, void *);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
static PyObject *pyhttp_client_request(struct pyhttp_client *, int,
PyObject *);
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
static PyObject *python_curlopt_set(struct pycurl_data *,
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
long, PyObject *);
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
static int python_curlopt_from_dict(struct pycurl_data *,
PyObject *);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
#endif
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void python_append_path(const char *);
static void python_push_integer(PyObject *, const char *, long);
static void python_push_type(const char *, PyObject *, PyTypeObject *);
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
static int python_validator_check(PyObject *);
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
static int python_runtime_resolve(const char *, const struct stat *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static int python_runtime_http_request(void *, struct http_request *);
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
static void python_runtime_http_request_free(void *, struct http_request *);
static void python_runtime_http_body_chunk(void *, struct http_request *,
const void *, size_t);
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
static int python_runtime_validator(void *, struct http_request *,
const void *);
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
static void python_runtime_wsmessage(void *, struct connection *,
u_int8_t, const void *, size_t);
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
static void python_runtime_execute(void *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static int python_runtime_onload(void *, int);
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
static void python_runtime_signal(void *, int);
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
static void python_runtime_configure(void *, int, char **);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void python_runtime_connect(void *, struct connection *);
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
static void python_module_load(struct kore_module *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void python_module_free(struct kore_module *);
static void python_module_reload(struct kore_module *);
static void *python_module_getsym(struct kore_module *, const char *);
let python use kore backed memory allocators.
2017-01-25 22:07:58 +01:00
static void *python_malloc(void *, size_t);
static void *python_calloc(void *, size_t, size_t);
static void *python_realloc(void *, void *, size_t);
static void python_free(void *, void *);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
struct kore_module_functions kore_python_module = {
.free = python_module_free,
.load = python_module_load,
.getsym = python_module_getsym,
.reload = python_module_reload
};
struct kore_runtime kore_python_runtime = {
KORE_RUNTIME_PYTHON,
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
.resolve = python_runtime_resolve,
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
.http_request = python_runtime_http_request,
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
.http_body_chunk = python_runtime_http_body_chunk,
.http_request_free = python_runtime_http_request_free,
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
.validator = python_runtime_validator,
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
.wsconnect = python_runtime_connect,
.wsmessage = python_runtime_wsmessage,
.wsdisconnect = python_runtime_connect,
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
.onload = python_runtime_onload,
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
.signal = python_runtime_signal,
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
.connect = python_runtime_connect,
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
.execute = python_runtime_execute,
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
.configure = python_runtime_configure,
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
};
push all integers exported to python in a struct.
2017-01-25 20:56:57 +01:00
static struct {
const char *symbol;
int value;
} python_integers[] = {
{ "LOG_ERR", LOG_ERR },
{ "LOG_INFO", LOG_INFO },
{ "LOG_NOTICE", LOG_NOTICE },
{ "RESULT_OK", KORE_RESULT_OK },
{ "RESULT_RETRY", KORE_RESULT_RETRY },
{ "RESULT_ERROR", KORE_RESULT_ERROR },
{ "MODULE_LOAD", KORE_MODULE_LOAD },
{ "MODULE_UNLOAD", KORE_MODULE_UNLOAD },
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
{ "TIMER_ONESHOT", KORE_TIMER_ONESHOT },
push all integers exported to python in a struct.
2017-01-25 20:56:57 +01:00
{ "CONN_PROTO_HTTP", CONN_PROTO_HTTP },
{ "CONN_PROTO_UNKNOWN", CONN_PROTO_UNKNOWN },
{ "CONN_PROTO_WEBSOCKET", CONN_PROTO_WEBSOCKET },
{ "CONN_STATE_ESTABLISHED", CONN_STATE_ESTABLISHED },
prefix HTTP defines when exporting them to python.
2018-04-10 14:35:52 +02:00
{ "HTTP_METHOD_GET", HTTP_METHOD_GET },
{ "HTTP_METHOD_PUT", HTTP_METHOD_PUT },
{ "HTTP_METHOD_HEAD", HTTP_METHOD_HEAD },
{ "HTTP_METHOD_POST", HTTP_METHOD_POST },
{ "HTTP_METHOD_DELETE", HTTP_METHOD_DELETE },
{ "HTTP_METHOD_OPTIONS", HTTP_METHOD_OPTIONS },
{ "HTTP_METHOD_PATCH", HTTP_METHOD_PATCH },
export WEBSOCKET_OP_TEXT and WEBSOCKET_OP_BINARY.
2017-01-30 21:02:58 +01:00
{ "WEBSOCKET_OP_TEXT", WEBSOCKET_OP_TEXT },
{ "WEBSOCKET_OP_BINARY", WEBSOCKET_OP_BINARY },
push all integers exported to python in a struct.
2017-01-25 20:56:57 +01:00
{ "WEBSOCKET_BROADCAST_LOCAL", WEBSOCKET_BROADCAST_LOCAL },
{ "WEBSOCKET_BROADCAST_GLOBAL", WEBSOCKET_BROADCAST_GLOBAL },
{ NULL, -1 }
};
let python use kore backed memory allocators.
2017-01-25 22:07:58 +01:00
static PyMemAllocatorEx allocator = {
.ctx = NULL,
.malloc = python_malloc,
.calloc = python_calloc,
.realloc = python_realloc,
.free = python_free
};
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
#if defined(__linux__)
#include "seccomp.h"
static struct sock_filter filter_python[] = {
/* Required for kore.proc */
Wrap certain syscalls inside of an ifdef. Makes kore with python and acme work on my pinebook pro.
2020-07-14 15:38:54 +02:00
#if defined(SYS_dup2)
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
KORE_SYSCALL_ALLOW(dup2),
Wrap certain syscalls inside of an ifdef. Makes kore with python and acme work on my pinebook pro.
2020-07-14 15:38:54 +02:00
#endif
#if defined(SYS_dup3)
KORE_SYSCALL_ALLOW(dup3),
#endif
#if defined(SYS_pipe)
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
KORE_SYSCALL_ALLOW(pipe),
Wrap certain syscalls inside of an ifdef. Makes kore with python and acme work on my pinebook pro.
2020-07-14 15:38:54 +02:00
#endif
#if defined(SYS_pipe2)
KORE_SYSCALL_ALLOW(pipe2),
#endif
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
KORE_SYSCALL_ALLOW(wait4),
KORE_SYSCALL_ALLOW(execve),
/* Socket related. */
allow more net related syscalls for python.
2019-09-26 16:13:42 +02:00
KORE_SYSCALL_ALLOW(bind),
KORE_SYSCALL_ALLOW(listen),
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
KORE_SYSCALL_ALLOW(sendto),
KORE_SYSCALL_ALLOW(recvfrom),
KORE_SYSCALL_ALLOW(getsockname),
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
KORE_SYSCALL_ALLOW(getpeername),
allow more net related syscalls for python.
2019-09-26 16:13:42 +02:00
KORE_SYSCALL_ALLOW_ARG(socket, 0, AF_INET),
KORE_SYSCALL_ALLOW_ARG(socket, 0, AF_INET6),
KORE_SYSCALL_ALLOW_ARG(socket, 0, AF_UNIX),
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
};
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
#define PYSECCOMP_ACTION_ALLOW 1
#define PYSECCOMP_ACTION_DENY 2
#define PYSECCOMP_SYSCALL_FILTER 1
#define PYSECCOMP_SYSCALL_ARG 2
#define PYSECCOMP_SYSCALL_MASK 3
#define PYSECCOMP_SYSCALL_FLAG 4
static int pyseccomp_filter_install(struct pyseccomp *,
const char *, int, int, int, int);
static PyObject *pyseccomp_common_action(struct pyseccomp *, PyObject *,
PyObject *, int, int);
static struct pyseccomp *py_seccomp = NULL;
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
#endif
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
static TAILQ_HEAD(, pyproc) procs;
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
static TAILQ_HEAD(, pyroute) routes;
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
static struct reqcall_list prereq;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static struct kore_pool coro_pool;
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
static struct kore_pool iterobj_pool;
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
static struct kore_pool queue_wait_pool;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
static struct kore_pool gather_coro_pool;
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
static struct kore_pool queue_object_pool;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
static struct kore_pool gather_result_pool;
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
static u_int64_t coro_id;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static int coro_count;
make coro tracing available at runtime, always. call kore.corotrace(True) to enable it and kore.corotrace(False) to disable.
2019-09-18 10:55:13 +02:00
static int coro_tracing;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
static struct coro_list coro_runnable;
static struct coro_list coro_suspended;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
extern const char *__progname;
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
static PyObject *pickle = NULL;
Python: Add kore.app(). This method allows you to set a Python object and obtain it by calling the method again without any arguments. eg: foo = SomeClass() kore.app(foo) foo = kore.app()
2020-07-05 21:47:22 +02:00
static PyObject *kore_app = NULL;
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
static PyObject *pickle_dumps = NULL;
static PyObject *pickle_loads = NULL;
static PyObject *python_tracer = NULL;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
/* XXX */
static struct python_coro *coro_running = NULL;
Improve python experience. - If Kore is built with PYTHON=1 you can now specify the module that should be loaded on the command-line. eg: $ kore -frn myapp - Add skeleton generation for python applications to kodev. eg: $ kodev create -p myapp This should make it a whole lot easier to get started with kore python.
2019-06-12 23:35:43 +02:00
#if !defined(KORE_SINGLE_BINARY)
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
static const char *kore_pymodule = NULL;
Improve python experience. - If Kore is built with PYTHON=1 you can now specify the module that should be loaded on the command-line. eg: $ kore -frn myapp - Add skeleton generation for python applications to kodev. eg: $ kodev create -p myapp This should make it a whole lot easier to get started with kore python.
2019-06-12 23:35:43 +02:00
#endif
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
void
kore_python_init(void)
{
don't call close() no underlying socket.
2018-11-12 09:00:36 +01:00
struct kore_runtime_call *rcall;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
coro_id = 0;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
coro_count = 0;
make coro tracing available at runtime, always. call kore.corotrace(True) to enable it and kore.corotrace(False) to disable.
2019-09-18 10:55:13 +02:00
coro_tracing = 0;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
make sure prereqs are in order
2019-06-07 21:33:00 +02:00
TAILQ_INIT(&prereq);
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
TAILQ_INIT(&procs);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
TAILQ_INIT(&routes);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
TAILQ_INIT(&coro_runnable);
TAILQ_INIT(&coro_suspended);
kore_pool_init(&coro_pool, "coropool", sizeof(struct python_coro), 100);
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
kore_pool_init(&iterobj_pool, "iterobj_pool",
sizeof(struct pyhttp_iterobj), 100);
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
kore_pool_init(&queue_wait_pool, "queue_wait_pool",
sizeof(struct pyqueue_waiting), 100);
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
kore_pool_init(&gather_coro_pool, "gather_coro_pool",
sizeof(struct pygather_coro), 100);
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
kore_pool_init(&queue_object_pool, "queue_object_pool",
sizeof(struct pyqueue_object), 100);
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
kore_pool_init(&gather_result_pool, "gather_result_pool",
sizeof(struct pygather_result), 100);
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
let python use kore backed memory allocators.
2017-01-25 22:07:58 +01:00
PyMem_SetAllocator(PYMEM_DOMAIN_OBJ, &allocator);
PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &allocator);
PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &allocator);
Only call PyMem_SetupDebugHooks() if DEBUG is 1. We do not need the memory debug hooks in production environments.
2020-08-13 09:10:19 +02:00
#if defined(KORE_DEBUG)
let python use kore backed memory allocators.
2017-01-25 22:07:58 +01:00
PyMem_SetupDebugHooks();
Only call PyMem_SetupDebugHooks() if DEBUG is 1. We do not need the memory debug hooks in production environments.
2020-08-13 09:10:19 +02:00
#endif
let python use kore backed memory allocators.
2017-01-25 22:07:58 +01:00
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
kore_msg_register(KORE_PYTHON_SEND_OBJ, python_kore_recvobj);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
if (PyImport_AppendInittab("kore", &python_module_init) == -1)
fatal("kore_python_init: failed to add new module");
don't call close() no underlying socket.
2018-11-12 09:00:36 +01:00
rcall = kore_runtime_getcall("kore_python_preinit");
if (rcall != NULL) {
kore_runtime_execute(rcall);
kore_free(rcall);
}
use Py_InitializeEx and do not install python signals.
2019-10-15 21:18:38 +02:00
Py_InitializeEx(0);
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
if ((pickle = PyImport_ImportModule("pickle")) == NULL)
fatal("failed to import pickle module");
if ((pickle_dumps = PyObject_GetAttrString(pickle, "dumps")) == NULL)
fatal("pickle module has no dumps method");
if ((pickle_loads = PyObject_GetAttrString(pickle, "loads")) == NULL)
fatal("pickle module has no loads method");
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
#if defined(__linux__)
kore_seccomp_filter("python", filter_python,
KORE_FILTER_LEN(filter_python));
#endif
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
Hide kore_pymodule behind !KORE_SINGLE_BINARY. Unbreaks building single binaries with Python support.
2022-03-21 08:51:05 +01:00
#if !defined(KORE_SINGLE_BINARY)
Only set initial python deployment if needed. We should only be setting this if an actual module was specified on the command-line that will be loaded.
2021-09-07 21:15:17 +02:00
if (kore_pymodule) {
if (!kore_configure_setting("deployment", "dev"))
fatal("failed to set initial deployment");
}
Hide kore_pymodule behind !KORE_SINGLE_BINARY. Unbreaks building single binaries with Python support.
2022-03-21 08:51:05 +01:00
#endif
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
void
kore_python_cleanup(void)
{
when exiting clear any lingering python exceptions.
2017-02-06 16:54:38 +01:00
if (Py_IsInitialized()) {
PyErr_Clear();
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Py_Finalize();
when exiting clear any lingering python exceptions.
2017-02-06 16:54:38 +01:00
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
Add python_path configuration option. Allows adding more items to the python path so imports from inside python scripts work more sensible depending on how you configure your app.
2018-03-01 12:55:33 +01:00
void
kore_python_path(const char *path)
{
python_append_path(path);
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
void
kore_python_coro_run(void)
{
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
struct pygather_op *op;
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
struct python_coro *coro;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
while ((coro = TAILQ_FIRST(&coro_runnable)) != NULL) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (coro->state != CORO_STATE_RUNNABLE)
fatal("non-runnable coro on coro_runnable");
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
if (python_coro_run(coro) == KORE_RESULT_OK) {
if (coro->gatherop != NULL) {
op = coro->gatherop;
if (op->coro->request != NULL)
http_request_wakeup(op->coro->request);
else
python_coro_wakeup(op->coro);
pygather_reap_coro(op, coro);
} else {
kore_python_coro_delete(coro);
}
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
/*
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
* Let Kore do HTTP processing so awoken coroutines run asap without
* having to wait for a tick from the event loop.
*
* Maybe it is more beneficial that we track if something related
* to HTTP requests was awoken and only run if true?
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
*/
http_process();
Stop a python coro from getting stuck with httpclient. In cases where a request is immediately completed in libcurl its multi handle and no additional i/o is happening a coro can get stuck waiting to be run. Prevent this by lowering netwait from KORE_WAIT_INFINITE if there are pending python coroutines.
2019-10-22 17:06:32 +02:00
#if defined(KORE_USE_CURL)
/*
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
* If a coroutine fired off a curl instance, immediately
Stop a python coro from getting stuck with httpclient. In cases where a request is immediately completed in libcurl its multi handle and no additional i/o is happening a coro can get stuck waiting to be run. Prevent this by lowering netwait from KORE_WAIT_INFINITE if there are pending python coroutines.
2019-10-22 17:06:32 +02:00
* let it make progress.
*/
kore_curl_do_timeout();
#endif
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
void
kore_python_coro_delete(void *obj)
{
struct python_coro *coro;
coro = obj;
coro_count--;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
python_coro_trace(coro->killed ? "killed" : "deleted", coro);
trace created/deleted.
2019-09-16 20:21:04 +02:00
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
coro_running = coro;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
if (coro->lockop != NULL) {
coro->lockop->active = 0;
TAILQ_REMOVE(&coro->lockop->lock->ops, coro->lockop, list);
Py_DECREF((PyObject *)coro->lockop);
coro->lockop = NULL;
}
Clear lockop before removing a reference from the coroutine.
2020-08-10 12:19:42 +02:00
Py_DECREF(coro->obj);
coro_running = NULL;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (coro->state == CORO_STATE_RUNNABLE)
TAILQ_REMOVE(&coro_runnable, coro, list);
else
TAILQ_REMOVE(&coro_suspended, coro, list);
allow coroutines to set friendly names.
2019-09-16 20:31:25 +02:00
kore_free(coro->name);
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
Py_XDECREF(coro->result);
allow coroutines to set friendly names.
2019-09-16 20:31:25 +02:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
kore_pool_put(&coro_pool, coro);
}
Stop a python coro from getting stuck with httpclient. In cases where a request is immediately completed in libcurl its multi handle and no additional i/o is happening a coro can get stuck waiting to be run. Prevent this by lowering netwait from KORE_WAIT_INFINITE if there are pending python coroutines.
2019-10-22 17:06:32 +02:00
int
kore_python_coro_pending(void)
{
return (!TAILQ_EMPTY(&coro_runnable));
}
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
void
kore_python_routes_resolve(void)
{
struct pyroute *route;
while ((route = TAILQ_FIRST(&routes)) != NULL) {
TAILQ_REMOVE(&routes, route, list);
if (!python_route_install(route))
fatalx("failed to install route for %s", route->path);
Py_DECREF((PyObject *)route);
}
}
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
void
kore_python_log_error(const char *function)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
const char *sval;
Allow user-supplied tracer callback.
2018-11-29 09:51:24 +01:00
PyObject *ret, *repr, *type, *value, *traceback;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if (!PyErr_Occurred() || PyErr_ExceptionMatches(PyExc_StopIteration))
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return;
PyErr_Fetch(&type, &value, &traceback);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
if (type == NULL || value == NULL) {
validators now work in python.
2017-01-25 21:17:12 +01:00
kore_log(LOG_ERR, "unknown python exception in '%s'", function);
return;
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
if (value == NULL || !PyObject_IsInstance(value, type))
PyErr_NormalizeException(&type, &value, &traceback);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
/*
* If we're in an active coroutine and it was tied to a gather
* operation we have to make sure we can use the Exception that
* was thrown as the result value so we can propagate it via the
* return list of kore.gather().
*/
if (coro_running != NULL && coro_running->gatherop != NULL) {
PyErr_SetObject(PyExc_StopIteration, value);
Allow user-supplied tracer callback.
2018-11-29 09:51:24 +01:00
} else if (python_tracer != NULL) {
/*
* Call the user-supplied tracer callback.
*/
ret = PyObject_CallFunctionObjArgs(python_tracer,
type, value, traceback, NULL);
Py_XDECREF(ret);
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
} else {
if ((repr = PyObject_Repr(value)) == NULL)
sval = "unknown";
else
sval = PyUnicode_AsUTF8(repr);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
kore_log(LOG_ERR,
"uncaught exception %s in '%s'", sval, function);
Py_XDECREF(repr);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Py_DECREF(type);
Py_DECREF(value);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Py_XDECREF(traceback);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
void
kore_python_proc_reap(void)
{
struct pyproc *proc;
Make sure we wakeup the coroutine that called proc.reap(). We actually woke up the coroutine that originally spawned the process when we reap it, but another coroutine may have taken over the object. This mimics how we do things for the pysock_op things.
2019-10-29 15:12:20 +01:00
struct python_coro *coro;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
pid_t child;
int status;
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
for (;;) {
if ((child = waitpid(-1, &status, WNOHANG)) == -1) {
if (errno == ECHILD)
return;
if (errno == EINTR)
continue;
kore_log(LOG_NOTICE, "waitpid: %s", errno_s);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
return;
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
if (child == 0)
return;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
proc = NULL;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
TAILQ_FOREACH(proc, &procs, list) {
if (proc->pid == child)
break;
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
if (proc == NULL)
continue;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
proc->pid = -1;
proc->reaped = 1;
proc->status = status;
if (proc->timer != NULL) {
kore_timer_remove(proc->timer);
proc->timer = NULL;
}
Make sure we wakeup the coroutine that called proc.reap(). We actually woke up the coroutine that originally spawned the process when we reap it, but another coroutine may have taken over the object. This mimics how we do things for the pysock_op things.
2019-10-29 15:12:20 +01:00
/*
* If someone is waiting on proc.reap() then wakeup that
* coroutine, otherwise wakeup the coroutine that created
* the process.
*/
if (proc->op != NULL)
coro = proc->op->coro;
else
coro = proc->coro;
if (coro->request != NULL)
http_request_wakeup(coro->request);
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
else
Make sure we wakeup the coroutine that called proc.reap(). We actually woke up the coroutine that originally spawned the process when we reap it, but another coroutine may have taken over the object. This mimics how we do things for the pysock_op things.
2019-10-29 15:12:20 +01:00
python_coro_wakeup(coro);
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
}
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
#if defined(__linux__)
void
let kore_python_seccomp_hook() take the hook name
2019-10-04 11:28:14 +02:00
kore_python_seccomp_hook(const char *method)
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
{
struct kore_runtime *rt;
PyObject *func, *result;
s/hook/method
2019-10-04 11:34:49 +02:00
if ((func = kore_module_getsym(method, &rt)) == NULL)
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
return;
if (rt->type != KORE_RUNTIME_PYTHON)
return;
py_seccomp = PyObject_New(struct pyseccomp, &pyseccomp_type);
if (py_seccomp == NULL)
fatal("failed to create seccomp object");
py_seccomp->elm = 0;
py_seccomp->filters = NULL;
result = PyObject_CallFunctionObjArgs(func,
(PyObject *)py_seccomp, NULL);
s/hook/method
2019-10-04 11:34:49 +02:00
kore_python_log_error(method);
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
kore_seccomp_filter("koreapp", py_seccomp->filters, py_seccomp->elm);
Py_XDECREF(result);
}
void
kore_python_seccomp_cleanup(void)
{
Py_XDECREF(py_seccomp);
py_seccomp = NULL;
}
static void
pyseccomp_dealloc(struct pyseccomp *seccomp)
{
kore_free(seccomp->filters);
seccomp->elm = 0;
seccomp->filters = NULL;
}
Add seccomp.bpf_stmt() method to Python api. Allows a developer to create their own statements, the bpf_jmp equivalent may follow later if I need it.
2019-10-07 13:44:31 +02:00
static PyObject *
pyseccomp_bpf_stmt(struct pyseccomp *seccomp, PyObject *args)
{
u_int32_t k;
u_int16_t code;
size_t len, off;
struct sock_filter filter[1];
if (!PyArg_ParseTuple(args, "HI", &code, &k))
return (NULL);
filter[0].k = k;
filter[0].jt = 0;
filter[0].jf = 0;
filter[0].code = code;
len = sizeof(struct sock_filter);
off = seccomp->elm * sizeof(struct sock_filter);
seccomp->filters = kore_realloc(seccomp->filters, off + len);
memcpy(seccomp->filters + off, filter, len);
seccomp->elm += 1;
Py_RETURN_NONE;
}
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
static PyObject *
pyseccomp_allow(struct pyseccomp *seccomp, PyObject *args)
{
const char *syscall;
if (!PyArg_ParseTuple(args, "s", &syscall))
return (NULL);
if (!pyseccomp_filter_install(seccomp, syscall,
PYSECCOMP_SYSCALL_FILTER, 0, 0, SECCOMP_RET_ALLOW))
return (NULL);
Py_RETURN_NONE;
}
static PyObject *
pyseccomp_allow_arg(struct pyseccomp *seccomp, PyObject *args)
{
return (pyseccomp_common_action(seccomp, args, NULL,
PYSECCOMP_SYSCALL_ARG, PYSECCOMP_ACTION_ALLOW));
}
static PyObject *
pyseccomp_allow_flag(struct pyseccomp *seccomp, PyObject *args)
{
return (pyseccomp_common_action(seccomp, args, NULL,
PYSECCOMP_SYSCALL_FLAG, PYSECCOMP_ACTION_ALLOW));
}
static PyObject *
pyseccomp_allow_mask(struct pyseccomp *seccomp, PyObject *args)
{
return (pyseccomp_common_action(seccomp, args, NULL,
PYSECCOMP_SYSCALL_MASK, PYSECCOMP_ACTION_ALLOW));
}
static PyObject *
pyseccomp_deny(struct pyseccomp *seccomp, PyObject *args, PyObject *kwargs)
{
long err;
const char *syscall;
if (!PyArg_ParseTuple(args, "s", &syscall))
return (NULL);
err = EACCES;
if (kwargs != NULL)
python_long_from_dict(kwargs, "errno", &err);
if (!pyseccomp_filter_install(seccomp, syscall,
PYSECCOMP_SYSCALL_FILTER, 0, 0, SECCOMP_RET_ERRNO | (int)err))
return (NULL);
Py_RETURN_NONE;
}
static PyObject *
pyseccomp_deny_arg(struct pyseccomp *seccomp, PyObject *args, PyObject *kwargs)
{
return (pyseccomp_common_action(seccomp, args, kwargs,
PYSECCOMP_SYSCALL_ARG, PYSECCOMP_ACTION_DENY));
}
static PyObject *
pyseccomp_deny_flag(struct pyseccomp *seccomp, PyObject *args, PyObject *kwargs)
{
return (pyseccomp_common_action(seccomp, args, kwargs,
PYSECCOMP_SYSCALL_FLAG, PYSECCOMP_ACTION_DENY));
}
static PyObject *
pyseccomp_deny_mask(struct pyseccomp *seccomp, PyObject *args, PyObject *kwargs)
{
return (pyseccomp_common_action(seccomp, args, kwargs,
PYSECCOMP_SYSCALL_MASK, PYSECCOMP_ACTION_DENY));
}
static PyObject *
pyseccomp_common_action(struct pyseccomp *sc, PyObject *args,
PyObject *kwargs, int which, int action)
{
long err;
const char *syscall;
int arg, val;
if (!PyArg_ParseTuple(args, "sii", &syscall, &arg, &val))
return (NULL);
switch (action) {
case PYSECCOMP_ACTION_ALLOW:
action = SECCOMP_RET_ALLOW;
break;
case PYSECCOMP_ACTION_DENY:
err = EACCES;
if (kwargs != NULL)
python_long_from_dict(kwargs, "errno", &err);
action = SECCOMP_RET_ERRNO | (int)err;
break;
default:
fatal("%s: bad action %d", __func__, action);
}
if (!pyseccomp_filter_install(sc, syscall, which, arg, val, action))
return (NULL);
Py_RETURN_NONE;
}
static int
pyseccomp_filter_install(struct pyseccomp *seccomp, const char *syscall,
int which, int arg, int val, int action)
{
struct sock_filter *filter;
size_t elm, len, off;
switch (which) {
case PYSECCOMP_SYSCALL_FILTER:
filter = kore_seccomp_syscall_filter(syscall, action);
break;
case PYSECCOMP_SYSCALL_ARG:
filter = kore_seccomp_syscall_arg(syscall, action, arg, val);
break;
case PYSECCOMP_SYSCALL_MASK:
filter = kore_seccomp_syscall_mask(syscall, action, arg, val);
break;
case PYSECCOMP_SYSCALL_FLAG:
filter = kore_seccomp_syscall_flag(syscall, action, arg, val);
break;
default:
fatal("%s: invalid syscall instruction %d", __func__, which);
}
if (filter == NULL) {
PyErr_Format(PyExc_RuntimeError,
"system call '%s' does not exist", syscall);
return (KORE_RESULT_ERROR);
}
elm = 0;
/*
* Find the number of elements in the BPF program, by looking for
* the KORE_BPF_GUARD element.
*/
for (;;) {
if (filter[elm].code == USHRT_MAX &&
filter[elm].jt == UCHAR_MAX &&
filter[elm].jf == UCHAR_MAX &&
filter[elm].k == UINT_MAX)
break;
elm++;
}
len = elm * sizeof(struct sock_filter);
off = seccomp->elm * sizeof(struct sock_filter);
seccomp->filters = kore_realloc(seccomp->filters, off + len);
memcpy(seccomp->filters + off, filter, len);
seccomp->elm += elm;
kore_free(filter);
return (KORE_RESULT_OK);
}
#endif
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
static int
python_long_from_dict(PyObject *dict, const char *key, long *result)
{
PyObject *obj;
if ((obj = PyDict_GetItemString(dict, key)) == NULL)
return (KORE_RESULT_ERROR);
if (!PyLong_CheckExact(obj))
return (KORE_RESULT_ERROR);
PyErr_Clear();
*result = PyLong_AsLong(obj);
if (*result == -1 && PyErr_Occurred()) {
PyErr_Clear();
return (KORE_RESULT_ERROR);
}
return (KORE_RESULT_OK);
}
static int
python_bool_from_dict(PyObject *dict, const char *key, int *result)
{
PyObject *obj;
if ((obj = PyDict_GetItemString(dict, key)) == NULL)
return (KORE_RESULT_ERROR);
if (!PyBool_Check(obj))
return (KORE_RESULT_ERROR);
*result = (obj == Py_True);
return (KORE_RESULT_OK);
}
static const char *
python_string_from_dict(PyObject *dict, const char *key)
{
PyObject *obj;
if ((obj = PyDict_GetItemString(dict, key)) == NULL)
return (NULL);
if (!PyUnicode_Check(obj))
return (NULL);
return (PyUnicode_AsUTF8AndSize(obj, NULL));
}
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
static PyObject *
python_cmsg_to_list(struct msghdr *msg)
{
struct cmsghdr *c;
Improvements for kore.recvmsg() in Python. The cmsghdr contains a length (cmsg_len) which indicates the length of the data in combination with the cmsghdr length itself. Remove the length of the cmsghdr before passing it back up to callers so they don't need to bother with it. This also fixes a mistake where we ended up copying extra data from the ancdata buffer that was unintended.
2022-04-22 17:01:06 +02:00
size_t len;
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
Py_ssize_t idx;
PyObject *list, *tuple;
if ((list = PyList_New(0)) == NULL)
return (NULL);
idx = 0;
for (c = CMSG_FIRSTHDR(msg); c != NULL; c = CMSG_NXTHDR(msg, c)) {
Improvements for kore.recvmsg() in Python. The cmsghdr contains a length (cmsg_len) which indicates the length of the data in combination with the cmsghdr length itself. Remove the length of the cmsghdr before passing it back up to callers so they don't need to bother with it. This also fixes a mistake where we ended up copying extra data from the ancdata buffer that was unintended.
2022-04-22 17:01:06 +02:00
len = c->cmsg_len - sizeof(*c);
tuple = Py_BuildValue("(Iiiy#)", len,
c->cmsg_level, c->cmsg_type, CMSG_DATA(c), len);
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
if (tuple == NULL) {
Py_DECREF(list);
return (NULL);
}
/* Steals a reference to tuple. */
if (PyList_Insert(list, idx++, tuple) == -1) {
Py_DECREF(tuple);
Py_DECREF(list);
return (NULL);
}
}
return (list);
}
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
static void *
python_malloc(void *ctx, size_t len)
{
return (kore_malloc(len));
}
static void *
python_calloc(void *ctx, size_t memb, size_t len)
{
return (kore_calloc(memb, len));
}
static void *
python_realloc(void *ctx, void *ptr, size_t len)
{
return (kore_realloc(ptr, len));
}
static void
python_free(void *ctx, void *ptr)
{
kore_free(ptr);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void
python_module_free(struct kore_module *module)
{
kore_free(module->path);
Py_DECREF(module->handle);
kore_free(module);
}
allow double quotes in kore.proc argument lists.
2019-02-22 16:56:34 +01:00
static void
python_split_arguments(char *args, char **argv, size_t elm)
{
size_t idx;
char *p, *line, *end;
if (elm <= 1)
fatal("not enough elements (%zu)", elm);
idx = 0;
line = args;
for (p = line; *p != '\0'; p++) {
if (idx >= elm - 1)
break;
if (*p == ' ') {
*p = '\0';
if (*line != '\0')
argv[idx++] = line;
line = p + 1;
continue;
}
if (*p != '"')
continue;
line = p + 1;
if ((end = strchr(line, '"')) == NULL)
break;
*end = '\0';
argv[idx++] = line;
line = end + 1;
while (isspace(*(unsigned char *)line))
line++;
p = line;
}
if (idx < elm - 1 && *line != '\0')
argv[idx++] = line;
argv[idx] = NULL;
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void
python_module_reload(struct kore_module *module)
{
allow reloading of python modules on SIGHUP.
2017-01-26 11:13:09 +01:00
PyObject *handle;
call PyErr_Clear() before PyImport_ReloadModule().
2017-02-22 20:54:57 +01:00
PyErr_Clear();
allow reloading of python modules on SIGHUP.
2017-01-26 11:13:09 +01:00
if ((handle = PyImport_ReloadModule(module->handle)) == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_module_reload");
allow reloading of python modules on SIGHUP.
2017-01-26 11:13:09 +01:00
return;
}
Py_DECREF(module->handle);
module->handle = handle;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
static void
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
python_module_load(struct kore_module *module)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
module->handle = python_import(module->path);
if (module->handle == NULL)
fatal("%s: failed to import module", module->path);
}
static void *
python_module_getsym(struct kore_module *module, const char *symbol)
{
return (python_callable(module->handle, symbol));
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static struct python_coro *
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
python_coro_create(PyObject *obj, struct http_request *req)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
struct python_coro *coro;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (!PyCoro_CheckExact(obj))
fatal("%s: object is not a coroutine", __func__);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
coro = kore_pool_get(&coro_pool);
coro_count++;
allow coroutines to set friendly names.
2019-09-16 20:31:25 +02:00
coro->name = NULL;
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
coro->result = NULL;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
coro->sockop = NULL;
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
coro->lockop = NULL;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
coro->gatherop = NULL;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
coro->exception = NULL;
coro->exception_msg = NULL;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
coro->obj = obj;
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
coro->killed = 0;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
coro->request = req;
coro->id = coro_id++;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
coro->state = CORO_STATE_RUNNABLE;
move new/wokenup/suspended coros to end of lists.
2019-09-16 19:34:47 +02:00
TAILQ_INSERT_TAIL(&coro_runnable, coro, list);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (coro->request != NULL)
http_request_sleep(coro->request);
trace created/deleted.
2019-09-16 20:21:04 +02:00
python_coro_trace("created", coro);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
return (coro);
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
static int
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
python_coro_run(struct python_coro *coro)
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
{
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
PySendResult res;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
PyObject *item;
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
PyObject *type, *traceback;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (coro->state != CORO_STATE_RUNNABLE)
fatal("non-runnable coro attempted to run");
coro_running = coro;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
for (;;) {
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
python_coro_trace("running", coro);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
PyErr_Clear();
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
#if PY_VERSION_HEX < 0x030A0000
res = PYGEN_RETURN;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
item = _PyGen_Send((PyGenObject *)coro->obj, NULL);
Get Kore to build with Python 3.10.0a3.
2021-01-28 10:17:02 +01:00
#else
Improve Python 3.10.x support. Python 3.10.x its PyIter_Send() will return PYGEN_ERROR if the generator returned (as opposed to yielding) if the result returned is Py_None and no argument was given to PyIter_Send(). Which is counter intuitive since it seems it should give us a PYGEN_RETURN as per its documentation. It also no longer sets PyExc_StopIteration in this case so we cannot depend on that anymore to figure out if a coroutine has finished. This caused Kore to respond with a 500 immediately after coroutines ran to completion as it was looking for the PyExc_StopIteration exception. Fix this by simply doing another check if an exception happened before we enter the code path where Kore would spew the 500.
2022-07-14 10:56:27 +02:00
/*
* Python 3.10.x its PyIter_Send() will return a PYGEN_ERROR
* if the coro returned (instead of yielding) and the result
* ends up being Py_None. This means the returned item is
* NULL but no StopIteration exception has occurred.
*/
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
res = PyIter_Send(coro->obj, NULL, &item);
Get Kore to build with Python 3.10.0a3.
2021-01-28 10:17:02 +01:00
#endif
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
if (item == NULL || res == PYGEN_ERROR) {
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
Py_XDECREF(item);
do not grab the result if it was in a gatherop
2019-03-19 13:58:00 +01:00
if (coro->gatherop == NULL && PyErr_Occurred() &&
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
PyErr_ExceptionMatches(PyExc_StopIteration)) {
PyErr_Fetch(&type, &coro->result, &traceback);
Py_DECREF(type);
Py_XDECREF(traceback);
Improve Python 3.10.x support. Python 3.10.x its PyIter_Send() will return PYGEN_ERROR if the generator returned (as opposed to yielding) if the result returned is Py_None and no argument was given to PyIter_Send(). Which is counter intuitive since it seems it should give us a PYGEN_RETURN as per its documentation. It also no longer sets PyExc_StopIteration in this case so we cannot depend on that anymore to figure out if a coroutine has finished. This caused Kore to respond with a 500 immediately after coroutines ran to completion as it was looking for the PyExc_StopIteration exception. Fix this by simply doing another check if an exception happened before we enter the code path where Kore would spew the 500.
2022-07-14 10:56:27 +02:00
} else if (PyErr_Occurred()) {
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
kore_python_log_error("coroutine");
Python: respond with 500 in case of a coroutine error. If a coroutine throws an exception, respond with a 500 after logging the exception itself.
2020-06-30 09:57:48 +02:00
if (coro->request != NULL) {
http_response(coro->request,
HTTP_STATUS_INTERNAL_ERROR,
NULL, 0);
}
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
coro_running = NULL;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (KORE_RESULT_OK);
}
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
#if PY_VERSION_HEX >= 0x030A0000
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
if (res == PYGEN_RETURN) {
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
coro->result = item;
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
coro_running = NULL;
return (KORE_RESULT_OK);
}
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
#endif
More Python 3.10.x changes. Handle PYGEN_RETURN in case our prerequest handlers return actual values instead of None. Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if (item == Py_None) {
Py_DECREF(item);
break;
}
Py_DECREF(item);
}
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
python_coro_suspend(coro);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
coro_running = NULL;
if (coro->request != NULL)
http_request_sleep(coro->request);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (KORE_RESULT_RETRY);
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static void
python_coro_wakeup(struct python_coro *coro)
{
if (coro->state != CORO_STATE_SUSPENDED)
return;
coro->state = CORO_STATE_RUNNABLE;
TAILQ_REMOVE(&coro_suspended, coro, list);
move new/wokenup/suspended coros to end of lists.
2019-09-16 19:34:47 +02:00
TAILQ_INSERT_TAIL(&coro_runnable, coro, list);
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
python_coro_trace("wokeup", coro);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
static void
python_coro_suspend(struct python_coro *coro)
{
if (coro->state != CORO_STATE_RUNNABLE)
return;
coro->state = CORO_STATE_SUSPENDED;
TAILQ_REMOVE(&coro_runnable, coro, list);
move new/wokenup/suspended coros to end of lists.
2019-09-16 19:34:47 +02:00
TAILQ_INSERT_TAIL(&coro_suspended, coro, list);
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
python_coro_trace("suspended", coro);
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
}
Hack around some hidden Python symbols. The _PyInterpreterFrame_GetLine() is hidden in dynamic libs so roll our own variant of it. Shuffle the old code so we always end up calling python_resolve_frame_line() no matter the Python version.
2022-09-26 08:48:29 +02:00
static int
python_resolve_frame_line(void *ptr)
{
int line;
#if PY_VERSION_HEX >= 0x030b0000
int addr;
_PyInterpreterFrame *frame;
frame = ptr;
addr = _PyInterpreterFrame_LASTI(frame) * sizeof(_Py_CODEUNIT);
line = PyCode_Addr2Line(frame->f_code, addr);
#else
line = PyFrame_GetLineNumber(ptr);
#endif
return (line);
}
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
static void
python_coro_trace(const char *label, struct python_coro *coro)
{
int line;
Python improvements: Rework corotracing for 3.11. In the upcoming Python 3.11 release the PyCoroObject no longer has a full PyFrameObject, but instead their internal frame struct _PyInterpreterFrame. Use that when we are building against 3.11 or higher so we can still provide useful tracing functionality (and so that it builds).
2022-09-25 00:29:08 +02:00
PyCoroObject *obj;
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
PyCodeObject *code;
Python improvements: Rework corotracing for 3.11. In the upcoming Python 3.11 release the PyCoroObject no longer has a full PyFrameObject, but instead their internal frame struct _PyInterpreterFrame. Use that when we are building against 3.11 or higher so we can still provide useful tracing functionality (and so that it builds).
2022-09-25 00:29:08 +02:00
#if PY_VERSION_HEX >= 0x030b0000
_PyInterpreterFrame *frame;
#else
PyFrameObject *frame;
#endif
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
const char *func, *fname, *file;
make coro tracing available at runtime, always. call kore.corotrace(True) to enable it and kore.corotrace(False) to disable.
2019-09-18 10:55:13 +02:00
if (coro_tracing == 0)
return;
Python improvements: Rework corotracing for 3.11. In the upcoming Python 3.11 release the PyCoroObject no longer has a full PyFrameObject, but instead their internal frame struct _PyInterpreterFrame. Use that when we are building against 3.11 or higher so we can still provide useful tracing functionality (and so that it builds).
2022-09-25 00:29:08 +02:00
obj = (PyCoroObject *)coro->obj;
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
Python improvements: Rework corotracing for 3.11. In the upcoming Python 3.11 release the PyCoroObject no longer has a full PyFrameObject, but instead their internal frame struct _PyInterpreterFrame. Use that when we are building against 3.11 or higher so we can still provide useful tracing functionality (and so that it builds).
2022-09-25 00:29:08 +02:00
#if PY_VERSION_HEX >= 0x030b0000
frame = (_PyInterpreterFrame *)obj->cr_iframe;
#else
frame = obj->cr_frame;
#endif
if (frame != NULL && frame->f_code != NULL) {
code = frame->f_code;
trace created/deleted.
2019-09-16 20:21:04 +02:00
func = PyUnicode_AsUTF8AndSize(code->co_name, NULL);
file = PyUnicode_AsUTF8AndSize(code->co_filename, NULL);
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
trace created/deleted.
2019-09-16 20:21:04 +02:00
if ((fname = strrchr(file, '/')) == NULL)
fname = file;
else
fname++;
} else {
func = "unknown";
fname = "unknown";
}
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
Hack around some hidden Python symbols. The _PyInterpreterFrame_GetLine() is hidden in dynamic libs so roll our own variant of it. Shuffle the old code so we always end up calling python_resolve_frame_line() no matter the Python version.
2022-09-26 08:48:29 +02:00
if (frame != NULL)
line = python_resolve_frame_line(frame);
else
trace created/deleted.
2019-09-16 20:21:04 +02:00
line = -1;
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
allow coroutines to set friendly names.
2019-09-16 20:31:25 +02:00
if (coro->name) {
kore_log(LOG_NOTICE, "coro '%s' %s <%s> @ [%s:%d]",
coro->name, label, func, fname, line);
} else {
Python coro under-the-hood improvements. - Change python coroutine id to a uint64_t. - Add kore.task_id() to return active coro its id.
2021-12-02 21:58:13 +01:00
kore_log(LOG_NOTICE, "coro %" PRIu64 " %s <%s> @ [%s:%d]",
allow coroutines to set friendly names.
2019-09-16 20:31:25 +02:00
coro->id, label, func, fname, line);
}
Add coroutine debugging. If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine traces while running. These traces include the filename, function and line number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static void
pyconnection_dealloc(struct pyconnection *pyc)
{
PyObject_Del((PyObject *)pyc);
}
static void
pyhttp_dealloc(struct pyhttp_request *pyreq)
{
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
Py_XDECREF(pyreq->dict);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Py_XDECREF(pyreq->data);
PyObject_Del((PyObject *)pyreq);
}
static void
pyhttp_file_dealloc(struct pyhttp_file *pyfile)
{
PyObject_Del((PyObject *)pyfile);
}
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
static int
python_runtime_resolve(const char *module, const struct stat *st)
{
const char *ext;
if (!S_ISDIR(st->st_mode) && !S_ISREG(st->st_mode))
return (KORE_RESULT_ERROR);
if (S_ISDIR(st->st_mode)) {
kore_module_load(module, NULL, KORE_MODULE_PYTHON);
if (chdir(module) == -1)
fatal("chdir(%s): %s", module, errno_s);
} else {
if ((ext = strrchr(module, '.')) == NULL)
return (KORE_RESULT_ERROR);
if (strcasecmp(ext, ".py"))
return (KORE_RESULT_ERROR);
kore_module_load(module, NULL, KORE_MODULE_PYTHON);
}
fix single binary builds using PYTHON flavor.
2023-04-02 18:54:36 +02:00
#if !defined(KORE_SINGLE_BINARY)
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
kore_pymodule = module;
fix single binary builds using PYTHON flavor.
2023-04-02 18:54:36 +02:00
#endif
Rework runtime init a little bit. It was hardcoded that if KORE_USE_PYTHON was defined we would look at the passed argument on the command-line as the python script or module to be run. This won't work when adding more runtimes. So instead call a kore_runtime_resolve() function that in turn calls each available runtime its resolve function. That resolve function will check if its a script / module that it can load, and if so will load it. This way we can remove all those KORE_USE_PYTHON blocks in the Kore startup path and we pave the way for lua.
2023-01-16 21:00:01 +01:00
kore_hooks_set(KORE_PYTHON_CONFIG_HOOK,
KORE_PYTHON_TEARDOWN_HOOK, KORE_PYTHON_DAEMONIZED_HOOK);
return (KORE_RESULT_OK);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static int
python_runtime_http_request(void *addr, struct http_request *req)
{
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
int ret, idx, cnt;
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
PyObject *pyret, *args, *callable;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
PyObject *cargs[HTTP_CAPTURE_GROUPS + 1];
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (req->py_coro != NULL) {
python_coro_wakeup(req->py_coro);
if (python_coro_run(req->py_coro) == KORE_RESULT_OK) {
kore_python_coro_delete(req->py_coro);
req->py_coro = NULL;
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
if (req->fsm_state != PYHTTP_STATE_PREPROCESS)
return (KORE_RESULT_OK);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
return (KORE_RESULT_RETRY);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
switch (req->fsm_state) {
case PYHTTP_STATE_INIT:
make sure prereqs are in order
2019-06-07 21:33:00 +02:00
req->py_rqnext = TAILQ_FIRST(&prereq);
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
req->fsm_state = PYHTTP_STATE_PREPROCESS;
if (req->py_req == NULL) {
if ((req->py_req = pyhttp_request_alloc(req)) == NULL)
fatal("%s: pyreq alloc failed", __func__);
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
}
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
/* fallthrough */
case PYHTTP_STATE_PREPROCESS:
ret = pyhttp_preprocess(req);
switch (ret) {
case KORE_RESULT_OK:
req->fsm_state = PYHTTP_STATE_RUN;
break;
case KORE_RESULT_RETRY:
return (KORE_RESULT_RETRY);
case KORE_RESULT_ERROR:
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
return (KORE_RESULT_OK);
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
default:
fatal("invalid state pyhttp state %d", req->fsm_state);
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
}
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
/* fallthrough */
case PYHTTP_STATE_RUN:
break;
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
cnt = 0;
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
callable = (PyObject *)addr;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
/* starts at 1 to skip the full path. */
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
if (req->rt->type == HANDLER_TYPE_DYNAMIC) {
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
for (idx = 1; idx < HTTP_CAPTURE_GROUPS - 1; idx++) {
if (req->cgroups[idx].rm_so == -1 ||
req->cgroups[idx].rm_eo == -1)
break;
cargs[cnt] = PyUnicode_FromStringAndSize(req->path +
req->cgroups[idx].rm_so,
req->cgroups[idx].rm_eo - req->cgroups[idx].rm_so);
if (cargs[cnt] == NULL) {
Fix unhappy path cleanup. The whole while (cnt-- >= 0) idiom is busted since cnt started at 0 and if the first call to PyUnicode_FromStringAndSize() fails then we're attempting to access -1.
2022-02-20 21:25:18 +01:00
while (cnt >= 0)
Py_XDECREF(cargs[cnt--]);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
kore_python_log_error("http request");
http_response(req,
HTTP_STATUS_INTERNAL_ERROR, NULL, 0);
return (KORE_RESULT_OK);
}
cnt++;
}
}
cargs[cnt] = NULL;
if ((args = PyTuple_New(cnt + 1)) == NULL)
fatal("%s: PyTuple_New failed", __func__);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
Py_INCREF(req->py_req);
if (PyTuple_SetItem(args, 0, req->py_req) != 0)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
fatal("python_runtime_http_request: PyTuple_SetItem failed");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
for (idx = 0; cargs[idx] != NULL; idx++) {
if (PyTuple_SetItem(args, 1 + idx, cargs[idx]) != 0)
fatal("%s: PyTuple_SetItem failed (%d)", __func__, idx);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
PyErr_Clear();
pyret = PyObject_Call(callable, args, NULL);
Py_DECREF(args);
if (pyret == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_runtime_http_request");
If a python handler failed, don't fatal on purpose. Instead log and send an internal error status back to the client. This should be OK as long as the exception doesn't happen after the caller called req.response() already.
2018-04-11 13:00:30 +02:00
http_response(req, HTTP_STATUS_INTERNAL_ERROR, NULL, 0);
return (KORE_RESULT_OK);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if (PyCoro_CheckExact(pyret)) {
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
req->py_coro = python_coro_create(pyret, req);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (python_coro_run(req->py_coro) == KORE_RESULT_OK) {
Wake up HTTP requests if python coros finish immediately. python_coro_create() puts the HTTP request to sleep, but if they finish immediately they will be removed from the list and should properly be woken up or they are removed from the wrong list.
2019-05-30 17:15:08 +02:00
http_request_wakeup(req);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
kore_python_coro_delete(req->py_coro);
req->py_coro = NULL;
return (KORE_RESULT_OK);
}
return (KORE_RESULT_RETRY);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if (pyret != Py_None)
fatal("python_runtime_http_request: unexpected return type");
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Py_DECREF(pyret);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (KORE_RESULT_OK);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
static void
python_runtime_http_request_free(void *addr, struct http_request *req)
{
PyObject *ret;
Python: allocate py_req in the http_request_free hook if needed. Makes it possible to mix on_free with other runtimes.
2021-12-15 12:19:04 +01:00
if (req->py_req == NULL) {
if ((req->py_req = pyhttp_request_alloc(req)) == NULL)
fatal("%s: pyreq alloc failed", __func__);
}
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
PyErr_Clear();
ret = PyObject_CallFunctionObjArgs(addr, req->py_req, NULL);
if (ret == NULL)
kore_python_log_error("python_runtime_http_request_free");
Py_XDECREF(ret);
}
static void
python_runtime_http_body_chunk(void *addr, struct http_request *req,
const void *data, size_t len)
{
PyObject *args, *ret;
if (req->py_req == NULL) {
if ((req->py_req = pyhttp_request_alloc(req)) == NULL)
fatal("%s: pyreq alloc failed", __func__);
}
if ((args = Py_BuildValue("(Oy#)", req->py_req, data, len)) == NULL) {
kore_python_log_error("python_runtime_http_body_chunk");
return;
}
PyErr_Clear();
ret = PyObject_Call(addr, args, NULL);
if (ret == NULL)
kore_python_log_error("python_runtime_http_body_chunk");
Py_XDECREF(ret);
Py_DECREF(args);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static int
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
python_runtime_validator(void *addr, struct http_request *req, const void *data)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
int ret;
struct python_coro *coro;
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
PyObject *pyret, *args, *callable, *arg;
if (req->py_req == NULL) {
if ((req->py_req = pyhttp_request_alloc(req)) == NULL)
fatal("%s: pyreq alloc failed", __func__);
}
validators now work in python.
2017-01-25 21:17:12 +01:00
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if (req->py_validator != NULL) {
coro = req->py_validator;
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
python_coro_wakeup(coro);
if (python_coro_run(coro) == KORE_RESULT_OK) {
ret = python_validator_check(coro->result);
kore_python_coro_delete(coro);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
req->py_validator = NULL;
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
return (ret);
}
validators now work in python.
2017-01-25 21:17:12 +01:00
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
return (KORE_RESULT_RETRY);
}
callable = (PyObject *)addr;
validators now work in python.
2017-01-25 21:17:12 +01:00
if (req->flags & HTTP_VALIDATOR_IS_REQUEST) {
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
if ((args = PyTuple_New(1)) == NULL)
fatal("%s: PyTuple_New failed", __func__);
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
Py_INCREF(req->py_req);
if (PyTuple_SetItem(args, 0, req->py_req) != 0)
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
fatal("%s: PyTuple_SetItem failed", __func__);
validators now work in python.
2017-01-25 21:17:12 +01:00
} else {
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
if ((arg = PyUnicode_FromString(data)) == NULL)
fatal("python_runtime_validator: PyUnicode failed");
validators now work in python.
2017-01-25 21:17:12 +01:00
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
if ((args = PyTuple_New(2)) == NULL)
fatal("%s: PyTuple_New failed", __func__);
validators now work in python.
2017-01-25 21:17:12 +01:00
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
Py_INCREF(req->py_req);
if (PyTuple_SetItem(args, 0, req->py_req) != 0 ||
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
PyTuple_SetItem(args, 1, arg) != 0)
fatal("%s: PyTuple_SetItem failed", __func__);
}
validators now work in python.
2017-01-25 21:17:12 +01:00
PyErr_Clear();
pyret = PyObject_Call(callable, args, NULL);
Py_DECREF(args);
if (pyret == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_runtime_validator");
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
fatal("failed to execute python call");
validators now work in python.
2017-01-25 21:17:12 +01:00
}
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
if (PyCoro_CheckExact(pyret)) {
coro = python_coro_create(pyret, req);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
req->py_validator = coro;
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
if (python_coro_run(coro) == KORE_RESULT_OK) {
Wake up HTTP requests if python coros finish immediately. python_coro_create() puts the HTTP request to sleep, but if they finish immediately they will be removed from the list and should properly be woken up or they are removed from the wrong list.
2019-05-30 17:15:08 +02:00
http_request_wakeup(req);
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
ret = python_validator_check(coro->result);
kore_python_coro_delete(coro);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
req->py_validator = NULL;
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
return (ret);
}
return (KORE_RESULT_RETRY);
}
validators now work in python.
2017-01-25 21:17:12 +01:00
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
ret = python_validator_check(pyret);
validators now work in python.
2017-01-25 21:17:12 +01:00
Py_DECREF(pyret);
return (ret);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
static int
python_validator_check(PyObject *obj)
{
int ret;
if (obj == NULL)
return (KORE_RESULT_ERROR);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if (!PyBool_Check(obj)) {
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
kore_log(LOG_WARNING,
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
"validator did not return True/False");
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
ret = KORE_RESULT_ERROR;
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if (obj == Py_True)
ret = KORE_RESULT_OK;
else
Allow python validator methods to be async.
2019-03-18 09:34:31 +01:00
ret = KORE_RESULT_ERROR;
return (ret);
}
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
static void
python_runtime_wsmessage(void *addr, struct connection *c, u_int8_t op,
const void *data, size_t len)
{
PyObject *callable, *args, *pyret, *pyc, *pyop, *pydata;
callable = (PyObject *)addr;
if ((pyc = pyconnection_alloc(c)) == NULL)
fatal("python_runtime_wsmessage: pyc alloc failed");
if ((pyop = PyLong_FromLong((long)op)) == NULL)
fatal("python_runtime_wsmessage: PyLong_FromLong failed");
Improve python websocket handling a bit. Convert the data parameter to a string if the op was WEBSOCKET_OP_TEXT or convert to bytes if op was WEBSOCKET_OP_BINARY so the callee does not have to do this anymore.
2017-03-13 12:29:42 +01:00
switch (op) {
case WEBSOCKET_OP_TEXT:
if ((pydata = PyUnicode_FromStringAndSize(data, len)) == NULL)
fatal("wsmessage: PyUnicode_AsUTF8AndSize failed");
break;
case WEBSOCKET_OP_BINARY:
if ((pydata = PyBytes_FromStringAndSize(data, len)) == NULL)
fatal("wsmessage: PyBytes_FromString failed");
break;
default:
fatal("python_runtime_wsmessage: invalid op");
}
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
if ((args = PyTuple_New(3)) == NULL)
fatal("python_runtime_wsmessage: PyTuple_New failed");
if (PyTuple_SetItem(args, 0, pyc) != 0 ||
PyTuple_SetItem(args, 1, pyop) != 0 ||
PyTuple_SetItem(args, 2, pydata) != 0)
fatal("python_runtime_wsmessage: PyTuple_SetItem failed");
PyErr_Clear();
pyret = PyObject_Call(callable, args, NULL);
Py_DECREF(args);
if (pyret == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_runtime_wsconnect");
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
fatal("failed to execute python call");
}
Py_DECREF(pyret);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
static void
python_runtime_execute(void *addr)
{
PyObject *callable, *args, *pyret;
callable = (PyObject *)addr;
if ((args = PyTuple_New(0)) == NULL)
fatal("python_runtime_execute: PyTuple_New failed");
PyErr_Clear();
pyret = PyObject_Call(callable, args, NULL);
Py_DECREF(args);
if (pyret == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_runtime_execute");
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
fatal("failed to execute python call");
}
Py_DECREF(pyret);
}
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
static void
python_runtime_configure(void *addr, int argc, char **argv)
{
int i;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyObject *callable, *args, *pyret, *pyarg, *list;
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
callable = (PyObject *)addr;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if ((args = PyTuple_New(1)) == NULL)
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
fatal("python_runtime_configure: PyTuple_New failed");
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if ((list = PyList_New(argc + 1)) == NULL)
fatal("python_runtime_configure: PyList_New failed");
if ((pyarg = PyUnicode_FromString(__progname)) == NULL)
fatal("python_runtime_configure: PyUnicode_FromString");
if (PyList_SetItem(list, 0, pyarg) == -1)
fatal("python_runtime_configure: PyList_SetItem");
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
for (i = 0; i < argc; i++) {
if ((pyarg = PyUnicode_FromString(argv[i])) == NULL)
fatal("python_runtime_configure: PyUnicode_FromString");
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (PyList_SetItem(list, i + 1, pyarg) == -1)
fatal("python_runtime_configure: PyList_SetItem");
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (PyTuple_SetItem(args, 0, list) != 0)
fatal("python_runtime_configure: PyTuple_SetItem");
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
PyErr_Clear();
pyret = PyObject_Call(callable, args, NULL);
Py_DECREF(args);
if (pyret == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_runtime_configure");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
fatal("failed to configure your application");
Change kore_parent_configure() for single binaries. This function now takes any remaining arguments passed on the command line after kore parsed its own. For C the new prototype looks like this: void kore_parent_configure(int argc, char **argv); For python code, kore will pass each argument to the function so you can do things like: def kore_parent_configure(arg1, arg2):
2018-04-09 12:51:20 +02:00
}
Py_DECREF(pyret);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static int
python_runtime_onload(void *addr, int action)
{
int ret;
PyObject *pyret, *args, *pyact, *callable;
callable = (PyObject *)addr;
if ((pyact = PyLong_FromLong(action)) == NULL)
fatal("python_runtime_onload: PyLong_FromLong failed");
if ((args = PyTuple_New(1)) == NULL)
fatal("python_runtime_onload: PyTuple_New failed");
if (PyTuple_SetItem(args, 0, pyact) != 0)
fatal("python_runtime_onload: PyTuple_SetItem failed");
validators now work in python.
2017-01-25 21:17:12 +01:00
PyErr_Clear();
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
pyret = PyObject_Call(callable, args, NULL);
Py_DECREF(args);
if (pyret == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_runtime_onload");
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return (KORE_RESULT_ERROR);
}
if (!PyLong_Check(pyret))
fatal("python_runtime_onload: unexpected return type");
ret = (int)PyLong_AsLong(pyret);
Py_DECREF(pyret);
return (ret);
}
static void
python_runtime_connect(void *addr, struct connection *c)
{
PyObject *pyc, *pyret, *args, *callable;
callable = (PyObject *)addr;
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
if ((pyc = pyconnection_alloc(c)) == NULL)
fatal("python_runtime_connect: pyc alloc failed");
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
if ((args = PyTuple_New(1)) == NULL)
fatal("python_runtime_connect: PyTuple_New failed");
if (PyTuple_SetItem(args, 0, pyc) != 0)
fatal("python_runtime_connect: PyTuple_SetItem failed");
validators now work in python.
2017-01-25 21:17:12 +01:00
PyErr_Clear();
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
pyret = PyObject_Call(callable, args, NULL);
Py_DECREF(args);
if (pyret == NULL) {
Make kore_python_log_error() public. While here also make kore_module_load() return the kore_module data structure pointer it just added to the modules list.
2018-10-16 13:16:36 +02:00
kore_python_log_error("python_runtime_connect");
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
kore_connection_disconnect(c);
}
Py_DECREF(pyret);
}
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
static void
python_runtime_signal(void *addr, int sig)
{
PyObject *obj, *ret;
if ((obj = Py_BuildValue("i", sig)) == NULL) {
kore_python_log_error("python_runtime_signal");
return;
}
ret = PyObject_CallFunctionObjArgs(addr, obj, NULL);
Py_DECREF(obj);
Py_XDECREF(ret);
}
make python_module_init() non static
2021-07-10 10:02:46 +02:00
PyMODINIT_FUNC
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
python_module_init(void)
{
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
int i;
struct pyconfig *config;
PyObject *pykore;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
if ((pykore = PyModule_Create(&pykore_module)) == NULL)
fatal("python_module_init: failed to setup pykore module");
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
python_push_type("pyproc", pykore, &pyproc_type);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
python_push_type("pylock", pykore, &pylock_type);
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
python_push_type("pytimer", pykore, &pytimer_type);
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
python_push_type("pyqueue", pykore, &pyqueue_type);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
python_push_type("pyroute", pykore, &pyroute_type);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
python_push_type("pysocket", pykore, &pysocket_type);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
python_push_type("pydomain", pykore, &pydomain_type);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
python_push_type("pyconnection", pykore, &pyconnection_type);
Allow configuring seccomp on Linux via the python api. A new hook in the koreapp class is called right before seccomp is enabled. This hook receives a Kore seccomp object which has the following methods: seccomp.allow("syscall") seccomp.allow_arg("syscall", arg, value) seccomp.allow_flag("syscall", arg, flag) seccomp.allow_mask("syscall", arg, mask) seccomp.deny("syscall") seccomp.deny_arg("syscall", arg, value, errno=EACCES) seccomp.deny_flag("syscall", arg, flag, errno=EACCES) seccomp.deny_mask("syscall", arg, mask, errno=EACCES) This allows you to finetune the seccomp filters for your application from inside your koreapp.
2019-10-04 10:59:48 +02:00
#if defined(__linux__)
python_push_type("pyseccomp", pykore, &pyseccomp_type);
#endif
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
#if defined(KORE_USE_CURL)
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
python_push_type("pycurlhandle", pykore, &pycurl_handle_type);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
python_push_type("pyhttpclient", pykore, &pyhttp_client_type);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
for (i = 0; py_curlopt[i].name != NULL; i++) {
python_push_integer(pykore, py_curlopt[i].name,
py_curlopt[i].value);
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
#endif
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
python_push_type("pyhttp_file", pykore, &pyhttp_file_type);
python_push_type("pyhttp_request", pykore, &pyhttp_request_type);
push all integers exported to python in a struct.
2017-01-25 20:56:57 +01:00
for (i = 0; python_integers[i].symbol != NULL; i++) {
python_push_integer(pykore, python_integers[i].symbol,
python_integers[i].value);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if ((config = PyObject_New(struct pyconfig, &pyconfig_type)) == NULL)
fatal("failed to create config object");
if (PyObject_SetAttrString(pykore, "config", (PyObject *)config) == -1)
fatal("failed to add config object");
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return (pykore);
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
static int
pyconfig_setattr(PyObject *self, PyObject *attr, PyObject *val)
{
char *v;
int ret;
PyObject *repr;
const char *name, *value;
ret = -1;
repr = NULL;
if (!PyUnicode_Check(attr))
fatal("setattr: attribute name not a unicode string");
if (PyLong_CheckExact(val)) {
if ((repr = PyObject_Repr(val)) == NULL)
return (-1);
value = PyUnicode_AsUTF8(repr);
} else if (PyUnicode_CheckExact(val)) {
value = PyUnicode_AsUTF8(val);
} else if (PyBool_Check(val)) {
if (val == Py_False)
value = "False";
else
value = "True";
} else {
fatal("invalid object, config expects integer, bool or string");
}
name = PyUnicode_AsUTF8(attr);
v = kore_strdup(value);
if (!kore_configure_setting(name, v)) {
ret = -1;
PyErr_SetString(PyExc_RuntimeError,
"configured cannot be changed at runtime");
} else {
ret = 0;
}
kore_free(v);
Py_XDECREF(repr);
return (ret);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static void
python_append_path(const char *path)
{
PyObject *mpath, *spath;
if ((mpath = PyUnicode_FromString(path)) == NULL)
fatal("python_append_path: PyUnicode_FromString failed");
if ((spath = PySys_GetObject("path")) == NULL)
fatal("python_append_path: PySys_GetObject failed");
PyList_Append(spath, mpath);
Py_DECREF(mpath);
}
static void
python_push_type(const char *name, PyObject *module, PyTypeObject *type)
{
if (PyType_Ready(type) == -1)
fatal("python_push_type: failed to ready %s", name);
Py_INCREF(type);
if (PyModule_AddObject(module, name, (PyObject *)type) == -1)
fatal("python_push_type: failed to push %s", name);
}
static void
python_push_integer(PyObject *module, const char *name, long value)
{
Simplify python_push_integer(). We're not doing anything with ret, so kill it.
2022-02-20 21:27:35 +01:00
if (PyModule_AddIntConstant(module, name, value) == -1)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
fatal("python_push_integer: failed to add %s", name);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
#if defined(KORE_USE_PGSQL)
static PyObject *
python_kore_pgsql_register(PyObject *self, PyObject *args)
{
const char *db, *conninfo;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "ss", &db, &conninfo))
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (NULL);
(void)kore_pgsql_register(db, conninfo);
Py_RETURN_TRUE;
}
#endif
Python: Add kore.app(). This method allows you to set a Python object and obtain it by calling the method again without any arguments. eg: foo = SomeClass() kore.app(foo) foo = kore.app()
2020-07-05 21:47:22 +02:00
static PyObject *
python_kore_app(PyObject *self, PyObject *args)
{
PyObject *obj;
if (!PyArg_ParseTuple(args, "O", &obj)) {
PyErr_Clear();
if (kore_app == NULL)
Py_RETURN_NONE;
Py_INCREF(kore_app);
return (kore_app);
}
Py_XDECREF(kore_app);
kore_app = obj;
Py_INCREF(kore_app);
Py_RETURN_TRUE;
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *
expose kore_server_bind() and fatal() to python
2017-02-01 17:12:52 +01:00
python_kore_log(PyObject *self, PyObject *args)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
int prio;
const char *message;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "is", &prio, &message))
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return (NULL);
kore_log(prio, "%s", message);
Py_RETURN_TRUE;
}
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
static PyObject *
python_kore_time(PyObject *self, PyObject *args)
{
u_int64_t now;
now = kore_time_ms();
return (PyLong_FromUnsignedLongLong(now));
}
expose kore_server_bind() and fatal() to python
2017-02-01 17:12:52 +01:00
static PyObject *
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
python_kore_server(PyObject *self, PyObject *args, PyObject *kwargs)
expose kore_server_bind() and fatal() to python
2017-02-01 17:12:52 +01:00
{
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
struct kore_server *srv;
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
const char *name, *ip, *port, *path;
expose kore_server_bind() and fatal() to python
2017-02-01 17:12:52 +01:00
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (kwargs == NULL) {
PyErr_SetString(PyExc_RuntimeError, "missing keyword args");
expose kore_server_bind() and fatal() to python
2017-02-01 17:12:52 +01:00
return (NULL);
}
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
ip = python_string_from_dict(kwargs, "ip");
path = python_string_from_dict(kwargs, "path");
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (ip == NULL && path == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"missing ip or path keywords");
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
return (NULL);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (ip != NULL && path != NULL) {
PyErr_SetString(PyExc_RuntimeError, "ip/path are exclusive");
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
return (NULL);
}
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
name = python_string_from_dict(kwargs, "name");
if (name == NULL)
name = "default";
if ((srv = kore_server_lookup(name)) != NULL) {
PyErr_Format(PyExc_RuntimeError,
"server '%s' already exist", name);
return (NULL);
}
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
srv = kore_server_create(name);
python_bool_from_dict(kwargs, "tls", &srv->tls);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
Throw exception is TLS backend isnt supported.
2022-02-18 09:17:58 +01:00
if (srv->tls && !kore_tls_supported()) {
kore_server_free(srv);
PyErr_SetString(PyExc_RuntimeError,
"TLS not supported in this Kore build");
return (NULL);
}
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (ip != NULL) {
if ((port = python_string_from_dict(kwargs, "port")) == NULL) {
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
kore_server_free(srv);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
PyErr_SetString(PyExc_RuntimeError,
"missing or invalid 'port' keyword");
return (NULL);
}
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
if (!kore_server_bind(srv, ip, port, NULL)) {
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
PyErr_Format(PyExc_RuntimeError,
"failed to bind to '%s:%s'", ip, port);
return (NULL);
}
} else {
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
if (!kore_server_bind_unix(srv, path, NULL)) {
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
PyErr_Format(PyExc_RuntimeError,
"failed to bind to '%s'", path);
return (NULL);
}
}
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
kore_server_finalize(srv);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
Py_RETURN_NONE;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Rework worker startup/privsep config. Starting with the privsep config, this commit changes the following: - Removes the root, runas, keymgr_root, keymgr_runas, acme_root and acme_runas configuration options. Instead these are now configured via a privsep configuration context: privsep worker { root /tmp runas nobody } This is also configurable via Python using the new kore.privsep() method: kore.privsep("worker", root="/tmp", runas="nobody", skip=["chroot"]) Tied into this we also better handle worker startup: - Per worker process, wait until it signalled it is ready. - If a worker fails at startup, display its last log lines more clearly. - Don't start acme process if no domain requires acme. - Remove each process its individual startup log message in favour of a generalized one that displays its PID, root and user. - At startup, log the kore version and built-ins in a nicer way. - The worker processes now check things they need to start running before signaling they are ready (such as access to CA certs for TLS client authentication).
2021-09-07 21:59:22 +02:00
static PyObject *
python_kore_privsep(PyObject *self, PyObject *args, PyObject *kwargs)
{
struct kore_privsep *ps;
const char *val;
PyObject *skip, *obj;
Py_ssize_t list_len, idx;
if (!PyArg_ParseTuple(args, "s", &val))
return (NULL);
if (!strcmp(val, "worker")) {
ps = &worker_privsep;
} else if (!strcmp(val, "keymgr")) {
ps = &keymgr_privsep;
#if defined(KORE_USE_ACME)
use the correct name for acme.
2021-09-07 22:19:21 +02:00
} else if (!strcmp(val, "acme")) {
Rework worker startup/privsep config. Starting with the privsep config, this commit changes the following: - Removes the root, runas, keymgr_root, keymgr_runas, acme_root and acme_runas configuration options. Instead these are now configured via a privsep configuration context: privsep worker { root /tmp runas nobody } This is also configurable via Python using the new kore.privsep() method: kore.privsep("worker", root="/tmp", runas="nobody", skip=["chroot"]) Tied into this we also better handle worker startup: - Per worker process, wait until it signalled it is ready. - If a worker fails at startup, display its last log lines more clearly. - Don't start acme process if no domain requires acme. - Remove each process its individual startup log message in favour of a generalized one that displays its PID, root and user. - At startup, log the kore version and built-ins in a nicer way. - The worker processes now check things they need to start running before signaling they are ready (such as access to CA certs for TLS client authentication).
2021-09-07 21:59:22 +02:00
ps = &acme_privsep;
#endif
} else {
PyErr_Format(PyExc_RuntimeError,
"unknown privsep process '%s'", val);
return (NULL);
}
if ((val = python_string_from_dict(kwargs, "root")) != NULL) {
kore_free(ps->root);
ps->root = kore_strdup(val);
}
if ((val = python_string_from_dict(kwargs, "runas")) != NULL) {
kore_free(ps->runas);
ps->runas = kore_strdup(val);
}
if ((skip = PyDict_GetItemString(kwargs, "skip")) != NULL) {
if (!PyList_CheckExact(skip)) {
PyErr_Format(PyExc_RuntimeError,
"privsep skip keyword needs to be a list");
return (NULL);
}
list_len = PyList_Size(skip);
for (idx = 0; idx < list_len; idx++) {
if ((obj = PyList_GetItem(skip, idx)) == NULL)
return (NULL);
if (!PyUnicode_Check(obj))
return (NULL);
if ((val = PyUnicode_AsUTF8AndSize(obj, NULL)) == NULL)
return (NULL);
if (!strcmp(val, "chroot")) {
ps->skip_chroot = 1;
} else {
PyErr_Format(PyExc_RuntimeError,
"unknown skip keyword '%s'", val);
return (NULL);
}
}
}
Py_RETURN_NONE;
}
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
static PyObject *
python_kore_prerequest(PyObject *self, PyObject *args)
{
PyObject *f;
struct reqcall *rq;
if (!PyArg_ParseTuple(args, "O", &f))
return (NULL);
rq = kore_calloc(1, sizeof(*rq));
rq->f = f;
Py_INCREF(f);
make sure prereqs are in order
2019-06-07 21:33:00 +02:00
TAILQ_INSERT_TAIL(&prereq, rq, list);
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
return (f);
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static PyObject *
python_kore_task_create(PyObject *self, PyObject *args)
{
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
PyObject *obj;
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
struct python_coro *coro;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (!PyArg_ParseTuple(args, "O", &obj))
return (NULL);
if (!PyCoro_CheckExact(obj))
fatal("%s: object is not a coroutine", __func__);
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
coro = python_coro_create(obj, NULL);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Py_INCREF(obj);
Python coro under-the-hood improvements. - Change python coroutine id to a uint64_t. - Add kore.task_id() to return active coro its id.
2021-12-02 21:58:13 +01:00
return (PyLong_FromUnsignedLongLong(coro->id));
}
static PyObject *
python_kore_task_id(PyObject *self, PyObject *args)
{
if (coro_running == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"no coroutine active");
return (NULL);
}
return (PyLong_FromUnsignedLongLong(coro_running->id));
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
}
static PyObject *
python_kore_task_kill(PyObject *self, PyObject *args)
{
Python coro under-the-hood improvements. - Change python coroutine id to a uint64_t. - Add kore.task_id() to return active coro its id.
2021-12-02 21:58:13 +01:00
u_int64_t id;
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
struct python_coro *coro, *active;
Python coro under-the-hood improvements. - Change python coroutine id to a uint64_t. - Add kore.task_id() to return active coro its id.
2021-12-02 21:58:13 +01:00
if (!PyArg_ParseTuple(args, "K", &id))
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
return (NULL);
coro_running could be NULL, check it.
2020-08-12 13:43:38 +02:00
if (coro_running != NULL && coro_running->id == id) {
Clear lockop before removing a reference from the coroutine.
2020-08-10 12:19:42 +02:00
PyErr_SetString(PyExc_RuntimeError,
"refusing to kill active coroutine");
return (NULL);
}
Add kore.task_kill() to the python api. Allows killing of coroutines, given their task id. The kore.task_create() method now returns the task id for a newly created task to the caller. While here, change the coroutine task id to a uint32 from uint64. There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
/* Remember active coro, as delete sets coro_running to NULL. */
active = coro_running;
TAILQ_FOREACH(coro, &coro_runnable, list) {
if (coro->id == id) {
coro->killed++;
kore_python_coro_delete(coro);
coro_running = active;
Py_RETURN_TRUE;
}
}
TAILQ_FOREACH(coro, &coro_suspended, list) {
if (coro->id == id) {
coro->killed++;
kore_python_coro_delete(coro);
coro_running = active;
Py_RETURN_TRUE;
}
}
Py_RETURN_FALSE;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
static PyObject *
python_kore_socket_wrap(PyObject *self, PyObject *args)
{
struct pysocket *sock;
PyObject *pysock, *pyfd, *pyfam, *pyproto;
sock = NULL;
pyfd = NULL;
pyfam = NULL;
pyproto = NULL;
if (!PyArg_ParseTuple(args, "O", &pysock))
return (NULL);
if ((pyfd = PyObject_CallMethod(pysock, "fileno", NULL)) == NULL)
return (NULL);
if ((pyfam = PyObject_GetAttrString(pysock, "family")) == NULL)
goto out;
if ((pyproto = PyObject_GetAttrString(pysock, "proto")) == NULL)
goto out;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if ((sock = pysocket_alloc()) == NULL)
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
goto out;
sock->socket = pysock;
Py_INCREF(sock->socket);
sock->fd = (int)PyLong_AsLong(pyfd);
sock->family = (int)PyLong_AsLong(pyfam);
sock->protocol = (int)PyLong_AsLong(pyproto);
memset(&sock->addr, 0, sizeof(sock->addr));
switch (sock->family) {
case AF_INET:
case AF_UNIX:
break;
default:
PyErr_SetString(PyExc_RuntimeError, "unsupported family");
Py_DECREF((PyObject *)sock);
sock = NULL;
goto out;
}
out:
Py_XDECREF(pyfd);
Py_XDECREF(pyfam);
Py_XDECREF(pyproto);
return ((PyObject *)sock);
}
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
static PyObject *
python_kore_queue(PyObject *self, PyObject *args)
{
struct pyqueue *queue;
if ((queue = PyObject_New(struct pyqueue, &pyqueue_type)) == NULL)
return (NULL);
TAILQ_INIT(&queue->objects);
TAILQ_INIT(&queue->waiting);
return ((PyObject *)queue);
}
add kore.worker() to the python api. returns the current worker number or None if on parent.
2019-06-16 19:57:48 +02:00
static PyObject *
python_kore_worker(PyObject *self, PyObject *args)
{
if (worker == NULL) {
Py_RETURN_NONE;
}
return (PyLong_FromLong(worker->id));
}
Allow user-supplied tracer callback.
2018-11-29 09:51:24 +01:00
static PyObject *
python_kore_tracer(PyObject *self, PyObject *args)
{
PyObject *obj;
if (python_tracer != NULL) {
PyErr_SetString(PyExc_RuntimeError, "tracer already set");
return (NULL);
}
if (!PyArg_ParseTuple(args, "O", &obj))
return (NULL);
if (!PyCallable_Check(obj)) {
PyErr_SetString(PyExc_RuntimeError, "object not callable");
Py_DECREF(obj);
return (NULL);
}
keep a reference around for tracer.
2018-11-29 09:53:26 +01:00
Py_INCREF(obj);
Allow user-supplied tracer callback.
2018-11-29 09:51:24 +01:00
python_tracer = obj;
Py_RETURN_TRUE;
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
static PyObject *
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
python_kore_domain(PyObject *self, PyObject *args, PyObject *kwargs)
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
{
Allow acme config via python api
2019-11-13 23:01:24 +01:00
#if defined(KORE_USE_ACME)
int acme;
char *acert, *akey;
#endif
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
struct kore_server *srv;
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
long depth;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
const char *name;
struct pydomain *domain;
Support crls for kore.domain() in Python. The crl keyword is parsed when the client_verify keyword has been set. eg: kore.domain("kore.io", cert="cert.pem", key="key.pem", client_verify="cacert.pem", verify_depth=1, crl="crl.pem")
2022-08-18 10:43:10 +02:00
const char *cert, *key, *ca, *attach, *crl;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
ca = NULL;
depth = -1;
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
key = NULL;
Support crls for kore.domain() in Python. The crl keyword is parsed when the client_verify keyword has been set. eg: kore.domain("kore.io", cert="cert.pem", key="key.pem", client_verify="cacert.pem", verify_depth=1, crl="crl.pem")
2022-08-18 10:43:10 +02:00
crl = NULL;
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
cert = NULL;
attach = NULL;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Make dumb compilers happy.
2019-11-15 07:49:16 +01:00
#if defined(KORE_USE_ACME)
acme = 0;
#endif
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (!PyArg_ParseTuple(args, "s", &name))
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (NULL);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if (kwargs != NULL)
attach = python_string_from_dict(kwargs, "attach");
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if (attach == NULL)
attach = "default";
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
if ((srv = kore_server_lookup(attach)) == NULL) {
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
PyErr_Format(PyExc_RuntimeError,
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
"server '%s' does not exist", attach);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
return (NULL);
}
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
if (srv->tls) {
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if (kwargs == NULL) {
PyErr_Format(PyExc_RuntimeError,
"no keywords for TLS enabled domain %s", name);
return (NULL);
}
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
key = python_string_from_dict(kwargs, "key");
cert = python_string_from_dict(kwargs, "cert");
Allow acme config via python api
2019-11-13 23:01:24 +01:00
#if defined(KORE_USE_ACME)
python_bool_from_dict(kwargs, "acme", &acme);
if (acme) {
kore_acme_get_paths(name, &akey, &acert);
count acme domains when configured with Python
2021-09-21 20:47:23 +02:00
acme_domains++;
Allow acme config via python api
2019-11-13 23:01:24 +01:00
key = akey;
cert = acert;
}
#endif
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (key == NULL || cert == NULL) {
PyErr_Format(PyExc_RuntimeError,
"missing key or cert keywords for TLS listener");
return (NULL);
}
change client_authority to client_verify.
2019-11-18 20:30:52 +01:00
ca = python_string_from_dict(kwargs, "client_verify");
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (ca != NULL) {
python_long_from_dict(kwargs, "verify_depth", &depth);
if (depth < 0) {
PyErr_Format(PyExc_RuntimeError,
"invalid depth '%d'", depth);
return (NULL);
}
Support crls for kore.domain() in Python. The crl keyword is parsed when the client_verify keyword has been set. eg: kore.domain("kore.io", cert="cert.pem", key="key.pem", client_verify="cacert.pem", verify_depth=1, crl="crl.pem")
2022-08-18 10:43:10 +02:00
crl = python_string_from_dict(kwargs, "crl");
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
}
} else if (key != NULL || cert != NULL || ca != NULL) {
kore_log(LOG_INFO, "ignoring tls settings for '%s'", name);
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
if (kore_domain_lookup(srv, name) != NULL) {
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
PyErr_SetString(PyExc_RuntimeError, "domain exists");
return (NULL);
}
if ((domain = PyObject_New(struct pydomain, &pydomain_type)) == NULL)
return (NULL);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
domain->next = NULL;
domain->kwargs = NULL;
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if ((domain->config = kore_domain_new(name)) == NULL)
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
fatal("failed to create new domain configuration");
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
if (!kore_domain_attach(domain->config, srv))
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
fatal("failed to attach domain configuration");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Allow multiple binds on new server directive.
2019-09-27 20:00:35 +02:00
if (srv->tls) {
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
domain->config->certkey = kore_strdup(key);
domain->config->certfile = kore_strdup(cert);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Allow acme config via python api
2019-11-13 23:01:24 +01:00
#if defined(KORE_USE_ACME)
domain->config->acme = acme;
if (domain->config->acme) {
kore_free(akey);
kore_free(acert);
}
#endif
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if (ca != NULL) {
domain->config->cafile = kore_strdup(ca);
domain->config->x509_verify_depth = depth;
Support crls for kore.domain() in Python. The crl keyword is parsed when the client_verify keyword has been set. eg: kore.domain("kore.io", cert="cert.pem", key="key.pem", client_verify="cacert.pem", verify_depth=1, crl="crl.pem")
2022-08-18 10:43:10 +02:00
if (crl != NULL)
domain->config->crlfile = kore_strdup(crl);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
return ((PyObject *)domain);
}
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
static PyObject *
python_kore_route(PyObject *self, PyObject *args, PyObject *kwargs)
{
const char *path;
PyObject *inner;
struct pyroute *route;
if ((route = PyObject_New(struct pyroute, &pyroute_type)) == NULL)
return (NULL);
if (!PyArg_ParseTuple(args, "s", &path))
return (NULL);
route->domain = NULL;
route->kwargs = kwargs;
route->path = kore_strdup(path);
Py_XINCREF(route->kwargs);
inner = PyObject_GetAttrString((PyObject *)route, "inner");
if (inner == NULL) {
Py_DECREF((PyObject *)route);
PyErr_SetString(PyExc_RuntimeError, "failed to find inner");
return (NULL);
}
return (inner);
}
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
static PyObject *
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
python_kore_gather(PyObject *self, PyObject *args, PyObject *kwargs)
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
{
struct pygather_op *op;
PyObject *obj;
struct pygather_coro *coro;
Py_ssize_t sz, idx;
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
int concurrency;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
if (coro_running == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"kore.gather only available in coroutines");
return (NULL);
}
sz = PyTuple_Size(args);
if (sz > INT_MAX) {
PyErr_SetString(PyExc_TypeError, "too many arguments");
return (NULL);
}
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
if (kwargs != NULL &&
(obj = PyDict_GetItemString(kwargs, "concurrency")) != NULL) {
if (!PyLong_Check(obj)) {
PyErr_SetString(PyExc_TypeError,
"concurrency level must be an integer");
return (NULL);
}
PyErr_Clear();
concurrency = (int)PyLong_AsLong(obj);
if (concurrency == -1 && PyErr_Occurred())
return (NULL);
if (concurrency == 0)
concurrency = sz;
} else {
concurrency = sz;
}
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
op = PyObject_New(struct pygather_op, &pygather_op_type);
if (op == NULL)
return (NULL);
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
op->running = 0;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
op->count = (int)sz;
op->coro = coro_running;
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
op->concurrency = concurrency;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
TAILQ_INIT(&op->results);
TAILQ_INIT(&op->coroutines);
for (idx = 0; idx < sz; idx++) {
don't leak op in error path
2018-10-29 21:18:38 +01:00
if ((obj = PyTuple_GetItem(args, idx)) == NULL) {
Py_DECREF((PyObject *)op);
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
return (NULL);
don't leak op in error path
2018-10-29 21:18:38 +01:00
}
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
if (!PyCoro_CheckExact(obj)) {
don't leak op in error path
2018-10-29 21:18:38 +01:00
Py_DECREF((PyObject *)op);
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
PyErr_SetString(PyExc_TypeError, "not a coroutine");
return (NULL);
}
Py_INCREF(obj);
coro = kore_pool_get(&gather_coro_pool);
coro->coro = python_coro_create(obj, NULL);
coro->coro->gatherop = op;
TAILQ_INSERT_TAIL(&op->coroutines, coro, list);
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
if (idx > concurrency - 1)
python_coro_suspend(coro->coro);
else
op->running++;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
}
return ((PyObject *)op);
}
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
static PyObject *
python_kore_lock(PyObject *self, PyObject *args)
{
struct pylock *lock;
if ((lock = PyObject_New(struct pylock, &pylock_type)) == NULL)
return (NULL);
lock->owner = NULL;
TAILQ_INIT(&lock->ops);
return ((PyObject *)lock);
}
expose kore_server_bind() and fatal() to python
2017-02-01 17:12:52 +01:00
static PyObject *
python_kore_fatal(PyObject *self, PyObject *args)
{
const char *reason;
if (!PyArg_ParseTuple(args, "s", &reason))
reason = "python_kore_fatal: PyArg_ParseTuple failed";
fatal("%s", reason);
/* not reached */
Py_RETURN_TRUE;
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static PyObject *
python_kore_fatalx(PyObject *self, PyObject *args)
{
const char *reason;
if (!PyArg_ParseTuple(args, "s", &reason))
reason = "python_kore_fatalx: PyArg_ParseTuple failed";
fatalx("%s", reason);
/* not reached */
Py_RETURN_TRUE;
}
allow python modules to set progname.
2019-09-04 20:37:33 +02:00
static PyObject *
python_kore_setname(PyObject *self, PyObject *args)
{
const char *name;
extern char *kore_progname;
if (!PyArg_ParseTuple(args, "s", &name))
return (NULL);
kore_free(kore_progname);
kore_progname = kore_strdup(name);
Py_RETURN_NONE;
}
Worker hook rework. This commit adds improved hooks for Python and a new signal delivery hook. For the Python API kore_worker_configure() and kore_worker_teardown() had to be implemented before this commit. Now one can create a workerstart and workerend method in their koreapp as those will be called when they exist. The new signal hook is either kore_worker_signal() or koreapp.signal. This new hook is called after the worker event code handles the received signal itself first. With this commit there is also a new kore_signal_trap() API call allowing you to more easily trap new signals. This API also also exported to the Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
static PyObject *
python_kore_sigtrap(PyObject *self, PyObject *args)
{
int sig;
if (!PyArg_ParseTuple(args, "i", &sig))
return (NULL);
kore_signal_trap(sig);
Py_RETURN_NONE;
}
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
static PyObject *
python_kore_sendobj(PyObject *self, PyObject *args, PyObject *kwargs)
{
long val;
u_int16_t dst;
char *ptr;
Py_ssize_t length;
PyObject *object, *bytes;
if (!PyArg_ParseTuple(args, "O", &object))
return (NULL);
bytes = PyObject_CallFunctionObjArgs(pickle_dumps, object, NULL);
if (bytes == NULL)
return (NULL);
if (PyBytes_AsStringAndSize(bytes, &ptr, &length) == -1) {
Py_DECREF(bytes);
return (NULL);
}
dst = KORE_MSG_WORKER_ALL;
if (kwargs != NULL) {
if (python_long_from_dict(kwargs, "worker", &val)) {
if (val <= 0 || val > worker_count ||
val >= KORE_WORKER_MAX) {
PyErr_Format(PyExc_RuntimeError,
"worker %ld invalid", val);
Py_DECREF(bytes);
return (NULL);
}
dst = val;
}
}
kore_msg_send(dst, KORE_PYTHON_SEND_OBJ, ptr, length);
Py_DECREF(bytes);
Py_RETURN_NONE;
}
static void
python_kore_recvobj(struct kore_msg *msg, const void *data)
{
struct kore_runtime *rt;
PyObject *onmsg, *ret, *bytes, *obj;
if ((onmsg = kore_module_getsym("koreapp.onmsg", &rt)) == NULL)
return;
if (rt->type != KORE_RUNTIME_PYTHON)
return;
if ((bytes = PyBytes_FromStringAndSize(data, msg->length)) == NULL) {
Py_DECREF(onmsg);
Make sure koreapp.onmsg can log errors from Python.
2020-12-18 19:48:17 +01:00
kore_python_log_error("koreapp.onmsg");
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
return;
}
obj = PyObject_CallFunctionObjArgs(pickle_loads, bytes, NULL);
Py_DECREF(bytes);
if (obj == NULL) {
Py_DECREF(onmsg);
Make sure koreapp.onmsg can log errors from Python.
2020-12-18 19:48:17 +01:00
kore_python_log_error("koreapp.onmsg");
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
return;
}
ret = PyObject_CallFunctionObjArgs(onmsg, obj, NULL);
Make sure koreapp.onmsg can log errors from Python.
2020-12-18 19:48:17 +01:00
kore_python_log_error("koreapp.onmsg");
Add kore.sendmsg(object, worker=None) to the python api. This allows you to send Python objects that can be run through pickle to other worker processes. If your application implements koreapp.onmsg() you will be able to receive these objects.
2019-10-16 12:05:27 +02:00
Py_DECREF(obj);
Py_DECREF(onmsg);
Py_XDECREF(ret);
}
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
static PyObject *
python_kore_suspend(PyObject *self, PyObject *args)
{
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
struct pysuspend_op *op;
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
int delay;
if (!PyArg_ParseTuple(args, "i", &delay))
return (NULL);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
op = PyObject_New(struct pysuspend_op, &pysuspend_op_type);
if (op == NULL)
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
return (NULL);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
op->timer = NULL;
op->delay = delay;
op->coro = coro_running;
op->state = PYSUSPEND_OP_INIT;
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
return ((PyObject *)op);
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
}
Add kore_shutdown(). Allows workers to cleanly initiate a shutdown of the entire server process.
2018-10-23 19:49:42 +02:00
static PyObject *
python_kore_shutdown(PyObject *self, PyObject *args)
{
kore_shutdown();
Py_RETURN_TRUE;
}
allow coroutines to set friendly names.
2019-09-16 20:31:25 +02:00
static PyObject *
python_kore_coroname(PyObject *self, PyObject *args)
{
const char *name;
if (coro_running == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"kore.coroname() only available in coroutines");
return (NULL);
}
if (!PyArg_ParseTuple(args, "s", &name))
return (NULL);
kore_free(coro_running->name);
coro_running->name = kore_strdup(name);
make coro tracing available at runtime, always. call kore.corotrace(True) to enable it and kore.corotrace(False) to disable.
2019-09-18 10:55:13 +02:00
Py_RETURN_NONE;
}
static PyObject *
python_kore_corotrace(PyObject *self, PyObject *args)
{
if (!PyArg_ParseTuple(args, "b", &coro_tracing))
return (NULL);
allow coroutines to set friendly names.
2019-09-16 20:31:25 +02:00
Py_RETURN_NONE;
}
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
static PyObject *
allow use udata in kore.timer() via the data kwarg.
2020-01-22 09:42:41 +01:00
python_kore_timer(PyObject *self, PyObject *args, PyObject *kwargs)
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
{
u_int64_t ms;
PyObject *obj;
int flags;
struct pytimer *timer;
Small improvement to the Python kore.timer() api. Do not allow kore.timer() to be called from the parent process as it shouldn't be run there. This makes Kore fail more gracefully.
2021-06-03 14:02:28 +02:00
if (worker == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"kore.timer not supported on parent process");
return (NULL);
}
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
if (!PyArg_ParseTuple(args, "OKi", &obj, &ms, &flags))
return (NULL);
if (flags & ~(KORE_TIMER_FLAGS)) {
PyErr_SetString(PyExc_RuntimeError, "invalid flags");
return (NULL);
}
if ((timer = PyObject_New(struct pytimer, &pytimer_type)) == NULL)
return (NULL);
allow use udata in kore.timer() via the data kwarg.
2020-01-22 09:42:41 +01:00
timer->udata = NULL;
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
timer->flags = flags;
timer->callable = obj;
timer->run = kore_timer_add(pytimer_run, ms, timer, flags);
make sure timers don't go out of scope.
2018-10-23 21:36:19 +02:00
Py_INCREF((PyObject *)timer);
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
Py_INCREF(timer->callable);
allow use udata in kore.timer() via the data kwarg.
2020-01-22 09:42:41 +01:00
if (kwargs != NULL) {
if ((obj = PyDict_GetItemString(kwargs, "data")) != NULL) {
Py_INCREF(obj);
timer->udata = obj;
}
}
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
return ((PyObject *)timer);
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
static PyObject *
Improve kore.proc in the Python API. Allow passing of an env keyword, allowing you to set environment variables that may be required by the subprocess. The env keyword must be a list with correctly formed environment variables. eg: proc = kore.proc("/bin/myproc", env=[ "LD_LIBRARY_PATH=/my/path" ] )
2022-10-19 08:04:16 +02:00
python_kore_proc(PyObject *self, PyObject *args, PyObject *kwargs)
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
{
Improve kore.proc in the Python API. Allow passing of an env keyword, allowing you to set environment variables that may be required by the subprocess. The env keyword must be a list with correctly formed environment variables. eg: proc = kore.proc("/bin/myproc", env=[ "LD_LIBRARY_PATH=/my/path" ] )
2022-10-19 08:04:16 +02:00
union deconst cp;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
const char *cmd;
struct pyproc *proc;
Improve kore.proc in the Python API. Allow passing of an env keyword, allowing you to set environment variables that may be required by the subprocess. The env keyword must be a list with correctly formed environment variables. eg: proc = kore.proc("/bin/myproc", env=[ "LD_LIBRARY_PATH=/my/path" ] )
2022-10-19 08:04:16 +02:00
Py_ssize_t idx, len;
PyObject *obj, *item;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
int timeo, in_pipe[2], out_pipe[2];
Improve kore.proc in the Python API. Allow passing of an env keyword, allowing you to set environment variables that may be required by the subprocess. The env keyword must be a list with correctly formed environment variables. eg: proc = kore.proc("/bin/myproc", env=[ "LD_LIBRARY_PATH=/my/path" ] )
2022-10-19 08:04:16 +02:00
char *copy, *argv[32], *env[PYTHON_PROC_MAX_ENV + 1];
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
timeo = -1;
if (coro_running == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"kore.proc only available in coroutines");
return (NULL);
}
if (!PyArg_ParseTuple(args, "s|i", &cmd, &timeo))
return (NULL);
Improve kore.proc in the Python API. Allow passing of an env keyword, allowing you to set environment variables that may be required by the subprocess. The env keyword must be a list with correctly formed environment variables. eg: proc = kore.proc("/bin/myproc", env=[ "LD_LIBRARY_PATH=/my/path" ] )
2022-10-19 08:04:16 +02:00
if (kwargs != NULL &&
(obj = PyDict_GetItemString(kwargs, "env")) != NULL) {
if (!PyList_CheckExact(obj)) {
PyErr_SetString(PyExc_RuntimeError,
"kore.proc: env is not of type 'list'");
return (NULL);
}
len = PyList_Size(obj);
if (len > PYTHON_PROC_MAX_ENV) {
PyErr_SetString(PyExc_RuntimeError,
"kore.proc: too many entries in 'env' keyword");
return (NULL);
}
for (idx = 0; idx < len; idx++) {
if ((item = PyList_GetItem(obj, idx)) == NULL)
return (NULL);
if (!PyUnicode_CheckExact(item))
return (NULL);
if ((cp.cp = PyUnicode_AsUTF8(item)) == NULL)
return (NULL);
env[idx] = cp.p;
}
env[idx] = NULL;
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (pipe(in_pipe) == -1) {
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
}
if (pipe(out_pipe) == -1) {
close(in_pipe[0]);
close(in_pipe[1]);
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
}
if ((proc = PyObject_New(struct pyproc, &pyproc_type)) == NULL) {
close(in_pipe[0]);
close(in_pipe[1]);
close(out_pipe[0]);
close(out_pipe[1]);
return (NULL);
}
proc->pid = -1;
Make sure we wakeup the coroutine that called proc.reap(). We actually woke up the coroutine that originally spawned the process when we reap it, but another coroutine may have taken over the object. This mimics how we do things for the pysock_op things.
2019-10-29 15:12:20 +01:00
proc->op = NULL;
store active pid into apid so it does not overwritten.
2019-10-15 14:25:57 +02:00
proc->apid = -1;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
proc->reaped = 0;
proc->status = 0;
proc->timer = NULL;
proc->coro = coro_running;
proc->in = pysocket_alloc();
proc->out = pysocket_alloc();
if (proc->in == NULL || proc->out == NULL) {
Py_DECREF((PyObject *)proc);
return (NULL);
}
TAILQ_INSERT_TAIL(&procs, proc, list);
proc->pid = fork();
if (proc->pid == -1) {
if (errno == ENOSYS) {
Py_DECREF((PyObject *)proc);
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
}
fatal("python_kore_proc: fork(): %s", errno_s);
}
if (proc->pid == 0) {
close(in_pipe[1]);
close(out_pipe[0]);
if (dup2(out_pipe[1], STDOUT_FILENO) == -1 ||
dup2(out_pipe[1], STDERR_FILENO) == -1 ||
dup2(in_pipe[0], STDIN_FILENO) == -1)
fatal("dup2: %s", errno_s);
copy = kore_strdup(cmd);
allow double quotes in kore.proc argument lists.
2019-02-22 16:56:34 +01:00
python_split_arguments(copy, argv, 32);
execve() its envp shouldn't be NULL.
2022-02-20 21:20:14 +01:00
(void)execve(argv[0], argv, env);
allow double quotes in kore.proc argument lists.
2019-02-22 16:56:34 +01:00
kore_log(LOG_ERR, "kore.proc failed to execute %s (%s)",
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
argv[0], errno_s);
exit(1);
}
close(in_pipe[0]);
close(out_pipe[1]);
if (!kore_connection_nonblock(in_pipe[1], 0) ||
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
!kore_connection_nonblock(out_pipe[0], 0))
fatal("failed to mark kore.proc pipes are non-blocking");
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
store active pid into apid so it does not overwritten.
2019-10-15 14:25:57 +02:00
proc->apid = proc->pid;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
proc->in->fd = in_pipe[1];
proc->out->fd = out_pipe[0];
if (timeo != -1) {
proc->timer = kore_timer_add(pyproc_timeout,
timeo, proc, KORE_TIMER_ONESHOT);
}
return ((PyObject *)proc);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *
python_import(const char *path)
{
Append full module path if it is a directory.
2019-03-16 16:13:52 +01:00
struct stat st;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
PyObject *module;
char *dir, *file, *copy, *p;
Append full module path if it is a directory.
2019-03-16 16:13:52 +01:00
if (stat(path, &st) == -1)
fatal("python_import: stat(%s): %s", path, errno_s);
if (!S_ISDIR(st.st_mode) && !S_ISREG(st.st_mode))
fatal("python_import: '%s' is not a file or directory", path);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
copy = kore_strdup(path);
work around different dirname()/basename() implementations.
2020-06-09 12:22:22 +02:00
if ((p = dirname(copy)) == NULL)
fatal("dirname: %s: %s", path, errno_s);
dir = kore_strdup(p);
kore_free(copy);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
work around different dirname()/basename() implementations.
2020-06-09 12:22:22 +02:00
copy = kore_strdup(path);
if ((p = basename(copy)) == NULL)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
fatal("basename: %s: %s", path, errno_s);
work around different dirname()/basename() implementations.
2020-06-09 12:22:22 +02:00
file = kore_strdup(p);
kore_free(copy);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
if ((p = strrchr(file, '.')) != NULL)
*p = '\0';
python_append_path(dir);
Append full module path if it is a directory.
2019-03-16 16:13:52 +01:00
if (S_ISDIR(st.st_mode))
python_append_path(path);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
module = PyImport_ImportModule(file);
if (module == NULL)
PyErr_Print();
work around different dirname()/basename() implementations.
2020-06-09 12:22:22 +02:00
kore_free(dir);
kore_free(file);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return (module);
}
static PyObject *
python_callable(PyObject *module, const char *symbol)
{
allow foo.method symbols in python. this way you can create page handlers that reside inside of other objects. eg: static / restapi.index
2018-11-16 11:07:21 +01:00
char *base, *method;
PyObject *res, *obj, *meth;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
allow foo.method symbols in python. this way you can create page handlers that reside inside of other objects. eg: static / restapi.index
2018-11-16 11:07:21 +01:00
res = NULL;
obj = NULL;
base = kore_strdup(symbol);
if ((method = strchr(base, '.')) != NULL)
*(method)++ = '\0';
if ((obj = PyObject_GetAttrString(module, base)) == NULL)
goto out;
if (method != NULL) {
if ((meth = PyObject_GetAttrString(obj, method)) == NULL)
goto out;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Py_DECREF(obj);
allow foo.method symbols in python. this way you can create page handlers that reside inside of other objects. eg: static / restapi.index
2018-11-16 11:07:21 +01:00
obj = meth;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
allow foo.method symbols in python. this way you can create page handlers that reside inside of other objects. eg: static / restapi.index
2018-11-16 11:07:21 +01:00
if (!PyCallable_Check(obj))
goto out;
res = obj;
obj = NULL;
out:
if (obj != NULL)
Py_DECREF(obj);
call PyErr_Clear() in python_callable().
2019-10-15 16:03:39 +02:00
PyErr_Clear();
allow foo.method symbols in python. this way you can create page handlers that reside inside of other objects. eg: static / restapi.index
2018-11-16 11:07:21 +01:00
kore_free(base);
return (res);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
static PyObject *
pyconnection_alloc(struct connection *c)
{
struct pyconnection *pyc;
pyc = PyObject_New(struct pyconnection, &pyconnection_type);
if (pyc == NULL)
return (NULL);
pyc->c = c;
return ((PyObject *)pyc);
}
expose c.disconnect() and c.fd to Python.
2017-03-13 12:22:50 +01:00
static PyObject *
pyconnection_disconnect(struct pyconnection *pyc, PyObject *args)
{
kore_connection_disconnect(pyc->c);
Py_RETURN_TRUE;
}
static PyObject *
pyconnection_get_fd(struct pyconnection *pyc, void *closure)
{
PyObject *fd;
if ((fd = PyLong_FromLong(pyc->c->fd)) == NULL)
return (PyErr_NoMemory());
return (fd);
}
expose connection address to python.
2017-03-24 12:42:40 +01:00
static PyObject *
pyconnection_get_addr(struct pyconnection *pyc, void *closure)
{
void *ptr;
PyObject *result;
char addr[INET6_ADDRSTRLEN];
missing addrtype -> family renames
2018-10-08 20:12:25 +02:00
switch (pyc->c->family) {
expose connection address to python.
2017-03-24 12:42:40 +01:00
case AF_INET:
ptr = &pyc->c->addr.ipv4.sin_addr;
break;
case AF_INET6:
ptr = &pyc->c->addr.ipv6.sin6_addr;
break;
default:
missing addrtype -> family renames
2018-10-08 20:12:25 +02:00
PyErr_SetString(PyExc_RuntimeError, "invalid family");
expose connection address to python.
2017-03-24 12:42:40 +01:00
return (NULL);
}
missing addrtype -> family renames
2018-10-08 20:12:25 +02:00
if (inet_ntop(pyc->c->family, ptr, addr, sizeof(addr)) == NULL) {
expose connection address to python.
2017-03-24 12:42:40 +01:00
PyErr_SetString(PyExc_RuntimeError, "inet_ntop failed");
return (NULL);
}
if ((result = PyUnicode_FromString(addr)) == NULL)
return (PyErr_NoMemory());
return (result);
}
Add support to obtain peer certificate from Python This will return the DER encoded bytes representing the peer certificate.
2018-11-28 11:28:07 +01:00
static PyObject *
pyconnection_get_peer_x509(struct pyconnection *pyc, void *closure)
{
Add kore_tls_x509_data(). Use it in the Python code, which requires it.
2022-02-18 09:14:30 +01:00
size_t len;
u_int8_t *der;
PyObject *bytes;
Add support to obtain peer certificate from Python This will return the DER encoded bytes representing the peer certificate.
2018-11-28 11:28:07 +01:00
rename ssl prefixed things to tls.
2022-02-18 10:20:28 +01:00
if (pyc->c->tls_cert == NULL) {
Add support to obtain peer certificate from Python This will return the DER encoded bytes representing the peer certificate.
2018-11-28 11:28:07 +01:00
Py_RETURN_NONE;
}
Add kore_tls_x509_data(). Use it in the Python code, which requires it.
2022-02-18 09:14:30 +01:00
if (!kore_tls_x509_data(pyc->c, &der, &len)) {
PyErr_SetString(PyExc_RuntimeError,
"failed to obtain certificate data");
Add support to obtain peer certificate from Python This will return the DER encoded bytes representing the peer certificate.
2018-11-28 11:28:07 +01:00
return (NULL);
}
bytes = PyBytes_FromStringAndSize((char *)der, len);
kore_free(der);
return (bytes);
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
}
static PyObject *
pyconnection_get_peer_x509dict(struct pyconnection *pyc, void *closure)
{
For each TLS backend let us use correct types.
2022-02-18 10:47:05 +01:00
KORE_X509_NAMES *name;
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
PyObject *dict, *issuer, *subject, *ret;
ret = NULL;
issuer = NULL;
subject = NULL;
rename ssl prefixed things to tls.
2022-02-18 10:20:28 +01:00
if (pyc->c->tls_cert == NULL) {
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
Py_RETURN_NONE;
}
if ((dict = PyDict_New()) == NULL)
goto out;
if ((issuer = PyDict_New()) == NULL)
goto out;
if (PyDict_SetItemString(dict, "issuer", issuer) == -1)
goto out;
if ((subject = PyDict_New()) == NULL)
goto out;
if (PyDict_SetItemString(dict, "subject", subject) == -1)
goto out;
PyErr_Clear();
rename ssl prefixed things to tls.
2022-02-18 10:20:28 +01:00
if ((name = kore_tls_x509_subject_name(pyc->c)) == NULL) {
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
PyErr_Format(PyExc_RuntimeError,
Add kore_tls_x509_data(). Use it in the Python code, which requires it.
2022-02-18 09:14:30 +01:00
"failed to obtain x509 subjectName");
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
goto out;
}
Unbreak x509dict. I accidentally swapped issuer and subject while constructing it.
2022-04-20 11:06:10 +02:00
if (!kore_tls_x509name_foreach(name, 0, subject,
pyconnection_x509_cb)) {
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
if (PyErr_Occurred() == NULL) {
PyErr_Format(PyExc_RuntimeError,
adjust log messages
2022-04-20 11:10:01 +02:00
"failed to add subject name to dictionary");
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
}
goto out;
}
rename ssl prefixed things to tls.
2022-02-18 10:20:28 +01:00
if ((name = kore_tls_x509_issuer_name(pyc->c)) == NULL) {
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
PyErr_Format(PyExc_RuntimeError,
Add kore_tls_x509_data(). Use it in the Python code, which requires it.
2022-02-18 09:14:30 +01:00
"failed to obtain x509 issuerName");
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
goto out;
}
Unbreak x509dict. I accidentally swapped issuer and subject while constructing it.
2022-04-20 11:06:10 +02:00
if (!kore_tls_x509name_foreach(name, 0, issuer, pyconnection_x509_cb)) {
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
if (PyErr_Occurred() == NULL) {
PyErr_Format(PyExc_RuntimeError,
adjust log messages
2022-04-20 11:10:01 +02:00
"failed to add issuer name to dictionary");
Python: Add an req.connection.x509dict This dictionary for now only contains the subject and issuer names from the client certificate (if one was provided) with their X509_NAME components. Eg: { "issuer": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" }, "subject": { "C": "SE", "O": "kore autogen: x509name-test", "CN": "localhost" } }
2021-12-11 22:37:15 +01:00
}
goto out;
}
ret = dict;
dict = NULL;
out:
Py_XDECREF(dict);
Py_XDECREF(issuer);
Py_XDECREF(subject);
return (ret);
}
static int
pyconnection_x509_cb(void *udata, int islast, int nid, const char *field,
const void *data, size_t len, int flags)
{
PyObject *dict, *obj;
dict = udata;
if ((obj = PyUnicode_FromStringAndSize(data, len)) == NULL)
return (KORE_RESULT_ERROR);
if (PyDict_SetItemString(dict, field, obj) == -1) {
Py_DECREF(obj);
return (KORE_RESULT_ERROR);
}
Py_DECREF(obj);
return (KORE_RESULT_OK);
Add support to obtain peer certificate from Python This will return the DER encoded bytes representing the peer certificate.
2018-11-28 11:28:07 +01:00
}
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
static void
pytimer_run(void *arg, u_int64_t now)
{
PyObject *ret;
struct pytimer *timer = arg;
PyErr_Clear();
allow use udata in kore.timer() via the data kwarg.
2020-01-22 09:42:41 +01:00
ret = PyObject_CallFunctionObjArgs(timer->callable, timer->udata, NULL);
deal with PyObject_CallObject() returning NULL.
2019-01-15 10:20:13 +01:00
Py_XDECREF(ret);
allow use udata in kore.timer() via the data kwarg.
2020-01-22 09:42:41 +01:00
Py_XDECREF(timer->udata);
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
allow use udata in kore.timer() via the data kwarg.
2020-01-22 09:42:41 +01:00
timer->udata = NULL;
small python improvements. - Check for an exception when in a timer context and log it if uncaught. - Make sure tlsverify is settable regardless of client certs being specified.
2019-05-16 21:07:42 +02:00
kore_python_log_error("pytimer_run");
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
if (timer->flags & KORE_TIMER_ONESHOT) {
timer->run = NULL;
Py_DECREF((PyObject *)timer);
}
}
static void
pytimer_dealloc(struct pytimer *timer)
{
if (timer->run != NULL) {
kore_timer_remove(timer->run);
timer->run = NULL;
}
if (timer->callable != NULL) {
Py_DECREF(timer->callable);
timer->callable = NULL;
}
PyObject_Del((PyObject *)timer);
}
static PyObject *
pytimer_close(struct pytimer *timer, PyObject *args)
{
if (timer->run != NULL) {
kore_timer_remove(timer->run);
timer->run = NULL;
}
if (timer->callable != NULL) {
Py_DECREF(timer->callable);
timer->callable = NULL;
}
allow use udata in kore.timer() via the data kwarg.
2020-01-22 09:42:41 +01:00
if (timer->udata != NULL) {
Py_DECREF(timer->udata);
timer->udata = NULL;
}
make sure timers don't go out of scope.
2018-10-23 21:36:19 +02:00
Py_INCREF((PyObject *)timer);
Hook kore timers into python.
2018-10-23 20:44:43 +02:00
Py_RETURN_TRUE;
}
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
static void
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
pysuspend_op_dealloc(struct pysuspend_op *op)
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
{
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (op->timer != NULL) {
kore_timer_remove(op->timer);
op->timer = NULL;
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
PyObject_Del((PyObject *)op);
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
}
static PyObject *
pysuspend_op_await(PyObject *sop)
{
Py_INCREF(sop);
return (sop);
}
static PyObject *
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
pysuspend_op_iternext(struct pysuspend_op *op)
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
{
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
switch (op->state) {
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
case PYSUSPEND_OP_INIT:
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
op->timer = kore_timer_add(pysuspend_wakeup, op->delay,
op, KORE_TIMER_ONESHOT);
op->state = PYSUSPEND_OP_WAIT;
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
break;
case PYSUSPEND_OP_WAIT:
break;
case PYSUSPEND_OP_CONTINUE:
PyErr_SetNone(PyExc_StopIteration);
return (NULL);
default:
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
fatal("unknown state %d for pysuspend_op", op->state);
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
}
Py_RETURN_NONE;
}
static void
pysuspend_wakeup(void *arg, u_int64_t now)
{
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
struct pysuspend_op *op = arg;
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
op->timer = NULL;
op->state = PYSUSPEND_OP_CONTINUE;
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (op->coro->request != NULL)
http_request_wakeup(op->coro->request);
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
else
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
python_coro_wakeup(op->coro);
}
static struct pysocket *
pysocket_alloc(void)
{
struct pysocket *sock;
if ((sock = PyObject_New(struct pysocket, &pysocket_type)) == NULL)
return (NULL);
sock->fd = -1;
sock->family = -1;
sock->protocol = -1;
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sock->scheduled = 0;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
sock->socket = NULL;
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sock->recvop = NULL;
sock->sendop = NULL;
sock->event.s = sock;
sock->event.evt.flags = 0;
sock->event.evt.type = KORE_TYPE_PYSOCKET;
sock->event.evt.handle = pysocket_evt_handle;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
return (sock);
Add kore.suspend(delay) to python. Will suspend the coroutine for a number of milliseconds. Example: async def page(req): await kore.suspend(1000) req.response(200, b'')
2018-10-23 21:32:08 +02:00
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static void
pysocket_dealloc(struct pysocket *sock)
{
only deschedule if not -1.
2019-03-27 22:15:35 +01:00
if (sock->scheduled && sock->fd != -1) {
remove the socket from the event queue on dealloc.
2019-03-13 16:01:42 +01:00
kore_platform_disable_read(sock->fd);
#if !defined(__linux__)
kore_platform_disable_write(sock->fd);
#endif
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (sock->socket != NULL) {
Py_DECREF(sock->socket);
} else if (sock->fd != -1) {
(void)close(sock->fd);
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyObject_Del((PyObject *)sock);
}
static PyObject *
pysocket_send(struct pysocket *sock, PyObject *args)
{
Py_buffer buf;
Python: Improve the kore.socket interface. - Make sure PyBuffer_Release() is called for y* arguments. - Make sure buffer is cleaned up for SENDTO socket ops as well.
2020-08-11 15:24:59 +02:00
PyObject *ret;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (!PyArg_ParseTuple(args, "y*", &buf))
return (NULL);
Python: Improve the kore.socket interface. - Make sure PyBuffer_Release() is called for y* arguments. - Make sure buffer is cleaned up for SENDTO socket ops as well.
2020-08-11 15:24:59 +02:00
ret = pysocket_op_create(sock, PYSOCKET_TYPE_SEND, buf.buf, buf.len);
PyBuffer_Release(&buf);
return (ret);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
static PyObject *
pysocket_sendto(struct pysocket *sock, PyObject *args)
{
Py_buffer buf;
struct pysocket_op *op;
PyObject *ret;
int port;
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
const char *ip, *sockaddr;
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
switch (sock->family) {
case AF_INET:
if (!PyArg_ParseTuple(args, "siy*", &ip, &port, &buf))
return (NULL);
if (port <= 0 || port >= USHRT_MAX) {
PyErr_SetString(PyExc_RuntimeError, "invalid port");
return (NULL);
}
break;
case AF_UNIX:
if (!PyArg_ParseTuple(args, "sy*", &sockaddr, &buf))
return (NULL);
break;
default:
PyErr_SetString(PyExc_RuntimeError, "unsupported family");
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
return (NULL);
}
ret = pysocket_op_create(sock, PYSOCKET_TYPE_SENDTO, buf.buf, buf.len);
Python: Improve the kore.socket interface. - Make sure PyBuffer_Release() is called for y* arguments. - Make sure buffer is cleaned up for SENDTO socket ops as well.
2020-08-11 15:24:59 +02:00
PyBuffer_Release(&buf);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
op = (struct pysocket_op *)ret;
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
switch (sock->family) {
case AF_INET:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->sendaddr.ipv4.sin_family = AF_INET;
op->sendaddr.ipv4.sin_port = htons(port);
op->sendaddr.ipv4.sin_addr.s_addr = inet_addr(ip);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
case AF_UNIX:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->sendaddr.sun.sun_family = AF_UNIX;
if (kore_strlcpy(op->sendaddr.sun.sun_path, sockaddr,
sizeof(op->sendaddr.sun.sun_path)) >=
sizeof(op->sendaddr.sun.sun_path)) {
cleanup.
2019-03-01 20:56:03 +01:00
Py_DECREF(ret);
PyErr_SetString(PyExc_RuntimeError,
"unix socket path too long");
return (NULL);
}
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
default:
cleanup.
2019-03-01 20:56:03 +01:00
Py_DECREF(ret);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
PyErr_SetString(PyExc_RuntimeError, "unsupported family");
return (NULL);
}
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
return (ret);
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static PyObject *
pysocket_recv(struct pysocket *sock, PyObject *args)
{
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
Py_ssize_t len;
struct pysocket_op *op;
PyObject *obj;
int timeo;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
timeo = -1;
if (!PyArg_ParseTuple(args, "n|i", &len, &timeo))
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
return (NULL);
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
obj = pysocket_op_create(sock, PYSOCKET_TYPE_RECV, NULL, len);
if (obj == NULL)
return (NULL);
op = (struct pysocket_op *)obj;
if (timeo != -1) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->timer = kore_timer_add(pysocket_op_timeout,
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
timeo, op, KORE_TIMER_ONESHOT);
}
return (obj);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
static PyObject *
pysocket_recvmsg(struct pysocket *sock, PyObject *args)
{
Py_ssize_t len;
if (!PyArg_ParseTuple(args, "n", &len))
return (NULL);
return (pysocket_op_create(sock, PYSOCKET_TYPE_RECVMSG, NULL, len));
}
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
static PyObject *
pysocket_recvfrom(struct pysocket *sock, PyObject *args)
{
Py_ssize_t len;
if (!PyArg_ParseTuple(args, "n", &len))
return (NULL);
return (pysocket_op_create(sock, PYSOCKET_TYPE_RECVFROM, NULL, len));
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static PyObject *
pysocket_accept(struct pysocket *sock, PyObject *args)
{
return (pysocket_op_create(sock, PYSOCKET_TYPE_ACCEPT, NULL, 0));
}
static PyObject *
pysocket_connect(struct pysocket *sock, PyObject *args)
{
const char *host;
int port, len;
port = 0;
if (!PyArg_ParseTuple(args, "s|i", &host, &port))
return (NULL);
if (port < 0 || port > USHRT_MAX) {
PyErr_SetString(PyExc_RuntimeError, "invalid port number");
return (NULL);
}
switch (sock->family) {
case AF_INET:
sock->addr.ipv4.sin_family = AF_INET;
sock->addr.ipv4.sin_port = htons(port);
if (inet_pton(sock->family, host,
&sock->addr.ipv4.sin_addr) == -1) {
PyErr_SetString(PyExc_RuntimeError, "invalid host");
return (NULL);
}
sock->addr_len = sizeof(sock->addr.ipv4);
break;
case AF_UNIX:
sock->addr.sun.sun_family = AF_UNIX;
len = snprintf(sock->addr.sun.sun_path,
sizeof(sock->addr.sun.sun_path), "%s", host);
if (len == -1 ||
(size_t)len >= sizeof(sock->addr.sun.sun_path)) {
PyErr_SetString(PyExc_RuntimeError, "path too long");
return (NULL);
}
#if defined(__linux__)
/* Assume abstract socket if prefixed with '@'. */
if (sock->addr.sun.sun_path[0] == '@')
sock->addr.sun.sun_path[0] = '\0';
#endif
sock->addr_len = sizeof(sock->addr.sun.sun_family) + len;
break;
default:
fatal("unsupported socket family %d", sock->family);
}
return (pysocket_op_create(sock, PYSOCKET_TYPE_CONNECT, NULL, 0));
}
static PyObject *
pysocket_close(struct pysocket *sock, PyObject *args)
{
pysock: call disable read/write only when needed
2019-03-14 09:27:13 +01:00
if (sock->scheduled) {
sock->scheduled = 0;
kore_platform_disable_read(sock->fd);
#if !defined(__linux__)
kore_platform_disable_write(sock->fd);
#endif
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (sock->socket != NULL) {
Py_DECREF(sock->socket);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
sock->socket = NULL;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
} else if (sock->fd != -1) {
(void)close(sock->fd);
}
set fd to -1 on close.
2019-03-21 10:09:24 +01:00
sock->fd = -1;
Improve closing of a kore.socket() in Python API. When a kore.socket() is closed from any coroutine, make sure any other coroutines waiting on events on the socket are awoken so they properly can return errors.
2021-12-02 22:47:17 +01:00
sock->event.evt.handle(&sock->event, 1);
set fd to -1 on close.
2019-03-21 10:09:24 +01:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Py_RETURN_TRUE;
}
static void
pysocket_op_dealloc(struct pysocket_op *op)
{
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->type == PYSOCKET_TYPE_RECV ||
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
op->type == PYSOCKET_TYPE_RECVMSG ||
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->type == PYSOCKET_TYPE_RECVFROM ||
Python: Improve the kore.socket interface. - Make sure PyBuffer_Release() is called for y* arguments. - Make sure buffer is cleaned up for SENDTO socket ops as well.
2020-08-11 15:24:59 +02:00
op->type == PYSOCKET_TYPE_SEND ||
op->type == PYSOCKET_TYPE_SENDTO)
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
kore_buf_cleanup(&op->buffer);
switch (op->type) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
case PYSOCKET_TYPE_RECV:
case PYSOCKET_TYPE_ACCEPT:
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
case PYSOCKET_TYPE_RECVMSG:
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
case PYSOCKET_TYPE_RECVFROM:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->socket->recvop != op)
fatal("recvop mismatch");
op->socket->recvop = NULL;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
break;
case PYSOCKET_TYPE_SEND:
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
case PYSOCKET_TYPE_SENDTO:
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
case PYSOCKET_TYPE_CONNECT:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->socket->sendop != op)
fatal("sendop mismatch");
op->socket->sendop = NULL;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
break;
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->timer != NULL) {
kore_timer_remove(op->timer);
op->timer = NULL;
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->coro->sockop = NULL;
Py_DECREF(op->socket);
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyObject_Del((PyObject *)op);
}
static PyObject *
pysocket_op_create(struct pysocket *sock, int type, const void *ptr, size_t len)
{
struct pysocket_op *op;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (coro_running->sockop != NULL)
fatal("pysocket_op_create: coro has active socketop");
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
switch (type) {
case PYSOCKET_TYPE_RECV:
case PYSOCKET_TYPE_ACCEPT:
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
case PYSOCKET_TYPE_RECVMSG:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
case PYSOCKET_TYPE_RECVFROM:
if (sock->recvop != NULL) {
PyErr_SetString(PyExc_RuntimeError,
"only one recv operation can be done per socket");
return (NULL);
}
break;
case PYSOCKET_TYPE_SEND:
case PYSOCKET_TYPE_SENDTO:
case PYSOCKET_TYPE_CONNECT:
if (sock->sendop != NULL) {
PyErr_SetString(PyExc_RuntimeError,
"only one send operation can be done per socket");
return (NULL);
}
break;
default:
fatal("unknown pysocket_op type %u", type);
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
op = PyObject_New(struct pysocket_op, &pysocket_op_type);
if (op == NULL)
return (NULL);
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->eof = 0;
op->self = op;
op->type = type;
op->timer = NULL;
op->socket = sock;
op->coro = coro_running;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
coro_running->sockop = op;
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
Py_INCREF(op->socket);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
switch (type) {
case PYSOCKET_TYPE_RECV:
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
case PYSOCKET_TYPE_RECVMSG:
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
case PYSOCKET_TYPE_RECVFROM:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sock->recvop = op;
kore_buf_init(&op->buffer, len);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
break;
case PYSOCKET_TYPE_SEND:
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
case PYSOCKET_TYPE_SENDTO:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sock->sendop = op;
kore_buf_init(&op->buffer, len);
kore_buf_append(&op->buffer, ptr, len);
kore_buf_reset(&op->buffer);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
break;
case PYSOCKET_TYPE_ACCEPT:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sock->recvop = op;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
break;
case PYSOCKET_TYPE_CONNECT:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sock->sendop = op;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
break;
default:
fatal("unknown pysocket_op type %u", type);
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (sock->scheduled == 0) {
sock->scheduled = 1;
kore_platform_event_all(sock->fd, &sock->event);
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
return ((PyObject *)op);
}
static PyObject *
pysocket_op_await(PyObject *obj)
{
Py_INCREF(obj);
return (obj);
}
static PyObject *
pysocket_op_iternext(struct pysocket_op *op)
{
PyObject *ret;
If fd is -1 in pysock_op_iternext just return None. None in pysock signals EOF and if it was closed under our feet we should relay that properly.
2019-03-21 14:44:41 +01:00
if (op->socket->fd == -1) {
PyErr_SetNone(PyExc_StopIteration);
return (NULL);
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->eof) {
if (op->coro->exception != NULL) {
PyErr_SetString(op->coro->exception,
op->coro->exception_msg);
op->coro->exception = NULL;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
return (NULL);
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->type != PYSOCKET_TYPE_RECV) {
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
PyErr_SetString(PyExc_RuntimeError, "socket EOF");
return (NULL);
}
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
/* Drain the recv socket. */
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->socket->event.evt.flags |= KORE_EVENT_READ;
several python improvements. - add kore.time() as equivalent for kore_time_ms(). - call waitpid() until no more children are available for reaping otherwise we risk missing a process if several die at the same time and only one SIGCHLD is delivered to us. - drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
return (pysocket_async_recv(op));
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
switch (op->type) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
case PYSOCKET_TYPE_CONNECT:
ret = pysocket_async_connect(op);
break;
case PYSOCKET_TYPE_ACCEPT:
ret = pysocket_async_accept(op);
break;
case PYSOCKET_TYPE_RECV:
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
case PYSOCKET_TYPE_RECVMSG:
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
case PYSOCKET_TYPE_RECVFROM:
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
ret = pysocket_async_recv(op);
break;
case PYSOCKET_TYPE_SEND:
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
case PYSOCKET_TYPE_SENDTO:
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
ret = pysocket_async_send(op);
break;
default:
PyErr_SetString(PyExc_RuntimeError, "invalid op type");
return (NULL);
}
return (ret);
}
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
static void
pysocket_op_timeout(void *arg, u_int64_t now)
{
struct pysocket_op *op = arg;
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->eof = 1;
op->timer = NULL;
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->coro->exception = PyExc_TimeoutError;
op->coro->exception_msg = "timeout before operation completed";
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->coro->request != NULL)
http_request_wakeup(op->coro->request);
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
else
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
python_coro_wakeup(op->coro);
Add an optional timeout to socketop.recv().
2019-02-25 10:35:00 +01:00
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
static PyObject *
pysocket_async_connect(struct pysocket_op *op)
{
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (connect(op->socket->fd, (struct sockaddr *)&op->socket->addr,
op->socket->addr_len) == -1) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (errno != EALREADY && errno != EINPROGRESS &&
Check for EAGAIN when calling connect(2). Linux does this under certain conditions.
2019-03-12 20:04:08 +01:00
errno != EISCONN && errno != EAGAIN) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
}
if (errno != EISCONN) {
Py_RETURN_NONE;
}
}
PyErr_SetNone(PyExc_StopIteration);
return (NULL);
}
static PyObject *
pysocket_async_accept(struct pysocket_op *op)
{
int fd;
struct pysocket *sock;
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (!(op->socket->event.evt.flags & KORE_EVENT_READ)) {
Py_RETURN_NONE;
}
if ((sock = pysocket_alloc()) == NULL)
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
return (NULL);
sock->addr_len = sizeof(sock->addr);
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if ((fd = accept(op->socket->fd,
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
(struct sockaddr *)&sock->addr, &sock->addr_len)) == -1) {
Py_DECREF((PyObject *)sock);
if (errno == EAGAIN || errno == EWOULDBLOCK) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->socket->event.evt.flags &= ~KORE_EVENT_READ;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Py_RETURN_NONE;
}
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
}
if (!kore_connection_nonblock(fd, 0)) {
Py_DECREF((PyObject *)sock);
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
}
sock->fd = fd;
sock->socket = NULL;
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sock->family = op->socket->family;
sock->protocol = op->socket->protocol;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyErr_SetObject(PyExc_StopIteration, (PyObject *)sock);
Py_DECREF((PyObject *)sock);
return (NULL);
}
static PyObject *
pysocket_async_recv(struct pysocket_op *op)
{
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
ssize_t ret;
size_t len;
u_int16_t port;
struct iovec iov;
struct msghdr msg;
socklen_t socklen;
struct sockaddr *sendaddr;
const char *ptr, *ip;
u_int8_t ancdata[1024];
PyObject *bytes, *result, *tuple, *list;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (!(op->socket->event.evt.flags & KORE_EVENT_READ)) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Py_RETURN_NONE;
}
Set socklen to 0 by default in pysocket_async_recv().
2022-02-20 21:30:17 +01:00
socklen = 0;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
for (;;) {
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
switch (op->type) {
case PYSOCKET_TYPE_RECV:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
ret = read(op->socket->fd, op->buffer.data,
op->buffer.length);
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
break;
case PYSOCKET_TYPE_RECVMSG:
memset(&msg, 0, sizeof(msg));
iov.iov_base = op->buffer.data;
iov.iov_len = op->buffer.length;
msg.msg_iov = &iov;
msg.msg_iovlen = 1;
msg.msg_name = &op->sendaddr;
msg.msg_namelen = sizeof(op->sendaddr);
msg.msg_control = ancdata;
msg.msg_controllen = sizeof(ancdata);
memset(&op->sendaddr, 0, sizeof(op->sendaddr));
ret = recvmsg(op->socket->fd, &msg, 0);
break;
case PYSOCKET_TYPE_RECVFROM:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sendaddr = (struct sockaddr *)&op->sendaddr;
switch (op->socket->family) {
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
case AF_INET:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
socklen = sizeof(op->sendaddr.ipv4);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
case AF_UNIX:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
socklen = sizeof(op->sendaddr.sun);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
default:
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
fatal("%s: non AF_INET/AF_UNIX", __func__);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
}
simplify af_unix recvfrom/sendto support.
2019-03-19 13:27:50 +01:00
memset(sendaddr, 0, socklen);
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
ret = recvfrom(op->socket->fd, op->buffer.data,
op->buffer.length, 0, sendaddr, &socklen);
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
break;
default:
fatal("%s: unknown type %d", __func__, op->type);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (ret == -1) {
if (errno == EINTR)
continue;
if (errno == EAGAIN || errno == EWOULDBLOCK) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->socket->event.evt.flags &= ~KORE_EVENT_READ;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
Py_RETURN_NONE;
}
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
break;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->coro->exception = NULL;
op->coro->exception_msg = NULL;
kill recv() timer if we have data. also reset any exception that is set when we have successfully read data from a socket.
2019-03-04 16:37:25 +01:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->timer != NULL) {
kore_timer_remove(op->timer);
op->timer = NULL;
kill recv() timer if we have data. also reset any exception that is set when we have successfully read data from a socket.
2019-03-04 16:37:25 +01:00
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->type == PYSOCKET_TYPE_RECV && ret == 0) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyErr_SetNone(PyExc_StopIteration);
return (NULL);
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
ptr = (const char *)op->buffer.data;
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
if ((bytes = PyBytes_FromStringAndSize(ptr, ret)) == NULL)
return (NULL);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
list = NULL;
switch (op->type) {
case PYSOCKET_TYPE_RECV:
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyErr_SetObject(PyExc_StopIteration, bytes);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Py_DECREF(bytes);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
return (NULL);
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
case PYSOCKET_TYPE_RECVMSG:
socklen = msg.msg_namelen;
Two small python improvements. - Decrement bytes count when python_cmsg_to_list() fails. - Use correct define for PYSOCKET_TYPE_RECVFROM.
2021-11-03 15:19:43 +01:00
if ((list = python_cmsg_to_list(&msg)) == NULL) {
Py_DECREF(bytes);
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
return (NULL);
Two small python improvements. - Decrement bytes count when python_cmsg_to_list() fails. - Use correct define for PYSOCKET_TYPE_RECVFROM.
2021-11-03 15:19:43 +01:00
}
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
break;
case PYSOCKET_TYPE_RECVFROM:
break;
default:
fatal("%s: unknown type %d", __func__, op->type);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
switch(op->socket->family) {
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
case AF_INET:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
port = ntohs(op->sendaddr.ipv4.sin_port);
ip = inet_ntoa(op->sendaddr.ipv4.sin_addr);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
Two small python improvements. - Decrement bytes count when python_cmsg_to_list() fails. - Use correct define for PYSOCKET_TYPE_RECVFROM.
2021-11-03 15:19:43 +01:00
if (op->type == PYSOCKET_TYPE_RECVFROM)
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
tuple = Py_BuildValue("(sHN)", ip, port, bytes);
else
tuple = Py_BuildValue("(sHNN)", ip, port, bytes, list);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
case AF_UNIX:
simplify af_unix recvfrom/sendto support.
2019-03-19 13:27:50 +01:00
len = strlen(op->sendaddr.sun.sun_path);
#if defined(__linux__)
if (len == 0 && socklen > 0) {
len = socklen - sizeof(sa_family_t);
op->sendaddr.sun.sun_path[0] = '@';
op->sendaddr.sun.sun_path[len] = '\0';
}
#endif
if (len == 0) {
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
if (op->type == PYSOCKET_TYPE_RECVFROM) {
tuple = Py_BuildValue("(ON)", Py_None, bytes);
} else {
tuple = Py_BuildValue("(ONN)",
Py_None, bytes, list);
}
} else {
if (op->type == PYSOCKET_TYPE_RECVFROM) {
tuple = Py_BuildValue("(sN)",
op->sendaddr.sun.sun_path, bytes);
} else {
tuple = Py_BuildValue("(sNN)",
op->sendaddr.sun.sun_path, bytes, list);
}
add python sendto recvfrom abstract socket support
2019-03-14 08:37:02 +01:00
}
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
default:
Python: Add kore.socket.recvmsg() Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
fatal("%s: non AF_INET/AF_UNIX", __func__);
}
if (tuple == NULL) {
Py_XDECREF(list);
Py_DECREF(bytes);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
return (NULL);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
}
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
result = PyObject_CallFunctionObjArgs(PyExc_StopIteration, tuple, NULL);
if (result == NULL) {
Py_DECREF(tuple);
return (NULL);
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
Py_DECREF(tuple);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
PyErr_SetObject(PyExc_StopIteration, result);
Py_DECREF(result);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
return (NULL);
}
static PyObject *
pysocket_async_send(struct pysocket_op *op)
{
cleanup.
2019-03-01 20:56:03 +01:00
ssize_t ret;
socklen_t socklen;
const struct sockaddr *sendaddr;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (!(op->socket->event.evt.flags & KORE_EVENT_WRITE)) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Py_RETURN_NONE;
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
for (;;) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->type == PYSOCKET_TYPE_SEND) {
ret = write(op->socket->fd,
op->buffer.data + op->buffer.offset,
op->buffer.length - op->buffer.offset);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
} else {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
sendaddr = (const struct sockaddr *)&op->sendaddr;
cleanup.
2019-03-01 20:56:03 +01:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
switch (op->socket->family) {
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
case AF_INET:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
socklen = sizeof(op->sendaddr.ipv4);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
case AF_UNIX:
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
socklen = sizeof(op->sendaddr.sun);
add python sendto recvfrom abstract socket support
2019-03-14 08:37:02 +01:00
#if defined(__linux__)
if (op->sendaddr.sun.sun_path[0] == '@') {
socklen = sizeof(sa_family_t) +
strlen(op->sendaddr.sun.sun_path);
op->sendaddr.sun.sun_path[0] = '\0';
}
#endif
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
break;
default:
cleanup.
2019-03-01 20:56:03 +01:00
fatal("non AF_INET/AF_UNIX in %s", __func__);
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
ret = sendto(op->socket->fd,
op->buffer.data + op->buffer.offset,
op->buffer.length - op->buffer.offset,
support AF_UNIX in recvfrom()/sendto() python sockets
2019-02-28 14:45:56 +01:00
0, sendaddr, socklen);
Support recvfrom()/sendto() on kore python sockets.
2019-01-08 17:49:00 +01:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (ret == -1) {
if (errno == EINTR)
continue;
if (errno == EAGAIN || errno == EWOULDBLOCK) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->socket->event.evt.flags &=
~KORE_EVENT_WRITE;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
Py_RETURN_NONE;
}
PyErr_SetString(PyExc_RuntimeError, errno_s);
return (NULL);
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
break;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->buffer.offset += (size_t)ret;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (op->buffer.offset == op->buffer.length) {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
PyErr_SetNone(PyExc_StopIteration);
return (NULL);
}
Py_RETURN_NONE;
}
static void
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
pysocket_evt_handle(void *arg, int eof)
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
{
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
struct pysocket_event *event = arg;
struct pysocket *socket = event->s;
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Make sure coros are woken up on eof.
2019-03-13 15:45:55 +01:00
if ((eof || (event->evt.flags & KORE_EVENT_READ)) &&
socket->recvop != NULL) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (socket->recvop->coro->request != NULL)
http_request_wakeup(socket->recvop->coro->request);
else
python_coro_wakeup(socket->recvop->coro);
socket->recvop->eof = eof;
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Make sure coros are woken up on eof.
2019-03-13 15:45:55 +01:00
if ((eof || (event->evt.flags & KORE_EVENT_WRITE)) &&
socket->sendop != NULL) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
if (socket->sendop->coro->request != NULL)
http_request_wakeup(socket->sendop->coro->request);
else
python_coro_wakeup(socket->sendop->coro);
socket->sendop->eof = eof;
}
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
}
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
static void
pyqueue_dealloc(struct pyqueue *queue)
{
pyqueue improvements. - cleanup queue if it gets deallocated. - make sure waitables on a queue get removed if their pyqueue_op dies.
2018-10-21 21:58:34 +02:00
struct pyqueue_object *object;
struct pyqueue_waiting *waiting;
while ((object = TAILQ_FIRST(&queue->objects)) != NULL) {
TAILQ_REMOVE(&queue->objects, object, list);
Py_DECREF(object->obj);
kore_pool_put(&queue_object_pool, object);
}
while ((waiting = TAILQ_FIRST(&queue->waiting)) != NULL) {
TAILQ_REMOVE(&queue->waiting, waiting, list);
if (waiting->op != NULL)
waiting->op->waiting = NULL;
kore_pool_put(&queue_wait_pool, waiting);
}
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
PyObject_Del((PyObject *)queue);
}
static PyObject *
pyqueue_pop(struct pyqueue *queue, PyObject *args)
{
struct pyqueue_op *op;
if ((op = PyObject_New(struct pyqueue_op, &pyqueue_op_type)) == NULL)
return (NULL);
op->queue = queue;
pyqueue improvements. - cleanup queue if it gets deallocated. - make sure waitables on a queue get removed if their pyqueue_op dies.
2018-10-21 21:58:34 +02:00
op->waiting = kore_pool_get(&queue_wait_pool);
op->waiting->op = op;
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
pyqueue improvements. - cleanup queue if it gets deallocated. - make sure waitables on a queue get removed if their pyqueue_op dies.
2018-10-21 21:58:34 +02:00
op->waiting->coro = coro_running;
TAILQ_INSERT_TAIL(&queue->waiting, op->waiting, list);
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
Py_INCREF((PyObject *)queue);
return ((PyObject *)op);
}
Give pyqueues "popnow".
2018-10-22 08:28:03 +02:00
static PyObject *
pyqueue_popnow(struct pyqueue *queue, PyObject *args)
{
PyObject *obj;
struct pyqueue_object *object;
if ((object = TAILQ_FIRST(&queue->objects)) == NULL) {
Py_RETURN_NONE;
}
TAILQ_REMOVE(&queue->objects, object, list);
obj = object->obj;
kore_pool_put(&queue_object_pool, object);
return (obj);
}
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
static PyObject *
pyqueue_push(struct pyqueue *queue, PyObject *args)
{
PyObject *obj;
struct pyqueue_object *object;
struct pyqueue_waiting *waiting;
if (!PyArg_ParseTuple(args, "O", &obj))
return (NULL);
Py_INCREF(obj);
object = kore_pool_get(&queue_object_pool);
object->obj = obj;
TAILQ_INSERT_TAIL(&queue->objects, object, list);
/* Wakeup first in line if any. */
if ((waiting = TAILQ_FIRST(&queue->waiting)) != NULL) {
TAILQ_REMOVE(&queue->waiting, waiting, list);
/* wakeup HTTP request if one is tied. */
if (waiting->coro->request != NULL)
http_request_wakeup(waiting->coro->request);
else
python_coro_wakeup(waiting->coro);
pyqueue improvements. - cleanup queue if it gets deallocated. - make sure waitables on a queue get removed if their pyqueue_op dies.
2018-10-21 21:58:34 +02:00
waiting->op->waiting = NULL;
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
kore_pool_put(&queue_wait_pool, waiting);
}
Py_RETURN_TRUE;
}
static void
pyqueue_op_dealloc(struct pyqueue_op *op)
{
pyqueue improvements. - cleanup queue if it gets deallocated. - make sure waitables on a queue get removed if their pyqueue_op dies.
2018-10-21 21:58:34 +02:00
if (op->waiting != NULL) {
TAILQ_REMOVE(&op->queue->waiting, op->waiting, list);
kore_pool_put(&queue_wait_pool, op->waiting);
op->waiting = NULL;
}
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
Py_DECREF((PyObject *)op->queue);
PyObject_Del((PyObject *)op);
}
static PyObject *
pyqueue_op_await(PyObject *obj)
{
Py_INCREF(obj);
return (obj);
}
static PyObject *
pyqueue_op_iternext(struct pyqueue_op *op)
{
PyObject *obj;
struct pyqueue_object *object;
struct pyqueue_waiting *waiting;
if ((object = TAILQ_FIRST(&op->queue->objects)) == NULL) {
Py_RETURN_NONE;
}
TAILQ_REMOVE(&op->queue->objects, object, list);
obj = object->obj;
kore_pool_put(&queue_object_pool, object);
TAILQ_FOREACH(waiting, &op->queue->waiting, list) {
fix for multiple consumers for the async queues.
2019-01-22 11:23:59 +01:00
if (waiting->coro->id == coro_running->id) {
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
TAILQ_REMOVE(&op->queue->waiting, waiting, list);
fix for multiple consumers for the async queues.
2019-01-22 11:23:59 +01:00
waiting->op->waiting = NULL;
Add an asynchronous queue mechanism. This allows coroutines to submit messages to and pop messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
kore_pool_put(&queue_wait_pool, waiting);
break;
}
}
PyErr_SetObject(PyExc_StopIteration, obj);
Py_DECREF(obj);
return (NULL);
}
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
static void
pylock_dealloc(struct pylock *lock)
{
struct pylock_op *op;
while ((op = TAILQ_FIRST(&lock->ops)) != NULL) {
TAILQ_REMOVE(&lock->ops, op, list);
op->active = 0;
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
op->coro->lockop = NULL;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Py_DECREF((PyObject *)op);
}
fix a few minor problems with pylock. - when pylocks are deallocated we ended up deleting the wrong object. - do not call Py_DECREF on the op when inside the op its iternext call.
2019-03-14 17:31:57 +01:00
PyObject_Del((PyObject *)lock);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
}
add lock.trylock() and make lock.release() sync.
2019-09-20 10:03:43 +02:00
static PyObject *
pylock_trylock(struct pylock *lock, PyObject *args)
{
if (lock->owner != NULL)
Py_RETURN_FALSE;
lock->owner = coro_running;
Py_RETURN_TRUE;
}
static PyObject *
pylock_release(struct pylock *lock, PyObject *args)
{
if (lock->owner == NULL) {
PyErr_SetString(PyExc_RuntimeError, "no lock owner set");
return (NULL);
}
if (lock->owner->id != coro_running->id) {
PyErr_SetString(PyExc_RuntimeError, "lock not owned by caller");
return (NULL);
}
pylock_do_release(lock);
Py_RETURN_NONE;
}
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
static PyObject *
pylock_aenter(struct pylock *lock, PyObject *args)
{
struct pylock_op *op;
fix more fallout from printf format attributes.
2022-08-17 13:19:25 +02:00
if (coro_running->lockop != NULL) {
fatal("%s: lockop not NULL for %" PRIu64,
__func__, coro_running->id);
}
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
if (lock->owner != NULL && lock->owner->id == coro_running->id) {
PyErr_SetString(PyExc_RuntimeError, "recursive lock detected");
return (NULL);
}
if ((op = PyObject_New(struct pylock_op, &pylock_op_type)) == NULL)
return (NULL);
op->active = 1;
op->lock = lock;
op->locking = 1;
op->coro = coro_running;
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
coro_running->lockop = op;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Py_INCREF((PyObject *)op);
Py_INCREF((PyObject *)lock);
TAILQ_INSERT_TAIL(&lock->ops, op, list);
return ((PyObject *)op);
}
static PyObject *
pylock_aexit(struct pylock *lock, PyObject *args)
{
struct pylock_op *op;
fix more fallout from printf format attributes.
2022-08-17 13:19:25 +02:00
if (coro_running->lockop != NULL) {
fatal("%s: lockop not NULL for %" PRIu64,
__func__, coro_running->id);
}
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
if (lock->owner == NULL || lock->owner->id != coro_running->id) {
PyErr_SetString(PyExc_RuntimeError, "invalid lock owner");
return (NULL);
}
if ((op = PyObject_New(struct pylock_op, &pylock_op_type)) == NULL)
return (NULL);
op->active = 1;
op->lock = lock;
op->locking = 0;
op->coro = coro_running;
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
coro_running->lockop = op;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Py_INCREF((PyObject *)op);
Py_INCREF((PyObject *)lock);
TAILQ_INSERT_TAIL(&lock->ops, op, list);
return ((PyObject *)op);
}
static void
pylock_do_release(struct pylock *lock)
{
struct pylock_op *op;
lock->owner = NULL;
TAILQ_FOREACH(op, &lock->ops, list) {
if (op->locking == 0)
continue;
Fix two concurrency issues with kore.lock(). 1) If a coroutine was waken up by another releasing a lock it was waiting on we would incorrectly remove the pylock_op when the newly awoken coroutine hits the iternext for pylock. 2) If a coroutine attempts to grab a lock it was woken up for only to fail we did not properly rearm the coroutine to be woken up later to try again.
2019-05-29 15:25:31 +02:00
op->active = 0;
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
op->coro->lockop = NULL;
TAILQ_REMOVE(&lock->ops, op, list);
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
if (op->coro->request != NULL)
http_request_wakeup(op->coro->request);
else
python_coro_wakeup(op->coro);
Py_DECREF((PyObject *)op);
break;
}
}
static void
pylock_op_dealloc(struct pylock_op *op)
{
if (op->active) {
TAILQ_REMOVE(&op->lock->ops, op, list);
op->active = 0;
}
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
op->coro->lockop = NULL;
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Py_DECREF((PyObject *)op->lock);
PyObject_Del((PyObject *)op);
}
static PyObject *
pylock_op_await(PyObject *obj)
{
Py_INCREF(obj);
return (obj);
}
static PyObject *
pylock_op_iternext(struct pylock_op *op)
{
if (op->locking == 0) {
if (op->lock->owner == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"no lock owner set");
return (NULL);
}
if (op->lock->owner->id != coro_running->id) {
PyErr_SetString(PyExc_RuntimeError,
"lock not owned by caller");
return (NULL);
}
pylock_do_release(op->lock);
} else {
if (op->lock->owner != NULL) {
Fix two concurrency issues with kore.lock(). 1) If a coroutine was waken up by another releasing a lock it was waiting on we would incorrectly remove the pylock_op when the newly awoken coroutine hits the iternext for pylock. 2) If a coroutine attempts to grab a lock it was woken up for only to fail we did not properly rearm the coroutine to be woken up later to try again.
2019-05-29 15:25:31 +02:00
/*
* We could be beat by another coroutine that grabbed
* the lock even if we were the one woken up for it.
*/
if (op->active == 0) {
op->active = 1;
Clear lockop before removing a reference from the coroutine.
2020-08-10 12:19:42 +02:00
op->coro->lockop = op;
Fix two concurrency issues with kore.lock(). 1) If a coroutine was waken up by another releasing a lock it was waiting on we would incorrectly remove the pylock_op when the newly awoken coroutine hits the iternext for pylock. 2) If a coroutine attempts to grab a lock it was woken up for only to fail we did not properly rearm the coroutine to be woken up later to try again.
2019-05-29 15:25:31 +02:00
TAILQ_INSERT_HEAD(&op->lock->ops, op, list);
Py_INCREF((PyObject *)op);
}
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Py_RETURN_NONE;
}
op->lock->owner = coro_running;
}
Fix two concurrency issues with kore.lock(). 1) If a coroutine was waken up by another releasing a lock it was waiting on we would incorrectly remove the pylock_op when the newly awoken coroutine hits the iternext for pylock. 2) If a coroutine attempts to grab a lock it was woken up for only to fail we did not properly rearm the coroutine to be woken up later to try again.
2019-05-29 15:25:31 +02:00
if (op->active) {
op->active = 0;
Python: improve kore.lock when handling cancelled coroutines. If a coroutine is killed from another coroutine and the killed coroutine was waiting on a kore.lock() object, it would have been incorrectly woken up again once said lock was released. This would cause a Python exception that a generator was already running and a crash due to the pool element already being freed. Track the active locking operation per coroutine so we can remove the coroutine if it is killed, fixing the problem.
2020-07-09 20:22:18 +02:00
op->coro->lockop = NULL;
Fix two concurrency issues with kore.lock(). 1) If a coroutine was waken up by another releasing a lock it was waiting on we would incorrectly remove the pylock_op when the newly awoken coroutine hits the iternext for pylock. 2) If a coroutine attempts to grab a lock it was woken up for only to fail we did not properly rearm the coroutine to be woken up later to try again.
2019-05-29 15:25:31 +02:00
TAILQ_REMOVE(&op->lock->ops, op, list);
Py_DECREF((PyObject *)op);
}
Add a locking mechanism in pykore. Support the async with syntax: lock = kore.lock() async with lock: # your block Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
PyErr_SetNone(PyExc_StopIteration);
return (NULL);
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
static void
pyproc_timeout(void *arg, u_int64_t now)
{
struct pyproc *proc = arg;
proc->timer = NULL;
if (proc->coro->sockop != NULL)
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
proc->coro->sockop->eof = 1;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
proc->coro->exception = PyExc_TimeoutError;
proc->coro->exception_msg = "timeout before process exited";
if (proc->coro->request != NULL)
http_request_wakeup(proc->coro->request);
else
python_coro_wakeup(proc->coro);
}
static void
pyproc_dealloc(struct pyproc *proc)
{
int status;
TAILQ_REMOVE(&procs, proc, list);
if (proc->timer != NULL) {
kore_timer_remove(proc->timer);
proc->timer = NULL;
}
if (proc->pid != -1) {
if (kill(proc->pid, SIGKILL) == -1) {
kore_log(LOG_NOTICE,
"kore.proc failed to send SIGKILL %d (%s)",
proc->pid, errno_s);
}
for (;;) {
if (waitpid(proc->pid, &status, 0) == -1) {
if (errno == EINTR)
continue;
kore_log(LOG_NOTICE,
"kore.proc failed to wait for %d (%s)",
proc->pid, errno_s);
}
break;
}
}
if (proc->in != NULL) {
Py_DECREF((PyObject *)proc->in);
proc->in = NULL;
}
if (proc->out != NULL) {
Py_DECREF((PyObject *)proc->out);
proc->out = NULL;
}
PyObject_Del((PyObject *)proc);
}
static PyObject *
pyproc_kill(struct pyproc *proc, PyObject *args)
{
if (proc->pid != -1 && kill(proc->pid, SIGKILL) == -1)
kore_log(LOG_NOTICE, "kill(%d): %s", proc->pid, errno_s);
Py_RETURN_TRUE;
}
static PyObject *
pyproc_reap(struct pyproc *proc, PyObject *args)
{
struct pyproc_op *op;
Make sure we wakeup the coroutine that called proc.reap(). We actually woke up the coroutine that originally spawned the process when we reap it, but another coroutine may have taken over the object. This mimics how we do things for the pysock_op things.
2019-10-29 15:12:20 +01:00
if (proc->op != NULL) {
PyErr_Format(PyExc_RuntimeError,
"process %d already being reaped", proc->apid);
return (NULL);
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (proc->timer != NULL) {
kore_timer_remove(proc->timer);
proc->timer = NULL;
}
if ((op = PyObject_New(struct pyproc_op, &pyproc_op_type)) == NULL)
return (NULL);
op->proc = proc;
Make sure we wakeup the coroutine that called proc.reap(). We actually woke up the coroutine that originally spawned the process when we reap it, but another coroutine may have taken over the object. This mimics how we do things for the pysock_op things.
2019-10-29 15:12:20 +01:00
op->coro = coro_running;
proc->op = op;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
Py_INCREF((PyObject *)proc);
return ((PyObject *)op);
}
static PyObject *
pyproc_recv(struct pyproc *proc, PyObject *args)
{
Add timeout support to proc.recv()
2019-02-26 15:22:55 +01:00
Py_ssize_t len;
struct pysocket_op *op;
PyObject *obj;
int timeo;
timeo = -1;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (proc->out == NULL) {
PyErr_SetString(PyExc_RuntimeError, "stdout closed");
return (NULL);
}
Add timeout support to proc.recv()
2019-02-26 15:22:55 +01:00
if (!PyArg_ParseTuple(args, "n|i", &len, &timeo))
return (NULL);
obj = pysocket_op_create(proc->out, PYSOCKET_TYPE_RECV, NULL, len);
if (obj == NULL)
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
return (NULL);
Add timeout support to proc.recv()
2019-02-26 15:22:55 +01:00
op = (struct pysocket_op *)obj;
if (timeo != -1) {
Rework pysocket async/await. Attach the events directly to the pysocket data structure instead of one event per pysocket_op. Makes the code easier, gives us a good performance boost and reduces the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
op->timer = kore_timer_add(pysocket_op_timeout,
Add timeout support to proc.recv()
2019-02-26 15:22:55 +01:00
timeo, op, KORE_TIMER_ONESHOT);
}
return (obj);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
}
static PyObject *
pyproc_send(struct pyproc *proc, PyObject *args)
{
Py_buffer buf;
PyObject *ret;
if (proc->in == NULL) {
PyErr_SetString(PyExc_RuntimeError, "stdin closed");
return (NULL);
}
if (!PyArg_ParseTuple(args, "y*", &buf))
return (NULL);
ret = pysocket_op_create(proc->in,
PYSOCKET_TYPE_SEND, buf.buf, buf.len);
Python: Improve the kore.socket interface. - Make sure PyBuffer_Release() is called for y* arguments. - Make sure buffer is cleaned up for SENDTO socket ops as well.
2020-08-11 15:24:59 +02:00
PyBuffer_Release(&buf);
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
return (ret);
}
static PyObject *
pyproc_close_stdin(struct pyproc *proc, PyObject *args)
{
if (proc->in != NULL) {
Py_DECREF((PyObject *)proc->in);
proc->in = NULL;
}
Py_RETURN_TRUE;
}
Add kore.proc.pid, returns the PID of the proc.
2019-10-15 14:23:49 +02:00
static PyObject *
pyproc_get_pid(struct pyproc *proc, void *closure)
{
store active pid into apid so it does not overwritten.
2019-10-15 14:25:57 +02:00
return (PyLong_FromLong(proc->apid));
Add kore.proc.pid, returns the PID of the proc.
2019-10-15 14:23:49 +02:00
}
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
static void
pyproc_op_dealloc(struct pyproc_op *op)
{
Py_DECREF((PyObject *)op->proc);
PyObject_Del((PyObject *)op);
}
static PyObject *
pyproc_op_await(PyObject *sop)
{
Py_INCREF(sop);
return (sop);
}
static PyObject *
pyproc_op_iternext(struct pyproc_op *op)
{
int ret;
PyObject *res;
if (op->proc->coro->exception != NULL) {
PyErr_SetString(op->proc->coro->exception,
op->proc->coro->exception_msg);
op->proc->coro->exception = NULL;
return (NULL);
}
if (op->proc->reaped == 0)
Py_RETURN_NONE;
if (WIFSTOPPED(op->proc->status)) {
op->proc->reaped = 0;
Py_RETURN_NONE;
}
if (WIFEXITED(op->proc->status)) {
ret = WEXITSTATUS(op->proc->status);
} else {
ret = op->proc->status;
}
if ((res = PyLong_FromLong(ret)) == NULL)
return (NULL);
PyErr_SetObject(PyExc_StopIteration, res);
Py_DECREF(res);
return (NULL);
}
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
static void
pygather_reap_coro(struct pygather_op *op, struct python_coro *reap)
{
struct pygather_coro *coro;
struct pygather_result *result;
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
#if PY_VERSION_HEX >= 0x030A0000
PyObject *type, *traceback;
#endif
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
TAILQ_FOREACH(coro, &op->coroutines, list) {
if (coro->coro->id == reap->id)
break;
}
if (coro == NULL)
fix more fallout from printf format attributes.
2022-08-17 13:19:25 +02:00
fatal("coroutine %" PRIu64 " not found in gather", reap->id);
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
op->running--;
if (op->running < 0)
fatal("gatherop: running miscount (%d)", op->running);
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
result = kore_pool_get(&gather_result_pool);
result->obj = NULL;
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
#if PY_VERSION_HEX < 0x030A0000
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
if (_PyGen_FetchStopIterationValue(&result->obj) == -1) {
result->obj = Py_None;
Py_INCREF(Py_None);
}
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
#else
if (PyErr_Occurred()) {
Py_XDECREF(coro->coro->result);
PyErr_Fetch(&type, &coro->coro->result, &traceback);
Py_DECREF(type);
Py_XDECREF(traceback);
If no result was available, use Py_None.
2022-08-04 13:19:02 +02:00
} else {
if (coro->coro->result == NULL) {
coro->coro->result = Py_None;
Py_INCREF(Py_None);
}
Python 3.10 improvements. The coroutines results are now relayed back via PyIter_Send() and no longer obtainable via _PyGen_FetchStopIterationValue(). This means that our kore.gather() would not be able to return any values from any of the coroutines it governed. Fix this by saving the object returned in PyIter_Send() and using it later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
}
result->obj = coro->coro->result;
Py_INCREF(result->obj);
#endif
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
TAILQ_INSERT_TAIL(&op->results, result, list);
TAILQ_REMOVE(&op->coroutines, coro, list);
kore_pool_put(&gather_coro_pool, coro);
kore_python_coro_delete(reap);
}
static void
pygather_op_dealloc(struct pygather_op *op)
{
save/restore coro_running in pygather_op_dealloc().
2018-10-30 12:37:30 +01:00
struct python_coro *old;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
struct pygather_coro *coro, *next;
struct pygather_result *res, *rnext;
save/restore coro_running in pygather_op_dealloc().
2018-10-30 12:37:30 +01:00
/*
* Since we are calling kore_python_coro_delete() on all the
* remaining coroutines in this gather op we must remember the
* original coroutine that is running as the removal will end
* up setting coro_running to NULL.
*/
old = coro_running;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
for (coro = TAILQ_FIRST(&op->coroutines); coro != NULL; coro = next) {
next = TAILQ_NEXT(coro, list);
TAILQ_REMOVE(&op->coroutines, coro, list);
/* Make sure we don't end up in pygather_reap_coro(). */
coro->coro->gatherop = NULL;
kore_python_coro_delete(coro->coro);
kore_pool_put(&gather_coro_pool, coro);
}
save/restore coro_running in pygather_op_dealloc().
2018-10-30 12:37:30 +01:00
coro_running = old;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
for (res = TAILQ_FIRST(&op->results); res != NULL; res = rnext) {
rnext = TAILQ_NEXT(res, list);
TAILQ_REMOVE(&op->results, res, list);
Py_DECREF(res->obj);
kore_pool_put(&gather_result_pool, res);
}
PyObject_Del((PyObject *)op);
}
static PyObject *
pygather_op_await(PyObject *obj)
{
Py_INCREF(obj);
return (obj);
}
static PyObject *
pygather_op_iternext(struct pygather_op *op)
{
int idx;
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
struct pygather_coro *coro;
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
struct pygather_result *res, *next;
PyObject *list, *obj;
if (!TAILQ_EMPTY(&op->coroutines)) {
Add a concurrency parameter to kore.gather()
2019-03-25 10:13:52 +01:00
if (op->running > 0)
Py_RETURN_NONE;
TAILQ_FOREACH(coro, &op->coroutines, list) {
if (op->running >= op->concurrency)
break;
python_coro_wakeup(coro->coro);
op->running++;
}
Add kore.gather() to the python api. Allows one to run coroutines concurrently and gather all their results in a single returned list. If any of the coroutines throw an exception the exception is returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
Py_RETURN_NONE;
}
if ((list = PyList_New(op->count)) == NULL)
return (NULL);
idx = 0;
for (res = TAILQ_FIRST(&op->results); res != NULL; res = next) {
next = TAILQ_NEXT(res, list);
TAILQ_REMOVE(&op->results, res, list);
obj = res->obj;
res->obj = NULL;
kore_pool_put(&gather_result_pool, res);
if (PyList_SetItem(list, idx++, obj) != 0) {
Py_DECREF(list);
return (NULL);
}
}
PyErr_SetObject(PyExc_StopIteration, list);
Py_DECREF(list);
return (NULL);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
pyhttp_request_alloc(const struct http_request *req)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
{
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
union { const void *cp; void *p; } ptr;
struct pyhttp_request *pyreq;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
pyreq = PyObject_New(struct pyhttp_request, &pyhttp_request_type);
if (pyreq == NULL)
return (NULL);
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
/*
* Hack around all http apis taking a non-const pointer and us having
* a const pointer for the req data structure. This is because we
* could potentially be called from a validator where the argument
* is a http_request pointer.
*/
ptr.cp = req;
pyreq->req = ptr.p;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
pyreq->data = NULL;
Python HTTP improvements. 1) Add @kore.prerequest python decorator. Using this decorator on a function will cause that function to always be executed *before* any page handler is run. eg: @kore.prerequest def _check(req): if req.method == kore.HTTP_METHOD_POST: req.populate_post() 2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
pyreq->dict = NULL;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return ((PyObject *)pyreq);
}
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
static PyObject *
pyhttp_file_alloc(struct http_file *file)
{
struct pyhttp_file *pyfile;
pyfile = PyObject_New(struct pyhttp_file, &pyhttp_file_type);
if (pyfile == NULL)
return (NULL);
pyfile->file = file;
return ((PyObject *)pyfile);
}
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
static int
pyhttp_preprocess(struct http_request *req)
{
struct reqcall *rq;
PyObject *ret;
rq = req->py_rqnext;
while (rq) {
make sure prereqs are in order
2019-06-07 21:33:00 +02:00
req->py_rqnext = TAILQ_NEXT(rq, list);
Allow kore.prerequests to be async.
2019-06-07 21:06:54 +02:00
PyErr_Clear();
ret = PyObject_CallFunctionObjArgs(rq->f, req->py_req, NULL);
if (ret == NULL) {
kore_python_log_error("preprocess");
http_response(req, HTTP_STATUS_INTERNAL_ERROR, NULL, 0);
return (KORE_RESULT_ERROR);
}
if (ret == Py_False) {
Py_DECREF(ret);
return (KORE_RESULT_ERROR);
}
if (PyCoro_CheckExact(ret)) {
req->py_coro = python_coro_create(ret, req);
if (python_coro_run(req->py_coro) == KORE_RESULT_OK) {
http_request_wakeup(req);
kore_python_coro_delete(req->py_coro);
req->py_coro = NULL;
rq = req->py_rqnext;
continue;
}
return (KORE_RESULT_RETRY);
}
Py_DECREF(ret);
rq = req->py_rqnext;
}
return (KORE_RESULT_OK);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *
pyhttp_response(struct pyhttp_request *pyreq, PyObject *args)
{
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
struct connection *c;
Always use http_response_stream() in req.response. Keep around the python bytes object while we stream the contents of it out over the wire. This avoids an entire copy of the data.
2018-11-14 10:03:32 +01:00
char *ptr;
Py_ssize_t length;
int status;
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
struct pyhttp_iterobj *iterobj;
PyObject *obj, *iterator;
Always use http_response_stream() in req.response. Keep around the python bytes object while we stream the contents of it out over the wire. This avoids an entire copy of the data.
2018-11-14 10:03:32 +01:00
length = -1;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
if (!PyArg_ParseTuple(args, "iO", &status, &obj))
Always use http_response_stream() in req.response. Keep around the python bytes object while we stream the contents of it out over the wire. This avoids an entire copy of the data.
2018-11-14 10:03:32 +01:00
return (NULL);
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
if (PyBytes_CheckExact(obj)) {
if (PyBytes_AsStringAndSize(obj, &ptr, &length) == -1)
return (NULL);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
if (length < 0) {
PyErr_SetString(PyExc_TypeError, "invalid length");
return (NULL);
}
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
Py_INCREF(obj);
Always use http_response_stream() in req.response. Keep around the python bytes object while we stream the contents of it out over the wire. This avoids an entire copy of the data.
2018-11-14 10:03:32 +01:00
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
http_response_stream(pyreq->req, status, ptr, length,
pyhttp_response_sent, obj);
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
} else if (obj == Py_None) {
http_response(pyreq->req, status, NULL, 0);
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
} else {
Improve iterator support for Python req.response(). If the connection on which we are about to send the response was marked as disconnecting, do not go ahead and hook into the disconnect callback (it will never be called, it is already disconnecting). Instead just return, the connection will be removed anyway.
2019-06-05 10:35:47 +02:00
c = pyreq->req->owner;
if (c->state == CONN_STATE_DISCONNECTING) {
Py_RETURN_FALSE;
}
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
if ((iterator = PyObject_GetIter(obj)) == NULL)
return (NULL);
iterobj = kore_pool_get(&iterobj_pool);
iterobj->iterator = iterator;
iterobj->connection = c;
iterobj->remove = 0;
kore_buf_init(&iterobj->buf, 4096);
c->hdlr_extra = iterobj;
c->flags |= CONN_IS_BUSY;
c->disconnect = pyhttp_iterobj_disconnect;
pyreq->req->flags |= HTTP_REQUEST_NO_CONTENT_LENGTH;
http_response_header(pyreq->req, "transfer-encoding",
"chunked");
http_response(pyreq->req, status, NULL, 0);
pyhttp_iterobj_next(iterobj);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
Py_RETURN_TRUE;
}
Always use http_response_stream() in req.response. Keep around the python bytes object while we stream the contents of it out over the wire. This avoids an entire copy of the data.
2018-11-14 10:03:32 +01:00
static int
pyhttp_response_sent(struct netbuf *nb)
{
PyObject *data;
data = nb->extra;
Py_DECREF(data);
return (KORE_RESULT_OK);
}
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
static int
pyhttp_iterobj_next(struct pyhttp_iterobj *iterobj)
{
struct netbuf *nb;
PyObject *obj;
const char *ptr;
Py_ssize_t length;
PyErr_Clear();
if ((obj = PyIter_Next(iterobj->iterator)) == NULL) {
if (PyErr_Occurred()) {
kore_python_log_error("pyhttp_iterobj_next");
return (KORE_RESULT_ERROR);
}
return (KORE_RESULT_OK);
}
if ((ptr = PyUnicode_AsUTF8AndSize(obj, &length)) == NULL) {
kore_python_log_error("pyhttp_iterobj_next");
return (KORE_RESULT_ERROR);
}
kore_buf_reset(&iterobj->buf);
fix more fallout from printf format attributes.
2022-08-17 13:19:25 +02:00
kore_buf_appendf(&iterobj->buf, "%lx\r\n", length);
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
kore_buf_append(&iterobj->buf, ptr, length);
kore_buf_appendf(&iterobj->buf, "\r\n");
Py_DECREF(obj);
net_send_stream(iterobj->connection, iterobj->buf.data,
iterobj->buf.offset, pyhttp_iterobj_chunk_sent, &nb);
nb->extra = iterobj;
return (KORE_RESULT_RETRY);
}
static int
pyhttp_iterobj_chunk_sent(struct netbuf *nb)
{
int ret;
struct pyhttp_iterobj *iterobj;
iterobj = nb->extra;
if (iterobj->remove) {
ret = KORE_RESULT_ERROR;
} else {
ret = pyhttp_iterobj_next(iterobj);
}
if (ret != KORE_RESULT_RETRY) {
iterobj->connection->hdlr_extra = NULL;
iterobj->connection->disconnect = NULL;
iterobj->connection->flags &= ~CONN_IS_BUSY;
if (iterobj->remove == 0)
http_start_recv(iterobj->connection);
kore_buf_reset(&iterobj->buf);
kore_buf_appendf(&iterobj->buf, "0\r\n\r\n");
net_send_queue(iterobj->connection,
iterobj->buf.data, iterobj->buf.offset);
Py_DECREF(iterobj->iterator);
kore_buf_cleanup(&iterobj->buf);
kore_pool_put(&iterobj_pool, iterobj);
} else {
ret = KORE_RESULT_OK;
}
return (ret);
}
static void
pyhttp_iterobj_disconnect(struct connection *c)
{
struct pyhttp_iterobj *iterobj;
iterobj = c->hdlr_extra;
iterobj->remove = 1;
set hdlr_extra to NULL in pyhttp_iterobj_disconnect.
2019-06-12 21:09:57 +02:00
c->hdlr_extra = NULL;
allow an iterator to be passed to req.response(). if an iterator is passed kore will send the response with transfer-encoding: chunked and call the iterator for every chunk that was sent. The iterator must return a utf-8 string. Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *
pyhttp_response_header(struct pyhttp_request *pyreq, PyObject *args)
{
const char *header, *value;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "ss", &header, &value))
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return (NULL);
http_response_header(pyreq->req, header, value);
Py_RETURN_TRUE;
}
static PyObject *
pyhttp_request_header(struct pyhttp_request *pyreq, PyObject *args)
{
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
const char *value;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
const char *header;
PyObject *result;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "s", &header))
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return (NULL);
if (!http_request_header(pyreq->req, header, &value)) {
Py_RETURN_NONE;
}
if ((result = PyUnicode_FromString(value)) == NULL)
return (PyErr_NoMemory());
return (result);
}
static PyObject *
pyhttp_body_read(struct pyhttp_request *pyreq, PyObject *args)
{
ssize_t ret;
size_t len;
Py_ssize_t pylen;
PyObject *result;
u_int8_t buf[1024];
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (!PyArg_ParseTuple(args, "n", &pylen) || pylen < 0)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
return (NULL);
len = (size_t)pylen;
if (len > sizeof(buf)) {
PyErr_SetString(PyExc_RuntimeError, "len > sizeof(buf)");
return (NULL);
}
ret = http_body_read(pyreq->req, buf, len);
if (ret == -1) {
PyErr_SetString(PyExc_RuntimeError, "http_body_read() failed");
return (NULL);
}
Cleanup integer types for # Python arguments. Define PY_SSIZE_T_CLEAN before pulling in Python.h
2021-01-28 13:34:43 +01:00
result = Py_BuildValue("ny#", ret, buf, ret);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
if (result == NULL)
return (PyErr_NoMemory());
return (result);
}
static PyObject *
pyhttp_populate_get(struct pyhttp_request *pyreq, PyObject *args)
{
http_populate_get(pyreq->req);
Py_RETURN_TRUE;
}
static PyObject *
pyhttp_populate_post(struct pyhttp_request *pyreq, PyObject *args)
{
http_populate_post(pyreq->req);
Py_RETURN_TRUE;
}
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
static PyObject *
pyhttp_populate_multi(struct pyhttp_request *pyreq, PyObject *args)
{
http_populate_multipart_form(pyreq->req);
Py_RETURN_TRUE;
}
expose new cookie stuff to python. req.populate_cookies() value = req.cookie("name")
2017-02-07 22:54:42 +01:00
static PyObject *
pyhttp_populate_cookies(struct pyhttp_request *pyreq, PyObject *args)
{
http_populate_cookies(pyreq->req);
Py_RETURN_TRUE;
}
Expose get/post params via req.argument for python.
2017-01-26 12:01:28 +01:00
static PyObject *
pyhttp_argument(struct pyhttp_request *pyreq, PyObject *args)
{
const char *name;
PyObject *value;
char *string;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "s", &name))
Expose get/post params via req.argument for python.
2017-01-26 12:01:28 +01:00
return (NULL);
if (!http_argument_get_string(pyreq->req, name, &string)) {
Py_RETURN_NONE;
}
if ((value = PyUnicode_FromString(string)) == NULL)
return (PyErr_NoMemory());
return (value);
}
expose new cookie stuff to python. req.populate_cookies() value = req.cookie("name")
2017-02-07 22:54:42 +01:00
static PyObject *
pyhttp_cookie(struct pyhttp_request *pyreq, PyObject *args)
{
const char *name;
PyObject *value;
char *string;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "s", &name))
expose new cookie stuff to python. req.populate_cookies() value = req.cookie("name")
2017-02-07 22:54:42 +01:00
return (NULL);
if (!http_request_cookie(pyreq->req, name, &string)) {
Py_RETURN_NONE;
}
if ((value = PyUnicode_FromString(string)) == NULL)
return NULL if PyUnicode_FromString fails.
2019-07-16 21:31:19 +02:00
return (NULL);
expose new cookie stuff to python. req.populate_cookies() value = req.cookie("name")
2017-02-07 22:54:42 +01:00
return (value);
}
Python HTTP API improvement: add req.headers() Calling req.headers() will return all the request headers in a dictionary to the caller.
2022-12-29 12:58:43 +01:00
static PyObject *
pyhttp_headers(struct pyhttp_request *pyreq, PyObject *args)
{
struct http_header *hdr;
struct http_request *req;
PyObject *obj, *dict, *ret;
ret = NULL;
obj = NULL;
dict = NULL;
req = pyreq->req;
if ((dict = PyDict_New()) == NULL)
goto cleanup;
if ((obj = PyUnicode_FromString(req->host)) == NULL)
goto cleanup;
if (PyDict_SetItemString(dict, "host", obj) == -1)
goto cleanup;
TAILQ_FOREACH(hdr, &req->req_headers, list) {
if ((obj = PyUnicode_FromString(hdr->value)) == NULL)
goto cleanup;
if (PyDict_SetItemString(dict, hdr->header, obj) == -1)
goto cleanup;
}
ret = dict;
obj = NULL;
dict = NULL;
cleanup:
Py_XDECREF(obj);
Py_XDECREF(dict);
return (ret);
}
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
static PyObject *
pyhttp_file_lookup(struct pyhttp_request *pyreq, PyObject *args)
{
const char *name;
struct http_file *file;
PyObject *pyfile;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "s", &name))
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
return (NULL);
if ((file = http_file_lookup(pyreq->req, name)) == NULL) {
Py_RETURN_NONE;
}
if ((pyfile = pyhttp_file_alloc(file)) == NULL)
return (PyErr_NoMemory());
return (pyfile);
}
static PyObject *
pyhttp_file_read(struct pyhttp_file *pyfile, PyObject *args)
{
ssize_t ret;
size_t len;
Py_ssize_t pylen;
PyObject *result;
u_int8_t buf[1024];
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (!PyArg_ParseTuple(args, "n", &pylen) || pylen < 0)
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
return (NULL);
len = (size_t)pylen;
if (len > sizeof(buf)) {
PyErr_SetString(PyExc_RuntimeError, "len > sizeof(buf)");
return (NULL);
}
ret = http_file_read(pyfile->file, buf, len);
if (ret == -1) {
PyErr_SetString(PyExc_RuntimeError, "http_file_read() failed");
return (NULL);
}
Cleanup integer types for # Python arguments. Define PY_SSIZE_T_CLEAN before pulling in Python.h
2021-01-28 13:34:43 +01:00
result = Py_BuildValue("ny#", ret, buf, ret);
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
if (result == NULL)
return (PyErr_NoMemory());
return (result);
}
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
static PyObject *
pyhttp_websocket_handshake(struct pyhttp_request *pyreq, PyObject *args)
{
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
struct connection *c;
PyObject *onconnect, *onmsg, *ondisconnect;
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if (!PyArg_ParseTuple(args, "OOO", &onconnect, &onmsg, &ondisconnect))
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
return (NULL);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
kore_websocket_handshake(pyreq->req, NULL, NULL, NULL);
c = pyreq->req->owner;
Py_INCREF(onconnect);
Py_INCREF(onmsg);
Py_INCREF(ondisconnect);
c->ws_connect = kore_calloc(1, sizeof(struct kore_runtime_call));
c->ws_connect->addr = onconnect;
c->ws_connect->runtime = &kore_python_runtime;
c->ws_message = kore_calloc(1, sizeof(struct kore_runtime_call));
c->ws_message->addr = onmsg;
c->ws_message->runtime = &kore_python_runtime;
c->ws_disconnect = kore_calloc(1, sizeof(struct kore_runtime_call));
c->ws_disconnect->addr = ondisconnect;
c->ws_disconnect->runtime = &kore_python_runtime;
python_runtime_connect(onconnect, c);
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
Py_RETURN_TRUE;
}
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
static PyObject *
pyconnection_websocket_send(struct pyconnection *pyc, PyObject *args)
{
Cleanup integer types for # Python arguments. Define PY_SSIZE_T_CLEAN before pulling in Python.h
2021-01-28 13:34:43 +01:00
int op;
ssize_t len;
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
const char *data;
if (pyc->c->proto != CONN_PROTO_WEBSOCKET) {
PyErr_SetString(PyExc_TypeError, "not a websocket connection");
return (NULL);
}
len = -1;
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
if (!PyArg_ParseTuple(args, "iy#", &op, &data, &len))
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
return (NULL);
if (len < 0) {
PyErr_SetString(PyExc_TypeError, "invalid length");
return (NULL);
}
switch (op) {
case WEBSOCKET_OP_TEXT:
case WEBSOCKET_OP_BINARY:
break;
default:
PyErr_SetString(PyExc_TypeError, "invalid op parameter");
return (NULL);
}
kore_websocket_send(pyc->c, op, data, len);
Py_RETURN_TRUE;
}
initial fudging to add websockets to python
2017-01-29 22:57:34 +01:00
static PyObject *
python_websocket_broadcast(PyObject *self, PyObject *args)
{
allow None as source for kore.websocket_broadcast. This allows kore.websocket_broadcast() to be used to relay the messages back to all clients rather then all except the sender.
2017-01-30 21:48:58 +01:00
struct connection *c;
Cleanup integer types for # Python arguments. Define PY_SSIZE_T_CLEAN before pulling in Python.h
2021-01-28 13:34:43 +01:00
ssize_t len;
implement kore_websockat_broadcast.
2017-01-30 20:30:42 +01:00
struct pyconnection *pyc;
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
const char *data;
allow None as source for kore.websocket_broadcast. This allows kore.websocket_broadcast() to be used to relay the messages back to all clients rather then all except the sender.
2017-01-30 21:48:58 +01:00
PyObject *pysrc;
Cleanup integer types for # Python arguments. Define PY_SSIZE_T_CLEAN before pulling in Python.h
2021-01-28 13:34:43 +01:00
int op, broadcast;
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
len = -1;
implement kore_websockat_broadcast.
2017-01-30 20:30:42 +01:00
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
if (!PyArg_ParseTuple(args, "Oiy#i", &pysrc, &op, &data, &len,
don't set our own exception on invalid parameters.
2018-04-11 13:32:56 +02:00
&broadcast))
implement kore_websockat_broadcast.
2017-01-30 20:30:42 +01:00
return (NULL);
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
if (len < 0) {
PyErr_SetString(PyExc_TypeError, "invalid length");
return (NULL);
}
switch (op) {
case WEBSOCKET_OP_TEXT:
case WEBSOCKET_OP_BINARY:
break;
default:
PyErr_SetString(PyExc_TypeError, "invalid op parameter");
return (NULL);
}
allow None as source for kore.websocket_broadcast. This allows kore.websocket_broadcast() to be used to relay the messages back to all clients rather then all except the sender.
2017-01-30 21:48:58 +01:00
if (pysrc == Py_None) {
c = NULL;
} else {
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
if (!PyObject_TypeCheck(pysrc, &pyconnection_type))
allow None as source for kore.websocket_broadcast. This allows kore.websocket_broadcast() to be used to relay the messages back to all clients rather then all except the sender.
2017-01-30 21:48:58 +01:00
return (NULL);
pyc = (struct pyconnection *)pysrc;
c = pyc->c;
}
Move websocket_send() to pyconnection. While here convert all instances of y* for incoming binary data to y# since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
kore_websocket_broadcast(c, op, data, len, broadcast);
expose kore_websocket_send() to python.
2017-01-30 22:00:03 +01:00
Py_RETURN_TRUE;
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *
pyhttp_get_host(struct pyhttp_request *pyreq, void *closure)
{
PyObject *host;
if ((host = PyUnicode_FromString(pyreq->req->host)) == NULL)
return (PyErr_NoMemory());
return (host);
}
static PyObject *
pyhttp_get_path(struct pyhttp_request *pyreq, void *closure)
{
PyObject *path;
if ((path = PyUnicode_FromString(pyreq->req->path)) == NULL)
return (PyErr_NoMemory());
return (path);
}
static PyObject *
pyhttp_get_body(struct pyhttp_request *pyreq, void *closure)
{
ssize_t ret;
struct kore_buf buf;
PyObject *body;
u_int8_t data[BUFSIZ];
kore_buf_init(&buf, 1024);
Set error if http_body_rewind() fails.
2017-03-13 11:19:58 +01:00
if (!http_body_rewind(pyreq->req)) {
PyErr_SetString(PyExc_RuntimeError,
"http_body_rewind() failed");
return (NULL);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
for (;;) {
ret = http_body_read(pyreq->req, data, sizeof(data));
if (ret == -1) {
kore_buf_cleanup(&buf);
PyErr_SetString(PyExc_RuntimeError,
"http_body_read() failed");
return (NULL);
}
if (ret == 0)
break;
kore_buf_append(&buf, data, (size_t)ret);
}
body = PyBytes_FromStringAndSize((char *)buf.data, buf.offset);
kore_buf_free(&buf);
if (body == NULL)
return (PyErr_NoMemory());
return (body);
}
static PyObject *
pyhttp_get_agent(struct pyhttp_request *pyreq, void *closure)
{
Python: Add req.body_digest. Returns the SHA256 digest of the uploaded body as a bytes object.
2021-12-13 10:45:00 +01:00
return (PyUnicode_FromString(pyreq->req->path));
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
static PyObject *
pyhttp_get_method(struct pyhttp_request *pyreq, void *closure)
{
Python: Add req.body_digest. Returns the SHA256 digest of the uploaded body as a bytes object.
2021-12-13 10:45:00 +01:00
return (PyLong_FromUnsignedLong(pyreq->req->method));
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
}
Python: add a protocol member to kore.httprequest This returns a string depending on the protocol used (https / http) for the HTTP request.
2021-12-17 16:52:13 +01:00
static PyObject *
pyhttp_get_protocol(struct pyhttp_request *pyreq, void *closure)
{
struct connection *c;
const char *proto;
c = pyreq->req->owner;
if (c->owner->server->tls)
proto = "https";
else
proto = "http";
return (PyUnicode_FromString(proto));
}
export http_body_path via python
2017-02-22 17:52:57 +01:00
static PyObject *
pyhttp_get_body_path(struct pyhttp_request *pyreq, void *closure)
{
if (pyreq->req->http_body_path == NULL) {
Py_RETURN_NONE;
}
Python: Add req.body_digest. Returns the SHA256 digest of the uploaded body as a bytes object.
2021-12-13 10:45:00 +01:00
return (PyUnicode_FromString(pyreq->req->http_body_path));
}
export http_body_path via python
2017-02-22 17:52:57 +01:00
Python: Add req.body_digest. Returns the SHA256 digest of the uploaded body as a bytes object.
2021-12-13 10:45:00 +01:00
static PyObject *
pyhttp_get_body_digest(struct pyhttp_request *pyreq, void *closure)
{
PyObject *digest;
digest = PyBytes_FromStringAndSize((char *)pyreq->req->http_body_digest,
sizeof(pyreq->req->http_body_digest));
return (digest);
export http_body_path via python
2017-02-22 17:52:57 +01:00
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
static PyObject *
pyhttp_get_connection(struct pyhttp_request *pyreq, void *closure)
{
PyObject *pyc;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if (pyreq->req->owner == NULL) {
Py_RETURN_NONE;
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
if ((pyc = pyconnection_alloc(pyreq->req->owner)) == NULL)
return (PyErr_NoMemory());
return (pyc);
}
expose http_file interface to python.
2017-02-01 21:23:56 +01:00
static PyObject *
pyhttp_file_get_name(struct pyhttp_file *pyfile, void *closure)
{
PyObject *name;
if ((name = PyUnicode_FromString(pyfile->file->name)) == NULL)
return (PyErr_NoMemory());
return (name);
}
static PyObject *
pyhttp_file_get_filename(struct pyhttp_file *pyfile, void *closure)
{
PyObject *name;
if ((name = PyUnicode_FromString(pyfile->file->filename)) == NULL)
return (PyErr_NoMemory());
return (name);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
void
pyroute_dealloc(struct pyroute *route)
{
kore_free(route->path);
Py_XDECREF(route->func);
Py_XDECREF(route->kwargs);
PyObject_Del((PyObject *)route);
}
static PyObject *
pyroute_inner(struct pyroute *route, PyObject *args)
{
PyObject *obj;
if (!PyArg_ParseTuple(args, "O", &obj))
return (NULL);
if (!PyCallable_Check(obj))
return (NULL);
route->func = obj;
Py_INCREF(route->func);
TAILQ_INSERT_TAIL(&routes, route, list);
return (route->func);
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
void
pydomain_dealloc(struct pydomain *domain)
{
PyObject_Del((PyObject *)domain);
}
static int
pydomain_set_accesslog(struct pydomain *domain, PyObject *arg, void *closure)
{
const char *path;
if (!PyUnicode_CheckExact(arg))
return (-1);
if (domain->config->accesslog != -1) {
PyErr_Format(PyExc_RuntimeError,
"domain %s accesslog already set", domain->config->domain);
return (-1);
}
path = PyUnicode_AsUTF8(arg);
domain->config->accesslog = open(path,
O_CREAT | O_APPEND | O_WRONLY,
S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
if (domain->config->accesslog == -1) {
PyErr_Format(PyExc_RuntimeError,
"failed to open accesslog for %s (%s:%s)",
domain->config->domain, path, errno_s);
return (-1);
}
return (0);
}
static PyObject *
pydomain_filemaps(struct pydomain *domain, PyObject *args)
{
Py_ssize_t idx;
Allow authenticators on filemaps. This commit introduces the ability to add authenticators to filemaps. Just like in normal routes, the authenticators will be resolved first before allowing access to the filemap entries. Configuration wise, the authenticator is an optional value after the filemap config directive: filemap / webroot myauth In the Python API you can now pass the authenticator for a filemap entry but turning the value of the filemap into a tuple with the first entry being the path and the second being the auth dict: AUTH AUTH={ "type": "cookie", "value": "cookiename", "redirect": "/auth/", "verify": verify_cookie } domain.filemaps({ "/css/": "webroot/css", "/secret/": ("webroot/secret", AUTH) })
2022-08-10 10:13:01 +02:00
struct kore_route *rt;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
const char *url, *path;
Allow authenticators on filemaps. This commit introduces the ability to add authenticators to filemaps. Just like in normal routes, the authenticators will be resolved first before allowing access to the filemap entries. Configuration wise, the authenticator is an optional value after the filemap config directive: filemap / webroot myauth In the Python API you can now pass the authenticator for a filemap entry but turning the value of the filemap into a tuple with the first entry being the path and the second being the auth dict: AUTH AUTH={ "type": "cookie", "value": "cookiename", "redirect": "/auth/", "verify": verify_cookie } domain.filemaps({ "/css/": "webroot/css", "/secret/": ("webroot/secret", AUTH) })
2022-08-10 10:13:01 +02:00
PyObject *dict, *key, *value, *auth;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if (!PyArg_ParseTuple(args, "O", &dict))
return (NULL);
The *_CheckExact() family sets no exceptions. So set a runtime exception if the objects passed mismatch.
2021-09-22 16:48:21 +02:00
if (!PyDict_CheckExact(dict)) {
PyErr_SetString(PyExc_RuntimeError, "filemaps not a dict");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (NULL);
The *_CheckExact() family sets no exceptions. So set a runtime exception if the objects passed mismatch.
2021-09-22 16:48:21 +02:00
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
idx = 0;
while (PyDict_Next(dict, &idx, &key, &value)) {
Allow authenticators on filemaps. This commit introduces the ability to add authenticators to filemaps. Just like in normal routes, the authenticators will be resolved first before allowing access to the filemap entries. Configuration wise, the authenticator is an optional value after the filemap config directive: filemap / webroot myauth In the Python API you can now pass the authenticator for a filemap entry but turning the value of the filemap into a tuple with the first entry being the path and the second being the auth dict: AUTH AUTH={ "type": "cookie", "value": "cookiename", "redirect": "/auth/", "verify": verify_cookie } domain.filemaps({ "/css/": "webroot/css", "/secret/": ("webroot/secret", AUTH) })
2022-08-10 10:13:01 +02:00
if (!PyUnicode_CheckExact(key)) {
The *_CheckExact() family sets no exceptions. So set a runtime exception if the objects passed mismatch.
2021-09-22 16:48:21 +02:00
PyErr_SetString(PyExc_RuntimeError,
Allow authenticators on filemaps. This commit introduces the ability to add authenticators to filemaps. Just like in normal routes, the authenticators will be resolved first before allowing access to the filemap entries. Configuration wise, the authenticator is an optional value after the filemap config directive: filemap / webroot myauth In the Python API you can now pass the authenticator for a filemap entry but turning the value of the filemap into a tuple with the first entry being the path and the second being the auth dict: AUTH AUTH={ "type": "cookie", "value": "cookiename", "redirect": "/auth/", "verify": verify_cookie } domain.filemaps({ "/css/": "webroot/css", "/secret/": ("webroot/secret", AUTH) })
2022-08-10 10:13:01 +02:00
"filemap key not a string");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (NULL);
}
url = PyUnicode_AsUTF8(key);
Allow authenticators on filemaps. This commit introduces the ability to add authenticators to filemaps. Just like in normal routes, the authenticators will be resolved first before allowing access to the filemap entries. Configuration wise, the authenticator is an optional value after the filemap config directive: filemap / webroot myauth In the Python API you can now pass the authenticator for a filemap entry but turning the value of the filemap into a tuple with the first entry being the path and the second being the auth dict: AUTH AUTH={ "type": "cookie", "value": "cookiename", "redirect": "/auth/", "verify": verify_cookie } domain.filemaps({ "/css/": "webroot/css", "/secret/": ("webroot/secret", AUTH) })
2022-08-10 10:13:01 +02:00
if (!PyUnicode_CheckExact(value) &&
!PyTuple_CheckExact(value)) {
PyErr_SetString(PyExc_RuntimeError,
"filemap value can be either be a string or tuple");
return (NULL);
}
if (PyTuple_CheckExact(value)) {
auth = PyTuple_GetItem(value, 1);
if (!PyDict_CheckExact(auth)) {
PyErr_SetString(PyExc_RuntimeError,
"filemap value tuple auth is not a dict");
return (NULL);
}
value = PyTuple_GetItem(value, 0);
if (!PyUnicode_CheckExact(value)) {
PyErr_SetString(PyExc_RuntimeError,
"filemap value tuple path is invalid");
return (NULL);
}
} else {
auth = NULL;
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
path = PyUnicode_AsUTF8(value);
Allow authenticators on filemaps. This commit introduces the ability to add authenticators to filemaps. Just like in normal routes, the authenticators will be resolved first before allowing access to the filemap entries. Configuration wise, the authenticator is an optional value after the filemap config directive: filemap / webroot myauth In the Python API you can now pass the authenticator for a filemap entry but turning the value of the filemap into a tuple with the first entry being the path and the second being the auth dict: AUTH AUTH={ "type": "cookie", "value": "cookiename", "redirect": "/auth/", "verify": verify_cookie } domain.filemaps({ "/css/": "webroot/css", "/secret/": ("webroot/secret", AUTH) })
2022-08-10 10:13:01 +02:00
rt = kore_filemap_create(domain->config, path, url, NULL);
if (rt == NULL) {
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
PyErr_Format(PyExc_RuntimeError,
"failed to create filemap %s->%s for %s",
url, path, domain->config->domain);
return (NULL);
}
Allow authenticators on filemaps. This commit introduces the ability to add authenticators to filemaps. Just like in normal routes, the authenticators will be resolved first before allowing access to the filemap entries. Configuration wise, the authenticator is an optional value after the filemap config directive: filemap / webroot myauth In the Python API you can now pass the authenticator for a filemap entry but turning the value of the filemap into a tuple with the first entry being the path and the second being the auth dict: AUTH AUTH={ "type": "cookie", "value": "cookiename", "redirect": "/auth/", "verify": verify_cookie } domain.filemaps({ "/css/": "webroot/css", "/secret/": ("webroot/secret", AUTH) })
2022-08-10 10:13:01 +02:00
if (auth != NULL) {
if (!python_route_auth(auth, rt)) {
kore_python_log_error("python_route_auth");
kore_route_free(rt);
return (KORE_RESULT_ERROR);
}
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
Py_RETURN_NONE;
}
Python API domain improvement. Add redirect() method to add a redirect on a domain much like in the Kore configuration file. eg: domain.redirect("^/account/(.*)$", 301, "https://site/account/$1")
2023-03-04 23:15:49 +01:00
static PyObject *
pydomain_redirect(struct pydomain *domain, PyObject *args)
{
int status;
const char *src, *dst;
if (!PyArg_ParseTuple(args, "sis", &src, &status, &dst))
return (NULL);
if (!http_redirect_add(domain->config, src, status, dst)) {
fatal("failed to add redirect '%s' on '%s'",
src, domain->config->domain);
}
Py_RETURN_NONE;
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
static PyObject *
pydomain_route(struct pydomain *domain, PyObject *args, PyObject *kwargs)
{
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
PyObject *obj;
const char *path;
struct pyroute *route;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if (!PyArg_ParseTuple(args, "sO", &path, &obj))
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (NULL);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if (!PyCallable_Check(obj))
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (NULL);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if ((route = PyObject_New(struct pyroute, &pyroute_type)) == NULL)
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (NULL);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
route->kwargs = kwargs;
route->domain = domain->config;
route->path = kore_strdup(path);
Py_XINCREF(route->kwargs);
route->func = obj;
Py_INCREF(route->func);
TAILQ_INSERT_TAIL(&routes, route, list);
Py_RETURN_NONE;
}
static int
python_route_install(struct pyroute *route)
{
const char *val;
struct kore_domain *domain;
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
struct kore_route *rt, *entry;
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
PyObject *kwargs, *repr, *obj;
if ((repr = PyObject_Repr(route->func)) == NULL) {
kore_python_log_error("python_route_install");
return (KORE_RESULT_ERROR);
}
domain = python_route_domain_resolve(route);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt = kore_calloc(1, sizeof(*rt));
rt->dom = domain;
rt->methods = HTTP_METHOD_ALL;
rt->path = kore_strdup(route->path);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
TAILQ_INIT(&rt->params);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
val = PyUnicode_AsUTF8(repr);
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt->func = kore_strdup(val);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
kwargs = route->kwargs;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt->rcall = kore_calloc(1, sizeof(struct kore_runtime_call));
rt->rcall->addr = route->func;
rt->rcall->runtime = &kore_python_runtime;
Py_INCREF(rt->rcall->addr);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if (kwargs != NULL) {
if ((obj = PyDict_GetItemString(kwargs, "methods")) != NULL) {
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
if (!python_route_methods(obj, kwargs, rt)) {
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
kore_python_log_error("python_route_install");
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
kore_route_free(rt);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
return (KORE_RESULT_ERROR);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
}
if ((obj = PyDict_GetItemString(kwargs, "auth")) != NULL) {
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
if (!python_route_auth(obj, rt)) {
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
kore_python_log_error("python_route_install");
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
kore_route_free(rt);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
return (KORE_RESULT_ERROR);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
}
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
if ((obj = PyDict_GetItemString(kwargs, "hooks")) != NULL) {
if (!python_route_hooks(obj, rt)) {
kore_python_log_error("python_route_install");
kore_route_free(rt);
return (KORE_RESULT_ERROR);
}
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
if (rt->path[0] == '/') {
rt->type = HANDLER_TYPE_STATIC;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
} else {
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt->type = HANDLER_TYPE_DYNAMIC;
if (regcomp(&rt->rctx, rt->path, REG_EXTENDED))
fatal("failed to compile regex for '%s'", rt->path);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
TAILQ_FOREACH(entry, &domain->routes, list) {
if (!strcmp(entry->path, rt->path) &&
(entry->methods & rt->methods))
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
fatal("duplicate route for '%s'", route->path);
}
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
TAILQ_INSERT_TAIL(&domain->routes, rt, list);
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
return (KORE_RESULT_OK);
}
static struct kore_domain *
python_route_domain_resolve(struct pyroute *route)
{
struct kore_server *srv;
const char *name;
struct kore_domain *domain;
if (route->domain != NULL)
return (route->domain);
if (route->kwargs != NULL)
name = python_string_from_dict(route->kwargs, "domain");
else
name = NULL;
if (name != NULL) {
domain = NULL;
LIST_FOREACH(srv, &kore_servers, list) {
TAILQ_FOREACH(domain, &srv->domains, list) {
if (!strcmp(domain->domain, name))
break;
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if (domain == NULL)
fatal("domain '%s' does not exist", name);
} else {
if ((domain = kore_domain_byid(1)) != NULL)
fatal("ambiguous domain on route, please specify one");
if ((domain = kore_domain_byid(0)) == NULL)
fatal("no domains configured, please configure one");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
return (domain);
}
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
static int
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
python_route_methods(PyObject *obj, PyObject *kwargs, struct kore_route *rt)
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
{
const char *val;
PyObject *item;
int method;
Py_ssize_t list_len, idx;
The *_CheckExact() family sets no exceptions. So set a runtime exception if the objects passed mismatch.
2021-09-22 16:48:21 +02:00
if (!PyList_CheckExact(obj)) {
PyErr_SetString(PyExc_RuntimeError, "methods not a list");
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
return (KORE_RESULT_ERROR);
The *_CheckExact() family sets no exceptions. So set a runtime exception if the objects passed mismatch.
2021-09-22 16:48:21 +02:00
}
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt->methods = 0;
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
list_len = PyList_Size(obj);
for (idx = 0; idx < list_len; idx++) {
if ((item = PyList_GetItem(obj, idx)) == NULL)
return (KORE_RESULT_ERROR);
if ((val = PyUnicode_AsUTF8(item)) == NULL)
return (KORE_RESULT_ERROR);
if ((method = http_method_value(val)) == 0) {
PyErr_Format(PyExc_RuntimeError,
"unknown HTTP method: %s", val);
return (KORE_RESULT_ERROR);
}
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt->methods |= method;
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
if (method == HTTP_METHOD_GET)
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt->methods |= HTTP_METHOD_HEAD;
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
if (!python_route_params(kwargs, rt, val, method, 0))
Add query string support to the Python validator API. Now you can specify the qs keyword in a route which can contain validators for the query string. Eg: @kore.route("/", methods=["post"], qs={"id": "^[0-9]+$"}) def index: ...
2021-06-18 13:00:57 +02:00
return (KORE_RESULT_ERROR);
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
if (!python_route_params(kwargs, rt, "qs", method, 1))
Major Python API improvements. 1) Add @kore.route as a decorator for Python. This decorator can be used on non-class methods to automatically declare their route and parameters. Takes the same arguments as the kore.domain.route function that exists today. Provides a nice clean way of setting up Kore if you dont want a whole class based approach. 2) Remove the requirement for the name for kore.server() and the kore.domain(attach=) keywords. Instead of no name was given, the name "default" is used in both places resulting in less boilerplating. 3) Allow multiple routes to be defined for the same URI as long as the methods are different. So you can have one method for GET / and another for POST /. All changes combined condense the initial experience of getting a Kore Python app up and running: eg: import kore kore.server(ip="127.0.0.1", port="8888", tls=False) kore.domain("*") @kore.route("/", methods=["get"]) async def index(req): req.response(200, b'get method') @kore.route("/", methods=["post"]) async def index_post(req) req.response(200, b'post method')
2021-05-02 00:23:11 +02:00
return (KORE_RESULT_ERROR);
}
return (KORE_RESULT_OK);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
static int
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
python_route_params(PyObject *kwargs, struct kore_route *rt,
Add query string support to the Python validator API. Now you can specify the qs keyword in a route which can contain validators for the query string. Eg: @kore.route("/", methods=["post"], qs={"id": "^[0-9]+$"}) def index: ...
2021-06-18 13:00:57 +02:00
const char *method, int type, int qs)
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
{
Py_ssize_t idx;
const char *val;
int vtype;
struct kore_validator *vldr;
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
struct kore_route_params *param;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
PyObject *obj, *key, *item;
if ((obj = PyDict_GetItemString(kwargs, method)) == NULL)
return (KORE_RESULT_OK);
if (!PyDict_CheckExact(obj))
return (KORE_RESULT_ERROR);
idx = 0;
while (PyDict_Next(obj, &idx, &key, &item)) {
if (!PyUnicode_CheckExact(key))
return (KORE_RESULT_ERROR);
val = PyUnicode_AsUTF8(key);
if (PyUnicode_CheckExact(item)) {
vtype = KORE_VALIDATOR_TYPE_REGEX;
} else if (PyCallable_Check(item)) {
vtype = KORE_VALIDATOR_TYPE_FUNCTION;
} else {
PyErr_Format(PyExc_RuntimeError,
"validator '%s' must be regex or function", val);
return (KORE_RESULT_ERROR);
}
vldr = kore_calloc(1, sizeof(*vldr));
vldr->type = vtype;
if (vtype == KORE_VALIDATOR_TYPE_REGEX) {
val = PyUnicode_AsUTF8(item);
if (regcomp(&(vldr->rctx),
val, REG_EXTENDED | REG_NOSUB)) {
PyErr_Format(PyExc_RuntimeError,
"Invalid regex (%s)", val);
kore_free(vldr);
return (KORE_RESULT_ERROR);
}
} else {
vldr->rcall = kore_calloc(1, sizeof(*vldr->rcall));
vldr->rcall->addr = item;
vldr->rcall->runtime = &kore_python_runtime;
Py_INCREF(item);
}
val = PyUnicode_AsUTF8(key);
vldr->name = kore_strdup(val);
param = kore_calloc(1, sizeof(*param));
param->flags = 0;
param->method = type;
param->validator = vldr;
param->name = kore_strdup(val);
Add query string support to the Python validator API. Now you can specify the qs keyword in a route which can contain validators for the query string. Eg: @kore.route("/", methods=["post"], qs={"id": "^[0-9]+$"}) def index: ...
2021-06-18 13:00:57 +02:00
if (type == HTTP_METHOD_GET || qs == 1)
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
param->flags = KORE_PARAMS_QUERY_STRING;
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
TAILQ_INSERT_TAIL(&rt->params, param, list);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
}
return (KORE_RESULT_OK);
}
static int
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
python_route_auth(PyObject *dict, struct kore_route *rt)
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
{
int type;
struct kore_auth *auth;
struct kore_validator *vldr;
PyObject *obj, *repr;
const char *value, *redir;
if (!PyDict_CheckExact(dict))
return (KORE_RESULT_ERROR);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if ((value = python_string_from_dict(dict, "type")) == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"missing or invalid 'type' keyword");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (KORE_RESULT_ERROR);
}
if (!strcmp(value, "cookie")) {
type = KORE_AUTH_TYPE_COOKIE;
} else if (!strcmp(value, "header")) {
type = KORE_AUTH_TYPE_HEADER;
} else {
PyErr_Format(PyExc_RuntimeError,
"invalid 'type' (%s) in auth dictionary for '%s'",
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
value, rt->path);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (KORE_RESULT_ERROR);
}
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if ((value = python_string_from_dict(dict, "value")) == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"missing or invalid 'value' keyword");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (KORE_RESULT_ERROR);
}
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
redir = python_string_from_dict(dict, "redirect");
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
if ((obj = PyDict_GetItemString(dict, "verify")) == NULL ||
!PyCallable_Check(obj)) {
PyErr_Format(PyExc_RuntimeError,
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
"missing 'verify' in auth dictionary for '%s'", rt->path);
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (KORE_RESULT_ERROR);
}
auth = kore_calloc(1, sizeof(*auth));
auth->type = type;
auth->value = kore_strdup(value);
if (redir != NULL)
auth->redirect = kore_strdup(redir);
vldr = kore_calloc(1, sizeof(*vldr));
vldr->type = KORE_VALIDATOR_TYPE_FUNCTION;
vldr->rcall = kore_calloc(1, sizeof(*vldr->rcall));
vldr->rcall->addr = obj;
vldr->rcall->runtime = &kore_python_runtime;
Py_INCREF(obj);
if ((repr = PyObject_Repr(obj)) == NULL) {
kore_free(vldr->rcall);
kore_free(vldr);
kore_free(auth);
return (KORE_RESULT_ERROR);
}
value = PyUnicode_AsUTF8(repr);
vldr->name = kore_strdup(value);
Py_DECREF(repr);
auth->validator = vldr;
Change how routes are configured in Kore. Routes are now configured in a context per route: route /path { handler handler_name methods get post head validate qs:get id v_id } All route related configurations are per-route, allowing multiple routes for the same path (for different methods). The param context is removed and merged into the route context now so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
rt->auth = auth;
Many many Python improvements. - Kore can now fully be configured via Python code if one wants nothing to do with configuration files. - Kore can now start single python files and no longer requires them to be inside a module directory. - Pass all regex capture groups to the handler methods, allowing you to get access to them immediately. - Change python websocket_handshake to take callable objects directly. - Added a new deployment configuration option. If set to "dev" or "development" Kore will automatically foreground, no chroot / etc. If set to "production" Kore *will* chroot, drop privs, etc. - Many more.. These are all backported from a project that I was working on a while ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
return (KORE_RESULT_OK);
}
Python: allow route hooks via kore.route(). Adding the hooks keyword with a dictionary attached to specify the relevant hooks will hook them for the given route. Eg: domain.route("/", self.index, methods=["get"], hooks={ "on_free": self.request_free } ) These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
static int
python_route_hooks(PyObject *dict, struct kore_route *rt)
{
if (!PyDict_CheckExact(dict))
return (KORE_RESULT_ERROR);
if (!python_route_hook_set(dict, "on_free", &rt->on_free))
return (KORE_RESULT_ERROR);
if (!python_route_hook_set(dict, "on_headers", &rt->on_headers))
return (KORE_RESULT_ERROR);
if (!python_route_hook_set(dict, "on_body_chunk", &rt->on_body_chunk))
return (KORE_RESULT_ERROR);
return (KORE_RESULT_OK);
}
static int
python_route_hook_set(PyObject *dict, const char *name,
struct kore_runtime_call **out)
{
PyObject *obj;
struct kore_runtime_call *rcall;
if ((obj = PyDict_GetItemString(dict, name)) == NULL)
return (KORE_RESULT_OK);
if (!PyCallable_Check(obj)) {
PyErr_Format(PyExc_RuntimeError,
"%s for a route not callable", name);
Py_DECREF(obj);
return (KORE_RESULT_ERROR);
}
rcall = kore_calloc(1, sizeof(struct kore_runtime_call));
rcall->addr = obj;
rcall->runtime = &kore_python_runtime;
Py_INCREF(rcall->addr);
*out = rcall;
return (KORE_RESULT_OK);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
#if defined(KORE_USE_PGSQL)
static PyObject *
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
python_kore_pgsql_query(PyObject *self, PyObject *args, PyObject *kwargs)
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
{
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
struct pykore_pgsql *op;
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
PyObject *obj;
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
const char *db, *query;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
if (!PyArg_ParseTuple(args, "ss", &db, &query))
return (NULL);
op = PyObject_New(struct pykore_pgsql, &pykore_pgsql_type);
if (op == NULL)
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (NULL);
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
op->binary = 0;
op->param.count = 0;
op->param.objs = NULL;
op->param.values = NULL;
op->param.lengths = NULL;
op->param.formats = NULL;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
op->result = NULL;
op->coro = coro_running;
op->db = kore_strdup(db);
op->query = kore_strdup(query);
op->state = PYKORE_PGSQL_PREINIT;
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
memset(&op->sql, 0, sizeof(op->sql));
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
if (kwargs != NULL) {
if ((obj = PyDict_GetItemString(kwargs, "params")) != NULL) {
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
if (!pykore_pgsql_params(op, obj)) {
Py_DECREF((PyObject *)op);
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
return (NULL);
}
}
if ((obj = PyDict_GetItemString(kwargs, "binary")) != NULL) {
if (obj == Py_True) {
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
op->binary = 1;
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
} else if (obj == Py_False) {
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
op->binary = 0;
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
} else {
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
Py_DECREF((PyObject *)op);
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
PyErr_SetString(PyExc_RuntimeError,
"pgsql: binary not True or False");
return (NULL);
}
}
}
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
return ((PyObject *)op);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
}
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
static int
pykore_pgsql_params(struct pykore_pgsql *op, PyObject *list)
{
union { const char *cp; char *p; } ptr;
PyObject *item;
int format;
Py_ssize_t i, len, vlen;
if (!PyList_CheckExact(list)) {
if (list == Py_None)
return (KORE_RESULT_OK);
PyErr_SetString(PyExc_RuntimeError,
"pgsql: params keyword must be a list");
return (KORE_RESULT_ERROR);
}
len = PyList_Size(list);
if (len == 0)
return (KORE_RESULT_OK);
if (len > INT_MAX) {
PyErr_SetString(PyExc_RuntimeError,
"pgsql: list length too large");
return (KORE_RESULT_ERROR);
}
op->param.count = len;
op->param.lengths = kore_calloc(len, sizeof(int));
op->param.formats = kore_calloc(len, sizeof(int));
op->param.values = kore_calloc(len, sizeof(char *));
op->param.objs = kore_calloc(len, sizeof(PyObject *));
for (i = 0; i < len; i++) {
if ((item = PyList_GetItem(list, i)) == NULL)
return (KORE_RESULT_ERROR);
if (PyUnicode_CheckExact(item)) {
format = 0;
ptr.cp = PyUnicode_AsUTF8AndSize(item, &vlen);
} else if (PyBytes_CheckExact(item)) {
format = 1;
if (PyBytes_AsStringAndSize(item, &ptr.p, &vlen) == -1)
ptr.p = NULL;
} else {
PyErr_Format(PyExc_RuntimeError,
"pgsql: item %zu is not a string or bytes", i);
return (KORE_RESULT_ERROR);
}
if (ptr.cp == NULL)
return (KORE_RESULT_ERROR);
op->param.lengths[i] = vlen;
op->param.values[i] = ptr.cp;
op->param.formats[i] = format;
/* Hold on to it since we are directly referencing its data. */
op->param.objs[i] = item;
Py_INCREF(item);
}
return (KORE_RESULT_OK);
}
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
static void
pykore_pgsql_dealloc(struct pykore_pgsql *pysql)
{
Py_ssize_t i;
kore_free(pysql->db);
kore_free(pysql->query);
kore_pgsql_cleanup(&pysql->sql);
if (pysql->result != NULL)
Py_DECREF(pysql->result);
for (i = 0; i < pysql->param.count; i++)
Py_XDECREF(pysql->param.objs[i]);
kore_free(pysql->param.objs);
kore_free(pysql->param.values);
kore_free(pysql->param.lengths);
kore_free(pysql->param.formats);
PyObject_Del((PyObject *)pysql);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
static PyObject *
pykore_pgsql_iternext(struct pykore_pgsql *pysql)
{
switch (pysql->state) {
Decouple pgsql from the http layer. When the pgsql layer was introduced it was tightly coupled with the http layer in order to make async work fluently. The time has come to split these up and follow the same method we used for tasks, allowing either http requests to be tied to a pgsql data structure or a simple callback function. This also reworks the internal queueing of pgsql requests until connections to the db are available again. The following API functions were changes: - kore_pgsql_query_init() -> kore_pgsql_setup() no longer takes an http_request parameter. - NEW kore_pgsql_init() must be called before operating on an kore_pgsql structure. - NEW kore_pgsql_bind_request() binds an http_request to a kore_pgsql data structure. - NEW kore_pgsql_bind_callback() binds a callback to a kore_pgsql data structure. With all of this you can now build kore with PGSQL=1 NOHTTP=1. The pgsql/ example has been updated to reflect these changes and new features.
2017-03-24 12:53:07 +01:00
case PYKORE_PGSQL_PREINIT:
kore_pgsql_init(&pysql->sql);
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
kore_pgsql_bind_callback(&pysql->sql,
pykore_pgsql_callback, pysql);
Decouple pgsql from the http layer. When the pgsql layer was introduced it was tightly coupled with the http layer in order to make async work fluently. The time has come to split these up and follow the same method we used for tasks, allowing either http requests to be tied to a pgsql data structure or a simple callback function. This also reworks the internal queueing of pgsql requests until connections to the db are available again. The following API functions were changes: - kore_pgsql_query_init() -> kore_pgsql_setup() no longer takes an http_request parameter. - NEW kore_pgsql_init() must be called before operating on an kore_pgsql structure. - NEW kore_pgsql_bind_request() binds an http_request to a kore_pgsql data structure. - NEW kore_pgsql_bind_callback() binds a callback to a kore_pgsql data structure. With all of this you can now build kore with PGSQL=1 NOHTTP=1. The pgsql/ example has been updated to reflect these changes and new features.
2017-03-24 12:53:07 +01:00
pysql->state = PYKORE_PGSQL_INITIALIZE;
/* fallthrough */
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
case PYKORE_PGSQL_INITIALIZE:
Decouple pgsql from the http layer. When the pgsql layer was introduced it was tightly coupled with the http layer in order to make async work fluently. The time has come to split these up and follow the same method we used for tasks, allowing either http requests to be tied to a pgsql data structure or a simple callback function. This also reworks the internal queueing of pgsql requests until connections to the db are available again. The following API functions were changes: - kore_pgsql_query_init() -> kore_pgsql_setup() no longer takes an http_request parameter. - NEW kore_pgsql_init() must be called before operating on an kore_pgsql structure. - NEW kore_pgsql_bind_request() binds an http_request to a kore_pgsql data structure. - NEW kore_pgsql_bind_callback() binds a callback to a kore_pgsql data structure. With all of this you can now build kore with PGSQL=1 NOHTTP=1. The pgsql/ example has been updated to reflect these changes and new features.
2017-03-24 12:53:07 +01:00
if (!kore_pgsql_setup(&pysql->sql, pysql->db,
KORE_PGSQL_ASYNC)) {
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if (pysql->sql.state == KORE_PGSQL_STATE_INIT)
break;
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
PyErr_Format(PyExc_RuntimeError, "pgsql error: %s",
pysql->sql.error);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (NULL);
}
/* fallthrough */
case PYKORE_PGSQL_QUERY:
Python: Only use parameters if needed. We always called kore_pgsql_query_param_fields() regardless if the params keyword was specified or not, instead only use it if actual parameters have been given. Otherwise use the kore_pgsql_query() function directly to execute the query.
2021-09-06 15:39:38 +02:00
if (pysql->param.count > 0) {
if (!kore_pgsql_query_param_fields(&pysql->sql,
pysql->query, pysql->binary,
pysql->param.count, pysql->param.values,
pysql->param.lengths, pysql->param.formats)) {
PyErr_Format(PyExc_RuntimeError,
"pgsql error: %s", pysql->sql.error);
return (NULL);
}
} else {
if (!kore_pgsql_query(&pysql->sql, pysql->query)) {
PyErr_Format(PyExc_RuntimeError,
"pgsql error: %s", pysql->sql.error);
return (NULL);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
}
pysql->state = PYKORE_PGSQL_WAIT;
break;
wait_again:
case PYKORE_PGSQL_WAIT:
switch (pysql->sql.state) {
case KORE_PGSQL_STATE_WAIT:
break;
case KORE_PGSQL_STATE_COMPLETE:
PyErr_SetNone(PyExc_StopIteration);
if (pysql->result != NULL) {
PyErr_SetObject(PyExc_StopIteration,
pysql->result);
Py_DECREF(pysql->result);
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
pysql->result = NULL;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
} else {
PyErr_SetObject(PyExc_StopIteration, Py_None);
}
return (NULL);
case KORE_PGSQL_STATE_ERROR:
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
PyErr_Format(PyExc_RuntimeError,
"failed to perform query: %s", pysql->sql.error);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (NULL);
case KORE_PGSQL_STATE_RESULT:
if (!pykore_pgsql_result(pysql))
return (NULL);
goto wait_again;
default:
Decouple pgsql from the http layer. When the pgsql layer was introduced it was tightly coupled with the http layer in order to make async work fluently. The time has come to split these up and follow the same method we used for tasks, allowing either http requests to be tied to a pgsql data structure or a simple callback function. This also reworks the internal queueing of pgsql requests until connections to the db are available again. The following API functions were changes: - kore_pgsql_query_init() -> kore_pgsql_setup() no longer takes an http_request parameter. - NEW kore_pgsql_init() must be called before operating on an kore_pgsql structure. - NEW kore_pgsql_bind_request() binds an http_request to a kore_pgsql data structure. - NEW kore_pgsql_bind_callback() binds a callback to a kore_pgsql data structure. With all of this you can now build kore with PGSQL=1 NOHTTP=1. The pgsql/ example has been updated to reflect these changes and new features.
2017-03-24 12:53:07 +01:00
kore_pgsql_continue(&pysql->sql);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
goto wait_again;
}
break;
default:
PyErr_SetString(PyExc_RuntimeError, "bad pykore_pgsql state");
return (NULL);
}
/* tell caller to wait. */
Py_RETURN_NONE;
}
python pgsql changes. - decouple pgsql from the HTTP request allowing it to be used in other contexts as well (such as a task, etc). - change names to dbsetup() and dbquery(). eg: result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
static void
pykore_pgsql_callback(struct kore_pgsql *pgsql, void *arg)
{
struct pykore_pgsql *op = arg;
if (op->coro->request != NULL)
http_request_wakeup(op->coro->request);
else
python_coro_wakeup(op->coro);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
static PyObject *
pykore_pgsql_await(PyObject *obj)
{
Py_INCREF(obj);
return (obj);
}
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
static int
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
pykore_pgsql_result(struct pykore_pgsql *pysql)
{
const char *val;
char key[64];
PyObject *list, *pyrow, *pyval;
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
int rows, row, field, fields, len;
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if ((list = PyList_New(0)) == NULL) {
PyErr_SetNone(PyExc_MemoryError);
return (KORE_RESULT_ERROR);
}
rows = kore_pgsql_ntuples(&pysql->sql);
fields = kore_pgsql_nfields(&pysql->sql);
for (row = 0; row < rows; row++) {
if ((pyrow = PyDict_New()) == NULL) {
Py_DECREF(list);
PyErr_SetNone(PyExc_MemoryError);
return (KORE_RESULT_ERROR);
}
for (field = 0; field < fields; field++) {
val = kore_pgsql_getvalue(&pysql->sql, row, field);
Improve pgsql support. - Add kore_pgsql_query_param_fields() which allows you to pass in the arrays for values, lengths and formats yourself. - Add kore_pgsql_column_binary() which will return 1 if the given column index contains a binary result or 0 if it contains a text result. - Change the query call in req.pgsql() for Python to always use the parameterized queries. This adds the 'params' and 'binary' keywords to the req.pgsql method. Eg: result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"), params=["this is my value"])
2019-04-25 23:13:13 +02:00
len = kore_pgsql_getlength(&pysql->sql, row, field);
if (kore_pgsql_column_binary(&pysql->sql, field)) {
pyval = PyBytes_FromStringAndSize(val, len);
} else {
pyval = PyUnicode_FromString(val);
}
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
if (pyval == NULL) {
Py_DECREF(pyrow);
Py_DECREF(list);
PyErr_SetNone(PyExc_MemoryError);
return (KORE_RESULT_ERROR);
}
(void)snprintf(key, sizeof(key), "%s",
kore_pgsql_fieldname(&pysql->sql, field));
if (PyDict_SetItemString(pyrow, key, pyval) == -1) {
Py_DECREF(pyval);
Py_DECREF(pyrow);
Py_DECREF(list);
PyErr_SetString(PyExc_RuntimeError,
"failed to add new value to row");
return (KORE_RESULT_ERROR);
}
Py_DECREF(pyval);
}
if (PyList_Insert(list, row, pyrow) == -1) {
Py_DECREF(pyrow);
Py_DECREF(list);
PyErr_SetString(PyExc_RuntimeError,
"failed to add new row to list");
return (KORE_RESULT_ERROR);
}
Py_DECREF(pyrow);
}
pysql->result = list;
Decouple pgsql from the http layer. When the pgsql layer was introduced it was tightly coupled with the http layer in order to make async work fluently. The time has come to split these up and follow the same method we used for tasks, allowing either http requests to be tied to a pgsql data structure or a simple callback function. This also reworks the internal queueing of pgsql requests until connections to the db are available again. The following API functions were changes: - kore_pgsql_query_init() -> kore_pgsql_setup() no longer takes an http_request parameter. - NEW kore_pgsql_init() must be called before operating on an kore_pgsql structure. - NEW kore_pgsql_bind_request() binds an http_request to a kore_pgsql data structure. - NEW kore_pgsql_bind_callback() binds a callback to a kore_pgsql data structure. With all of this you can now build kore with PGSQL=1 NOHTTP=1. The pgsql/ example has been updated to reflect these changes and new features.
2017-03-24 12:53:07 +01:00
kore_pgsql_continue(&pysql->sql);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
return (KORE_RESULT_OK);
}
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#endif
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
#if defined(KORE_USE_CURL)
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
static PyObject *
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
python_curlopt_set(struct pycurl_data *data, long opt, PyObject *value)
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
{
int i;
for (i = 0; py_curlopt[i].name != NULL; i++) {
if (py_curlopt[i].value == opt)
break;
}
if (py_curlopt[i].name == NULL) {
use correct format specifier.
2021-08-27 10:06:45 +02:00
PyErr_Format(PyExc_RuntimeError, "invalid option '%ld'", opt);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
return (NULL);
}
if (py_curlopt[i].cb == NULL) {
PyErr_Format(PyExc_RuntimeError, "option '%s' not implemented",
py_curlopt[i].name);
return (NULL);
}
return (py_curlopt[i].cb(data, i, value));
}
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
static int
python_curlopt_from_dict(struct pycurl_data *data, PyObject *dict)
{
long opt;
Py_ssize_t idx;
PyObject *key, *value, *obj;
idx = 0;
if (!PyDict_CheckExact(dict)) {
PyErr_SetString(PyExc_RuntimeError,
"curlopt must be a dictionary");
return (KORE_RESULT_ERROR);
}
while (PyDict_Next(dict, &idx, &key, &value)) {
if (!PyLong_CheckExact(key)) {
PyErr_Format(PyExc_RuntimeError,
"invalid key in curlopt keyword");
return (KORE_RESULT_ERROR);
}
opt = PyLong_AsLong(key);
if ((obj = python_curlopt_set(data, opt, value)) == NULL)
return (KORE_RESULT_ERROR);
Py_DECREF(obj);
}
return (KORE_RESULT_OK);
}
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
static PyObject *
python_kore_curl_handle(PyObject *self, PyObject *args)
{
const char *url;
struct pycurl_handle *handle;
if (!PyArg_ParseTuple(args, "s", &url))
return (NULL);
handle = PyObject_New(struct pycurl_handle, &pycurl_handle_type);
if (handle == NULL)
return (NULL);
handle->url = kore_strdup(url);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
memset(&handle->data.curl, 0, sizeof(handle->data.curl));
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
handle->body = NULL;
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
LIST_INIT(&handle->data.slists);
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
if (!kore_curl_init(&handle->data.curl, handle->url, KORE_CURL_ASYNC)) {
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
Py_DECREF((PyObject *)handle);
PyErr_SetString(PyExc_RuntimeError, "failed to setup call");
return (NULL);
}
return ((PyObject *)handle);
}
static void
pycurl_handle_dealloc(struct pycurl_handle *handle)
{
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
struct pycurl_slist *psl;
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
while ((psl = LIST_FIRST(&handle->data.slists))) {
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
LIST_REMOVE(psl, list);
curl_slist_free_all(psl->slist);
kore_free(psl);
}
if (handle->body != NULL)
kore_buf_free(handle->body);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
kore_free(handle->url);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_cleanup(&handle->data.curl);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
PyObject_Del((PyObject *)handle);
}
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
static PyObject *
pycurl_handle_setbody(struct pycurl_handle *handle, PyObject *args)
{
PyObject *obj;
char *ptr;
Py_ssize_t length;
if (!PyArg_ParseTuple(args, "O", &obj))
return (NULL);
if (handle->body != NULL) {
PyErr_SetString(PyExc_RuntimeError,
"curl handle already has body attached");
return (NULL);
}
if (!PyBytes_CheckExact(obj)) {
PyErr_SetString(PyExc_RuntimeError,
"curl.setbody expects bytes");
return (NULL);
}
if (PyBytes_AsStringAndSize(obj, &ptr, &length) == -1)
return (NULL);
if (length < 0) {
PyErr_SetString(PyExc_TypeError, "invalid length");
return (NULL);
}
handle->body = kore_buf_alloc(length);
kore_buf_append(handle->body, ptr, length);
kore_buf_reset(handle->body);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(handle->data.curl.handle,
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
CURLOPT_READFUNCTION, kore_curl_frombuf);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(handle->data.curl.handle,
CURLOPT_READDATA, handle->body);
curl_easy_setopt(handle->data.curl.handle, CURLOPT_UPLOAD, 1);
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
Py_RETURN_TRUE;
}
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
static PyObject *
pycurl_handle_setopt(struct pycurl_handle *handle, PyObject *args)
{
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
int opt;
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
PyObject *value;
if (!PyArg_ParseTuple(args, "iO", &opt, &value))
return (NULL);
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
return (python_curlopt_set(&handle->data, opt, value));
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
}
static PyObject *
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
pycurl_handle_setopt_string(struct pycurl_data *data, int idx, PyObject *obj)
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
{
const char *str;
work around curl 7.87 curl_easy_setopt() being broken.
2022-12-28 16:04:18 +01:00
CURLoption option;
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
if (!PyUnicode_Check(obj)) {
PyErr_Format(PyExc_RuntimeError,
"option '%s' requires a string as argument",
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
py_curlopt[idx].name);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
return (NULL);
}
if ((str = PyUnicode_AsUTF8(obj)) == NULL)
return (NULL);
work around curl 7.87 curl_easy_setopt() being broken.
2022-12-28 16:04:18 +01:00
option = CURLOPTTYPE_OBJECTPOINT + py_curlopt[idx].value;
curl_easy_setopt(data->curl.handle, option, str);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
Py_RETURN_TRUE;
}
static PyObject *
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
pycurl_handle_setopt_long(struct pycurl_data *data, int idx, PyObject *obj)
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
{
long val;
more workarounds for curl 7.87
2022-12-28 16:05:14 +01:00
CURLoption option;
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
if (!PyLong_CheckExact(obj)) {
PyErr_Format(PyExc_RuntimeError,
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
"option '%s' requires a long as argument",
py_curlopt[idx].name);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
return (NULL);
}
PyErr_Clear();
val = PyLong_AsLong(obj);
if (val == -1 && PyErr_Occurred())
return (NULL);
more workarounds for curl 7.87
2022-12-28 16:05:14 +01:00
option = CURLOPTTYPE_LONG + py_curlopt[idx].value;
curl_easy_setopt(data->curl.handle, option, val);
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
Py_RETURN_TRUE;
}
static PyObject *
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
pycurl_handle_setopt_slist(struct pycurl_data *data, int idx, PyObject *obj)
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
{
struct pycurl_slist *psl;
PyObject *item;
const char *sval;
struct curl_slist *slist;
more workarounds for curl 7.87
2022-12-28 16:05:14 +01:00
CURLoption option;
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
Py_ssize_t list_len, i;
if (!PyList_CheckExact(obj)) {
PyErr_Format(PyExc_RuntimeError,
"option '%s' requires a list as argument",
py_curlopt[idx].name);
return (NULL);
}
slist = NULL;
list_len = PyList_Size(obj);
for (i = 0; i < list_len; i++) {
if ((item = PyList_GetItem(obj, i)) == NULL)
return (NULL);
if (!PyUnicode_Check(item))
return (NULL);
if ((sval = PyUnicode_AsUTF8AndSize(item, NULL)) == NULL)
return (NULL);
if ((slist = curl_slist_append(slist, sval)) == NULL)
fatal("%s: curl_slist_append failed", __func__);
}
psl = kore_calloc(1, sizeof(*psl));
psl->slist = slist;
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
LIST_INSERT_HEAD(&data->slists, psl, list);
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
more workarounds for curl 7.87
2022-12-28 16:05:14 +01:00
option = CURLOPTTYPE_OBJECTPOINT + py_curlopt[idx].value;
curl_easy_setopt(data->curl.handle, option, slist);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
Py_RETURN_TRUE;
}
static PyObject *
pycurl_handle_run(struct pycurl_handle *handle, PyObject *args)
{
struct pycurl_handle_op *op;
op = PyObject_New(struct pycurl_handle_op, &pycurl_handle_op_type);
if (op == NULL)
return (NULL);
Py_INCREF(handle);
op->handle = handle;
op->coro = coro_running;
op->state = CURL_CLIENT_OP_RUN;
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_bind_callback(&handle->data.curl,
python_curl_handle_callback, op);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
return ((PyObject *)op);
}
static void
pycurl_handle_op_dealloc(struct pycurl_handle_op *op)
{
Py_DECREF(op->handle);
PyObject_Del((PyObject *)op);
}
static PyObject *
pycurl_handle_op_await(PyObject *op)
{
Py_INCREF(op);
return (op);
}
static PyObject *
pycurl_handle_op_iternext(struct pycurl_handle_op *op)
{
size_t len;
PyObject *result;
const u_int8_t *response;
if (op->state == CURL_CLIENT_OP_RUN) {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_run(&op->handle->data.curl);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
op->state = CURL_CLIENT_OP_RESULT;
Py_RETURN_NONE;
}
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
if (op->handle->body != NULL) {
kore_buf_free(op->handle->body);
op->handle->body = NULL;
}
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
if (!kore_curl_success(&op->handle->data.curl)) {
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
/* Do not log the url here, may contain some sensitive data. */
PyErr_Format(PyExc_RuntimeError, "request failed: %s",
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_strerror(&op->handle->data.curl));
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
return (NULL);
}
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_response_as_bytes(&op->handle->data.curl, &response, &len);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
if ((result = PyBytes_FromStringAndSize((const char *)response,
len)) == NULL)
return (NULL);
PyErr_SetObject(PyExc_StopIteration, result);
Py_DECREF(result);
return (NULL);
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
static PyObject *
python_kore_httpclient(PyObject *self, PyObject *args, PyObject *kwargs)
{
struct pyhttp_client *client;
const char *url, *v;
if (!PyArg_ParseTuple(args, "s", &url))
return (NULL);
client = PyObject_New(struct pyhttp_client, &pyhttp_client_type);
if (client == NULL)
return (NULL);
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
client->unix = NULL;
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
client->tlskey = NULL;
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
client->curlopt = NULL;
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
client->tlscert = NULL;
add cabundle keyword to kore.httpclient()
2019-04-24 21:09:24 +02:00
client->cabundle = NULL;
client->tlsverify = 1;
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
client->url = kore_strdup(url);
if (kwargs != NULL) {
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if ((v = python_string_from_dict(kwargs, "tlscert")) != NULL)
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
client->tlscert = kore_strdup(v);
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if ((v = python_string_from_dict(kwargs, "tlskey")) != NULL)
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
client->tlskey = kore_strdup(v);
add cabundle keyword to kore.httpclient()
2019-04-24 21:09:24 +02:00
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if ((v = python_string_from_dict(kwargs, "cabundle")) != NULL)
add cabundle keyword to kore.httpclient()
2019-04-24 21:09:24 +02:00
client->cabundle = kore_strdup(v);
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
if ((v = python_string_from_dict(kwargs, "unix")) != NULL)
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
client->unix = kore_strdup(v);
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
client->curlopt = PyDict_GetItemString(kwargs, "curlopt");
Py_XINCREF(client->curlopt);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
python_bool_from_dict(kwargs, "tlsverify", &client->tlsverify);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
}
if ((client->tlscert != NULL && client->tlskey == NULL) ||
(client->tlskey != NULL && client->tlscert == NULL)) {
Py_DECREF((PyObject *)client);
PyErr_SetString(PyExc_RuntimeError,
"invalid TLS client configuration");
return (NULL);
}
return ((PyObject *)client);
}
static void
pyhttp_client_dealloc(struct pyhttp_client *client)
{
kore_free(client->url);
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
kore_free(client->unix);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
kore_free(client->tlskey);
kore_free(client->tlscert);
add cabundle keyword to kore.httpclient()
2019-04-24 21:09:24 +02:00
kore_free(client->cabundle);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
Py_XDECREF(client->curlopt);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
PyObject_Del((PyObject *)client);
}
static PyObject *
pyhttp_client_get(struct pyhttp_client *client, PyObject *args,
PyObject *kwargs)
{
return (pyhttp_client_request(client, HTTP_METHOD_GET, kwargs));
}
static PyObject *
pyhttp_client_put(struct pyhttp_client *client, PyObject *args,
PyObject *kwargs)
{
return (pyhttp_client_request(client, HTTP_METHOD_PUT, kwargs));
}
static PyObject *
pyhttp_client_post(struct pyhttp_client *client, PyObject *args,
PyObject *kwargs)
{
return (pyhttp_client_request(client, HTTP_METHOD_POST, kwargs));
}
static PyObject *
pyhttp_client_head(struct pyhttp_client *client, PyObject *args,
PyObject *kwargs)
{
return (pyhttp_client_request(client, HTTP_METHOD_HEAD, kwargs));
}
static PyObject *
pyhttp_client_patch(struct pyhttp_client *client, PyObject *args,
PyObject *kwargs)
{
return (pyhttp_client_request(client, HTTP_METHOD_PATCH, kwargs));
}
static PyObject *
pyhttp_client_delete(struct pyhttp_client *client, PyObject *args,
PyObject *kwargs)
{
return (pyhttp_client_request(client, HTTP_METHOD_DELETE, kwargs));
}
static PyObject *
pyhttp_client_options(struct pyhttp_client *client, PyObject *args,
PyObject *kwargs)
{
return (pyhttp_client_request(client, HTTP_METHOD_OPTIONS, kwargs));
}
static PyObject *
pyhttp_client_request(struct pyhttp_client *client, int m, PyObject *kwargs)
{
struct pyhttp_client_op *op;
char *ptr;
const char *k, *v;
Py_ssize_t length, idx;
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
PyObject *data, *headers, *key, *obj;
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
ptr = NULL;
length = 0;
headers = NULL;
if (kwargs != NULL &&
((headers = PyDict_GetItemString(kwargs, "headers")) != NULL)) {
if (!PyDict_CheckExact(headers)) {
PyErr_SetString(PyExc_RuntimeError,
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
"headers keyword must be a dict");
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
return (NULL);
}
}
switch (m) {
case HTTP_METHOD_GET:
case HTTP_METHOD_HEAD:
case HTTP_METHOD_OPTIONS:
break;
case HTTP_METHOD_PUT:
case HTTP_METHOD_POST:
case HTTP_METHOD_PATCH:
Allow DELETE for kore.httpclient() to have body. The DELETE method could have an HTTP body, so allow it in the kore.httpclient() python call.
2021-12-06 14:16:58 +01:00
case HTTP_METHOD_DELETE:
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
length = -1;
if (kwargs == NULL) {
Allow DELETE for kore.httpclient() to have body. The DELETE method could have an HTTP body, so allow it in the kore.httpclient() python call.
2021-12-06 14:16:58 +01:00
if (m == HTTP_METHOD_DELETE) {
length = 0;
break;
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
PyErr_Format(PyExc_RuntimeError,
"no keyword arguments given, but body expected ",
http_method_text(m));
return (NULL);
}
if ((data = PyDict_GetItemString(kwargs, "body")) == NULL)
return (NULL);
if (PyBytes_AsStringAndSize(data, &ptr, &length) == -1)
return (NULL);
if (length < 0) {
PyErr_SetString(PyExc_TypeError, "invalid length");
return (NULL);
}
break;
default:
fatal("%s: unknown method %d", __func__, m);
}
whitespace fixes
2021-08-27 10:12:11 +02:00
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
op = PyObject_New(struct pyhttp_client_op, &pyhttp_client_op_type);
if (op == NULL)
return (NULL);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
if (!kore_curl_init(&op->data.curl, client->url, KORE_CURL_ASYNC)) {
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
Py_DECREF((PyObject *)op);
PyErr_SetString(PyExc_RuntimeError, "failed to setup call");
return (NULL);
}
op->headers = 0;
op->coro = coro_running;
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
op->state = CURL_CLIENT_OP_RUN;
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
LIST_INIT(&op->data.slists);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
Py_INCREF(client);
Remove memleak from Python httpclient. We grab a reference to the pyhttp_client for the client_op data structure but never removed it. This caused the pyhttp_client object to never be released when out of scope.
2019-06-05 10:27:20 +02:00
op->client = client;
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_http_setup(&op->data.curl, m, ptr, length);
kore_curl_bind_callback(&op->data.curl, python_curl_http_callback, op);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
/* Go in with our own bare hands. */
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
if (client->unix != NULL) {
#if defined(__linux__)
if (client->unix[0] == '@') {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(op->data.curl.handle,
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
CURLOPT_ABSTRACT_UNIX_SOCKET, client->unix + 1);
} else {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(op->data.curl.handle,
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
CURLOPT_UNIX_SOCKET_PATH, client->unix);
}
#else
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(op->data.curl.handle, CURLOPT_UNIX_SOCKET_PATH,
add option to use unix paths via kore.httpclient
2019-05-03 15:31:16 +02:00
client->unix);
#endif
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
if (client->tlskey != NULL && client->tlscert != NULL) {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(op->data.curl.handle, CURLOPT_SSLCERT,
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
client->tlscert);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(op->data.curl.handle, CURLOPT_SSLKEY,
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
client->tlskey);
small python improvements. - Check for an exception when in a timer context and log it if uncaught. - Make sure tlsverify is settable regardless of client certs being specified.
2019-05-16 21:07:42 +02:00
}
add tlsverify keyword to kore.httpclient().
2019-04-24 18:28:23 +02:00
small python improvements. - Check for an exception when in a timer context and log it if uncaught. - Make sure tlsverify is settable regardless of client certs being specified.
2019-05-16 21:07:42 +02:00
if (client->tlsverify == 0) {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(op->data.curl.handle,
CURLOPT_SSL_VERIFYHOST, 0);
curl_easy_setopt(op->data.curl.handle,
CURLOPT_SSL_VERIFYPEER, 0);
}
if (client->curlopt != NULL) {
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
if (!python_curlopt_from_dict(&op->data, client->curlopt)) {
Py_DECREF((PyObject *)op);
return (NULL);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
}
add cabundle keyword to kore.httpclient()
2019-04-24 21:09:24 +02:00
if (client->cabundle != NULL) {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
curl_easy_setopt(op->data.curl.handle, CURLOPT_CAINFO,
add cabundle keyword to kore.httpclient()
2019-04-24 21:09:24 +02:00
client->cabundle);
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
if (headers != NULL) {
idx = 0;
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
while (PyDict_Next(headers, &idx, &key, &obj)) {
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
if ((k = PyUnicode_AsUTF8(key)) == NULL) {
Py_DECREF((PyObject *)op);
return (NULL);
}
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
if ((v = PyUnicode_AsUTF8(obj)) == NULL) {
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
Py_DECREF((PyObject *)op);
return (NULL);
}
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_http_set_header(&op->data.curl, k, v);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
}
}
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
if (kwargs != NULL) {
if ((obj = PyDict_GetItemString(kwargs, "curlopt")) != NULL) {
if (!python_curlopt_from_dict(&op->data, obj)) {
Py_DECREF((PyObject *)op);
return (NULL);
}
}
Allow listening of tls/notls ports at the same time. Before kore needed to be built with NOTLS=1 to be able to do non TLS connections. This has been like this for years. It is time to allow non TLS listeners without having to rebuild Kore. This commit changes your configuration format and will break existing applications their config. Configurations now get listener {} contexts: listen default { bind 127.0.0.1 8888 } The above will create a listener on 127.0.0.1, port 8888 that will serve TLS (still the default). If you want to turn off TLS on that listener, specify "tls no" in that context. Domains now need to be attached to a listener: Eg: domain * { attach default } For the Python API this kills kore.bind(), and kore.bind_unix(). They are replaced with: kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:22:35 +02:00
python_bool_from_dict(kwargs, "return_headers", &op->headers);
Allow curlopt keyword at httpclient request level. Now you can set curlopt on kore.httpclient at both the global httpclient object level and individual requests. Eg: client = kore.httpclient("https://kore.io", curlopt={ kore.CURLOPT_VERBOSE: 1 } ) status, body = await client.get( curlopt={ kore.CURLOPT_VERBOSE: 0 } )
2021-08-27 10:42:40 +02:00
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
return ((PyObject *)op);
}
static void
pyhttp_client_op_dealloc(struct pyhttp_client_op *op)
{
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
struct pycurl_slist *psl;
while ((psl = LIST_FIRST(&op->data.slists))) {
LIST_REMOVE(psl, list);
curl_slist_free_all(psl->slist);
kore_free(psl);
}
Remove memleak from Python httpclient. We grab a reference to the pyhttp_client for the client_op data structure but never removed it. This caused the pyhttp_client object to never be released when out of scope.
2019-06-05 10:27:20 +02:00
Py_DECREF(op->client);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_cleanup(&op->data.curl);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
PyObject_Del((PyObject *)op);
}
static PyObject *
pyhttp_client_op_await(PyObject *op)
{
Py_INCREF(op);
return (op);
}
static PyObject *
pyhttp_client_op_iternext(struct pyhttp_client_op *op)
{
size_t len;
struct http_header *hdr;
const u_int8_t *response;
PyObject *result, *tuple, *dict, *value;
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
if (op->state == CURL_CLIENT_OP_RUN) {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_run(&op->data.curl);
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
op->state = CURL_CLIENT_OP_RESULT;
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
Py_RETURN_NONE;
}
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
if (!kore_curl_success(&op->data.curl)) {
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
PyErr_Format(PyExc_RuntimeError, "request to '%s' failed: %s",
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
op->data.curl.url, kore_curl_strerror(&op->data.curl));
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
return (NULL);
}
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_response_as_bytes(&op->data.curl, &response, &len);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
if (op->headers) {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
kore_curl_http_parse_headers(&op->data.curl);
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
if ((dict = PyDict_New()) == NULL)
return (NULL);
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
TAILQ_FOREACH(hdr, &op->data.curl.http.resp_hdrs, list) {
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
value = PyUnicode_FromString(hdr->value);
if (value == NULL) {
Py_DECREF(dict);
return (NULL);
}
if (PyDict_SetItemString(dict,
hdr->header, value) == -1) {
Py_DECREF(dict);
Py_DECREF(value);
return (NULL);
}
Py_DECREF(value);
}
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
if ((tuple = Py_BuildValue("(iOy#)", op->data.curl.http.status,
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
dict, (const char *)response, len)) == NULL)
return (NULL);
Py_DECREF(dict);
} else {
Add support for setting curlopts in kore.httpclient. Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:00:50 +02:00
if ((tuple = Py_BuildValue("(iy#)", op->data.curl.http.status,
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
(const char *)response, len)) == NULL)
return (NULL);
}
result = PyObject_CallFunctionObjArgs(PyExc_StopIteration, tuple, NULL);
if (result == NULL) {
Py_DECREF(tuple);
return (NULL);
}
Py_DECREF(tuple);
PyErr_SetObject(PyExc_StopIteration, result);
Py_DECREF(result);
return (NULL);
}
static void
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
python_curl_http_callback(struct kore_curl *curl, void *arg)
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
{
struct pyhttp_client_op *op = arg;
if (op->coro->request != NULL)
http_request_wakeup(op->coro->request);
else
python_coro_wakeup(op->coro);
}
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
static void
python_curl_handle_callback(struct kore_curl *curl, void *arg)
{
struct pycurl_handle_op *op = arg;
if (op->coro->request != NULL)
http_request_wakeup(op->coro->request);
else
python_coro_wakeup(op->coro);
}
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
#endif