Commit Graph

209 Commits

Author SHA1 Message Date
Joris Vink 73be741bfd Allow authenticators on filemaps.
This commit introduces the ability to add authenticators to filemaps.
Just like in normal routes, the authenticators will be resolved first
before allowing access to the filemap entries.

Configuration wise, the authenticator is an optional value after the
filemap config directive:

	filemap / webroot myauth

In the Python API you can now pass the authenticator for a filemap entry
but turning the value of the filemap into a tuple with the first entry
being the path and the second being the auth dict:

	AUTH AUTH={
	    "type": "cookie",
	    "value": "cookiename",
	    "redirect": "/auth/",
	    "verify": verify_cookie
	}

	domain.filemaps({
	    "/css/": "webroot/css",
	    "/secret/": ("webroot/secret", AUTH)
	})
2022-08-10 10:13:01 +02:00
Joris Vink 7316f372d1 If no result was available, use Py_None. 2022-08-04 13:19:02 +02:00
Joris Vink fd94fba3c7 Python 3.10 improvements.
The coroutines results are now relayed back via PyIter_Send() and
no longer obtainable via _PyGen_FetchStopIterationValue().

This means that our kore.gather() would not be able to return any
values from any of the coroutines it governed.

Fix this by saving the object returned in PyIter_Send() and using it
later in pygather_reap_coro().
2022-08-04 09:56:41 +02:00
Joris Vink 30c8652534 More Python 3.10.x changes.
Handle PYGEN_RETURN in case our prerequest handlers return actual
values instead of None.

Mimic PySendResult for older versions for now.
2022-07-14 11:48:32 +02:00
Joris Vink f6c54042c5 Improve Python 3.10.x support.
Python 3.10.x its PyIter_Send() will return PYGEN_ERROR if the
generator returned (as opposed to yielding) if the result returned
is Py_None and no argument was given to PyIter_Send(). Which is counter
intuitive since it seems it should give us a PYGEN_RETURN as per its
documentation.

It also no longer sets PyExc_StopIteration in this case so we cannot depend
on that anymore to figure out if a coroutine has finished.

This caused Kore to respond with a 500 immediately after coroutines
ran to completion as it was looking for the PyExc_StopIteration exception.

Fix this by simply doing another check if an exception happened before
we enter the code path where Kore would spew the 500.
2022-07-14 10:56:27 +02:00
Joris Vink f7a76f7e96 Improvements for kore.recvmsg() in Python.
The cmsghdr contains a length (cmsg_len) which indicates the length
of the data in combination with the cmsghdr length itself.

Remove the length of the cmsghdr before passing it back up to callers
so they don't need to bother with it.

This also fixes a mistake where we ended up copying extra data
from the ancdata buffer that was unintended.
2022-04-22 17:01:06 +02:00
Joris Vink 3516d50be3 adjust log messages 2022-04-20 11:10:01 +02:00
Joris Vink dcdfce5763 Unbreak x509dict.
I accidentally swapped issuer and subject while constructing it.
2022-04-20 11:06:10 +02:00
Joris Vink 5bfd61d136 Hide kore_pymodule behind !KORE_SINGLE_BINARY.
Unbreaks building single binaries with Python support.
2022-03-21 08:52:38 +01:00
Joris Vink 78d667abc7 Set socklen to 0 by default in pysocket_async_recv(). 2022-02-20 21:30:17 +01:00
Joris Vink a65be853f0 Simplify python_push_integer().
We're not doing anything with ret, so kill it.
2022-02-20 21:27:35 +01:00
Joris Vink a7aa51d8d5 Fix unhappy path cleanup.
The whole while (cnt-- >= 0) idiom is busted since cnt started
at 0 and if the first call to PyUnicode_FromStringAndSize() fails
then we're attempting to access -1.
2022-02-20 21:25:18 +01:00
Joris Vink daca80f1cb execve() its envp shouldn't be NULL. 2022-02-20 21:20:14 +01:00
Joris Vink 80383024a3 For each TLS backend let us use correct types. 2022-02-18 10:47:05 +01:00
Joris Vink a9f7bd7faf rename ssl prefixed things to tls. 2022-02-18 10:20:28 +01:00
Joris Vink b20d26e839 Throw exception is TLS backend isnt supported. 2022-02-18 09:17:58 +01:00
Joris Vink feb90208ef Add kore_tls_x509_data().
Use it in the Python code, which requires it.
2022-02-18 09:14:30 +01:00
Joris Vink 833ca646e7 i forgot, it's 2022. 2022-01-31 22:02:06 +01:00
Joris Vink 93a4fe2a15 Worker hook rework.
This commit adds improved hooks for Python and a new signal delivery hook.

For the Python API kore_worker_configure() and kore_worker_teardown() had
to be implemented before this commit. Now one can create a workerstart
and workerend method in their koreapp as those will be called when
they exist.

The new signal hook is either kore_worker_signal() or koreapp.signal.

This new hook is called after the worker event code handles the received
signal itself first.

With this commit there is also a new kore_signal_trap() API call allowing
you to more easily trap new signals. This API also also exported to the
Python part of the code under kore.sigtrap()
2021-12-22 09:50:26 +01:00
Joris Vink ff19ce7652 Python: add a protocol member to kore.httprequest
This returns a string depending on the protocol used (https / http) for
the HTTP request.
2021-12-17 16:52:13 +01:00
Joris Vink a3800fa57e Python: allocate py_req in the http_request_free hook if needed.
Makes it possible to mix on_free with other runtimes.
2021-12-15 12:19:04 +01:00
Joris Vink e8e01980fc Python: allow route hooks via kore.route().
Adding the hooks keyword with a dictionary attached to specify
the relevant hooks will hook them for the given route.

Eg:

domain.route("/", self.index, methods=["get"],
    hooks={
        "on_free": self.request_free
    }
)

These are the same hooks available via a normal Kore route configuration.
2021-12-14 23:15:21 +01:00
Joris Vink 9845c8bbe1 Python: Add req.body_digest.
Returns the SHA256 digest of the uploaded body as a bytes object.
2021-12-13 10:45:00 +01:00
Joris Vink 774cc56ed2 Python: Add an req.connection.x509dict
This dictionary for now only contains the subject and issuer names
from the client certificate (if one was provided) with their
X509_NAME components.

Eg:

{
  "issuer": {
    "C": "SE",
    "O": "kore autogen: x509name-test",
    "CN": "localhost"
  },
  "subject": {
    "C": "SE",
    "O": "kore autogen: x509name-test",
    "CN": "localhost"
  }
}
2021-12-11 22:37:15 +01:00
Joris Vink 6b2609c2b8 Allow DELETE for kore.httpclient() to have body.
The DELETE method could have an HTTP body, so allow it in the
kore.httpclient() python call.
2021-12-06 14:16:58 +01:00
Joris Vink a9ee15bff6 Improve closing of a kore.socket() in Python API.
When a kore.socket() is closed from any coroutine, make sure any other
coroutines waiting on events on the socket are awoken so they properly
can return errors.
2021-12-02 22:47:17 +01:00
Joris Vink 5ac62b17bc Python coro under-the-hood improvements.
- Change python coroutine id to a uint64_t.
- Add kore.task_id() to return active coro its id.
2021-12-02 21:58:13 +01:00
Joris Vink 01e85fd717 Two small python improvements.
- Decrement bytes count when python_cmsg_to_list() fails.
- Use correct define for PYSOCKET_TYPE_RECVFROM.
2021-11-03 15:19:43 +01:00
Joris Vink 3e85d36532 The *_CheckExact() family sets no exceptions.
So set a runtime exception if the objects passed mismatch.
2021-09-22 16:48:21 +02:00
Joris Vink af45284641 count acme domains when configured with Python 2021-09-21 20:47:23 +02:00
Joris Vink e98a4ddab5 Change how routes are configured in Kore.
Routes are now configured in a context per route:

route /path {
	handler handler_name
	methods get post head
	validate qs:get id v_id
}

All route related configurations are per-route, allowing multiple
routes for the same path (for different methods).

The param context is removed and merged into the route context now
so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
Joris Vink 116f935e10 use the correct name for acme. 2021-09-07 22:19:21 +02:00
Joris Vink 3b20cda11c Rework worker startup/privsep config.
Starting with the privsep config, this commit changes the following:

- Removes the root, runas, keymgr_root, keymgr_runas, acme_root and
  acme_runas configuration options.

  Instead these are now configured via a privsep configuration context:

  privsep worker {
      root /tmp
      runas nobody
  }

  This is also configurable via Python using the new kore.privsep() method:

      kore.privsep("worker", root="/tmp", runas="nobody", skip=["chroot"])

Tied into this we also better handle worker startup:

- Per worker process, wait until it signalled it is ready.
- If a worker fails at startup, display its last log lines more clearly.
- Don't start acme process if no domain requires acme.
- Remove each process its individual startup log message in favour
  of a generalized one that displays its PID, root and user.
- At startup, log the kore version and built-ins in a nicer way.
- The worker processes now check things they need to start running
  before signaling they are ready (such as access to CA certs for
  TLS client authentication).
2021-09-07 21:59:22 +02:00
Joris Vink c8c9a24d99 Only set initial python deployment if needed.
We should only be setting this if an actual module was specified
on the command-line that will be loaded.
2021-09-07 21:15:17 +02:00
Joris Vink 599835e7fd Python: Only use parameters if needed.
We always called kore_pgsql_query_param_fields() regardless if the
params keyword was specified or not, instead only use it if actual
parameters have been given.

Otherwise use the kore_pgsql_query() function directly to execute the query.
2021-09-06 15:39:38 +02:00
Joris Vink 3c4acd9ac3 Allow curlopt keyword at httpclient request level.
Now you can set curlopt on kore.httpclient at both the
global httpclient object level and individual requests.

Eg:

client = kore.httpclient("https://kore.io",
    curlopt={
        kore.CURLOPT_VERBOSE: 1
    }
)

status, body = await client.get(
    curlopt={
        kore.CURLOPT_VERBOSE: 0
    }
)
2021-08-27 10:42:40 +02:00
Joris Vink 3eff4b9790 whitespace fixes 2021-08-27 10:12:11 +02:00
Joris Vink 355cf87b93 use correct format specifier. 2021-08-27 10:06:45 +02:00
Joris Vink 55aaef875d Add support for setting curlopts in kore.httpclient.
Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:05:30 +02:00
Joris Vink 90056dbdcb make python_module_init() non static 2021-07-10 10:02:46 +02:00
Joris Vink 95139925ec Add query string support to the Python validator API.
Now you can specify the qs keyword in a route which can contain
validators for the query string.

Eg:

@kore.route("/", methods=["post"], qs={"id": "^[0-9]+$"})
def index:
    ...
2021-06-18 13:00:57 +02:00
Joris Vink f1a65ef236 Small improvement to the Python kore.timer() api.
Do not allow kore.timer() to be called from the parent process
as it shouldn't be run there.

This makes Kore fail more gracefully.
2021-06-03 14:03:45 +02:00
Joris Vink fb335e1e0c Major Python API improvements.
1) Add @kore.route as a decorator for Python.

This decorator can be used on non-class methods to automatically
declare their route and parameters.

Takes the same arguments as the kore.domain.route function that
exists today.

Provides a nice clean way of setting up Kore if you dont want
a whole class based approach.

2) Remove the requirement for the name for kore.server() and the
kore.domain(attach=) keywords.

Instead of no name was given, the name "default" is used in both
places resulting in less boilerplating.

3) Allow multiple routes to be defined for the same URI as long
as the methods are different. So you can have one method for GET /
and another for POST /.

All changes combined condense the initial experience of getting
a Kore Python app up and running:

eg:

import kore

kore.server(ip="127.0.0.1", port="8888", tls=False)
kore.domain("*")

@kore.route("/", methods=["get"])
async def index(req):
    req.response(200, b'get method')

@kore.route("/", methods=["post"])
async def index_post(req)
    req.response(200, b'post method')
2021-05-02 00:32:47 +02:00
Joris Vink 0549295f30 Cleanup integer types for # Python arguments.
Define PY_SSIZE_T_CLEAN before pulling in Python.h
2021-01-28 13:34:43 +01:00
Joris Vink c36a7759f1 Get Kore to build with Python 3.10.0a3. 2021-01-28 10:17:02 +01:00
Joris Vink cef5ac4003 bump copyright year. 2021-01-11 23:46:08 +01:00
Joris Vink cb0f5a4137 Make sure koreapp.onmsg can log errors from Python. 2020-12-18 19:48:17 +01:00
Joris Vink 4313c0eab5 Python: Add kore.socket.recvmsg()
Returns the ancillary data to the caller as a list.
2020-10-02 11:27:52 +02:00
Joris Vink eaef4b654a Only call PyMem_SetupDebugHooks() if DEBUG is 1.
We do not need the memory debug hooks in production environments.
2020-08-13 09:10:19 +02:00
Joris Vink 6b3347ae3a coro_running could be NULL, check it. 2020-08-12 13:43:38 +02:00