Commit Graph

1897 Commits

Author SHA1 Message Date
Joris Vink e98a4ddab5 Change how routes are configured in Kore.
Routes are now configured in a context per route:

route /path {
	handler handler_name
	methods get post head
	validate qs:get id v_id
}

All route related configurations are per-route, allowing multiple
routes for the same path (for different methods).

The param context is removed and merged into the route context now
so that you use the validate keyword to specify what needs validating.
2021-09-15 11:09:52 +02:00
Joris Vink 2576427dc0 Simplify if a worker needs to be added to msg.
Just look if the kore_worker data structure its ps member is non NULL.
If it is, it was started by the parent process at some point.
2021-09-14 15:45:34 +02:00
Joris Vink a2d48feeb7 Lets make sure milliseconds are formatted nicely. 2021-09-14 09:36:33 +02:00
Joris Vink 41511c1683 Log timestamps in UTC for, add milliseconds.
This is when using the normal foreground logs or a specified logfile.
2021-09-14 09:30:17 +02:00
Joris Vink b6ec4081d5 Oops, remove debug. 2021-09-13 22:55:09 +02:00
Joris Vink 824d6421d5 Use correct format for strftime(). 2021-09-13 22:54:05 +02:00
Joris Vink 983f5a03f5 Initial mem and log earlier.
Kill the kodev mention.
2021-09-13 15:33:42 +02:00
Joris Vink 450aabbea1 Add timestamp prefix to log when not using syslog. 2021-09-13 15:07:43 +02:00
Joris Vink 77848e0708 Always use logfile when set, even if !foreground. 2021-09-12 15:19:37 +02:00
Joris Vink 4b4767ac39 Missed committing http.h from previous commit. 2021-09-12 15:12:48 +02:00
Joris Vink 862bf1a5f6 Add http_response_json().
Since its an HTTP response function it functions like http_response() but
takes a kore_json_item pointer that it will automatically convert to a kore_buf
and send/free using http_response_stream().

While here fix a problem with http_response_stream() which could end up
not calling the cb() in case of HTTP_METHOD_HEAD. Since the behaviour is
that it should call cb() when done it should do so immediately.
2021-09-12 15:10:06 +02:00
Joris Vink cdd681d602 Let http_response_header() handle duplicates.
If a response header was previously by an application for an HTTP request
http_response_header() will now overwrite the previous value.
2021-09-12 14:30:33 +02:00
Joris Vink eb0b8f21e3 Add http_response_close() to the C API.
This is the same as http_response() except it will automatically
close the connection after the response is sent.

This is a bit easier than setting CONN_CLOSE_EMPTY yourself manually.
2021-09-12 14:13:24 +02:00
Joris Vink b77d727f72 Add a logfile configuration option.
This will log all output from Kore processes to the specified file.
2021-09-10 13:34:57 +02:00
Joris Vink ff6bae6513 move startup log back into kore_server_start(). 2021-09-07 23:26:36 +02:00
Joris Vink a6677b873f On linux, keep track of seccomp tracing properly.
With the new process startup code we must handle the SIGSTOP
from the processes if seccomp_tracing is enabled. Otherwise
they just hang indefinitely and we assume they failed to start,
which is somewhat true.
2021-09-07 23:05:25 +02:00
Joris Vink 1af0772ebe Don't add acme to msg framework if not needed. 2021-09-07 22:27:02 +02:00
Joris Vink 116f935e10 use the correct name for acme. 2021-09-07 22:19:21 +02:00
Joris Vink 9fd30db598 Change timeout for worker startup a bit.
Also give some feedback we are waiting for process startup.
2021-09-07 22:14:28 +02:00
Joris Vink bbae4be6f1 remove unused kore_progname. 2021-09-07 22:06:18 +02:00
Joris Vink 3b20cda11c Rework worker startup/privsep config.
Starting with the privsep config, this commit changes the following:

- Removes the root, runas, keymgr_root, keymgr_runas, acme_root and
  acme_runas configuration options.

  Instead these are now configured via a privsep configuration context:

  privsep worker {
      root /tmp
      runas nobody
  }

  This is also configurable via Python using the new kore.privsep() method:

      kore.privsep("worker", root="/tmp", runas="nobody", skip=["chroot"])

Tied into this we also better handle worker startup:

- Per worker process, wait until it signalled it is ready.
- If a worker fails at startup, display its last log lines more clearly.
- Don't start acme process if no domain requires acme.
- Remove each process its individual startup log message in favour
  of a generalized one that displays its PID, root and user.
- At startup, log the kore version and built-ins in a nicer way.
- The worker processes now check things they need to start running
  before signaling they are ready (such as access to CA certs for
  TLS client authentication).
2021-09-07 21:59:22 +02:00
Joris Vink 1c33ce01d0 Add kore_build_date to version.c 2021-09-07 21:58:53 +02:00
Joris Vink c8c9a24d99 Only set initial python deployment if needed.
We should only be setting this if an actual module was specified
on the command-line that will be loaded.
2021-09-07 21:15:17 +02:00
Joris Vink 599835e7fd Python: Only use parameters if needed.
We always called kore_pgsql_query_param_fields() regardless if the
params keyword was specified or not, instead only use it if actual
parameters have been given.

Otherwise use the kore_pgsql_query() function directly to execute the query.
2021-09-06 15:39:38 +02:00
Joris Vink 0ac54eb48d Add a kore.config.skipchroot to the Python API.
If set to True, will skip the chroot() of the Kore workers.

This can be handy in case you want to set your deployment target to
production or docker so you get user changes but you don't want
to chroot the processes.
2021-09-06 14:35:04 +02:00
Joris Vink 00ef837d62 call explicit fflush() on stdout 2021-09-06 14:16:09 +02:00
Joris Vink 06991d22d5 remove norwegian debug 2021-09-06 13:40:33 +02:00
Joris Vink 7f56c7dbf2 Change how worker processes do logging.
Before each worker process would either directly print to stdout if
Kore was running in foreground mode, or syslog otherwise.

With this commit the workers will submit their log messages to the
parent process who will either put it onto stdout or syslog.

This change in completely under the hood and users shouldn't care about it.
2021-09-06 13:28:38 +02:00
Joris Vink 8661aee2f4 only log key info if !kore_quiet 2021-09-06 13:26:54 +02:00
Joris Vink ebe090833d add Matthew to BEERS 2021-09-05 18:04:54 +02:00
Joris Vink 5f11f796a8 Allow configuration to pickup values from environment.
Eg:

certfile $CERTFILE

will pickup the value from the set $CERTFILE environment variable.

This works for _any_ Kore configuration option.
2021-09-05 17:53:09 +02:00
Joris Vink 3c4acd9ac3 Allow curlopt keyword at httpclient request level.
Now you can set curlopt on kore.httpclient at both the
global httpclient object level and individual requests.

Eg:

client = kore.httpclient("https://kore.io",
    curlopt={
        kore.CURLOPT_VERBOSE: 1
    }
)

status, body = await client.get(
    curlopt={
        kore.CURLOPT_VERBOSE: 0
    }
)
2021-08-27 10:42:40 +02:00
Joris Vink 3eff4b9790 whitespace fixes 2021-08-27 10:12:11 +02:00
Joris Vink 355cf87b93 use correct format specifier. 2021-08-27 10:06:45 +02:00
Joris Vink 55aaef875d Add support for setting curlopts in kore.httpclient.
Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:05:30 +02:00
Joris Vink 89085246e5 style nits 2021-07-10 10:03:01 +02:00
Joris Vink 6d7c774ba2 remove support for older openssl apis in keymgr. 2021-07-10 10:02:53 +02:00
Joris Vink 90056dbdcb make python_module_init() non static 2021-07-10 10:02:46 +02:00
Joris Vink 95139925ec Add query string support to the Python validator API.
Now you can specify the qs keyword in a route which can contain
validators for the query string.

Eg:

@kore.route("/", methods=["post"], qs={"id": "^[0-9]+$"})
def index:
    ...
2021-06-18 13:00:57 +02:00
Joris Vink f1a65ef236 Small improvement to the Python kore.timer() api.
Do not allow kore.timer() to be called from the parent process
as it shouldn't be run there.

This makes Kore fail more gracefully.
2021-06-03 14:03:45 +02:00
Joris Vink f39919e98c Add readlinkat to seccomp whitelist. 2021-05-10 23:05:58 +02:00
Joris Vink d4a78de5fc Add madvise to keymgr seccomp rules. 2021-05-10 14:51:30 +02:00
Joris Vink 12909b98bb be explicit 2021-05-10 10:32:54 +02:00
Joris Vink 61c06291b6 Add a docker python kore.config.deployment setting.
This keeps kore in the foreground will still doing privsep.
Useful with upcoming official kore docker images.
2021-05-10 10:27:32 +02:00
Joris Vink 0abc9b19ff simplify the python-async example 2021-05-02 16:25:46 +02:00
Joris Vink 66dd856bdc and add the echo.py file of course.. 2021-05-02 16:08:32 +02:00
Joris Vink 5e84ebdab2 Simplify the echo example, kore can just run the file nowadays. 2021-05-02 16:07:41 +02:00
Frederic Cambus 582e18d2ec Stop hardcoding HTTP error codes in http_response() calls.
Use predefined HTTP_STATUS_* macros instead.
2021-05-02 16:03:18 +02:00
Joris Vink fb335e1e0c Major Python API improvements.
1) Add @kore.route as a decorator for Python.

This decorator can be used on non-class methods to automatically
declare their route and parameters.

Takes the same arguments as the kore.domain.route function that
exists today.

Provides a nice clean way of setting up Kore if you dont want
a whole class based approach.

2) Remove the requirement for the name for kore.server() and the
kore.domain(attach=) keywords.

Instead of no name was given, the name "default" is used in both
places resulting in less boilerplating.

3) Allow multiple routes to be defined for the same URI as long
as the methods are different. So you can have one method for GET /
and another for POST /.

All changes combined condense the initial experience of getting
a Kore Python app up and running:

eg:

import kore

kore.server(ip="127.0.0.1", port="8888", tls=False)
kore.domain("*")

@kore.route("/", methods=["get"])
async def index(req):
    req.response(200, b'get method')

@kore.route("/", methods=["post"])
async def index_post(req)
    req.response(200, b'post method')
2021-05-02 00:32:47 +02:00
Joris Vink c4a60c54bb resolve tls_dhparam after configure. 2021-04-21 22:39:35 +02:00