Commit Graph

1777 Commits

Author SHA1 Message Date
Joris Vink 3b20cda11c Rework worker startup/privsep config.
Starting with the privsep config, this commit changes the following:

- Removes the root, runas, keymgr_root, keymgr_runas, acme_root and
  acme_runas configuration options.

  Instead these are now configured via a privsep configuration context:

  privsep worker {
      root /tmp
      runas nobody
  }

  This is also configurable via Python using the new kore.privsep() method:

      kore.privsep("worker", root="/tmp", runas="nobody", skip=["chroot"])

Tied into this we also better handle worker startup:

- Per worker process, wait until it signalled it is ready.
- If a worker fails at startup, display its last log lines more clearly.
- Don't start acme process if no domain requires acme.
- Remove each process its individual startup log message in favour
  of a generalized one that displays its PID, root and user.
- At startup, log the kore version and built-ins in a nicer way.
- The worker processes now check things they need to start running
  before signaling they are ready (such as access to CA certs for
  TLS client authentication).
2021-09-07 21:59:22 +02:00
Joris Vink 1c33ce01d0 Add kore_build_date to version.c 2021-09-07 21:58:53 +02:00
Joris Vink c8c9a24d99 Only set initial python deployment if needed.
We should only be setting this if an actual module was specified
on the command-line that will be loaded.
2021-09-07 21:15:17 +02:00
Joris Vink 599835e7fd Python: Only use parameters if needed.
We always called kore_pgsql_query_param_fields() regardless if the
params keyword was specified or not, instead only use it if actual
parameters have been given.

Otherwise use the kore_pgsql_query() function directly to execute the query.
2021-09-06 15:39:38 +02:00
Joris Vink 0ac54eb48d Add a kore.config.skipchroot to the Python API.
If set to True, will skip the chroot() of the Kore workers.

This can be handy in case you want to set your deployment target to
production or docker so you get user changes but you don't want
to chroot the processes.
2021-09-06 14:35:04 +02:00
Joris Vink 00ef837d62 call explicit fflush() on stdout 2021-09-06 14:16:09 +02:00
Joris Vink 06991d22d5 remove norwegian debug 2021-09-06 13:40:33 +02:00
Joris Vink 7f56c7dbf2 Change how worker processes do logging.
Before each worker process would either directly print to stdout if
Kore was running in foreground mode, or syslog otherwise.

With this commit the workers will submit their log messages to the
parent process who will either put it onto stdout or syslog.

This change in completely under the hood and users shouldn't care about it.
2021-09-06 13:28:38 +02:00
Joris Vink 8661aee2f4 only log key info if !kore_quiet 2021-09-06 13:26:54 +02:00
Joris Vink ebe090833d add Matthew to BEERS 2021-09-05 18:04:54 +02:00
Joris Vink 5f11f796a8 Allow configuration to pickup values from environment.
Eg:

certfile $CERTFILE

will pickup the value from the set $CERTFILE environment variable.

This works for _any_ Kore configuration option.
2021-09-05 17:53:09 +02:00
Joris Vink 3c4acd9ac3 Allow curlopt keyword at httpclient request level.
Now you can set curlopt on kore.httpclient at both the
global httpclient object level and individual requests.

Eg:

client = kore.httpclient("https://kore.io",
    curlopt={
        kore.CURLOPT_VERBOSE: 1
    }
)

status, body = await client.get(
    curlopt={
        kore.CURLOPT_VERBOSE: 0
    }
)
2021-08-27 10:42:40 +02:00
Joris Vink 3eff4b9790 whitespace fixes 2021-08-27 10:12:11 +02:00
Joris Vink 355cf87b93 use correct format specifier. 2021-08-27 10:06:45 +02:00
Joris Vink 55aaef875d Add support for setting curlopts in kore.httpclient.
Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:05:30 +02:00
Joris Vink 89085246e5 style nits 2021-07-10 10:03:01 +02:00
Joris Vink 6d7c774ba2 remove support for older openssl apis in keymgr. 2021-07-10 10:02:53 +02:00
Joris Vink 90056dbdcb make python_module_init() non static 2021-07-10 10:02:46 +02:00
Joris Vink 95139925ec Add query string support to the Python validator API.
Now you can specify the qs keyword in a route which can contain
validators for the query string.

Eg:

@kore.route("/", methods=["post"], qs={"id": "^[0-9]+$"})
def index:
    ...
2021-06-18 13:00:57 +02:00
Joris Vink f1a65ef236 Small improvement to the Python kore.timer() api.
Do not allow kore.timer() to be called from the parent process
as it shouldn't be run there.

This makes Kore fail more gracefully.
2021-06-03 14:03:45 +02:00
Joris Vink f39919e98c Add readlinkat to seccomp whitelist. 2021-05-10 23:05:58 +02:00
Joris Vink d4a78de5fc Add madvise to keymgr seccomp rules. 2021-05-10 14:51:30 +02:00
Joris Vink 12909b98bb be explicit 2021-05-10 10:32:54 +02:00
Joris Vink 61c06291b6 Add a docker python kore.config.deployment setting.
This keeps kore in the foreground will still doing privsep.
Useful with upcoming official kore docker images.
2021-05-10 10:27:32 +02:00
Joris Vink 0abc9b19ff simplify the python-async example 2021-05-02 16:25:46 +02:00
Joris Vink 66dd856bdc and add the echo.py file of course.. 2021-05-02 16:08:32 +02:00
Joris Vink 5e84ebdab2 Simplify the echo example, kore can just run the file nowadays. 2021-05-02 16:07:41 +02:00
Frederic Cambus 582e18d2ec Stop hardcoding HTTP error codes in http_response() calls.
Use predefined HTTP_STATUS_* macros instead.
2021-05-02 16:03:18 +02:00
Joris Vink fb335e1e0c Major Python API improvements.
1) Add @kore.route as a decorator for Python.

This decorator can be used on non-class methods to automatically
declare their route and parameters.

Takes the same arguments as the kore.domain.route function that
exists today.

Provides a nice clean way of setting up Kore if you dont want
a whole class based approach.

2) Remove the requirement for the name for kore.server() and the
kore.domain(attach=) keywords.

Instead of no name was given, the name "default" is used in both
places resulting in less boilerplating.

3) Allow multiple routes to be defined for the same URI as long
as the methods are different. So you can have one method for GET /
and another for POST /.

All changes combined condense the initial experience of getting
a Kore Python app up and running:

eg:

import kore

kore.server(ip="127.0.0.1", port="8888", tls=False)
kore.domain("*")

@kore.route("/", methods=["get"])
async def index(req):
    req.response(200, b'get method')

@kore.route("/", methods=["post"])
async def index_post(req)
    req.response(200, b'post method')
2021-05-02 00:32:47 +02:00
Joris Vink c4a60c54bb resolve tls_dhparam after configure. 2021-04-21 22:39:35 +02:00
Joris Vink c77ec598e7 Remove dh parameters from kodev. 2021-04-21 10:52:02 +02:00
Joris Vink cf9e97f087 Improve TLS settings and dependencies.
- Kore now only supports OpenSSL 1.1.1 and LibreSSL 3.x.
- Revise the default TLS ciphersuites.
- Kore now carries ffdhe4096.pem and installs it under PREFIX/share/kore.
- Kore its tls_dhparam config setting defaults to the path mentioned above
  so you no longer have to set it.
2021-04-21 10:48:00 +02:00
Joris Vink 960fe5afd3 drop unused __init__ in cli generation 2021-04-20 10:00:46 +02:00
Joris Vink a27227d37f Rework how kodev create does python apps.
Drop the kore.conf for python apps, all configuration
can be done from inside the python code since kore4.

Adds all the basic goo in the app.py file to get up and running.
2021-04-19 09:47:18 +02:00
Joris Vink b6cb6c14f2 kore_json_strerror() no longer takes a param. 2021-04-18 11:57:53 +02:00
Frederic Cambus 087da688dd Stop hardcoding HTTP error codes in http_error_response() calls.
Use predefined HTTP_STATUS_* macros instead.
2021-04-18 11:36:23 +02:00
Joris Vink eea4177657 new BEERs available for grabs 2021-04-09 20:16:57 +02:00
Frederic Cambus a92f6d17cc Stop hardcoding HTTP/1.1 in access logs, Kore also supports HTTP/1.0. 2021-04-08 10:20:37 +02:00
Joris Vink 44bffcb1c9 Unix socket path improvements.
- Unlink the socket path if possible before attempting to bind to it.
- Unlink the socket path if possible when Kore is shutting down.

inspired by a diff from Joel Arbring via patches@
2021-04-08 09:25:19 +02:00
Frederic Cambus b6570e10a9 Do not send the HSTS header if tls is not enabled in the server context. 2021-04-08 09:17:06 +02:00
Joris Vink cc276e2471 Add kore_json_item_attach().
Allows a JSON subtree to be engrafted after creation.

from Joel Arbring via patches@
2021-04-08 09:10:58 +02:00
Joris Vink 81a09a04d6 Use route instead of old static/dynamic. 2021-04-07 14:52:41 +02:00
Joris Vink a160a9e7ff Use the correct directive for unix binds. 2021-04-07 14:51:11 +02:00
Joris Vink 9cfcd9a4be JSON API improvements.
- Try harder to mark integers as KORE_JSON_TYPE_INTEGER, especially if
  they fit in the internal representation of one (int64_t).

- Move error codes into the JSON code itself, rather then requiring
  a kore_json data structure. This allows the JSON API to relay errors
  such as "item not found" or "type mismatch" properly when looking at items.

- When asking for a KORE_JSON_TYPE_INTEGER_U64 and a KORE_JSON_TYPE_INTEGER
  was found with the same name, check if it could be returned properly and do
  so if possible.
2021-03-30 14:19:48 +02:00
Joris Vink 0031f0271e update counts to reflect reality
much beer is owed.
2021-03-26 10:54:47 +01:00
Joris Vink 85b93f7957 KORE_SECCOMP_FILTER() does not require a semicolumn.
It's a wrapper around a function.
2021-03-23 15:03:47 +01:00
Joris Vink 754eab5b4b sync config with reality 2021-03-10 17:30:01 +01:00
Joris Vink 25d47db9e7 Let VERSION depend on OBJDIR. 2021-03-09 15:13:45 +01:00
Joris Vink 26fdbc4030 Add -fno-common to CFLAGS.
Newer compilers have this enabled by default, instead of -fcommon
so enable it here to catch any problems early.
2021-01-31 14:54:26 +01:00
Joris Vink 0549295f30 Cleanup integer types for # Python arguments.
Define PY_SSIZE_T_CLEAN before pulling in Python.h
2021-01-28 13:34:43 +01:00