Commit Graph

155 Commits

Author SHA1 Message Date
Joris Vink 8235759bca Python: Add kore.app().
This method allows you to set a Python object and obtain it
by calling the method again without any arguments.

eg:

foo = SomeClass()

kore.app(foo)

foo = kore.app()
2020-07-05 21:47:22 +02:00
Joris Vink e38c6e5d30 Python: Several fixes for our async curl support.
- Fix the curl-extract-opt.sh generation script to work on newer
  curl releases as the header changed slightly.
- Use the correct handles when calling curl_easy_setopt() inside
  of our setopt functions exported via Python.
- Add a curl.setbody() method, allowing a body to be sent to be set.
  (eg when sending mail via SMTP).
- Regen of our python_curlopt.h from 7.71.1
2020-07-02 08:41:17 +02:00
Joris Vink 122a86013b Python: respond with 500 in case of a coroutine error.
If a coroutine throws an exception, respond with a 500
after logging the exception itself.
2020-06-30 09:57:48 +02:00
Joris Vink 8a39d18196 work around different dirname()/basename() implementations. 2020-06-09 12:22:22 +02:00
Joris Vink 9d0aef0079 bump copyright 2020-02-10 14:47:33 +01:00
Joris Vink d86a10afa1 allow use udata in kore.timer() via the data kwarg. 2020-01-22 09:42:41 +01:00
Joris Vink 2d380cac3f Expose our async libcurl support to the Python api.
Kore already exposed parts of this via the kore.httpclient() method but
this commit takes it a bit further and exposes the libcurl interface
completely (including the setopt options).

tldr:

handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README")
handle.setopt(kore.CURLOPT_TIMEOUT, 5)

data = await handle.run()
print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
Joris Vink 56c33f85d4 change client_authority to client_verify. 2019-11-18 20:30:52 +01:00
Joris Vink 73757a29d5 Make dumb compilers happy. 2019-11-15 07:49:16 +01:00
Joris Vink b3b5aa37b7 Allow acme config via python api 2019-11-13 23:01:24 +01:00
Joris Vink 7b5046873a Make sure we wakeup the coroutine that called proc.reap().
We actually woke up the coroutine that originally spawned the process
when we reap it, but another coroutine may have taken over the object.

This mimics how we do things for the pysock_op things.
2019-10-29 15:12:20 +01:00
Joris Vink 790d020ce9 Stop a python coro from getting stuck with httpclient.
In cases where a request is immediately completed in libcurl its multi
handle and no additional i/o is happening a coro can get stuck waiting
to be run.

Prevent this by lowering netwait from KORE_WAIT_INFINITE if there
are pending python coroutines.
2019-10-22 17:06:32 +02:00
Joris Vink a119f18a23 Adjust to new kore_curl_init(). 2019-10-21 13:29:26 +02:00
Joris Vink cdc3347120 Add kore.sendmsg(object, worker=None) to the python api.
This allows you to send Python objects that can be run through pickle
to other worker processes.

If your application implements koreapp.onmsg() you will be able to receive
these objects.
2019-10-16 12:05:27 +02:00
Joris Vink e90e3749a7 use Py_InitializeEx and do not install python signals. 2019-10-15 21:18:38 +02:00
Joris Vink bdc75556fe call PyErr_Clear() in python_callable(). 2019-10-15 16:03:39 +02:00
Joris Vink a1c1245ef7 don't call Py_DECREF on list in configure 2019-10-15 15:53:08 +02:00
Joris Vink 0f4d5294ad store active pid into apid so it does not overwritten. 2019-10-15 14:25:57 +02:00
Joris Vink bc33a5def4 Add kore.proc.pid, returns the PID of the proc. 2019-10-15 14:23:49 +02:00
Joris Vink 3dcf94d1ae Add seccomp.bpf_stmt() method to Python api.
Allows a developer to create their own statements, the bpf_jmp equivalent
may follow later if I need it.
2019-10-07 13:44:31 +02:00
Joris Vink 7bc1a92192 remove norwegian debug 2019-10-04 12:44:37 +02:00
Joris Vink 6c8970651c s/hook/method 2019-10-04 11:34:49 +02:00
Joris Vink 11cc5d304f let kore_python_seccomp_hook() take the hook name 2019-10-04 11:28:14 +02:00
Joris Vink 8bbdaedf94 Allow configuring seccomp on Linux via the python api.
A new hook in the koreapp class is called right before seccomp
is enabled. This hook receives a Kore seccomp object which has
the following methods:

	seccomp.allow("syscall")
	seccomp.allow_arg("syscall", arg, value)
	seccomp.allow_flag("syscall", arg, flag)
	seccomp.allow_mask("syscall", arg, mask)

	seccomp.deny("syscall")
	seccomp.deny_arg("syscall", arg, value, errno=EACCES)
	seccomp.deny_flag("syscall", arg, flag, errno=EACCES)
	seccomp.deny_mask("syscall", arg, mask, errno=EACCES)

This allows you to finetune the seccomp filters for your application
from inside your koreapp.
2019-10-04 10:59:48 +02:00
Joris Vink 46375303cb Allow multiple binds on new server directive. 2019-09-27 20:00:35 +02:00
Joris Vink be70118bf2 move l->tls check upwards 2019-09-27 12:33:48 +02:00
Joris Vink 7350131232 Allow listening of tls/notls ports at the same time.
Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.

It is time to allow non TLS listeners without having to rebuild Kore.

This commit changes your configuration format and will break existing
applications their config.

Configurations now get listener {} contexts:

listen default {
	bind 127.0.0.1 8888
}

The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).

If you want to turn off TLS on that listener, specify "tls no" in that
context.

Domains now need to be attached to a listener:

Eg:
	domain * {
		attach	default
	}

For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:

	kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:27:04 +02:00
Joris Vink d4ba6f3485 allow more net related syscalls for python. 2019-09-26 16:13:42 +02:00
Joris Vink 937c39f041 Many many Python improvements.
- Kore can now fully be configured via Python code if one wants nothing to
  do with configuration files.

- Kore can now start single python files and no longer requires them to be
  inside a module directory.

- Pass all regex capture groups to the handler methods, allowing you to
  get access to them immediately.

- Change python websocket_handshake to take callable objects directly.

- Added a new deployment configuration option. If set to "dev" or
  "development" Kore will automatically foreground, no chroot / etc.
  If set to "production" Kore *will* chroot, drop privs, etc.

- Many more..

These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Joris Vink cd9971247c Add seccomp syscall filtering to kore.
With this commit all Kore processes (minus the parent) are running
under seccomp.

The worker processes get the bare minimum allowed syscalls while each module
like curl, pgsql, etc will add their own filters to allow what they require.

New API functions:
    int kore_seccomp_filter(const char *name, void *filter, size_t len);

    Adds a filter into the seccomp system (must be called before
    seccomp is enabled).

New helpful macro:
    define KORE_SYSCALL_ALLOW(name)

    Allow the syscall with a given name, should be used in
    a sock_filter data structure.

New hooks:
    void kore_seccomp_hook(void);

    Called before seccomp is enabled, allows developers to add their
    own BPF filters into seccomp.
2019-09-25 14:31:20 +02:00
Joris Vink d99d0b2d77 add lock.trylock() and make lock.release() sync. 2019-09-20 10:03:43 +02:00
Joris Vink 868b0a71a2 make coro tracing available at runtime, always.
call kore.corotrace(True) to enable it and kore.corotrace(False) to disable.
2019-09-18 10:55:13 +02:00
Joris Vink 95506204e8 Add kore.task_kill() to the python api.
Allows killing of coroutines, given their task id.

The kore.task_create() method now returns the task id for a newly
created task to the caller.

While here, change the coroutine task id to a uint32 from uint64.
There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
Joris Vink c1653a5254 hide kore.coroname() behind PYTHON_CORO_TRACE. 2019-09-16 20:33:14 +02:00
Joris Vink 019006620d allow coroutines to set friendly names. 2019-09-16 20:31:25 +02:00
Joris Vink 960fabe44c s/PYTHON_CORO_DEBUG/PYTHON_CORO_TRACE 2019-09-16 20:25:34 +02:00
Joris Vink 1ee1021aba trace created/deleted. 2019-09-16 20:21:04 +02:00
Joris Vink 9bcccb9483 Add coroutine debugging.
If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine
traces while running. These traces include the filename, function and line
number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
Joris Vink dbb99a56d1 move new/wokenup/suspended coros to end of lists. 2019-09-16 19:34:47 +02:00
Joris Vink f2472ba485 allow python modules to set progname. 2019-09-04 20:37:33 +02:00
Joris Vink 8e858983bf python pgsql changes.
- decouple pgsql from the HTTP request allowing it to be used in other
  contexts as well (such as a task, etc).

- change names to dbsetup() and dbquery().

eg:

result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
Joris Vink f2e17c1384 return NULL if PyUnicode_FromString fails. 2019-07-16 21:31:19 +02:00
Joris Vink 5055f3c872 add kore.worker() to the python api.
returns the current worker number or None if on parent.
2019-06-16 19:57:48 +02:00
Joris Vink 3114f8d8d0 Improve python experience.
- If Kore is built with PYTHON=1 you can now specify the module that
  should be loaded on the command-line.

     eg: $ kore -frn myapp

- Add skeleton generation for python applications to kodev.

     eg: $ kodev create -p myapp

This should make it a whole lot easier to get started with kore python.
2019-06-12 23:35:43 +02:00
Joris Vink a46447b1f9 set hdlr_extra to NULL in pyhttp_iterobj_disconnect. 2019-06-12 21:09:57 +02:00
Joris Vink d4f4e007e0 make sure prereqs are in order 2019-06-07 21:33:00 +02:00
Joris Vink 6ed3ac4658 kill debug 2019-06-07 21:25:18 +02:00
Joris Vink 3cc7d6e238 Allow kore.prerequests to be async. 2019-06-07 21:06:54 +02:00
Joris Vink 9cc58d45c1 Python HTTP improvements.
1) Add @kore.prerequest python decorator.

Using this decorator on a function will cause that function
to always be executed *before* any page handler is run.

eg:

@kore.prerequest
def _check(req):
    if req.method == kore.HTTP_METHOD_POST:
        req.populate_post()

2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
Joris Vink 89e58fa474 Improve iterator support for Python req.response().
If the connection on which we are about to send the response was
marked as disconnecting, do not go ahead and hook into the disconnect
callback (it will never be called, it is already disconnecting).

Instead just return, the connection will be removed anyway.
2019-06-05 10:35:47 +02:00