Commit Graph

1538 Commits

Author SHA1 Message Date
Joris Vink d4ba6f3485 allow more net related syscalls for python. 2019-09-26 16:13:42 +02:00
Joris Vink 9d7ef805f0 hide rcall properly if needed 2019-09-26 16:05:01 +02:00
Joris Vink 937c39f041 Many many Python improvements.
- Kore can now fully be configured via Python code if one wants nothing to
  do with configuration files.

- Kore can now start single python files and no longer requires them to be
  inside a module directory.

- Pass all regex capture groups to the handler methods, allowing you to
  get access to them immediately.

- Change python websocket_handshake to take callable objects directly.

- Added a new deployment configuration option. If set to "dev" or
  "development" Kore will automatically foreground, no chroot / etc.
  If set to "production" Kore *will* chroot, drop privs, etc.

- Many more..

These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00
Joris Vink 296fe7a6d4 seccomp improvements.
More BPF helper macros, more helper for granular syscall checking.
Use these throughout kore where it makes sense.

The new helpers:

- KORE_SYSCALL_DENY_ARG(name, arg, value, errno):
	Deny the system call with errno if the argument matches value.

- KORE_SYSCALL_DENY_MASK(name, arg, mask, errno):
	Deny the system call with errno if the mask argument does not match
	the exact mask given.

- KORE_SYSCALL_DENY_WITH_FLAG(name, arg, flag, errno):
	Deny the system call with errno if the argument contains the
	given flag.

The reverse also exists:

- KORE_SYSCALL_ALLOW_ARG()
- KORE_SYSCALL_ALLOW_MASK()
- KORE_SYSCALL_ALLOW_WITH_FLAG()
2019-09-26 13:51:53 +02:00
Joris Vink 4ed6136693 add 2 more missing syscalls for musl to keymgr. 2019-09-26 10:20:30 +02:00
Joris Vink 9dc12af1a4 add rt_sigprocmask for keymgr seccomp policy 2019-09-26 10:06:32 +02:00
Joris Vink a64be43ccf add writev to keymgr, needed on some platforms 2019-09-26 10:02:58 +02:00
Joris Vink c6c253305a swap sockets to use send/recv and update seccomp. 2019-09-26 09:53:51 +02:00
Joris Vink 0314521658 shuffle some stuff around 2019-09-26 09:28:12 +02:00
Joris Vink aafecb9485 Make sure filters from the hook are added first.
This allows user seccomp filters to be added before the kore ones which
means developers can override our own settings.
2019-09-26 09:22:33 +02:00
Joris Vink a5f6805484 seccomp improvements for developers.
- Add KORE_SECCOMP_FILTER() as a helpful shortcut to create your
  application seccomp filter. You can still roll your own and hook
  into kore_seccomp_hook() yourself to load your filters.

- Add KORE_SYSCALL_ALLOW_LOG(_name)
	Allows a system call but will log it.
2019-09-26 09:20:57 +02:00
Joris Vink c1affbbd74 simplify bpf rule generation, add deny macro. 2019-09-25 23:41:43 +02:00
Joris Vink 61b8a9e90d fix tpyo 2019-09-25 16:14:58 +02:00
Joris Vink de97318a95 hide seccomp logs behind !quiet 2019-09-25 16:14:17 +02:00
Joris Vink 233415a7bb close the wrapped socket instead of the os one. 2019-09-25 15:47:14 +02:00
Joris Vink 4bc39f19af add lstat to worker seccomp filter 2019-09-25 15:44:03 +02:00
Joris Vink eab94eda49 move libressl support to minimum 3.3.0. 2019-09-25 15:39:39 +02:00
Joris Vink 54ca865c11 add a few more syscalls for keymgr.
different platforms may do different things
2019-09-25 15:04:12 +02:00
Joris Vink 68e90507f4 properly seccomp keymgr 2019-09-25 14:41:09 +02:00
Joris Vink e7352a3634 fix - that snuck in 2019-09-25 14:32:17 +02:00
Joris Vink cd9971247c Add seccomp syscall filtering to kore.
With this commit all Kore processes (minus the parent) are running
under seccomp.

The worker processes get the bare minimum allowed syscalls while each module
like curl, pgsql, etc will add their own filters to allow what they require.

New API functions:
    int kore_seccomp_filter(const char *name, void *filter, size_t len);

    Adds a filter into the seccomp system (must be called before
    seccomp is enabled).

New helpful macro:
    define KORE_SYSCALL_ALLOW(name)

    Allow the syscall with a given name, should be used in
    a sock_filter data structure.

New hooks:
    void kore_seccomp_hook(void);

    Called before seccomp is enabled, allows developers to add their
    own BPF filters into seccomp.
2019-09-25 14:31:20 +02:00
Joris Vink d99d0b2d77 add lock.trylock() and make lock.release() sync. 2019-09-20 10:03:43 +02:00
Joris Vink 06fd5ca2f2 Add aqcuire() and release() to pylock.
This allows it to be used from code without requiring the async with
context manager approach.
2019-09-20 09:56:21 +02:00
Joris Vink c3b2a8b2a2 fix NOHTTP builds 2019-09-20 09:37:02 +02:00
Joris Vink 868b0a71a2 make coro tracing available at runtime, always.
call kore.corotrace(True) to enable it and kore.corotrace(False) to disable.
2019-09-18 10:55:13 +02:00
Joris Vink 95506204e8 Add kore.task_kill() to the python api.
Allows killing of coroutines, given their task id.

The kore.task_create() method now returns the task id for a newly
created task to the caller.

While here, change the coroutine task id to a uint32 from uint64.
There is no need for it to be 64bit. (famous last words)
2019-09-17 20:04:35 +02:00
Joris Vink b107485ea4 if built with python+debug enable coro tracing 2019-09-16 20:36:23 +02:00
Joris Vink c1653a5254 hide kore.coroname() behind PYTHON_CORO_TRACE. 2019-09-16 20:33:14 +02:00
Joris Vink 019006620d allow coroutines to set friendly names. 2019-09-16 20:31:25 +02:00
Joris Vink 960fabe44c s/PYTHON_CORO_DEBUG/PYTHON_CORO_TRACE 2019-09-16 20:25:34 +02:00
Joris Vink 1ee1021aba trace created/deleted. 2019-09-16 20:21:04 +02:00
Joris Vink 9bcccb9483 Add coroutine debugging.
If built with PYTHON_CORO_DEBUG in CFLAGS Kore will spew out coroutine
traces while running. These traces include the filename, function and line
number where the coroutines are waking up, running and suspended.
2019-09-16 20:11:42 +02:00
Joris Vink dbb99a56d1 move new/wokenup/suspended coros to end of lists. 2019-09-16 19:34:47 +02:00
Joris Vink 58a6b4e331 trailing space fix 2019-09-13 23:22:38 +02:00
Joris Vink 3207ce8526 pgsql improvements round 2
- kill PQsetnonblocking(), it's not what one thinks.
- keep doing the PQconsumeInput()/PQisBusy() dance until
  the former clears all read data.
2019-09-13 23:20:51 +02:00
Frederic Cambus a8c6ecbb89 Set default MAN_DIR value only if it's not already set. 2019-09-05 09:42:54 +02:00
Joris Vink f2472ba485 allow python modules to set progname. 2019-09-04 20:37:33 +02:00
Joris Vink 88bd3ce045 tiny comment on koreapp 2019-09-04 20:10:48 +02:00
Joris Vink c10813dc44 turn python-pgsql into a real kore python app 2019-09-04 20:07:04 +02:00
Joris Vink 8e858983bf python pgsql changes.
- decouple pgsql from the HTTP request allowing it to be used in other
  contexts as well (such as a task, etc).

- change names to dbsetup() and dbquery().

eg:

result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
Joris Vink f3b7cba58c Call PQConsumeInput() again after PQisBusy().
Prevents a stall in case there is still data in the read end of the socket
but PQisBusy() told us to not fetch a result yet. In that case we end up
stalling due to epoll not giving us another EPOLLIN event due to EPOLLET.
2019-09-04 19:19:52 +02:00
Joris Vink b54b93536d Use strdup() when munging environment pointers. 2019-08-27 13:12:44 +02:00
Joris Vink c32880e12b http_argument_decode() can fail. 2019-08-02 11:34:45 +02:00
Joris Vink 0b9e174af4 http_argument_urldecode() could fail. 2019-07-20 20:35:58 +02:00
Joris Vink 0f58de36f4 Be a little less strict on url decoding. 2019-07-16 21:31:32 +02:00
Joris Vink f2e17c1384 return NULL if PyUnicode_FromString fails. 2019-07-16 21:31:19 +02:00
Joris Vink 5055f3c872 add kore.worker() to the python api.
returns the current worker number or None if on parent.
2019-06-16 19:57:48 +02:00
Joris Vink 4a64b4f07b Improve curl timeout handling.
In case libcurl instructs us to call the timeout function as soon
as possible (timeout == 0 in curl_timeout), don't try to be clever
with a timeout value of 10ms.

Instead call the timeout function once we get back in the worker
event loop. This makes things a lot snappier as we don't depend
on epoll/kqueue waiting for io for 10ms (which actually isn't 10ms...).
2019-06-13 12:59:17 +02:00
Joris Vink 574c9a7084 make sure kodev run works in python modules. 2019-06-13 11:18:10 +02:00
Joris Vink 3114f8d8d0 Improve python experience.
- If Kore is built with PYTHON=1 you can now specify the module that
  should be loaded on the command-line.

     eg: $ kore -frn myapp

- Add skeleton generation for python applications to kodev.

     eg: $ kodev create -p myapp

This should make it a whole lot easier to get started with kore python.
2019-06-12 23:35:43 +02:00