Commit Graph

1585 Commits

Author SHA1 Message Date
Joris Vink da959e8275 reset curl timer if nothing is running anymore. 2019-10-23 12:38:31 +02:00
Joris Vink 890eb78fd8 Improve our new JSON api a bit.
Allow JSON to be constructed via kore_json_create_item and its
handy macro family:
	- kore_json_create_object()
	- kore_json_create_array()
	- kore_json_create_string()
	- kore_json_create_number()
	- kore_json_create_literal().

Adds kore_json_item_tobuf() to convert a JSON item into a string
representation in a kore_buf data structure.

Renames the kore_json_get* family to kore_json_find* instead.

Allows for quite clean code:

	struct kore_buf		buf;
	struct kore_json_item	*root;

	root = kore_json_create_object(NULL, NULL);
	kore_json_create_string(root, "hello", "world");
	kore_json_create_number(root, "value", 2.241);

	kore_buf_init(&buf, 128);
	kore_json_item_tobuf(root, &buf);

	kore_json_item_free(root);
	kore_buf_cleanup(&buf);
2019-10-22 23:56:47 +02:00
Joris Vink 790d020ce9 Stop a python coro from getting stuck with httpclient.
In cases where a request is immediately completed in libcurl its multi
handle and no additional i/o is happening a coro can get stuck waiting
to be run.

Prevent this by lowering netwait from KORE_WAIT_INFINITE if there
are pending python coroutines.
2019-10-22 17:06:32 +02:00
Joris Vink 0430c36a08 Adjust for new kore_curl_init() 2019-10-21 21:20:16 +02:00
Joris Vink a119f18a23 Adjust to new kore_curl_init(). 2019-10-21 13:29:26 +02:00
Joris Vink b64ae5d111 Allow kore_curl calls to be synchronous.
Changes kore_curl_init() to take a flag parameter, much like pgsql api
in which you specify KORE_CURL_ASYNC or KORE_CURL_SYNC.

If KORE_CURL_ASYNC is specified, Kore will behave as before.
If Kore_CURL_SYNC is specified, Kore will execute the libcurl immediately
and return once it has been completed.
2019-10-21 13:26:44 +02:00
Joris Vink 22ebfae240 missing kore_free() 2019-10-20 23:55:08 +02:00
Joris Vink 00f505e3f3 simplify strtod() error checking.
we can just check if errno is ERANGE and drop d == 0 comparison.
2019-10-20 23:43:54 +02:00
Joris Vink 88330a5fcd missing kore_free() 2019-10-20 23:42:37 +02:00
Joris Vink dc55a48d87 Add native JSON parser example 2019-10-20 23:40:08 +02:00
Joris Vink e94cc2f3a8 Yikes, add the actual JSON parser code.. 2019-10-20 23:30:10 +02:00
Joris Vink 06fa452c96 Add a full native JSON parser to Kore.
Mostly compliant, ignores \uXXXX in strings for now.

New API functions:

void kore_json_init(struct kore_json *json, const u_int8_t *data, size_t len);
  - Prepares JSON data for parsing.

int kore_json_parse(struct kore_json *json)
  - Parses the JSON data prepared via kore_json_init. Returns KORE_RESULT_ERROR
    if parsing failed or KORE_RESULT_OK if it succeeded.

struct kore_json_item *kore_json_get(struct kore_json *json, const char *path,
                                     int type);
  - Try to find the object matching a given search patch and type.

  eg, given a JSON structure of:
    {
      "reasons": {
        "strings": [
          "first reason",
          "second"
        ]
      }
    }

  one can obtain the second element in the reasons.strings array via:

    item = kore_json_get(json, "reasons/strings[0]", KORE_JSON_TYPE_STRING);

  Returns NULL if the item was not found or a type mismatch was hit,
  otherwise will return the item of that type.

  The kore_json_item data structure has a data member that contains the
  relevant bits depending on the type:

    KORE_JSON_TYPE_ARRAY, KORE_JSON_TYPE_OBJECT:
      the data.items member is valid.

    KORE_JSON_TYPE_STRING:
      the data.string member is valid.

    KORE_JSON_TYPE_NUMBER:
      the data.number member is valid.

    KORE_JSON_TYPE_LITERAL:
      the data.literal member is valid.

void kore_json_cleanup(struct kore_json *json);
  - Cleanup any resources

const char *kore_json_strerror(struct kore_json *json);
  - Return pointer to human readable error string.
2019-10-20 23:22:11 +02:00
Joris Vink 7da7141c21 remove FILEREF_DEBUG that sneaked in 2019-10-16 12:13:21 +02:00
Joris Vink cdc3347120 Add kore.sendmsg(object, worker=None) to the python api.
This allows you to send Python objects that can be run through pickle
to other worker processes.

If your application implements koreapp.onmsg() you will be able to receive
these objects.
2019-10-16 12:05:27 +02:00
Joris Vink ffb9fd2fc5 Align kore memory functions on 16-byte boundaries. 2019-10-15 21:33:43 +02:00
Joris Vink e90e3749a7 use Py_InitializeEx and do not install python signals. 2019-10-15 21:18:38 +02:00
Joris Vink bdc75556fe call PyErr_Clear() in python_callable(). 2019-10-15 16:03:39 +02:00
Joris Vink a1c1245ef7 don't call Py_DECREF on list in configure 2019-10-15 15:53:08 +02:00
Joris Vink 0f4d5294ad store active pid into apid so it does not overwritten. 2019-10-15 14:25:57 +02:00
Joris Vink bc33a5def4 Add kore.proc.pid, returns the PID of the proc. 2019-10-15 14:23:49 +02:00
Joris Vink ec249390b1 Allow building with python3.8 2019-10-15 10:16:53 +02:00
Joris Vink 55f5b34dd9 Generate platform.h into OBJDIR if need be. 2019-10-07 16:23:21 +02:00
Joris Vink 3dcf94d1ae Add seccomp.bpf_stmt() method to Python api.
Allows a developer to create their own statements, the bpf_jmp equivalent
may follow later if I need it.
2019-10-07 13:44:31 +02:00
Joris Vink 0eb11794f5 Do not add keymgr its msg fd if not started.
Reshuffles the keymgr_active flag to keymgr.c and let it be figured out
from inside kore_server_start() instead of the worker init code.
2019-10-07 10:31:35 +02:00
Joris Vink 7209a67d47 unbreak DEBUG builds 2019-10-04 19:24:57 +02:00
Joris Vink 97523e2768 only register tls related msg callbacks if needed 2019-10-04 19:20:37 +02:00
Joris Vink 7bc1a92192 remove norwegian debug 2019-10-04 12:44:37 +02:00
Joris Vink 6c8970651c s/hook/method 2019-10-04 11:34:49 +02:00
Joris Vink b0cf42726d Do not start keymgr if no tls enabled servers are present 2019-10-04 11:29:45 +02:00
Joris Vink 11cc5d304f let kore_python_seccomp_hook() take the hook name 2019-10-04 11:28:14 +02:00
Joris Vink 8bbdaedf94 Allow configuring seccomp on Linux via the python api.
A new hook in the koreapp class is called right before seccomp
is enabled. This hook receives a Kore seccomp object which has
the following methods:

	seccomp.allow("syscall")
	seccomp.allow_arg("syscall", arg, value)
	seccomp.allow_flag("syscall", arg, flag)
	seccomp.allow_mask("syscall", arg, mask)

	seccomp.deny("syscall")
	seccomp.deny_arg("syscall", arg, value, errno=EACCES)
	seccomp.deny_flag("syscall", arg, flag, errno=EACCES)
	seccomp.deny_mask("syscall", arg, mask, errno=EACCES)

This allows you to finetune the seccomp filters for your application
from inside your koreapp.
2019-10-04 10:59:48 +02:00
Joris Vink bcf0355704 Add clock_gettime to keymgr process 2019-10-03 15:55:19 +02:00
Frederic Cambus 85981e22cc Allow the nanosleep syscall, it is used in kore_accesslog(). 2019-10-03 15:53:36 +02:00
Joris Vink 5e6a8d42cf include sys/socket.h in net.c 2019-10-03 12:19:36 +02:00
Joris Vink e59dcfb236 fix doc link 2019-09-27 23:55:42 +02:00
Joris Vink 4cfff12b46 mention sandboxing 2019-09-27 23:53:15 +02:00
Joris Vink d0e46adfb1 kill norwegian debug 2019-09-27 20:08:16 +02:00
Joris Vink 46375303cb Allow multiple binds on new server directive. 2019-09-27 20:00:35 +02:00
Joris Vink 5c18f4ac41 python acquire method mispelled.
from fcambus@
2019-09-27 13:49:09 +02:00
Joris Vink be70118bf2 move l->tls check upwards 2019-09-27 12:33:48 +02:00
Joris Vink 46127459d9 fix BSDs after last commit 2019-09-27 12:31:17 +02:00
Joris Vink 7350131232 Allow listening of tls/notls ports at the same time.
Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.

It is time to allow non TLS listeners without having to rebuild Kore.

This commit changes your configuration format and will break existing
applications their config.

Configurations now get listener {} contexts:

listen default {
	bind 127.0.0.1 8888
}

The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).

If you want to turn off TLS on that listener, specify "tls no" in that
context.

Domains now need to be attached to a listener:

Eg:
	domain * {
		attach	default
	}

For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:

	kore.listen("name", ip=None, port=None, path=None, tls=True).
2019-09-27 12:27:04 +02:00
Joris Vink 0d72f11902 make sure we can still run normal apps even with PYTHON=1 2019-09-26 20:38:02 +02:00
Joris Vink f725ca228c alter python skeleton from kodev create -p.
adds the kore.config.file setting (required a fix for -c) and the
kore.config.deployment option is set to "development".
2019-09-26 19:58:13 +02:00
Joris Vink 1f7405b1d3 2019 was here 9 months ago 2019-09-26 16:44:42 +02:00
Joris Vink 555856ab0a fix usage for python builds.
while here, force a module or script as a cli argument and
fix kodev run to pass the config if inside of a module.
2019-09-26 16:41:52 +02:00
Joris Vink b7163f35ce allow readlink syscall 2019-09-26 16:29:45 +02:00
Joris Vink d4ba6f3485 allow more net related syscalls for python. 2019-09-26 16:13:42 +02:00
Joris Vink 9d7ef805f0 hide rcall properly if needed 2019-09-26 16:05:01 +02:00
Joris Vink 937c39f041 Many many Python improvements.
- Kore can now fully be configured via Python code if one wants nothing to
  do with configuration files.

- Kore can now start single python files and no longer requires them to be
  inside a module directory.

- Pass all regex capture groups to the handler methods, allowing you to
  get access to them immediately.

- Change python websocket_handshake to take callable objects directly.

- Added a new deployment configuration option. If set to "dev" or
  "development" Kore will automatically foreground, no chroot / etc.
  If set to "production" Kore *will* chroot, drop privs, etc.

- Many more..

These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
2019-09-26 15:49:00 +02:00