1
0
mirror of https://git.kore.io/kore.git synced 2024-11-18 16:08:29 +01:00
Commit Graph

1347 Commits

Author SHA1 Message Date
Joris Vink
cb482d8e8f Always use http_response_stream() in req.response.
Keep around the python bytes object while we stream the contents
of it out over the wire. This avoids an entire copy of the data.
2018-11-14 10:03:32 +01:00
Joris Vink
966eaf8f7a Add a kore_python_preinit() hook.
This is called before the python initialization is completed
and allows developers to inject their own built-in methods.
2018-11-12 09:01:05 +01:00
Joris Vink
3925ba60ad don't call close() no underlying socket. 2018-11-12 09:00:36 +01:00
Joris Vink
4cc9e216bb Don't call close() on the python socket on dealloc 2018-11-06 16:54:26 +01:00
Joris Vink
2dd66586ff several python improvements.
- add kore.time() as equivalent for kore_time_ms().
- call waitpid() until no more children are available for reaping otherwise
  we risk missing a process if several die at the same time and only one
  SIGCHLD is delivered to us.
- drain a RECV socket operation if eof is set but no exception was given.
2018-10-30 20:28:27 +01:00
Joris Vink
5456f2e1d5 save/restore coro_running in pygather_op_dealloc(). 2018-10-30 12:37:30 +01:00
Joris Vink
062c80c1ac start enforcing a mininum number of elms in pools. 2018-10-30 10:41:49 +01:00
Joris Vink
71c145932c grow kore_pools at a slower rate.
Before we just doubled in size the second we exhausted a pool instead
of doing a more controlled expansion.

Now we will expand at 25% of the initial elm count whenever we need to.

Will help with memory pressure in certain scenarios.
2018-10-30 10:36:18 +01:00
Joris Vink
ae2ea0be72 specify what openssl versions are expected to work. 2018-10-30 08:45:11 +01:00
Joris Vink
bb00deca2c don't leak op in error path 2018-10-29 21:18:38 +01:00
Joris Vink
1c30da855c Add kore.gather() to the python api.
Allows one to run coroutines concurrently and gather all their
results in a single returned list.

If any of the coroutines throw an exception the exception is
returned as the value of that coroutine in the returned list.
2018-10-29 21:16:08 +01:00
Joris Vink
740acb4760 make sure we can still build vs 1.1.0 2018-10-29 21:11:29 +01:00
Joris Vink
818c7847cd simplify even more 2018-10-29 20:43:40 +01:00
Joris Vink
ae72b21d35 make sure older openssl/libressl builds work. 2018-10-29 20:42:28 +01:00
Joris Vink
339df66fd5 Add support for TLS 1.3 via OpenSSL 1.1.1.
This commit removes TLS 1.0 support no matter what OpenSSL
you are linking against.

Changes the value of tls_version from 1.2 to both. Meaning if
you link with OpenSSL 1.1.1 you will get 1.2 + 1.3.
2018-10-29 20:38:58 +01:00
Joris Vink
dda2e1fb2c Some things still talk http/1.0. 2018-10-26 21:24:51 +02:00
Joris Vink
fc246e5552 defaults were changed 2018-10-26 19:24:51 +02:00
Joris Vink
e2651889e0 Add asynchronous subprocess support.
This adds kore.proc to the python runtime allowing async processing
handling:

The kore.proc method takes the command to run and an optional timeout
parameter in milliseconds. If the process did not exit normally after
that amount of time a TimeoutError exception is raised.

For instance:

async def run(cmd):
	proc = kore.proc(cmd, 1000)

	try:
		await proc.send("hello")
		proc.close_stdin()
	except TimeoutError:
		proc.kill()

	retcode = await proc.reap()

	return retcode
2018-10-26 19:19:47 +02:00
Joris Vink
ea7ea48840 actually just query for HW_NCPU on all BSDs. 2018-10-25 19:31:48 +02:00
Joris Vink
9d9e9dcec2 set cpu_count to 1 for unknown BSDs 2018-10-25 19:30:10 +02:00
Joris Vink
892814e353 Add kore_[parent|worker]_teardown().
If exists these functions are called when the worker is exiting
and when right before the parent exists.

Allows for cleanup code for applications if need to do cleanup on exit.
2018-10-23 21:46:34 +02:00
Joris Vink
fc5fc4f4ab make sure timers don't go out of scope. 2018-10-23 21:36:19 +02:00
Joris Vink
8ea32983ae Add kore.suspend(delay) to python.
Will suspend the coroutine for a number of milliseconds.

Example:

async def page(req):
	await kore.suspend(1000)
	req.response(200, b'')
2018-10-23 21:32:08 +02:00
Joris Vink
47776a9fbb Hook kore timers into python. 2018-10-23 20:44:43 +02:00
Joris Vink
c41c1db303 Add kore_shutdown().
Allows workers to cleanly initiate a shutdown of the
entire server process.
2018-10-23 19:49:42 +02:00
Joris Vink
4c35073232 Use number of cpu cores if no worker count is set. 2018-10-22 21:27:17 +02:00
Joris Vink
b70d1ee80f Add a locking mechanism in pykore.
Support the async with syntax:

	lock = kore.lock()

	async with lock:
		# your block

Fix some small issues with other parts of the python system.
2018-10-22 20:09:23 +02:00
Joris Vink
1ac131c48f If we hit the accept threshold, unlock worker. 2018-10-22 09:01:05 +02:00
Joris Vink
8be316ac0c Let kore_worker_make_busy() set next_lock. 2018-10-22 09:00:55 +02:00
Joris Vink
fad5c6ea6f Give pyqueues "popnow". 2018-10-22 08:28:03 +02:00
Joris Vink
c8795b7d7f pyqueue improvements.
- cleanup queue if it gets deallocated.
- make sure waitables on a queue get removed if their pyqueue_op dies.
2018-10-21 21:58:34 +02:00
Joris Vink
4ae3d23c7e s/INCREF/DECREF 2018-10-18 22:24:58 +02:00
Joris Vink
0cda9ecfb0 Add an asynchronous queue mechanism.
This allows coroutines to submit messages to and pop
messages from a queue in an asynchronous way.
2018-10-18 22:15:21 +02:00
Joris Vink
b5958f7d7d Add kore_parent_daemonized().
This is called for single binaries after the parent
process has called daemon().

Also fix kore_parent_configure() for !single binaries.
2018-10-18 17:18:41 +02:00
Joris Vink
c2c4e55149 Makefile improvements.
Add KORE_PYTHON_LIB and KORE_PYTHON_INC which can be set
by a caller in case the libraries exist somewhere else.

Add KORE_CRYPTO to be able to override the name of the default
crypto library Kore would link with.
2018-10-17 11:31:36 +02:00
Joris Vink
c172d49567 kodev improvements.
- Take CFLAGS, CXXFLAGS and LDFLAGS from environment
  if present and append them to the build.conf cflags

- Allow overriding of the OBJDIR via KORE_OBJDIR.

- Allow overriding of build flavor via KORE_BUILD_FLAVOR.
2018-10-17 11:30:14 +02:00
Joris Vink
300daeadef fix comment 2018-10-16 13:17:44 +02:00
Joris Vink
29202d7330 Make kore_python_log_error() public.
While here also make kore_module_load() return the
kore_module data structure pointer it just added
to the modules list.
2018-10-16 13:16:36 +02:00
Joris Vink
6080bb1c35 echo server example in Python with new async/await. 2018-10-15 20:37:51 +02:00
Joris Vink
545d48e65d revert chunk that wasn't suppose to be changed. 2018-10-15 20:33:34 +02:00
Joris Vink
20a0103f1e Add async/await support for socket i/o in python.
This means you can now do things like:

	resp = await koresock.recv(1024)
	await koresock.send(resp)

directly from page handlers if they are defined as async.

Adds lots more to the python goo such as fatalx(), bind_unix(),
task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
Joris Vink
c12f296743 missing EPOLLRDHUP check since event loop rework 2018-10-10 14:33:26 +02:00
Joris Vink
c463ecb3cb Changes to the event loop inside of Kore.
Now anyone can schedule events and get a callback to work as long
as the user data structure that is added for the event begins
with a kore_event data structure.

All event state is now kept in that kore_event structure and renamed
CONN_[READ|WRITE]_POSSIBLE to KORE_EVENT_[READ|WRITE].
2018-10-09 19:34:40 +02:00
Joris Vink
2449a86085 missing addrtype -> family renames 2018-10-08 20:12:25 +02:00
Joris Vink
884d6d722e add listener to the list earlier. 2018-10-07 21:25:00 +02:00
Joris Vink
19044919b2 don't set nodelay on unix listener sockets 2018-10-07 21:21:37 +02:00
Joris Vink
20c4036e5f add bind_unix configuration option to example conf 2018-10-07 21:08:04 +02:00
Joris Vink
9427ed8a2e rename unix to sun 2018-10-07 21:03:12 +02:00
Joris Vink
442bdef79b allow kore to bind to unix sockets via bind_unix. 2018-10-07 20:49:16 +02:00
Joris Vink
f87624a459 Add missing KORE_NO_TLS guards.
Fixes NOTLS=1 builds on openssl 1.1.1
2018-10-01 10:36:33 +02:00