Commit Graph

117 Commits

Author SHA1 Message Date
Joris Vink dbb99a56d1 move new/wokenup/suspended coros to end of lists. 2019-09-16 19:34:47 +02:00
Joris Vink f2472ba485 allow python modules to set progname. 2019-09-04 20:37:33 +02:00
Joris Vink 8e858983bf python pgsql changes.
- decouple pgsql from the HTTP request allowing it to be used in other
  contexts as well (such as a task, etc).

- change names to dbsetup() and dbquery().

eg:

result = kore.dbquery("db", "select foo from bar")
2019-09-04 19:57:28 +02:00
Joris Vink f2e17c1384 return NULL if PyUnicode_FromString fails. 2019-07-16 21:31:19 +02:00
Joris Vink 5055f3c872 add kore.worker() to the python api.
returns the current worker number or None if on parent.
2019-06-16 19:57:48 +02:00
Joris Vink 3114f8d8d0 Improve python experience.
- If Kore is built with PYTHON=1 you can now specify the module that
  should be loaded on the command-line.

     eg: $ kore -frn myapp

- Add skeleton generation for python applications to kodev.

     eg: $ kodev create -p myapp

This should make it a whole lot easier to get started with kore python.
2019-06-12 23:35:43 +02:00
Joris Vink a46447b1f9 set hdlr_extra to NULL in pyhttp_iterobj_disconnect. 2019-06-12 21:09:57 +02:00
Joris Vink d4f4e007e0 make sure prereqs are in order 2019-06-07 21:33:00 +02:00
Joris Vink 6ed3ac4658 kill debug 2019-06-07 21:25:18 +02:00
Joris Vink 3cc7d6e238 Allow kore.prerequests to be async. 2019-06-07 21:06:54 +02:00
Joris Vink 9cc58d45c1 Python HTTP improvements.
1) Add @kore.prerequest python decorator.

Using this decorator on a function will cause that function
to always be executed *before* any page handler is run.

eg:

@kore.prerequest
def _check(req):
    if req.method == kore.HTTP_METHOD_POST:
        req.populate_post()

2) Allow attributes to be set on the pyhttp object.
2019-06-05 23:45:45 +02:00
Joris Vink 89e58fa474 Improve iterator support for Python req.response().
If the connection on which we are about to send the response was
marked as disconnecting, do not go ahead and hook into the disconnect
callback (it will never be called, it is already disconnecting).

Instead just return, the connection will be removed anyway.
2019-06-05 10:35:47 +02:00
Joris Vink 93b1d621d7 Remove memleak from Python httpclient.
We grab a reference to the pyhttp_client for the client_op data structure
but never removed it. This caused the pyhttp_client object to never
be released when out of scope.
2019-06-05 10:27:20 +02:00
Joris Vink ff7c85460c Wake up HTTP requests if python coros finish immediately.
python_coro_create() puts the HTTP request to sleep, but if they
finish immediately they will be removed from the list and should
properly be woken up or they are removed from the wrong list.
2019-05-30 17:15:08 +02:00
Joris Vink a8aff8b737 Improve curl error string handling.
Introduce kore_curl_strerror(), use this in kore_curl_logerror()
instead of assuming our errbuf has been populated.

Also use it in the python httpclient when throwing an exception rather
then looking at the errbuf member which may or may not be empty.
2019-05-30 14:25:04 +02:00
Joris Vink c2d9f1413c Fix two concurrency issues with kore.lock().
1) If a coroutine was waken up by another releasing a lock it was waiting
on we would incorrectly remove the pylock_op when the newly awoken coroutine
hits the iternext for pylock.

2) If a coroutine attempts to grab a lock it was woken up for only to fail
we did not properly rearm the coroutine to be woken up later to try again.
2019-05-29 15:25:31 +02:00
Joris Vink b4aba48fad small python improvements.
- Check for an exception when in a timer context and log it if uncaught.
- Make sure tlsverify is settable regardless of client certs being specified.
2019-05-16 21:07:42 +02:00
Joris Vink fe84997ce9 add option to use unix paths via kore.httpclient 2019-05-03 15:31:16 +02:00
Joris Vink 3c9a141cd0 allow an iterator to be passed to req.response().
if an iterator is passed kore will send the response with
transfer-encoding: chunked and call the iterator for every
chunk that was sent.

The iterator must return a utf-8 string.

Works wonderful with TemplateStream from jinja2.
2019-05-03 13:42:34 +02:00
Joris Vink e54886960e objs is a pointer, and use Py_XDECREF on it. 2019-04-25 23:25:19 +02:00
Joris Vink d0d0bdeb4f Improve pgsql support.
- Add kore_pgsql_query_param_fields() which allows you to pass in the
  arrays for values, lengths and formats yourself.

- Add kore_pgsql_column_binary() which will return 1 if the given column
  index contains a binary result or 0 if it contains a text result.

- Change the query call in req.pgsql() for Python to always use the
  parameterized queries.

This adds the 'params' and 'binary' keywords to the req.pgsql method.

Eg:
	result = await req.pgsql("db", "INSERT INTO foo (field) VALUES($1"),
	    params=["this is my value"])
2019-04-25 23:13:13 +02:00
Joris Vink 994f428a8d add cabundle keyword to kore.httpclient() 2019-04-24 21:09:24 +02:00
Joris Vink 6bd3d92fe2 unbreak. 2019-04-24 18:30:07 +02:00
Joris Vink a6af458cd2 add tlsverify keyword to kore.httpclient(). 2019-04-24 18:28:23 +02:00
Joris Vink 2c88bc6120 Add asynchronous libcurl support.
This commit adds the CURL=1 build option. When enabled allows
you to schedule CURL easy handles onto the Kore event loop.

It also adds an easy to use HTTP client API that abstracts away the
settings required from libcurl to make HTTP requests.

Tied together with HTTP request state machines this means you can
write fully asynchronous HTTP client requests in an easy way.

Additionally this exposes that API to the Python code as well
allowing you do to things like:

	client = kore.httpclient("https://kore.io")
	status, body = await client.get()

Introduces 2 configuration options:
	- curl_recv_max
		Max incoming bytes for a response.

	- curl_timeout
		Timeout in seconds before a transfer is cancelled.

This API also allows you to take the CURL easy handle and send emails
with it, run FTP, etc. All asynchronously.
2019-04-24 00:15:17 +02:00
Joris Vink 92fb4974b1 only deschedule if not -1. 2019-03-27 22:15:35 +01:00
Joris Vink 4ca7f29649 Add a concurrency parameter to kore.gather() 2019-03-25 10:13:52 +01:00
Joris Vink 58c2739dee If fd is -1 in pysock_op_iternext just return None.
None in pysock signals EOF and if it was closed under our feet we should
relay that properly.
2019-03-21 14:44:41 +01:00
Joris Vink ce012e7bd5 set fd to -1 on close. 2019-03-21 10:09:24 +01:00
Joris Vink d371454dad do not grab the result if it was in a gatherop 2019-03-19 13:58:00 +01:00
Joris Vink 681a88d535 simplify af_unix recvfrom/sendto support. 2019-03-19 13:27:50 +01:00
Erik Karlsson d60994abb5 add python sendto recvfrom abstract socket support 2019-03-19 11:11:44 +01:00
Joris Vink 9caa45a050 Allow python validator methods to be async. 2019-03-18 09:34:31 +01:00
Joris Vink c3ab570f56 Append full module path if it is a directory. 2019-03-16 16:13:52 +01:00
Joris Vink ec7c8b8e1d fix a few minor problems with pylock.
- when pylocks are deallocated we ended up deleting the wrong object.
- do not call Py_DECREF on the op when inside the op its iternext call.
2019-03-14 17:31:57 +01:00
Joris Vink 2217c7a2c8 pysock: call disable read/write only when needed 2019-03-14 09:27:13 +01:00
Joris Vink d41054bd26 remove the socket from the event queue on dealloc. 2019-03-13 16:01:42 +01:00
Joris Vink b06160c768 Make sure coros are woken up on eof. 2019-03-13 15:45:55 +01:00
Joris Vink 3b4574d791 Rework pysocket async/await.
Attach the events directly to the pysocket data structure instead of
one event per pysocket_op.

Makes the code easier, gives us a good performance boost and reduces
the number of system calls required when doing an await on a socket.
2019-03-13 11:07:15 +01:00
Joris Vink 01f9b4fcde Check for EAGAIN when calling connect(2).
Linux does this under certain conditions.
2019-03-12 20:04:08 +01:00
Joris Vink 61863bfd3a kill recv() timer if we have data.
also reset any exception that is set when we have successfully
read data from a socket.
2019-03-04 16:37:25 +01:00
Joris Vink a0c203f507 remove norwegian debug 2019-03-01 20:57:31 +01:00
Joris Vink 31a9a70d5a cleanup. 2019-03-01 20:56:03 +01:00
Erik Karlsson 01c1a8f8f8 support AF_UNIX in recvfrom()/sendto() python sockets 2019-03-01 20:13:28 +01:00
Joris Vink 1ebd82969c Add timeout support to proc.recv() 2019-02-26 15:22:55 +01:00
Joris Vink f4cd70956b Add an optional timeout to socketop.recv(). 2019-02-25 10:35:00 +01:00
Joris Vink bf1e8e5ffb bump copyright to 2019 2019-02-22 16:57:28 +01:00
Joris Vink 3dbb80463a allow double quotes in kore.proc argument lists. 2019-02-22 16:56:34 +01:00
Joris Vink 8040103458 fix for multiple consumers for the async queues. 2019-01-22 11:23:59 +01:00
Joris Vink d1e87c1a54 deal with PyObject_CallObject() returning NULL. 2019-01-15 10:20:13 +01:00