When the pgsql layer was introduced it was tightly coupled with the
http layer in order to make async work fluently.
The time has come to split these up and follow the same method we
used for tasks, allowing either http requests to be tied to a pgsql
data structure or a simple callback function.
This also reworks the internal queueing of pgsql requests until
connections to the db are available again.
The following API functions were changes:
- kore_pgsql_query_init() -> kore_pgsql_setup()
no longer takes an http_request parameter.
- NEW kore_pgsql_init()
must be called before operating on an kore_pgsql structure.
- NEW kore_pgsql_bind_request()
binds an http_request to a kore_pgsql data structure.
- NEW kore_pgsql_bind_callback()
binds a callback to a kore_pgsql data structure.
With all of this you can now build kore with PGSQL=1 NOHTTP=1.
The pgsql/ example has been updated to reflect these changes and
new features.
Also let this function reset offset and lengths for http_body_read().
Make sure of this function in the python code so req.body can be called
multiple times in succession.
The only reason you would want to directly modify the cookie
after creating it should be to unset the HTTPONLY or SECURE flags
if that is what you *really* want to do.
Change http_response_cookie() to take all required parameters instead
of having to marshall those in yourself after.
Now you set a sane default cookie in one shot:
http_response_cookie(req, "key", "value", "/", 0, -1, NULL);
Which would create a session cookie key=value for / under the current domain.
We now default to httponly & secure for newly created cookies.
This should've been the default all along.
The http_response_cookie() no longer returns a pointer to http_cookie
but rather takes it as a parameter and will populate the pointer with
the newly created http_cookie if not NULL.
Additionally http_response_cookie() automatically sets the domain
based on the http_request passed into the function.
Having the create, build, run tools baked into the kore binary
made things harder then they had to be for multiple projects with
each different build flavors.
So move away this functionality into a new "kodev" (name may change)
binary that is installed next to kore.
The new build tools will automatically pick up the correct flavors
the kore binary it points to is installed with. Or for single builds
what flavors where enabled.
The new tool also will honor looking into PREFIX for the kore binary
when doing a `kodev run`.
Additionally add a new command "info" that shows some basic info
about your project and how it will be built. For example it will
show you the flavors of the kore binary installed on the system
or the flavors you configured for a single binary build.
Obligitory, hacking on a plane comment.
- split up writing of cookies into its own function.
- turn maxage into a signed int and use -1 for it not being set.
- lots of style fixes
- remove HTTP_COOKIE_DEFAULT, just pass 0 if you don't want flags.
This commit adds the ability to use python "await" to suspend
execution of your page handler until the query sent to postgresql
has returned a result.
This is built upon the existing asynchrous query framework Kore had.
With this you can now write stuff like:
async def page(req):
result = await req.pgsql("db", "SELECT name FROM table");
req.response(200, json.dumps(result).encode("utf-8"))
The above code will fire off a query and suspend itself so Kore can
take care of business as usual until the query is successful at which
point Kore will jump back into the handler and resume.
This does not use threading, it's purely based on Python's excellent
coroutines and generators and Kore its built-in pgsql support.
These functions are created by the cli tool when building
and follow the naming format: asset_serve_<name>_<ext>().
Those serving functions can be used directly in handlers and
callthrough to a http_serveable() function that uses the SHA1
of the asset as its ETag and automatically checks for if-none-match.
- Change pools to use mmap() for allocating regions.
- Change kore_malloc() to use pools for commonly sized objects.
(split into multiple of 2 buckets, starting at 8 bytes up to 8192).
- Rename kore_mem_free() to kore_free().
The preallocated pools will hold up to 128K of elements per block size.
In case a larger object is to be allocated kore_malloc() will use
malloc() instead.
- Build with -O2 unless NOOPT is set to 1.
- Hide -g behind DEBUG instead of always building with it.
- Explicitely set the standard used to c99, use pedantic.
This commit is a flag day, your old modules will almost certainly
need to be updated in order to build properly with these changes.
Summary of changes:
- Offload HTTP bodies to disk if they are large (inspired by #100).
(disabled by default)
- The http_argument_get* macros now takes an explicit http_request parameter.
- Kore will now throw 404 errors almost immediately after an HTTP request
has come in instead of waiting until all data has arrived.
API changes:
- http_argument_get* macros now require an explicit http_request parameter.
(no more magic invokations).
- http_generic_404() is gone
- http_populate_arguments() is gone
- http_body_bytes() is gone
- http_body_text() is gone
- http_body_read() has been added
- http_populate_post() has been added
- http_populate_get() has been added
- http_file_read() has been added
- http_file_rewind() has been added
- http_file_lookup() no longer takes name, fname, data and len parameters.
- http_file_lookup() now returns a struct http_file pointer.
- http_populate_multipart_form() no longer takes an secondary parameter.
New configuration options:
- http_body_disk_offload:
Number of bytes after which Kore will offload the HTTP body to
disk instead of retaining it in memory. If 0 this feature is
disabled. (Default: 0)
- http_body_disk_path:
The path where Kore will store temporary HTTP body files.
(this directory does not get created if http_body_disk_offload is 0).
New example:
The upload example has been added, demonstrating how to deal with file
uploads from a multipart form.
Kore pre-allocates a kore_buf for the full size of the
incoming HTTP body ... but also was passing the full
size to the net_recv_reset() function.
Instead of this, properly read smaller chunks from the
network and append them to the body buffer as they roll in.
Allow setting it to 0 which will disable HTTP requests
that have a body (POST/PUT).
Reduce default http_body_max to 1MB by default, 10MB seems large.
Revisit to this code inspired by #100.
Joris Vink
Stanislav Yudin
Yorick de Wid
Ángel González
Stig Telfer