Also let this function reset offset and lengths for http_body_read().
Make sure of this function in the python code so req.body can be called
multiple times in succession.
The only reason you would want to directly modify the cookie
after creating it should be to unset the HTTPONLY or SECURE flags
if that is what you *really* want to do.
Change http_response_cookie() to take all required parameters instead
of having to marshall those in yourself after.
Now you set a sane default cookie in one shot:
http_response_cookie(req, "key", "value", "/", 0, -1, NULL);
Which would create a session cookie key=value for / under the current domain.
We now default to httponly & secure for newly created cookies.
This should've been the default all along.
The http_response_cookie() no longer returns a pointer to http_cookie
but rather takes it as a parameter and will populate the pointer with
the newly created http_cookie if not NULL.
Additionally http_response_cookie() automatically sets the domain
based on the http_request passed into the function.
Having the create, build, run tools baked into the kore binary
made things harder then they had to be for multiple projects with
each different build flavors.
So move away this functionality into a new "kodev" (name may change)
binary that is installed next to kore.
The new build tools will automatically pick up the correct flavors
the kore binary it points to is installed with. Or for single builds
what flavors where enabled.
The new tool also will honor looking into PREFIX for the kore binary
when doing a `kodev run`.
Additionally add a new command "info" that shows some basic info
about your project and how it will be built. For example it will
show you the flavors of the kore binary installed on the system
or the flavors you configured for a single binary build.
Obligitory, hacking on a plane comment.
- split up writing of cookies into its own function.
- turn maxage into a signed int and use -1 for it not being set.
- lots of style fixes
- remove HTTP_COOKIE_DEFAULT, just pass 0 if you don't want flags.
This commit adds the ability to use python "await" to suspend
execution of your page handler until the query sent to postgresql
has returned a result.
This is built upon the existing asynchrous query framework Kore had.
With this you can now write stuff like:
async def page(req):
result = await req.pgsql("db", "SELECT name FROM table");
req.response(200, json.dumps(result).encode("utf-8"))
The above code will fire off a query and suspend itself so Kore can
take care of business as usual until the query is successful at which
point Kore will jump back into the handler and resume.
This does not use threading, it's purely based on Python's excellent
coroutines and generators and Kore its built-in pgsql support.
These functions are created by the cli tool when building
and follow the naming format: asset_serve_<name>_<ext>().
Those serving functions can be used directly in handlers and
callthrough to a http_serveable() function that uses the SHA1
of the asset as its ETag and automatically checks for if-none-match.
- Change pools to use mmap() for allocating regions.
- Change kore_malloc() to use pools for commonly sized objects.
(split into multiple of 2 buckets, starting at 8 bytes up to 8192).
- Rename kore_mem_free() to kore_free().
The preallocated pools will hold up to 128K of elements per block size.
In case a larger object is to be allocated kore_malloc() will use
malloc() instead.
- Build with -O2 unless NOOPT is set to 1.
- Hide -g behind DEBUG instead of always building with it.
- Explicitely set the standard used to c99, use pedantic.
This commit is a flag day, your old modules will almost certainly
need to be updated in order to build properly with these changes.
Summary of changes:
- Offload HTTP bodies to disk if they are large (inspired by #100).
(disabled by default)
- The http_argument_get* macros now takes an explicit http_request parameter.
- Kore will now throw 404 errors almost immediately after an HTTP request
has come in instead of waiting until all data has arrived.
API changes:
- http_argument_get* macros now require an explicit http_request parameter.
(no more magic invokations).
- http_generic_404() is gone
- http_populate_arguments() is gone
- http_body_bytes() is gone
- http_body_text() is gone
- http_body_read() has been added
- http_populate_post() has been added
- http_populate_get() has been added
- http_file_read() has been added
- http_file_rewind() has been added
- http_file_lookup() no longer takes name, fname, data and len parameters.
- http_file_lookup() now returns a struct http_file pointer.
- http_populate_multipart_form() no longer takes an secondary parameter.
New configuration options:
- http_body_disk_offload:
Number of bytes after which Kore will offload the HTTP body to
disk instead of retaining it in memory. If 0 this feature is
disabled. (Default: 0)
- http_body_disk_path:
The path where Kore will store temporary HTTP body files.
(this directory does not get created if http_body_disk_offload is 0).
New example:
The upload example has been added, demonstrating how to deal with file
uploads from a multipart form.
Kore pre-allocates a kore_buf for the full size of the
incoming HTTP body ... but also was passing the full
size to the net_recv_reset() function.
Instead of this, properly read smaller chunks from the
network and append them to the body buffer as they roll in.
Allow setting it to 0 which will disable HTTP requests
that have a body (POST/PUT).
Reduce default http_body_max to 1MB by default, 10MB seems large.
Revisit to this code inspired by #100.
This basically turns off the HTTP layer for Kore. It does not
compile in anything for HTTP.
This allows Kore to be used as a network application platform as well.
Added an example for this called nohttp.
Other changes that sneaked in while hacking on this:
* Use calloc(), kill pendantic malloc option.
* Killed off SPDY/3.1 support completely, will be superseded by http2
Note that comes with massive changes to a lot of the core API
functions provided by Kore, these might break your application.
Add HTTP_REQUEST_NO_CONTENT_LENGTH which can be set by
a handler before calling http_response() to avoid Kore
from setting the content-length altogether.
If we are on a SPDY connection do not close the stream
if we do not pass data to http_response().
Introduces two new configuration knobs:
* socket_backlog (backlog for listen(2))
* http_request_limit
The second one is the most interesting one.
Before, kore would iterate over all received HTTP requests
in its queue before returning out of http_process().
Under heavy load this queue can cause Kore to spend a considerable
amount of time iterating over said queue. With the http_request_limit,
kore will process at MOST http_request_limit requests before returning
back to the event loop.
This means responses to processed requests are sent out much quicker
and allows kore to handle any other incoming requests more gracefully.
Signals Kore to not free any pointer set in req->hdlr_extra.
Useful in certain scenarios where you have data per request
bound to something in memory but do not want to lose it when
the request is freed by Kore.
Set this flag before your handler returns.
Introduces a few new api functions:
- kore_websocket_handshake(struct http_request *):
Performs the handshake on an HTTP request (coming from page handler)
- kore_websocket_send(struct connection *, u_int8_t, void *, size_t):
Sends data to a websocket connection.
- kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int):
Broadcast the given websocket op and data to all connected
websocket clients on the worker. Note that as of right now
the WEBSOCKET_BROADCAST_GLOBAL scope option does not work
yet and messages broadcasted will be restricted to workers
only.
- kore_worker_websocket_broadcast(struct connection *, void *, void *):
Backend function used by kore_websocket_broadcast().
Could prove useful for developers to have access to.
A simple example is given under examples/websocket.
Known issues:
Kore does not support PING or CONT frames just yet.
- The net code no longer has a recv_queue, instead reuse same recv buffer.
- Introduce net_recv_reset() to reset the recv buffer when needed.
- Have the workers spread the load better between them by slightly
delaying their next accept lock and giving them an accept treshold
so they don't go ahead and keep accepting connections if they end
up winning the race constantly between the workers.
- The kore_worker_acceptlock_release() is no longer available.
- Prepopulate the HTTP server response header that is added to each
response in both normal HTTP and SPDY modes.
- The path and host members of http_request are now allocated on the heap.
These changes overall result better performance on a multicore machine,
especially the worker load changes shine through.
This commit renames certain POST centric variable and configuration
naming to the correct HTTP body stuff.
API changes include http_postbody_text() and http_postbody_bytes() to
have become http_body_text() and http_body_bytes().
The developer is still responsible for validating the method their
page handler is called with. Hopefully this becomes a configuration
option soon enough.
Instead of letting http_requests spin, if we cannot allocate
a connection for the request we will queue them up put them to sleep.
When a connection becomes available, we'll wake up a request that
was waiting for a connection and let it continue.
This completely avoids consuming massive amounts of cpu time
when dealing with thousands of requests waiting for a pgsql
worker to become ready.
Gone is the ugly KORE_PGSQL macro that hid an overly complex
state machine for the pgsql api.
Gone is the pgsql array that was attached to http_requests.
Gone are the callback hacks inside the pgsql api.
Instead, I strongly encourage people to use the new state machine
api Kore offers to properly deal with asynchronous queries.
The pgsql example in examples/pgsql has been updated to reflect
these changes.
In order to use this, define states for your page handler:
struct http_state mystates[] = {
{ "PAGE_STATE_INIT", page_init },
{ "PAGE_STATE_RESULT", page_result },
};
In your page handler you can then simply call http_state_run() with
your states and http_request. This will cause Kore to start calling
your state callbacks beginning at index 0.
State callbacks have the same prototype as page handlers:
int func(struct http_request *);
However, unlike page handlers they MUST return one of the following:
- HTTP_STATE_OK: All good, just continue the fsm.
- HTTP_STATE_ERROR: Abort fsm and return KORE_RESULT_OK to Kore
(This will cancel the http request).
- HTTP_STATE_RETRY: Return KORE_RESULT_RETRY to Kore.
(Kore will retry your page handler next event loop).
- HTTP_STATE_COMPLETE: The fsm completed, break out cleanly.
Note that using this is completely optional and you can still
use the traditional way of writing page handlers.
The fsm is designed to get rid of the clutter that exists today
in Kore when dealing with non blocking tasks or pgsql calls.
This way we can get our code called whenever a stream is
completed. This cb handler does stand alone from an http_request
and is passed a netbuf data structure.
* Always make sure we end the stream properly
* Check for SPDY_FLOW_WINDOW_MAX on window frame updates
* Kill SPDY_STREAM_BLOCKING, once flow control kicks in its per session
Joris Vink
Stanislav Yudin
Yorick de Wid
Ángel González
Stig Telfer
Cleve Lendon
Quentin Perez
Thordur Bjornsson