A true application dependant limit check would require stream parsing.
As the limit enforcement was done, it added nothing of value compared
to HTTP request limit check, which is in Kore already.
The API surface is very limited. Jsonrpc support reads request from HTTP
body and such can't be activated if NOHTTP=1. At the moment there is no
websocket support either (which is a shame). It depends upon the
third-party Yajl library.
Errors can be emitted using jsonrpc_error() and results using
jsonrpc_result(), for the later you'll have to provide a callback which
will write the inner of the result object.
If errors happen during the response write process, no further error
output will be attempted and an HTTP error 500 will be returned.
Read the provided example for getting a better idea of the API.
- Change pools to use mmap() for allocating regions.
- Change kore_malloc() to use pools for commonly sized objects.
(split into multiple of 2 buckets, starting at 8 bytes up to 8192).
- Rename kore_mem_free() to kore_free().
The preallocated pools will hold up to 128K of elements per block size.
In case a larger object is to be allocated kore_malloc() will use
malloc() instead.
Kore applications now get a build.conf which may contain different
build flavors. Each flavor can get its own cflags or ldflags.
This was in parts inspired by #106.
A new cli command has been added:
kore flavor
This command allows you to see all flavors and switch between them.
The kore build command now also takes a -v argument which if given
dumps the used CFLAGS and LDFLAGS out to stdout.
For existing applications the build.conf is automatically generated
next time you run kore build or kore run.
Also fixed a bug in the json_yajl example, sneaky change here.
This commit is a flag day, your old modules will almost certainly
need to be updated in order to build properly with these changes.
Summary of changes:
- Offload HTTP bodies to disk if they are large (inspired by #100).
(disabled by default)
- The http_argument_get* macros now takes an explicit http_request parameter.
- Kore will now throw 404 errors almost immediately after an HTTP request
has come in instead of waiting until all data has arrived.
API changes:
- http_argument_get* macros now require an explicit http_request parameter.
(no more magic invokations).
- http_generic_404() is gone
- http_populate_arguments() is gone
- http_body_bytes() is gone
- http_body_text() is gone
- http_body_read() has been added
- http_populate_post() has been added
- http_populate_get() has been added
- http_file_read() has been added
- http_file_rewind() has been added
- http_file_lookup() no longer takes name, fname, data and len parameters.
- http_file_lookup() now returns a struct http_file pointer.
- http_populate_multipart_form() no longer takes an secondary parameter.
New configuration options:
- http_body_disk_offload:
Number of bytes after which Kore will offload the HTTP body to
disk instead of retaining it in memory. If 0 this feature is
disabled. (Default: 0)
- http_body_disk_path:
The path where Kore will store temporary HTTP body files.
(this directory does not get created if http_body_disk_offload is 0).
New example:
The upload example has been added, demonstrating how to deal with file
uploads from a multipart form.
Semantics for using pgsql API have changed quite heavily
with this commit. See the examples for more information.
Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt)
with several modifications by me.
Setting the handle callback allows your application
to take care of network events for the connection.
Look at the connection state and flags to determine
if read/write is possible and go from there.
See kore_connection_handle() for more details.
* The cli tools must know when building as KORE_NO_HTTP.
* Reshuffle some structs around to avoid forward declarations.
* Move wscbs under !KORE_NO_HTTP as its for websockets.
* Remove unused members from struct connection.
Applications that use the connect callbacks for new connections
must now set the connection state themselves, see nohttp example.
This basically turns off the HTTP layer for Kore. It does not
compile in anything for HTTP.
This allows Kore to be used as a network application platform as well.
Added an example for this called nohttp.
Other changes that sneaked in while hacking on this:
* Use calloc(), kill pendantic malloc option.
* Killed off SPDY/3.1 support completely, will be superseded by http2
Note that comes with massive changes to a lot of the core API
functions provided by Kore, these might break your application.
Allows one to bind a callback to a Kore task which is called
everytime activity happens on the task channel.
Add an example as well on how this works.
Inspired by issue #68.
Change the callback prototypes to:
void callback(struct kore_msg *msg, const void *data);
This allows the callbacks to receive the full kore_msg data structure
as sent over the wire (including length and id). Useful for future
additions to the kore_msg structure (such as worker origin).
Several other improvements:
* Accesslog now uses the msg framework as well.
* Websocket WEBSOCKET_BROADCAST_GLOBAL now works.
Small websocket improvement in this commit:
* Build the frame to be sent only once when broadcasting
instead of per connection we are broadcasting towards.
With this framework apps can now send messages between worker processes.
A new API function exists:
int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t);
This API call allows your app to register a new message callback for a given ID.
You can then send messages on this ID to other workers using:
void kore_msg_send(u_int8_t id, void *data, u_int32_t length);
This framework will interally be used for a few things such as allowing
websocket data to broadcasted between all workers, adding unified caching
and hopefully eventually moving the access log to this as well.
Some internals have changed with this commit:
* worker_clients has been called connections.
* the parent now initializes the net, and event subsystems.
* kore_worker_websocket_broadcast() is dead.
Before Kore would spawn a task thread per task started
if none were available. This was an obvious bad idiom
but never really hit me hard until now.
Kore will now only spawn as many task threads as configured
by "task_threads" and queue up any newly started tasks ontop
of already running threads if the limit was hit.
At times it seems relevant that worker their modules should not
be reloaded when receiving a SIGHUP. Developers can now control
this by returning anything else but KORE_RESULT_OK from their
initialization methods.
The parent module will always be reloaded.
Add configuration setting tls_version to specify if you
either want TLSv1.2 or TLSv1.0 or both.
The configuration options ssl_cipher and ssl_dhparam
have changed name to tls_cipher and tls_dhparam. There is
no fallback so you might have to update your configs.
Signals Kore to not free any pointer set in req->hdlr_extra.
Useful in certain scenarios where you have data per request
bound to something in memory but do not want to lose it when
the request is freed by Kore.
Set this flag before your handler returns.
Introduces a few new api functions:
- kore_websocket_handshake(struct http_request *):
Performs the handshake on an HTTP request (coming from page handler)
- kore_websocket_send(struct connection *, u_int8_t, void *, size_t):
Sends data to a websocket connection.
- kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int):
Broadcast the given websocket op and data to all connected
websocket clients on the worker. Note that as of right now
the WEBSOCKET_BROADCAST_GLOBAL scope option does not work
yet and messages broadcasted will be restricted to workers
only.
- kore_worker_websocket_broadcast(struct connection *, void *, void *):
Backend function used by kore_websocket_broadcast().
Could prove useful for developers to have access to.
A simple example is given under examples/websocket.
Known issues:
Kore does not support PING or CONT frames just yet.
- The net code no longer has a recv_queue, instead reuse same recv buffer.
- Introduce net_recv_reset() to reset the recv buffer when needed.
- Have the workers spread the load better between them by slightly
delaying their next accept lock and giving them an accept treshold
so they don't go ahead and keep accepting connections if they end
up winning the race constantly between the workers.
- The kore_worker_acceptlock_release() is no longer available.
- Prepopulate the HTTP server response header that is added to each
response in both normal HTTP and SPDY modes.
- The path and host members of http_request are now allocated on the heap.
These changes overall result better performance on a multicore machine,
especially the worker load changes shine through.
This commit renames certain POST centric variable and configuration
naming to the correct HTTP body stuff.
API changes include http_postbody_text() and http_postbody_bytes() to
have become http_body_text() and http_body_bytes().
The developer is still responsible for validating the method their
page handler is called with. Hopefully this becomes a configuration
option soon enough.
This function uses PQsendQueryParams() instead of the normal PQsendQuery()
allowing you to pass binary data in a cleaner fashion.
A basic call would look something like:
char *mydata = "Hello";
size_t mydata_len = strlen(mydata);
kore_pgsql_query_params(&pgsql, req,
"INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1
mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT);
kore_pgsql_query_params() is variadic, allowing you to pass any
count of parameters where each parameter has the following:
data pointer, data length, type of parameter.
I rather keep the old idioms instead of adding more complex things
on top of the async ones. Especially since the simple layer would
interfear with existing http state machines from your handler.
This simple query allows you to ditch rolling your own
state machine for handling async pgsql states and instead
asks you to provide 3 functions:
- init
- results
- done
You can see the different in complexity in the pgsql example,
which now contains a pgsql_simple.c holding the same asynchronous
query as in pgsql.c but using the simple pgsql api.
You can of course still roll your own in case you want more control.
Gone is the ugly KORE_PGSQL macro that hid an overly complex
state machine for the pgsql api.
Gone is the pgsql array that was attached to http_requests.
Gone are the callback hacks inside the pgsql api.
Instead, I strongly encourage people to use the new state machine
api Kore offers to properly deal with asynchronous queries.
The pgsql example in examples/pgsql has been updated to reflect
these changes.
Raphaël Monrouzeau
Joris Vink
Corbin Hughes
Shih-Yuan Lee (FourDollars)
Pascal Borreli
Geenz