These 2 functions can be used to move an HTTP request
from/to the active http_requests list. Effectively
putting them to "sleep" or "waking them up".
Sprinkle this through the pgsql and task code.
If used correctly greatly reduces overhead for
managing sleeping tasks.
Synchronize access to state/result properly so one
can access these from inside the task as well.
Introduce KORE_TASK_STATE_ABORT which will be set
when a task needs to be abort. You can use this
to create tasks that run in a loop until aborted.
Instead if a task is used from inside a request
you MUST call kore_task_bind_request() on it.
This way we can move forward for tasks that
don't belong to page handlers.
Also, some bug fixes for removing http_requests
that are indeed linked to a currently running task.
Tasks are now assigned to available threads instead
of a global task list.
You can now pass messages between your page handler
and the created task using the kore_task_channel_*
functions.
Only one task per time can be assigned to a request
but I feel this is probably a bad design choice.
Preferably we'd want to be able to start tasks
regardless of being in a page handler or not,
this not only ads flexibility but seems like
a better choice overall as it opens a lot more
possibilities about how tasks can be used.
Don't wait for a full event loop until we call the page handler
for a received pgsql result. This speeds up page loads using
KORE_PGSQL by quite a lot, especially on a non busy server.
Including but not limited to:
- Correctly use PQerrorMessage() in case we cleanup with PQfinish
- If we get a network error, cleanup the connection
- No longer call the page handler from inside kore_pgsql_handle()
but instead just put it to sleep in case we don't need it.
This does grow the http_requests list quite a bit with sleeping
connections and can perhaps be improved later on.
- Allow us to on error return OK from a page handler from inside
the completetion block for KORE_PGSQL().
- Count the cummulative time for a request to finish instead
of the latest run time for the handler.
Has support for full async pgsql queries. Most of the logic
is hidden behind a KORE_PGSQL() macro allowing you to insert
these pgsql calls in your page handlers without blocking the
kore worker while the query is going off.
There is place for improvement here, and perhaps KORE_PGSQL won't
stay as I feel this might overcomplicate things instead of making
them simpler as I thought it would.
Using authentication blocks one can define "authentication" mechanisms
in Kore for page handlers.
This can be used to require a session cookie (validated by your own validator)
for certain page handlers, and hopefully in the future provide a framework
for adding more authentication things (like HTTP Auth).
Right now only cookie checking is available.
- Parameter validation is now done only when http_process_*()
is called and upon http_argument_add().
- You MUST have defined your params in a param block or they will
be filtered out.
- http_argument_lookup() is dead, welcome http_argument_get() and
its brothers and sisters:
http_argument_get_string()
http_argument_get_uint16()
http_argument_get_int16()
http_argument_get_uint32()
http_argument_get_int32()
They will automatically do bounds checking on integers for you
and return proper integers or a NUL-terminated string.
- The http_argument_get* functions no longer create an additional
copy of the string which you need to free. Easier going.
- http_multiple_args() is dead, byebye
- Make some stuff we don't want to share with the modules static.
- Attempt to chain as much as we can in the send netbufs
(keeps down the SSL_write calls, silly seeing it go out with 8 bytes)
- Change NETBUF_SEND_PAYLOAD_MAX to 4K
- Call SSL_write() with max NETBUF_SEND_PAYLOAD_MAX in size
- Go back to flushing the send buffers after each request
- No more need for a fixed pool for nb->buf, go back to the heap for now
- Disable Nagle, we're doing the chaining now anyway
No longer takes callbacks, flags, or *out arguments.
Update rest of the code that called these callbacks whenever sending
was completed, instead call them right away now.
Allows you to configure maximum amount of seconds an HTTP connection
can stay open (does not affect SPDY connections). If set to 0 it will
disable keep-alive all together.
Add some inttypes fluff.
If enabled Kore adds the HSTS header to every response.
- Additionally, fix some typos in the example configuration.
- Change default SSL cipher list again, no more RC4 and almost PFS for all browsers.
- http_header_max:
Maximum size of HTTP headers (in non SPDY connections).
- http_postbody_max:
Maximum size of an HTTP POST body (both in SPDY and HTTP mode).
Right now Kore will simply DC the client, ideally we want to send
a 413 (entity too large) to the client however.
See modules/examples/module.conf for more.
New API functions (docs need to be updated):
- http_file_lookup()
- http_file_add()
- http_argument_add()
- kore_strip_chars()
- kore_mem_find()
- Add an example under the example module on how files can be read.
- Keep HTTP requests in connection, so we can delete them if the connection
ends before the requests do (this way we don't leak them).
- When spdy_stream_close() is called, delete the attached http request.
(This shouldn't hurt to do, so hopefully won't cause major fallout).
- When parsing HTTP, find the first occurence of end-of-headers so uploads
with multipart/form-data can succeed properly.
- Add a test upload page to the example module.
Prototypes:
int http_argument_multiple_lookup(struct http_req *req,
struct http_arg *args);
void http_argument_multiple_free(struct http_arg *args);
These functions can be used to lookup arguments in a single call.
args points to an array of struct http_arg elements. Each of them
have the argument name set and its value set to NULL.
The array must have its last element name field set to NULL.
Upon return http_argument_multiple_lookup() gives the caller the
number of arguments that were successfully found. It makes their values
available under the value field in the struct http_arg array passed.
Example:
int v;
struct http_args args[4];
memset(args, 0, sizeof(args));
args[0].name = "email";
args[1].name = "password1";
args[2].name = "password2";
args[3].name = NULL;
v = http_argument_multiple_lookup(req, args);
if (v != 3) {
kore_debug("argument %s was not present", args[v].name);
} else {
for (v = 0; args[v].name != NULL; v++)
kore_debug("%s -> %s", args[v].name, args[v].value);
}
http_argument_multiple_free(args);
This is useful to track down any issues you might have in your module.
A log entry with a page handler causing issues looks like:
Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11
Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone
Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
- Introduce own memory management system on top of malloc to keep track
of all our allocations and free's. Later we should introduce a pooling
mechanism for fixed size allocations (http_request comes to mind).
- Introduce ssl_cipher in configuration.
Memory usage is kind of high right now, but it seems its OpenSSL
doing it rather then Kore.
- make sure we dont free nb->buf if its NULL. (semantics).
- remove some superfluffy debug.
- make sure we call [inflate|deflate]End when we dc a client that used zlib.
http_populate_arguments() before using http_argument_lookup() to lookup a given argument.
population uses the appropriate query string (from GET or POST) automatically.
by calling http_response_header_add().
fix wrong overflow check in spdy_stream_get_header().
html_inject now exports last modified timestamp for the files that are
compiled into the module in the format static_mtime_<type>_<name>.
modules can now look into the request headers using http_request_header_get().
introduce net_send_flush() and net_recv_flush() for this purpose, we attempt to make as much headway as possible until we reach EAGAIN or until we can simply pickup again later.
should merge all the stuff in http_response() into a single send buffer, **out is in place in net_send_queue() for that purpose.
Joris Vink