Commit Graph

440 Commits

Author SHA1 Message Date
Joris Vink a1e6588085 Repopulate now when we are doing idle timer checks.
Otherwise in a weird scenario the idle timer could
be started with a more recent time then the time we
test against causing a connection to disconnect
before its idle timer was over.
2014-11-21 16:38:23 +01:00
Joris Vink 9184257d97 Properly order linker arguments.
Fixes broken linkages with LDFLAGS on certain platforms see #23.
2014-11-19 14:20:31 +01:00
Joris Vink 27b4447a30 Allow workers to bind to specific CPUs on FreeBSD. 2014-10-26 22:41:55 +01:00
Joris Vink 9512805c04 spacing 2014-10-26 22:11:13 +01:00
Joris Vink 6de0f8568a Rework net, worker and some http internals.
- The net code no longer has a recv_queue, instead reuse same recv buffer.
- Introduce net_recv_reset() to reset the recv buffer when needed.
- Have the workers spread the load better between them by slightly
  delaying their next accept lock and giving them an accept treshold
  so they don't go ahead and keep accepting connections if they end
  up winning the race constantly between the workers.
- The kore_worker_acceptlock_release() is no longer available.

- Prepopulate the HTTP server response header that is added to each
  response in both normal HTTP and SPDY modes.
- The path and host members of http_request are now allocated on the heap.

These changes overall result better performance on a multicore machine,
especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
Thordur Bjornsson e980dcdd5c Simplify environment variables, abide by POLA.
KORE_COMPILER => CC
KORE_LDFLAGS => LDFLAGS
2014-10-22 17:44:43 +02:00
Joris Vink 9bb7001813 Make sure we can still compile with BENCHMARK=1 2014-10-18 02:38:21 +02:00
Joris Vink b49622bb1a Add CRL support.
Allow Kore to use per domain CRLs when requiring client certificates.
The require_client_cert configuration option has been renamed to a more
sane client_certificates and can optionally take a second argument
which is the CRL in pem format.

You'll need a restart in case the CRLs get updated.
2014-10-18 02:32:05 +02:00
Joris Vink 523bc521c7 Generate certs with sha256 digests from now on. 2014-10-18 01:28:47 +02:00
Joris Vink 79b3ccfe7d kqueue event list should hold worker_max_connections * 2 2014-10-16 21:45:43 +02:00
Joris Vink 693a07250f bump copyright years 2014-10-14 16:18:23 +02:00
Joris Vink 16f2d00d37 No more SSLv3 allowed. 2014-10-14 15:03:04 +02:00
Joris Vink 5855a88862 Increase frequency of idle_check. 2014-10-13 16:34:42 +02:00
Joris Vink 2b2e765fe6 Allow 0 parameters for kore_pgsql_query_params() 2014-10-12 01:17:35 +02:00
Joris Vink a603b77e24 Add PUT/DELETE/HEAD methods (finally).
This commit renames certain POST centric variable and configuration
naming to the correct HTTP body stuff.

API changes include http_postbody_text() and http_postbody_bytes() to
have become http_body_text() and http_body_bytes().

The developer is still responsible for validating the method their
page handler is called with. Hopefully this becomes a configuration
option soon enough.
2014-10-08 11:03:14 +02:00
James Turner 215a4d2da2 Add required header for proper OpenBSD checking 2014-10-01 13:08:38 -04:00
Joris Vink e1be630b84 Add kore_pgsql_getlength() which will return length of a column. 2014-09-28 23:03:49 +02:00
Joris Vink 3b09683f5c Add kore_pgsql_query_params().
This function uses PQsendQueryParams() instead of the normal PQsendQuery()
allowing you to pass binary data in a cleaner fashion.

A basic call would look something like:

char *mydata = "Hello";
size_t mydata_len = strlen(mydata);

kore_pgsql_query_params(&pgsql, req,
    "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1
    mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT);

kore_pgsql_query_params() is variadic, allowing you to pass any
count of parameters where each parameter has the following:
	data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
Joris Vink 0909e6bac1 Write our assets.h with a fixed include guard. 2014-09-26 15:50:56 +02:00
Joris Vink 1c83af511d Make kore_mem_free() behave like free() and don't do anything if NULL is passed in. 2014-09-25 22:20:47 +02:00
Joris Vink e3417dea16 Remove the pgsql simple layer again.
I rather keep the old idioms instead of adding more complex things
on top of the async ones. Especially since the simple layer would
interfear with existing http state machines from your handler.
2014-09-19 15:53:22 +02:00
Joris Vink f73557c545 Use proper error message 2014-09-19 15:01:56 +02:00
Joris Vink efec7bf8cf No need to set l here 2014-09-19 15:01:45 +02:00
Joris Vink c0c3e9fb05 Use s instead of req->stream. 2014-09-19 15:01:25 +02:00
Joris Vink c4c60e1960 Oops, make sure the new simple api can handle > 1 request without borking. 2014-09-19 14:49:12 +02:00
Joris Vink f6f37437da In pgsql_simple_state_result() be sure to progress the sql state.
Drop back into waiting after our result so we can properly read out
the completed state.
2014-09-19 12:51:37 +02:00
Joris Vink 8e9c3da764 Add a new "simple query" layer to our pgsql api.
This simple query allows you to ditch rolling your own
state machine for handling async pgsql states and instead
asks you to provide 3 functions:
	- init
	- results
	- done

You can see the different in complexity in the pgsql example,
which now contains a pgsql_simple.c holding the same asynchronous
query as in pgsql.c but using the simple pgsql api.

You can of course still roll your own in case you want more control.
2014-09-19 12:32:49 +02:00
Joris Vink 73055662de Remove start from pgsql_job 2014-09-18 10:49:15 +02:00
Joris Vink 59a1a05afe No need to set now at the start, thanks clang. 2014-09-18 08:42:41 +02:00
Joris Vink 49e97ae6f8 Call kore_connection_disconnect() when draining connections.
This way the disconnect callbacks are called properly when
workers are exiting.
2014-09-17 08:46:55 +02:00
Joris Vink 7771adbec2 Allow applications to create new connections in our event loop. 2014-09-17 08:25:45 +02:00
Joris Vink f3e863a482 Properly end headers when doing status codes only. 2014-08-25 10:58:34 +02:00
Joris Vink fa69f2a49c Attempt vsnprintf() up to BUFSIZ in kore_buf_appendv().
If the result was too large fall back to vasprintf instead.
2014-08-22 14:40:19 +02:00
Joris Vink 0542d70a71 Use a single header_buf in http_response_normal()
Avoids constantly allocating and freeing buffers all the time.
This can be done safely due to the fact workers are single threaded.
2014-08-22 14:38:48 +02:00
Joris Vink 71fb54aab7 Replace - as well when building asset names 2014-08-21 23:06:54 +02:00
Joris Vink 7e74cd6e62 Introduce a wait queue for our pgsql code.
Instead of letting http_requests spin, if we cannot allocate
a connection for the request we will queue them up put them to sleep.

When a connection becomes available, we'll wake up a request that
was waiting for a connection and let it continue.

This completely avoids consuming massive amounts of cpu time
when dealing with thousands of requests waiting for a pgsql
worker to become ready.
2014-08-21 16:36:12 +02:00
Joris Vink c063dd52f6 epoll improvements.
* check for EPOLLRDHUP
* in kore_platform_schedule_read() don't use EPOLLET
2014-08-21 16:33:11 +02:00
Joris Vink d3149a2a71 We need at most 2 tokens here 2014-08-18 21:13:16 +02:00
Joris Vink 69f98ef158 Add root/src/includes as an include path when building 2014-08-18 18:56:22 +02:00
Joris Vink 68e0697f2e s/appl/rootdir in cli_compile_file 2014-08-18 15:31:10 +02:00
Joris Vink 83a16763c8 Always NUL-terminate an asset its data.
This way developers can cast assets to char * without first
having to convert it to a proper C string.

The NUL character is NOT included in the length of the asset.
2014-08-18 13:57:06 +02:00
Joris Vink 105804e325 Make http_status_text() return a const char 2014-08-18 10:37:13 +02:00
Joris Vink 873c68b515 Make http_status_text() available everywhere 2014-08-18 10:34:47 +02:00
Joris Vink 8565b47800 Attach tasks/pgsqls to http_requests once more.
This way if an http request is removed while tasks or
pgsqls are still active they are free'd out and cancelled
properly.
2014-08-14 22:05:34 +02:00
Joris Vink f021c8db44 pgsql improvements
* set pgsql->result to NULL when we freed it
* drain the connection pipe just in case in pgsql_conn_release()
2014-08-14 16:43:40 +02:00
Joris Vink 9ea1e3cb57 Properly release pgsql connection if needed 2014-08-14 14:46:21 +02:00
Joris Vink f93747828c Major pgsql rework.
Gone is the ugly KORE_PGSQL macro that hid an overly complex
state machine for the pgsql api.
Gone is the pgsql array that was attached to http_requests.
Gone are the callback hacks inside the pgsql api.

Instead, I strongly encourage people to use the new state machine
api Kore offers to properly deal with asynchronous queries.

The pgsql example in examples/pgsql has been updated to reflect
these changes.
2014-08-14 14:34:23 +02:00
Joris Vink 856d7b0cb2 Rename HTTP_STATE_OK to HTTP_STATE_CONTINUE 2014-08-14 11:20:04 +02:00
Joris Vink 210fb05a35 Add a hopefully easy to use and optional fsm for page handlers
In order to use this, define states for your page handler:

struct http_state mystates[] = {
	{ "PAGE_STATE_INIT",	page_init },
	{ "PAGE_STATE_RESULT",	page_result },
};

In your page handler you can then simply call http_state_run() with
your states and http_request. This will cause Kore to start calling
your state callbacks beginning at index 0.

State callbacks have the same prototype as page handlers:

int	func(struct http_request *);

However, unlike page handlers they MUST return one of the following:

- HTTP_STATE_OK:	All good, just continue the fsm.
- HTTP_STATE_ERROR:	Abort fsm and return KORE_RESULT_OK to Kore
			(This will cancel the http request).
- HTTP_STATE_RETRY:	Return KORE_RESULT_RETRY to Kore.
			(Kore will retry your page handler next event loop).
- HTTP_STATE_COMPLETE:	The fsm completed, break out cleanly.

Note that using this is completely optional and you can still
use the traditional way of writing page handlers.

The fsm is designed to get rid of the clutter that exists today
in Kore when dealing with non blocking tasks or pgsql calls.
2014-08-13 23:34:30 +02:00
Joris Vink 2e7d6a12a9 closedir() when we're ready traversing it 2014-08-13 14:32:57 +02:00
Joris Vink 29afdd0b21 Make sure opt always contains what we want 2014-08-13 14:32:49 +02:00
Joris Vink 0b907312ef let kore build generate certs with a more unique issuer
also fix spacing and asset print information.
2014-08-11 16:11:37 +02:00
Joris Vink d74f039a0c When building assets replace bad characters with an underscore 2014-08-11 13:08:26 +02:00
Joris Vink 97c17f724b Add kore_snprintf() as a wrapper around snprintf(). 2014-08-11 10:45:10 +02:00
Joris Vink 4a4fa4889e Another round of spdy/3.1 improvements.
* Implement flow control window updates if we're receiving data (POSTs)
2014-08-10 21:39:47 +02:00
Joris Vink c665b7d926 Add a callback to http_response_stream().
This way we can get our code called whenever a stream is
completed. This cb handler does stand alone from an http_request
and is passed a netbuf data structure.
2014-08-10 18:46:44 +02:00
Joris Vink 10284d59b6 Another round of spdy/3.1 improvements.
* Always make sure we end the stream properly
* Check for SPDY_FLOW_WINDOW_MAX on window frame updates
* Kill SPDY_STREAM_BLOCKING, once flow control kicks in its per session
2014-08-10 18:18:27 +02:00
Joris Vink 5cca2f1f78 properly implement spdy/3.1 flow control 2014-08-08 14:18:15 +02:00
Joris Vink d2c65b4f62 Change http_response_stream() to only take a base + len 2014-08-07 14:31:45 +02:00
Joris Vink cda09b6065 Add http_response_stream() which can stream data from a buffer to the client. 2014-08-07 14:23:26 +02:00
Joris Vink e3ae1b4e2d Implement some great SPDY improvements.
* Bump spdy announcement to the correct draft version
* When receiving a RST, clean out the netbuf chain of that stream
2014-08-07 10:22:54 +02:00
Joris Vink 966ed3d20d Rename the two functions used to read/set headers.
Rename http_request_header_get() to http_request_header().
Rename http_response_header_add() to http_response_header().
2014-08-05 14:26:31 +02:00
Joris Vink 0413de6a8f fatal if the return value from a page handler isn't what we want. 2014-08-05 14:17:23 +02:00
Joris Vink b27322933b The default pid file is kore.pid so no need to set that explicitely 2014-08-05 13:11:53 +02:00
Joris Vink 577462379d Deprecate ssl_no_compression config option, its always on. 2014-08-05 13:07:32 +02:00
Joris Vink b0700162c4 Make http_post_data_bytes() and http_post_data_text() available 2014-08-04 21:03:26 +02:00
Joris Vink 9b3ff7db53 Lowercase the HTTP headers sent in normal HTTP responses.
SPDY already had this, as required by the spec.
2014-08-04 20:12:31 +02:00
Joris Vink 4010bdd58d Remove kore_cb and its related settings.
After revisiting why this exists in Kore I decided it
does not belong in this platform and instead of letting
it sit there staring at me I rather just kill it.
2014-08-04 19:54:32 +02:00
Joris Vink 0b36877736 Change kore_buf_appendv() to use vasprintf(). 2014-08-04 18:59:59 +02:00
Joris Vink 7b9b1d5c2c sprinkle more const chars 2014-08-04 12:40:21 +02:00
Joris Vink 31eac77cd2 sprinkle const char where appropriate 2014-08-04 09:48:41 +02:00
Joris Vink d39246e0e2 static.h -> assets.h 2014-08-03 18:22:12 +02:00
Joris Vink 3bb77bbb9f Even if we skip files to be built, they still gotta be linked in 2014-08-03 18:18:48 +02:00
Joris Vink e906aab445 Be careful that f isn't uninitialized 2014-08-03 17:50:04 +02:00
Joris Vink f4f8b1e8b0 Allow users to set ldflags via KORE_LDFLAGS 2014-08-03 17:44:29 +02:00
Joris Vink 2b40672ba1 When compiling with PGSQL, note the include dir for "kore build" 2014-08-03 17:33:40 +02:00
Joris Vink 73b0569fe0 Missing fclose() 2014-08-03 17:17:11 +02:00
Joris Vink 8bcfb6cbb3 Be consistent with the help text 2014-08-03 17:11:02 +02:00
Joris Vink 93fd33e00e Big changes to how we handle "static" files.
* Rename static files to assets
* static_* has become asset_*
* Always generate asset.h with the right external symbols
2014-08-03 16:55:16 +02:00
Joris Vink 380b7dc1d7 Improvement on the cli commands a bit.
* kore build will now only rebuild what was changed (checking on
  last modified timestamp, not 100% accurate but it'll do).

* introduce kore clean which cleans out object files and the
  resulting .so file
2014-08-03 16:36:16 +02:00
Joris Vink e74921222c Check if conf/appl.conf exists before continuing with build or run 2014-08-03 15:30:53 +02:00
Joris Vink f538e92172 Correct typo 2014-08-03 15:22:08 +02:00
Joris Vink 0e2f478c75 Fix http_argument_get_*() integer functions. 2014-08-03 15:20:20 +02:00
Joris Vink a2897f790c Better sane defaults for kore build (gen certs + ssl_no_compression) 2014-08-03 15:17:03 +02:00
Joris Vink 5b8c880a62 Don't write the static output to a tmp buf, just write it to disk 2014-08-01 20:25:02 +02:00
Joris Vink 6495e84005 Show the uri to the running kore instance in foreground mode 2014-08-01 20:00:09 +02:00
Joris Vink 23164f636f Sprinkle KORE_BENCHMARK through cli.c 2014-08-01 17:17:34 +02:00
Joris Vink 38edc7d23b Add a warning about the generated certificates 2014-08-01 17:13:38 +02:00
Joris Vink 47ad4fbc28 Have "kore build" generate temp certs as well 2014-08-01 17:10:47 +02:00
Joris Vink 448f1a7912 Don't run cli_cleanup_files() in cli_fatal if theres no rootdir 2014-08-01 16:39:12 +02:00
Joris Vink 5fa964c5c0 gettimeofday() requires sys/time.h 2014-08-01 16:10:37 +02:00
Joris Vink 67c3bd84ba Make cli compile under freebsd 2014-08-01 16:01:33 +02:00
Joris Vink e29a5461ca Allow kore build to compile static files 2014-08-01 15:55:09 +02:00
Joris Vink 61f56b8da0 Bump copyright 2014-08-01 14:14:46 +02:00
Joris Vink 5d8c29af13 Allow kore build to take compiler via KORE_COMPILER 2014-08-01 14:09:57 +02:00
Joris Vink ea5b89d20b Move orbit functionality into kore directly.
Makes more sense and reads easier:

kore create myapp
kore build myapp
kore run myapp

Note that kore retains its cli options (if no command was given),
meaning you can still start kore in the traditional way as well.

The command options are simply to make development easier.
2014-08-01 13:59:47 +02:00
Joris Vink 818dd84acd No longer generate .objs as a default for orbit create. 2014-08-01 12:45:31 +02:00
Joris Vink e57a91a4e0 Create .objs when doing orbit build if it doesnt exist 2014-08-01 12:40:37 +02:00
Joris Vink 0c21c08b6d Hook orbit into the build 2014-08-01 10:46:50 +02:00
Joris Vink a2a87adf19 Euhm, revert last commit. 2014-08-01 10:33:26 +02:00