Commit Graph

187 Commits

Author SHA1 Message Date
Joris Vink 4ad50caa29 Large changes to the memory subsystem in kore.
- Change pools to use mmap() for allocating regions.
- Change kore_malloc() to use pools for commonly sized objects.
  (split into multiple of 2 buckets, starting at 8 bytes up to 8192).
- Rename kore_mem_free() to kore_free().

The preallocated pools will hold up to 128K of elements per block size.

In case a larger object is to be allocated kore_malloc() will use
malloc() instead.
2016-07-12 13:54:14 +02:00
Joris Vink d30921103b Code cleanup, several API breaking changes in here 2016-07-08 10:03:41 +02:00
Joris Vink 2dfd22a79a Change kore_buf_stringify() a bit.
Takes a size_t pointer as its second argument now, if not
NULL this will be populated with the length of the string
that is being returned.
2016-06-02 07:08:19 +02:00
Joris Vink fb6512259c small style changes. 2016-03-21 15:30:11 +01:00
Ángel González 1bda217d57 Based on #115 pull request, adds http_method_text, similar to http_status_text 2016-03-21 15:12:27 +01:00
Joris Vink f7b0b7c7ac Sometimes content_length wasn't properly updated. 2016-02-22 14:08:21 +01:00
Joris Vink 8fcf4762f4 Improve http_body_recv().
- If we fail to write to our temporary file error instead of fatal.
- Return KORE_RESULT_ERROR on other errors as well.
2016-02-13 15:41:37 +01:00
Joris Vink 12e1820a4f Unbreak normal POST parsing.
It was broken for > 2 fields due to recent large changes,
unfortunately i didn't catch this until now.
2016-02-06 15:32:23 +01:00
Joris Vink f4d00645ed Merge pull request #99 from oneswig/master
Add resource management as part of the kore shutdown process.
2016-02-01 19:51:27 +01:00
Joris Vink 1f5e482b8a Build option changes.
- Build with -O2 unless NOOPT is set to 1.
- Hide -g behind DEBUG instead of always building with it.
- Explicitely set the standard used to c99, use pedantic.
2016-02-01 15:33:40 +01:00
Stig Telfer 55d2451b90 Add cleanup of http_body_path pool 2016-02-01 10:10:04 +00:00
Stig Telfer 2ac6e7d41d Merge branch 'master' into oneswig 2016-01-24 13:46:38 +00:00
Stig Telfer ec73c35952 Add missing C library headers 2016-01-22 11:08:13 +00:00
Joris Vink 192f458f97 remove unused variable. 2016-01-18 11:43:44 +01:00
Joris Vink fcb86ddb8b Massive rework of HTTP layer.
This commit is a flag day, your old modules will almost certainly
need to be updated in order to build properly with these changes.

Summary of changes:

- Offload HTTP bodies to disk if they are large (inspired by #100).
  (disabled by default)
- The http_argument_get* macros now takes an explicit http_request parameter.
- Kore will now throw 404 errors almost immediately after an HTTP request
  has come in instead of waiting until all data has arrived.

API changes:

- http_argument_get* macros now require an explicit http_request parameter.
  (no more magic invokations).
- http_generic_404() is gone
- http_populate_arguments() is gone
- http_body_bytes() is gone
- http_body_text() is gone
- http_body_read() has been added
- http_populate_post() has been added
- http_populate_get() has been added
- http_file_read() has been added
- http_file_rewind() has been added
- http_file_lookup() no longer takes name, fname, data and len parameters.
- http_file_lookup() now returns a struct http_file pointer.
- http_populate_multipart_form() no longer takes an secondary parameter.

New configuration options:

- http_body_disk_offload:
	Number of bytes after which Kore will offload the HTTP body to
	disk instead of retaining it in memory. If 0 this feature is
	disabled. (Default: 0)

- http_body_disk_path:
	The path where Kore will store temporary HTTP body files.
	(this directory does not get created if http_body_disk_offload is 0).

New example:

The upload example has been added, demonstrating how to deal with file
uploads from a multipart form.
2016-01-18 11:30:22 +01:00
Joris Vink e580e6678e Simplify checking filename component. 2016-01-08 21:19:37 +01:00
Joris Vink ce874f09ea Cut memory usage for incoming HTTP bodies in half.
Kore pre-allocates a kore_buf for the full size of the
incoming HTTP body ... but also was passing the full
size to the net_recv_reset() function.

Instead of this, properly read smaller chunks from the
network and append them to the body buffer as they roll in.
2016-01-08 20:56:21 +01:00
Joris Vink 0647901ef5 Improve http_body_max directive a bit.
Allow setting it to 0 which will disable HTTP requests
that have a body (POST/PUT).

Reduce default http_body_max to 1MB by default, 10MB seems large.

Revisit to this code inspired by #100.
2016-01-08 17:54:40 +01:00
Joris Vink 0c47574fe9 If a body is too large send a 413 instead of 411. 2016-01-08 17:47:15 +01:00
Joris Vink f2d8834e8e After receiving a body, prime the netbuf again.
Not doing this results in nothing being read anymore
from the connection after a POST with a content-length of > 0.

Found by Xuning Niu.
2016-01-07 10:40:33 +01:00
Stig Telfer 18d3cc032d rename *_fini to *_cleanup 2016-01-04 21:40:14 +00:00
Joris Vink c4b1206ae3 Bump copyright to 2016. 2016-01-04 12:58:51 +01:00
Stig Telfer 0c51d9da53 Add resource management as part of the kore shutdown process. 2015-12-29 19:39:39 +00:00
Joris Vink 6ac15d37cb Make sure user agent is still set properly. 2015-12-10 09:25:22 +01:00
Joris Vink 4fd6d8a7a4 Correct usage of http_request_header().
Since latest change we no longer need free its result.
2015-11-29 17:22:30 +01:00
Joris Vink 7bdae240cf Change semantics for http_request_header().
The result returned by this function no longer needs to
be freed by the caller.
2015-11-29 14:19:44 +01:00
Joris Vink 769c78a6e8 Introduce NOHTTP=1 build option.
This basically turns off the HTTP layer for Kore. It does not
compile in anything for HTTP.

This allows Kore to be used as a network application platform as well.
Added an example for this called nohttp.

Other changes that sneaked in while hacking on this:
* Use calloc(), kill pendantic malloc option.
* Killed off SPDY/3.1 support completely, will be superseded by http2

Note that comes with massive changes to a lot of the core API
functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
Joris Vink a20fb00789 req can be NULL if we are coming from an error path. 2015-08-02 16:53:40 +02:00
Joris Vink be3fa17201 Do not send connection:close for websocket upgrades. 2015-06-30 16:43:21 +02:00
Cleve Lendon 18becccd5a Fix indentation. 2015-06-30 18:07:45 +09:00
Cleve Lendon fef3ee8efe Fix websockets. Connection must be Upgrade, not keep-alive, Upgrade 2015-06-26 12:14:01 +09:00
Joris Vink a97e18bbe9 Shuffle some HTTP_REQUEST_NO_CONTENT_LENGTH around 2015-05-19 09:13:29 +02:00
Joris Vink e33710b26a Wrap expression with some additional parenthesis. 2015-05-19 09:04:46 +02:00
Quentin Perez ff4faa98b7 fix-null-dereference
http.c:1214, req->method
http.c:1272, req->flags
2015-05-18 23:42:50 +02:00
Joris Vink 1dd7407a41 Close SPDY streams if len == 0 and SPDY_NO_CLOSE is not set. 2015-05-15 19:16:35 +02:00
Joris Vink cefeaf7992 HTTP layer improvements.
Add HTTP_REQUEST_NO_CONTENT_LENGTH which can be set by
a handler before calling http_response() to avoid Kore
from setting the content-length altogether.

If we are on a SPDY connection do not close the stream
if we do not pass data to http_response().
2015-05-15 19:12:18 +02:00
Joris Vink 09ed61187b Unfuck content-length changes.
We want a content-length unless its 204 or status < 200.
2015-05-13 09:16:06 +02:00
Joris Vink 7ce742f131 Merge pull request #48 from thorduri/content-length
Always write out content-length;
2015-05-13 09:02:58 +02:00
Thordur Bjornsson a62bce3e88 Always write out content-length;
Unless status is 204 or 1xx (informational).
2015-05-12 20:23:47 +02:00
Joris Vink 4f36b534aa Reset fsm_state back to 0 after we're done with it. 2015-05-08 21:19:20 +02:00
Joris Vink e5f6d37cd2 Make parameter validation less noisy. 2015-04-27 15:35:02 +02:00
Joris Vink dc6e6fb1b4 Kill NETBUF_CALL_CB_ALWAYS for incoming HTTP body
Do not let the NETBUF_CALL_CB_ALWAYS flag linger in
the netbuf we reset for the incoming HTTP body.
2015-04-21 11:30:59 +02:00
Joris Vink dfea3cf483 Place path and host in http_request into pools.
Also change the amount of times we end of doing unneccessairy strlen() calls.
2015-04-15 12:40:15 +02:00
Joris Vink 097a1166df Improve very heavy load handling.
Introduces two new configuration knobs:
	* socket_backlog (backlog for listen(2))
	* http_request_limit

The second one is the most interesting one.

Before, kore would iterate over all received HTTP requests
in its queue before returning out of http_process().

Under heavy load this queue can cause Kore to spend a considerable
amount of time iterating over said queue. With the http_request_limit,
kore will process at MOST http_request_limit requests before returning
back to the event loop.

This means responses to processed requests are sent out much quicker
and allows kore to handle any other incoming requests more gracefully.
2015-04-09 15:29:44 +02:00
Joris Vink af865abede Bump copyright to 2015 2015-04-07 13:08:26 +02:00
Joris Vink cf028f20b6 Rename kore_auth() to kore_auth_run(). 2015-04-02 13:45:42 +02:00
Joris Vink b234b7ed65 Add HTTP_REQUEST_RETAIN_EXTRA flag to http_request
Signals Kore to not free any pointer set in req->hdlr_extra.
Useful in certain scenarios where you have data per request
bound to something in memory but do not want to lose it when
the request is freed by Kore.

Set this flag before your handler returns.
2015-03-27 23:23:21 +01:00
Joris Vink ad5d662556 Use net_recv_reset() if more body bytes are coming.
Unbreaks clients that send the headers first and the
actual HTTP body after X amount of time.
2015-03-16 16:45:13 +01:00
Joris Vink 91c44af6f0 Always include includeSubDomains for HSTS. 2015-02-10 17:14:07 +01:00
Joris Vink 1507f88ec7 Respect keepalive when throwing HTTP errors.
Report from github issue #34
2015-02-06 10:59:15 +01:00