a1batross
/
kore
forked from mirrors/kore
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
kore/includes/kore.h

646 lines
17 KiB
C
Raw Normal View History

first commit
2013-04-17 22:34:27 +02:00
/*
Bump copyright to 2016.
2016-01-04 12:58:51 +01:00
* Copyright (c) 2013-2016 Joris Vink <joris@coders.se>
first commit
2013-04-17 22:34:27 +02:00
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __H_KORE_H
#define __H_KORE_H
Kill daemon deprication warning on osx. We get it.
2014-04-12 18:49:01 +02:00
#if defined(__APPLE__)
#define daemon portability_is_king
#endif
Include sys/types.h and unistd.h for close(), getopt(), getpid(), getuid(), setproctitle(), unlink() and declaration of optarg. Makes Kore compatible with FreeBSD too. From Ayan George <ayan@ayan.net>
2013-07-09 09:45:16 +02:00
#include <sys/types.h>
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#include <sys/time.h>
Clean up header includes, based on a diff from Ewan Higgs via github. And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
#include <sys/queue.h>
#include <netinet/in.h>
#include <arpa/inet.h>
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#if !defined(KORE_NO_TLS)
Clean up header includes, based on a diff from Ewan Higgs via github. And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
#include <openssl/err.h>
Add support for ephemeral key exchange mechanisms, ssl_dhparam configuration option must be set (and point to a file containing a generated DH key).
2013-08-07 16:51:39 +02:00
#include <openssl/dh.h>
Clean up header includes, based on a diff from Ewan Higgs via github. And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
#include <openssl/ssl.h>
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#endif
Clean up header includes, based on a diff from Ewan Higgs via github. And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
#include <errno.h>
#include <regex.h>
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#include <stdarg.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
Clean up header includes, based on a diff from Ewan Higgs via github. And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
#include <syslog.h>
Include sys/types.h and unistd.h for close(), getopt(), getpid(), getuid(), setproctitle(), unlink() and declaration of optarg. Makes Kore compatible with FreeBSD too. From Ayan George <ayan@ayan.net>
2013-07-09 09:45:16 +02:00
#include <unistd.h>
Add missing C library headers
2016-01-22 12:08:13 +01:00
#include <stdarg.h>
Clean up header includes, based on a diff from Ewan Higgs via github. And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
Add __cplusplus macros to headers.
2015-04-01 13:25:10 +02:00
#if defined(__cplusplus)
extern "C" {
#endif
Kill daemon deprication warning on osx. We get it.
2014-04-12 18:49:01 +02:00
#if defined(__APPLE__)
#undef daemon
extern int daemon(int, int);
#endif
first commit
2013-04-17 22:34:27 +02:00
#define KORE_RESULT_ERROR 0
#define KORE_RESULT_OK 1
allow handlers to return KORE_RESULT_RETRY. This will tell the worker to reschedule the page request again at the end of its list. (Allows module creators to write truely nonblocking modules).
2013-05-31 00:40:06 +02:00
#define KORE_RESULT_RETRY 2
first commit
2013-04-17 22:34:27 +02:00
More incompatible changes incoming, make it clear Semver demands it.
2015-12-09 16:29:35 +01:00
#define KORE_VERSION_MAJOR 2
#define KORE_VERSION_MINOR 0
Move towards 1.3.0-devel target, skipping 1.2.4. Mostly done due to incoming library changes.
2015-11-12 09:20:14 +01:00
#define KORE_VERSION_PATCH 0
#define KORE_VERSION_STATE "devel"
everybody loves to tout their own horn.. so introduce a server response header
2013-06-26 16:58:01 +02:00
Default to only TLSv1.2 from now on. Add configuration setting tls_version to specify if you either want TLSv1.2 or TLSv1.0 or both. The configuration options ssl_cipher and ssl_dhparam have changed name to tls_cipher and tls_dhparam. There is no fallback so you might have to update your configs.
2015-05-06 10:59:43 +02:00
#define KORE_TLS_VERSION_1_2 0
#define KORE_TLS_VERSION_1_0 1
#define KORE_TLS_VERSION_BOTH 2
first commit
2013-04-17 22:34:27 +02:00
#define errno_s strerror(errno)
Add initial SSL stuff + bare bones NPN advertisement. So far chrome connects and shows our connection as SPDY/3, good start.
2013-04-21 20:21:46 +02:00
#define ssl_errno_s ERR_error_string(ERR_get_error(), NULL)
move to a worker based threading approach where we delegate http requests to workers in a round robin basis (later this should be swapped to find the laziest worker and assign the request to that instead).
2013-05-30 19:36:42 +02:00
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#define KORE_DOMAINNAME_LEN 255
Move the default pid file to kore.pid instead of /var/run/kore.pid
2014-08-05 13:10:34 +02:00
#define KORE_PIDFILE_DEFAULT "kore.pid"
By default Kore now uses ECDH/DHE for TLS key exchanges. This commit disables RSA key exchanges for TLS completely, while introducing the requirement for always having DH parameters (ssl_dhparam). Judging from ciphersuites most modern browsers now prefer this change should be more than ok.
2015-02-03 13:17:59 +01:00
#define KORE_DEFAULT_CIPHER_LIST "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!kRSA:!kDSA"
write main process pid to /var/run/kore.pid (changable in configuration)
2013-06-04 16:53:30 +02:00
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
#if defined(KORE_DEBUG)
Improve http_body_max directive a bit. Allow setting it to 0 which will disable HTTP requests that have a body (POST/PUT). Reduce default http_body_max to 1MB by default, 10MB seems large. Revisit to this code inspired by #100.
2016-01-08 17:54:40 +01:00
#define kore_debug(...) \
configuration files are now passed using the -c option. allow debug output when the -d flag is specified.
2013-06-05 08:55:07 +02:00
if (kore_debug) \
Make debug builds build again.
2016-02-13 14:20:31 +01:00
kore_debug_internal(__FILE__, __LINE__, __VA_ARGS__)
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
#else
Improve http_body_max directive a bit. Allow setting it to 0 which will disable HTTP requests that have a body (POST/PUT). Reduce default http_body_max to 1MB by default, 10MB seems large. Revisit to this code inspired by #100.
2016-01-08 17:54:40 +01:00
#define kore_debug(...)
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
#endif
first commit
2013-04-17 22:34:27 +02:00
Improve on sending data to the client. - Attempt to chain as much as we can in the send netbufs (keeps down the SSL_write calls, silly seeing it go out with 8 bytes) - Change NETBUF_SEND_PAYLOAD_MAX to 4K - Call SSL_write() with max NETBUF_SEND_PAYLOAD_MAX in size - Go back to flushing the send buffers after each request - No more need for a fixed pool for nb->buf, go back to the heap for now - Disable Nagle, we're doing the chaining now anyway
2013-10-26 00:48:09 +02:00
#define NETBUF_RECV 0
#define NETBUF_SEND 1
properly implement spdy/3.1 flow control
2014-08-08 14:18:15 +02:00
#define NETBUF_SEND_PAYLOAD_MAX 8192
#define NETBUF_LAST_CHAIN 0
#define NETBUF_BEFORE_CHAIN 1
Add initial SSL stuff + bare bones NPN advertisement. So far chrome connects and shows our connection as SPDY/3, good start.
2013-04-21 20:21:46 +02:00
the net qeueuing functions don't need to return status codes, simplifies a lot of other code that revolves around those. prepare for POST support
2013-05-02 14:47:02 +02:00
#define NETBUF_CALL_CB_ALWAYS 0x01
#define NETBUF_FORCE_REMOVE 0x02
Implement some great SPDY improvements. * Bump spdy announcement to the correct draft version * When receiving a RST, clean out the netbuf chain of that stream
2014-08-07 10:22:54 +02:00
#define NETBUF_MUST_RESEND 0x04
Add http_response_stream() which can stream data from a buffer to the client.
2014-08-07 14:23:26 +02:00
#define NETBUF_IS_STREAM 0x10
the net qeueuing functions don't need to return status codes, simplifies a lot of other code that revolves around those. prepare for POST support
2013-05-02 14:47:02 +02:00
Improvements for client certificates. Double check we actually get a certificate if we are asking for one. Even though we set SSL_VERIFY_FAIL_IF_NO_PEER_CERT it's a sane thing to do. Start logging the CN for the received client certificate in the access logs. As a bonus re-arrange some accesslog stuff for sanity.
2014-03-05 11:38:47 +01:00
#define X509_GET_CN(c, o, l) \
X509_NAME_get_text_by_NID(X509_get_subject_name(c), \
NID_commonName, o, l)
#define X509_CN_LENGTH (ub_common_name + 1)
Handle SPDY streams better when sending data.
2014-01-14 21:43:45 +01:00
/* XXX hackish. */
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
Handle SPDY streams better when sending data.
2014-01-14 21:43:45 +01:00
struct http_request;
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
Handle SPDY streams better when sending data.
2014-01-14 21:43:45 +01:00
first commit
2013-04-17 22:34:27 +02:00
struct netbuf {
begin with the ability to read control frames. something feels fishy with epoll() and its triggering of events. I probably got it wrong.
2013-04-28 19:11:44 +02:00
u_int8_t *buf;
Chain our sending netbufs together. Gives us a lot less SSL_write() calls.
2013-10-25 14:22:29 +02:00
u_int32_t s_off;
u_int32_t b_len;
u_int32_t m_len;
allow us to expand receive buffers automatically so we can keep chaining data into the same netbuf. This gives us the possibility to retain the ctrl_frame by the time we reach the proper cb for the actual frame message.
2013-04-28 23:42:13 +02:00
u_int8_t type;
add flags to net_recv_queue() and net_send_queue(),. NETBUF_CALL_CB_ALWAYS flag, calls the registered cb everytime data has been received or sent.
2013-05-02 00:28:49 +02:00
u_int8_t flags;
allow us to expand receive buffers automatically so we can keep chaining data into the same netbuf. This gives us the possibility to retain the ctrl_frame by the time we reach the proper cb for the actual frame message.
2013-04-28 23:42:13 +02:00
first commit
2013-04-17 22:34:27 +02:00
void *owner;
Handle SPDY streams better when sending data.
2014-01-14 21:43:45 +01:00
support for POST in both spdy and normal connections
2013-05-03 07:35:11 +02:00
void *extra;
first commit
2013-04-17 22:34:27 +02:00
int (*cb)(struct netbuf *);
TAILQ_ENTRY(netbuf) list;
};
Chain our sending netbufs together. Gives us a lot less SSL_write() calls.
2013-10-25 14:22:29 +02:00
TAILQ_HEAD(netbuf_head, netbuf);
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
#define KORE_TYPE_LISTENER 1
#define KORE_TYPE_CONNECTION 2
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
#define KORE_TYPE_PGSQL_CONN 3
Add initial stab at asynchronous background tasks. More to follow.
2014-06-28 16:17:18 +02:00
#define KORE_TYPE_TASK 4
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
rework disconnecting clients and fix bug where spdy ocnnections could segfault after disconnecting
2013-05-02 09:10:35 +02:00
#define CONN_STATE_UNKNOWN 0
#define CONN_STATE_SSL_SHAKE 1
#define CONN_STATE_ESTABLISHED 2
#define CONN_STATE_DISCONNECTING 3
Add initial SSL stuff + bare bones NPN advertisement. So far chrome connects and shows our connection as SPDY/3, good start.
2013-04-21 20:21:46 +02:00
read next protocol properly and remember it for later.
2013-04-21 20:35:47 +02:00
#define CONN_PROTO_UNKNOWN 0
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#define CONN_PROTO_HTTP 1
#define CONN_PROTO_WEBSOCKET 2
#define CONN_PROTO_MSG 3
read next protocol properly and remember it for later.
2013-04-21 20:35:47 +02:00
rework the way events trigger and the way we handle them. follow the manual for once, as edge trigger does not refire until we got an EAGAIN. introduce net_send_flush() and net_recv_flush() for this purpose, we attempt to make as much headway as possible until we reach EAGAIN or until we can simply pickup again later. should merge all the stuff in http_response() into a single send buffer, **out is in place in net_send_queue() for that purpose.
2013-05-01 13:43:47 +02:00
#define CONN_READ_POSSIBLE 0x01
#define CONN_WRITE_POSSIBLE 0x02
Implement SPDY WINDOW_UPDATE and SETTINGS.
2013-07-01 11:30:18 +02:00
#define CONN_WRITE_BLOCK 0x04
- add idle timer for normal connections (ie: !SPDY), max 20 seconds. - use idle timer to make sure we don't block SPDY connections indefinately when our window size has reached 0.
2013-07-01 12:08:51 +02:00
#define CONN_IDLE_TIMER_ACT 0x10
Add spdy_session_teardown() which can properly teardown a SPDY session. Use this throughout the spdy code to propagate session errors (if any) occur. At the same time fix BSD's missing CONN_WRITE_BLOCK
2013-07-13 19:56:38 +02:00
#define CONN_READ_BLOCK 0x20
Major improvements to HTTP/1.1 behaviour, error reporting and handling. This comes with changes to http_response() (no longer returns a result).
2014-01-29 22:48:51 +01:00
#define CONN_CLOSE_EMPTY 0x40
- add idle timer for normal connections (ie: !SPDY), max 20 seconds. - use idle timer to make sure we don't block SPDY connections indefinately when our window size has reached 0.
2013-07-01 12:08:51 +02:00
#define KORE_IDLE_TIMER_MAX 20000
rework the way events trigger and the way we handle them. follow the manual for once, as edge trigger does not refire until we got an EAGAIN. introduce net_send_flush() and net_recv_flush() for this purpose, we attempt to make as much headway as possible until we reach EAGAIN or until we can simply pickup again later. should merge all the stuff in http_response() into a single send buffer, **out is in place in net_send_queue() for that purpose.
2013-05-01 13:43:47 +02:00
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
#define WEBSOCKET_OP_CONT 0x00
#define WEBSOCKET_OP_TEXT 0x01
#define WEBSOCKET_OP_BINARY 0x02
#define WEBSOCKET_OP_CLOSE 0x08
#define WEBSOCKET_OP_PING 0x09
#define WEBSOCKET_OP_PONG 0x10
#define WEBSOCKET_BROADCAST_LOCAL 1
#define WEBSOCKET_BROADCAST_GLOBAL 2
Add rudimentary timers to Kore. Timers are in ms resolution and are added using kore_timer_add(cb, interval, flags). Both oneshot timers and continious timers are supported.
2015-04-06 18:54:35 +02:00
#define KORE_TIMER_ONESHOT 0x01
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
#define KORE_CONNECTION_PRUNE_DISCONNECT 0
#define KORE_CONNECTION_PRUNE_ALL 1
first commit
2013-04-17 22:34:27 +02:00
struct connection {
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
u_int8_t type;
first commit
2013-04-17 22:34:27 +02:00
int fd;
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
u_int8_t state;
u_int8_t proto;
first commit
2013-04-17 22:34:27 +02:00
void *owner;
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#if !defined(KORE_NO_TLS)
X509 *cert;
Add initial SSL stuff + bare bones NPN advertisement. So far chrome connects and shows our connection as SPDY/3, good start.
2013-04-21 20:21:46 +02:00
SSL *ssl;
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
int tls_reneg;
#endif
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
u_int8_t flags;
cb_extra -> hdlr_extra and add one to struct connection as well for persistent data across requests.
2013-08-14 15:56:44 +02:00
void *hdlr_extra;
first commit
2013-04-17 22:34:27 +02:00
Allow apps to override connection handling. Setting the handle callback allows your application to take care of network events for the connection. Look at the connection state and flags to determine if read/write is possible and go from there. See kore_connection_handle() for more details.
2015-12-01 20:55:00 +01:00
int (*handle)(struct connection *);
Allow applications to create new connections in our event loop.
2014-09-17 08:25:45 +02:00
void (*disconnect)(struct connection *);
int (*read)(struct connection *, int *);
int (*write)(struct connection *, int, int *);
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
u_int8_t addrtype;
union {
struct sockaddr_in ipv4;
struct sockaddr_in6 ipv6;
} addr;
- add idle timer for normal connections (ie: !SPDY), max 20 seconds. - use idle timer to make sure we don't block SPDY connections indefinately when our window size has reached 0.
2013-07-01 12:08:51 +02:00
struct {
u_int64_t length;
u_int64_t start;
} idle_timer;
Implement some great SPDY improvements. * Bump spdy announcement to the correct draft version * When receiving a RST, clean out the netbuf chain of that stream
2014-08-07 10:22:54 +02:00
struct netbuf_head send_queue;
properly implement spdy/3.1 flow control
2014-08-08 14:18:15 +02:00
struct netbuf *snb;
struct netbuf *rnb;
add beginnings of zlib inflation, not really working. nap time.
2013-04-29 23:35:36 +02:00
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
More cleanup after introducing NOHTTP=1. * The cli tools must know when building as KORE_NO_HTTP. * Reshuffle some structs around to avoid forward declarations. * Move wscbs under !KORE_NO_HTTP as its for websockets. * Remove unused members from struct connection. Applications that use the connect callbacks for new connections must now set the connection state themselves, see nohttp example.
2015-11-30 16:23:34 +01:00
void *wscbs;
- Add SPDY RST control frame handler. - Keep HTTP requests in connection, so we can delete them if the connection ends before the requests do (this way we don't leak them). - When spdy_stream_close() is called, delete the attached http request. (This shouldn't hurt to do, so hopefully won't cause major fallout). - When parsing HTTP, find the first occurence of end-of-headers so uploads with multipart/form-data can succeed properly. - Add a test upload page to the example module.
2013-09-09 10:59:56 +02:00
TAILQ_HEAD(, http_request) http_requests;
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
rework disconnecting clients and fix bug where spdy ocnnections could segfault after disconnecting
2013-05-02 09:10:35 +02:00
TAILQ_ENTRY(connection) list;
first commit
2013-04-17 22:34:27 +02:00
};
Make worker_clients available everywhere.
2014-12-03 20:40:44 +01:00
TAILQ_HEAD(connection_list, connection);
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
extern struct connection_list connections;
extern struct connection_list disconnected;
Make worker_clients available everywhere.
2014-12-03 20:40:44 +01:00
More cleanup after introducing NOHTTP=1. * The cli tools must know when building as KORE_NO_HTTP. * Reshuffle some structs around to avoid forward declarations. * Move wscbs under !KORE_NO_HTTP as its for websockets. * Remove unused members from struct connection. Applications that use the connect callbacks for new connections must now set the connection state themselves, see nohttp example.
2015-11-30 16:23:34 +01:00
struct listener {
u_int8_t type;
u_int8_t addrtype;
int fd;
void (*connect)(struct connection *);
union {
struct sockaddr_in ipv4;
struct sockaddr_in6 ipv6;
} addr;
LIST_ENTRY(listener) list;
};
LIST_HEAD(listener_head, listener);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
Kore now supports GET parameters and automatic validation of GET/POST parameters. Kore will automatically removes invalid parameters as a security measure. See modules/examples/module.conf for an example of how this works.
2013-11-10 15:17:15 +01:00
struct kore_handler_params {
char *name;
u_int8_t method;
struct kore_validator *validator;
TAILQ_ENTRY(kore_handler_params) list;
};
Add authentication blocks for Kore. Using authentication blocks one can define "authentication" mechanisms in Kore for page handlers. This can be used to require a session cookie (validated by your own validator) for certain page handlers, and hopefully in the future provide a framework for adding more authentication things (like HTTP Auth). Right now only cookie checking is available.
2014-01-22 22:55:10 +01:00
#define KORE_AUTH_TYPE_COOKIE 1
Add header as an option for authentication blocks
2014-01-22 23:11:52 +01:00
#define KORE_AUTH_TYPE_HEADER 2
Add request as an authentication_type. request can be used for when you want to validate something for authentication bmanually. Youur validator will receive the http_reques passed down. A practical use of this is doing IP based ACL's.
2014-03-01 19:18:30 +01:00
#define KORE_AUTH_TYPE_REQUEST 3
Add authentication blocks for Kore. Using authentication blocks one can define "authentication" mechanisms in Kore for page handlers. This can be used to require a session cookie (validated by your own validator) for certain page handlers, and hopefully in the future provide a framework for adding more authentication things (like HTTP Auth). Right now only cookie checking is available.
2014-01-22 22:55:10 +01:00
struct kore_auth {
u_int8_t type;
char *name;
char *value;
char *redirect;
struct kore_validator *validator;
TAILQ_ENTRY(kore_auth) list;
};
initial module support
2013-05-01 16:03:48 +02:00
Add authentication blocks for Kore. Using authentication blocks one can define "authentication" mechanisms in Kore for page handlers. This can be used to require a session cookie (validated by your own validator) for certain page handlers, and hopefully in the future provide a framework for adding more authentication things (like HTTP Auth). Right now only cookie checking is available.
2014-01-22 22:55:10 +01:00
#define HANDLER_TYPE_STATIC 1
#define HANDLER_TYPE_DYNAMIC 2
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
#define KORE_MODULE_LOAD 1
#define KORE_MODULE_UNLOAD 2
Allow Kore to load multiple modules at once.
2013-12-15 01:11:56 +01:00
struct kore_module {
void *handle;
char *path;
char *onload;
Let modules decide if they want to be reloaded. At times it seems relevant that worker their modules should not be reloaded when receiving a SIGHUP. Developers can now control this by returning anything else but KORE_RESULT_OK from their initialization methods. The parent module will always be reloaded.
2015-05-07 13:03:10 +02:00
int (*ocb)(int);
Call the onload function whenever a module is loaded/reload. Allows one to teardown whatever they setup properly when the module gets a full reload. See example module for how it works.
2013-12-21 13:37:34 +01:00
Allow Kore to load multiple modules at once.
2013-12-15 01:11:56 +01:00
time_t mtime;
TAILQ_ENTRY(kore_module) list;
};
initial module support
2013-05-01 16:03:48 +02:00
struct kore_module_handle {
switch dynamic handlers to regex based patterns, much easier.
2013-05-29 14:29:46 +02:00
char *path;
allow sending SIGHUP to kore, which will then reload its content module
2013-05-03 00:04:06 +02:00
char *func;
void *addr;
initial module support
2013-05-01 16:03:48 +02:00
int type;
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
int errors;
switch dynamic handlers to regex based patterns, much easier.
2013-05-29 14:29:46 +02:00
regex_t rctx;
Make accesslogs optional
2014-07-30 09:11:21 +02:00
struct kore_domain *dom;
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
Add authentication blocks for Kore. Using authentication blocks one can define "authentication" mechanisms in Kore for page handlers. This can be used to require a session cookie (validated by your own validator) for certain page handlers, and hopefully in the future provide a framework for adding more authentication things (like HTTP Auth). Right now only cookie checking is available.
2014-01-22 22:55:10 +01:00
struct kore_auth *auth;
Kore now supports GET parameters and automatic validation of GET/POST parameters. Kore will automatically removes invalid parameters as a security measure. See modules/examples/module.conf for an example of how this works.
2013-11-10 15:17:15 +01:00
TAILQ_HEAD(, kore_handler_params) params;
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
initial module support
2013-05-01 16:03:48 +02:00
TAILQ_ENTRY(kore_module_handle) list;
};
move to a worker based threading approach where we delegate http requests to workers in a round robin basis (later this should be swapped to find the laziest worker and assign the request to that instead).
2013-05-30 19:36:42 +02:00
struct kore_worker {
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
u_int8_t id;
u_int8_t cpu;
move from multithreads to single threaded worker processes.
2013-06-04 11:55:38 +02:00
pid_t pid;
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
int pipe[2];
struct connection *msg[2];
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
u_int8_t has_lock;
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
struct kore_module_handle *active_hdlr;
move to a worker based threading approach where we delegate http requests to workers in a round robin basis (later this should be swapped to find the laziest worker and assign the request to that instead).
2013-05-30 19:36:42 +02:00
};
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
struct kore_domain {
Add access logging to Kore.
2013-06-24 09:36:40 +02:00
char *domain;
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
int accesslog;
#if !defined(KORE_NO_TLS)
Add support for client certificates
2013-12-14 16:31:07 +01:00
char *cafile;
Add CRL support. Allow Kore to use per domain CRLs when requiring client certificates. The require_client_cert configuration option has been renamed to a more sane client_certificates and can optionally take a second argument which is the CRL in pem format. You'll need a restart in case the CRLs get updated.
2014-10-18 02:32:05 +02:00
char *crlfile;
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
char *certfile;
char *certkey;
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
SSL_CTX *ssl_ctx;
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#endif
Add access logging to Kore.
2013-06-24 09:36:40 +02:00
TAILQ_HEAD(, kore_module_handle) handlers;
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
TAILQ_ENTRY(kore_domain) list;
Add access logging to Kore.
2013-06-24 09:36:40 +02:00
};
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
TAILQ_HEAD(kore_domain_h, kore_domain);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
Add validators to kore, specified in the configuration using 'validator' keyword. Example: validator v_id function v_id_function validator v_url regex ^/url/path/[a-z]*$ You can then call these using kore_validator_run(char *, char *), example: if (!kore_validator_run("v_url", req->path)) [req->path is bad];
2013-11-09 16:21:52 +01:00
#define KORE_VALIDATOR_TYPE_REGEX 1
#define KORE_VALIDATOR_TYPE_FUNCTION 2
struct kore_validator {
Pass the http_request responsible for calling the validator.
2014-02-01 17:47:58 +01:00
u_int8_t type;
char *name;
char *arg;
regex_t rctx;
int (*func)(struct http_request *, char *);
Add validators to kore, specified in the configuration using 'validator' keyword. Example: validator v_id function v_id_function validator v_url regex ^/url/path/[a-z]*$ You can then call these using kore_validator_run(char *, char *), example: if (!kore_validator_run("v_url", req->path)) [req->path is bad];
2013-11-09 16:21:52 +01:00
TAILQ_ENTRY(kore_validator) list;
};
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
Add validators to kore, specified in the configuration using 'validator' keyword. Example: validator v_id function v_id_function validator v_url regex ^/url/path/[a-z]*$ You can then call these using kore_validator_run(char *, char *), example: if (!kore_validator_run("v_url", req->path)) [req->path is bad];
2013-11-09 16:21:52 +01:00
Massive rework of HTTP layer. This commit is a flag day, your old modules will almost certainly need to be updated in order to build properly with these changes. Summary of changes: - Offload HTTP bodies to disk if they are large (inspired by #100). (disabled by default) - The http_argument_get* macros now takes an explicit http_request parameter. - Kore will now throw 404 errors almost immediately after an HTTP request has come in instead of waiting until all data has arrived. API changes: - http_argument_get* macros now require an explicit http_request parameter. (no more magic invokations). - http_generic_404() is gone - http_populate_arguments() is gone - http_body_bytes() is gone - http_body_text() is gone - http_body_read() has been added - http_populate_post() has been added - http_populate_get() has been added - http_file_read() has been added - http_file_rewind() has been added - http_file_lookup() no longer takes name, fname, data and len parameters. - http_file_lookup() now returns a struct http_file pointer. - http_populate_multipart_form() no longer takes an secondary parameter. New configuration options: - http_body_disk_offload: Number of bytes after which Kore will offload the HTTP body to disk instead of retaining it in memory. If 0 this feature is disabled. (Default: 0) - http_body_disk_path: The path where Kore will store temporary HTTP body files. (this directory does not get created if http_body_disk_offload is 0). New example: The upload example has been added, demonstrating how to deal with file uploads from a multipart form.
2016-01-18 11:30:22 +01:00
#define KORE_BUF_INCREMENT 4096
move buf stuff back into kore.h, wont need it separately
2013-05-02 15:14:00 +02:00
struct kore_buf {
u_int8_t *data;
Add http_header_max and http_postbody_max configuration variables. - http_header_max: Maximum size of HTTP headers (in non SPDY connections). - http_postbody_max: Maximum size of an HTTP POST body (both in SPDY and HTTP mode). Right now Kore will simply DC the client, ideally we want to send a 413 (entity too large) to the client however. See modules/examples/module.conf for more.
2013-09-22 20:05:24 +02:00
u_int64_t length;
u_int64_t offset;
move buf stuff back into kore.h, wont need it separately
2013-05-02 15:14:00 +02:00
};
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
struct kore_pool_region {
void *start;
LIST_ENTRY(kore_pool_region) list;
Kill __packed__ attributes on some data structs
2014-04-18 17:41:56 +02:00
};
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
struct kore_pool_entry {
u_int8_t state;
struct kore_pool_region *region;
LIST_ENTRY(kore_pool_entry) list;
Kill __packed__ attributes on some data structs
2014-04-18 17:41:56 +02:00
};
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
struct kore_pool {
u_int32_t elen;
u_int32_t slen;
u_int32_t elms;
u_int32_t inuse;
char *name;
LIST_HEAD(, kore_pool_region) regions;
LIST_HEAD(, kore_pool_entry) freelist;
Kill __packed__ attributes on some data structs
2014-04-18 17:41:56 +02:00
};
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
struct kore_wscbs {
void (*connect)(struct connection *);
void (*message)(struct connection *, u_int8_t,
void *, size_t);
void (*disconnect)(struct connection *);
};
Kore timer improvements. Introduces kore_timer_remove() and updates kore_timer_add() to return the newly added timer as a struct kore_timer. Also allow arguments to be passed to timers.
2015-05-15 19:11:10 +02:00
struct kore_timer {
u_int64_t nextrun;
u_int64_t interval;
int flags;
void *arg;
Remove 'behind' argument from timers. Also calculate next run for a timer as now + interval instead of last nextrun + interval - delta.
2015-08-06 08:20:41 +02:00
void (*cb)(void *, u_int64_t);
Kore timer improvements. Introduces kore_timer_remove() and updates kore_timer_add() to return the newly added timer as a struct kore_timer. Also allow arguments to be passed to timers.
2015-05-15 19:11:10 +02:00
TAILQ_ENTRY(kore_timer) list;
};
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#define KORE_WORKER_KEYMGR 0
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
/* Reserved message ids, registered on workers. */
#define KORE_MSG_ACCESSLOG 1
#define KORE_MSG_WEBSOCKET 2
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#define KORE_MSG_KEYMGR_REQ 3
#define KORE_MSG_KEYMGR_RESP 4
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
Update message framework with src/dst for workers. One can now send messages to specific workers and receiving workers can see the origin of the messages.
2015-07-06 21:08:36 +02:00
/* Predefined message targets. */
#define KORE_MSG_PARENT 1000
#define KORE_MSG_WORKER_ALL 1001
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
struct kore_msg {
u_int8_t id;
Update message framework with src/dst for workers. One can now send messages to specific workers and receiving workers can see the origin of the messages.
2015-07-06 21:08:36 +02:00
u_int16_t src;
u_int16_t dst;
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
u_int32_t length;
};
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#if !defined(KORE_NO_TLS)
struct kore_keyreq {
int padding;
char domain[KORE_DOMAINNAME_LEN];
u_int8_t domain_len;
u_int16_t data_len;
u_int8_t data[];
};
#endif
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
#if !defined(KORE_SINGLE_BINARY)
extern char *config_file;
#endif
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
extern pid_t kore_pid;
Add foreground
2014-07-31 13:43:37 +02:00
extern int foreground;
configuration files are now passed using the -c option. allow debug output when the -d flag is specified.
2013-06-05 08:55:07 +02:00
extern int kore_debug;
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
extern int skip_chroot;
add chroot and runas directives so we can chroot and drop privilegs properly
2013-05-04 22:18:27 +02:00
extern char *chroot_path;
Make runas behave similarly to chroot. Add new command line knob '-r', that disables runas similar to '-n', it's implied as well for kore command runs. Add default runas (nobody) user and chroot (/var/empty) path, if none are specified, fallback to these.
2015-05-18 21:34:39 +02:00
extern int skip_runas;
add chroot and runas directives so we can chroot and drop privilegs properly
2013-05-04 22:18:27 +02:00
extern char *runas_user;
write main process pid to /var/run/kore.pid (changable in configuration)
2013-06-04 16:53:30 +02:00
extern char *kore_pidfile;
Default to only TLSv1.2 from now on. Add configuration setting tls_version to specify if you either want TLSv1.2 or TLSv1.0 or both. The configuration options ssl_cipher and ssl_dhparam have changed name to tls_cipher and tls_dhparam. There is no fallback so you might have to update your configs.
2015-05-06 10:59:43 +02:00
extern char *kore_tls_cipher_list;
extern int tls_version;
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#if !defined(KORE_NO_TLS)
Default to only TLSv1.2 from now on. Add configuration setting tls_version to specify if you either want TLSv1.2 or TLSv1.0 or both. The configuration options ssl_cipher and ssl_dhparam have changed name to tls_cipher and tls_dhparam. There is no fallback so you might have to update your configs.
2015-05-06 10:59:43 +02:00
extern DH *tls_dhparam;
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#endif
Add BSD kqueue(2) support. Compile with make bsd (or make linux for linux)
2013-06-17 23:39:17 +02:00
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
extern u_int8_t nlisteners;
Add BSD kqueue(2) support. Compile with make bsd (or make linux for linux)
2013-06-17 23:39:17 +02:00
extern u_int16_t cpu_count;
extern u_int8_t worker_count;
Add new configuration option: worker_set_affinity. Controls wether or not worker processes sets its affinity mask to bind themselves to a single CPU. Defaults to on (1).
2015-04-27 10:36:33 +02:00
extern u_int8_t worker_set_affinity;
Add worker_rlimit_nofiles as a configurable option.
2014-07-31 09:14:03 +02:00
extern u_int32_t worker_rlimit_nofiles;
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
extern u_int32_t worker_max_connections;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
extern u_int32_t worker_active_connections;
Fix typo in configuration option worker_accept_treshold. There is no backwards comptabile option available. Fixes #53
2015-05-18 12:20:28 +02:00
extern u_int32_t worker_accept_threshold;
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
extern u_int64_t kore_websocket_maxframe;
extern u_int64_t kore_websocket_timeout;
Improve very heavy load handling. Introduces two new configuration knobs: * socket_backlog (backlog for listen(2)) * http_request_limit The second one is the most interesting one. Before, kore would iterate over all received HTTP requests in its queue before returning out of http_process(). Under heavy load this queue can cause Kore to spend a considerable amount of time iterating over said queue. With the http_request_limit, kore will process at MOST http_request_limit requests before returning back to the event loop. This means responses to processed requests are sent out much quicker and allows kore to handle any other incoming requests more gracefully.
2015-04-09 15:29:44 +02:00
extern u_int32_t kore_socket_backlog;
Add BSD kqueue(2) support. Compile with make bsd (or make linux for linux)
2013-06-17 23:39:17 +02:00
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
extern struct listener_head listeners;
Add access logging to Kore.
2013-06-24 09:36:40 +02:00
extern struct kore_worker *worker;
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
extern struct kore_domain_h domains;
extern struct kore_domain *primary_dom;
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
extern struct kore_pool nb_pool;
Add BSD kqueue(2) support. Compile with make bsd (or make linux for linux)
2013-06-17 23:39:17 +02:00
Move orbit functionality into kore directly. Makes more sense and reads easier: kore create myapp kore build myapp kore run myapp Note that kore retains its cli options (if no command was given), meaning you can still start kore in the traditional way as well. The command options are simply to make development easier.
2014-08-01 13:59:47 +02:00
void kore_cli_usage(int);
int kore_cli_main(int, char **);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
void kore_signal(int);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
void kore_worker_wait(int);
Add BSD kqueue(2) support. Compile with make bsd (or make linux for linux)
2013-06-17 23:39:17 +02:00
void kore_worker_init(void);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
void kore_worker_shutdown(void);
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
void kore_worker_privdrop(void);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
void kore_worker_dispatch_signal(int);
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
void kore_worker_spawn(u_int16_t, u_int16_t);
void kore_worker_entry(struct kore_worker *);
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
struct kore_worker *kore_worker_data(u_int8_t);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
void kore_platform_init(void);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
void kore_platform_event_init(void);
rename *_fini to *_cleanup
2016-01-04 22:40:14 +01:00
void kore_platform_event_cleanup(void);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
void kore_platform_proctitle(char *);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
void kore_platform_disable_read(int);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
void kore_platform_enable_accept(void);
void kore_platform_disable_accept(void);
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
int kore_platform_event_wait(u_int64_t);
Allow applications to create new connections in our event loop.
2014-09-17 08:25:45 +02:00
void kore_platform_event_all(int, void *);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
void kore_platform_schedule_read(int, void *);
Add kore_platform_schedule_write().
2015-12-09 21:29:44 +01:00
void kore_platform_schedule_write(int, void *);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
void kore_platform_event_schedule(int, int, int, void *);
void kore_platform_worker_setcpu(struct kore_worker *);
Add access logging to Kore.
2013-06-24 09:36:40 +02:00
void kore_accesslog_init(void);
void kore_accesslog_worker_init(void);
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
int kore_accesslog_write(const void *, u_int32_t);
add IPv6 support and support for multiple listeners.
2013-07-27 20:56:15 +02:00
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
Rename kore_auth() to kore_auth_run().
2015-04-02 13:45:42 +02:00
int kore_auth_run(struct http_request *, struct kore_auth *);
void kore_auth_init(void);
int kore_auth_new(const char *);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
struct kore_auth *kore_auth_lookup(const char *);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
Add authentication blocks for Kore. Using authentication blocks one can define "authentication" mechanisms in Kore for page handlers. This can be used to require a session cookie (validated by your own validator) for certain page handlers, and hopefully in the future provide a framework for adding more authentication things (like HTTP Auth). Right now only cookie checking is available.
2014-01-22 22:55:10 +01:00
Add rudimentary timers to Kore. Timers are in ms resolution and are added using kore_timer_add(cb, interval, flags). Both oneshot timers and continious timers are supported.
2015-04-06 18:54:35 +02:00
void kore_timer_init(void);
u_int64_t kore_timer_run(u_int64_t);
Kore timer improvements. Introduces kore_timer_remove() and updates kore_timer_add() to return the newly added timer as a struct kore_timer. Also allow arguments to be passed to timers.
2015-05-15 19:11:10 +02:00
void kore_timer_remove(struct kore_timer *);
Remove 'behind' argument from timers. Also calculate next run for a timer as now + interval instead of last nextrun + interval - delta.
2015-08-06 08:20:41 +02:00
struct kore_timer *kore_timer_add(void (*cb)(void *, u_int64_t),
u_int64_t, void *, int);
Add rudimentary timers to Kore. Timers are in ms resolution and are added using kore_timer_add(cb, interval, flags). Both oneshot timers and continious timers are supported.
2015-04-06 18:54:35 +02:00
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
void kore_listener_cleanup(void);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
int kore_server_bind(const char *, const char *, const char *);
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#if !defined(KORE_NO_TLS)
int kore_tls_sni_cb(SSL *, int *, void *);
Stop client initiated TLS renegotiations completely.
2015-05-20 16:36:13 +02:00
void kore_tls_info_callback(const SSL *, int, int);
Reduce memory footprint for NOTLS builds.
2016-01-07 09:20:09 +01:00
#endif
initial module support
2013-05-01 16:03:48 +02:00
Allow applications to create new connections in our event loop.
2014-09-17 08:25:45 +02:00
void kore_connection_init(void);
rename *_fini to *_cleanup
2016-01-04 22:40:14 +01:00
void kore_connection_cleanup(void);
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
void kore_connection_prune(int);
Allow applications to create new connections in our event loop.
2014-09-17 08:25:45 +02:00
struct connection *kore_connection_new(void *);
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
void kore_connection_check_timeout(void);
Kill TCP_NODELAY warnings for socketpair() fds.
2015-06-22 22:11:03 +02:00
int kore_connection_nonblock(int, int);
Allow applications to create new connections in our event loop.
2014-09-17 08:25:45 +02:00
int kore_connection_handle(struct connection *);
void kore_connection_remove(struct connection *);
void kore_connection_disconnect(struct connection *);
void kore_connection_start_idletimer(struct connection *);
void kore_connection_stop_idletimer(struct connection *);
void kore_connection_check_idletimer(u_int64_t,
struct connection *);
int kore_connection_accept(struct listener *,
struct connection **);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Add access logging to Kore.
2013-06-24 09:36:40 +02:00
u_int64_t kore_time_ms(void);
use syslog() for informative messages from all parts of kore.
2013-06-04 23:24:47 +02:00
void kore_log_init(void);
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
first commit
2013-04-17 22:34:27 +02:00
void *kore_malloc(size_t);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
void kore_parse_config(void);
first commit
2013-04-17 22:34:27 +02:00
void *kore_calloc(size_t, size_t);
void *kore_realloc(void *, size_t);
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
void kore_mem_free(void *);
void kore_mem_init(void);
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
void *kore_pool_get(struct kore_pool *);
void kore_pool_put(struct kore_pool *, void *);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
void kore_pool_init(struct kore_pool *, const char *,
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
u_int32_t, u_int32_t);
rename *_fini to *_cleanup
2016-01-04 22:40:14 +01:00
void kore_pool_cleanup(struct kore_pool *);
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
introduce kore_split_string() to properly split strings apart. introduce kore_date_to_time() to conver http-date formatted strings to time_t.
2013-05-01 20:10:45 +02:00
time_t kore_date_to_time(char *);
allow modules to add headers to the response before calling http_response() by calling http_response_header_add(). fix wrong overflow check in spdy_stream_get_header(). html_inject now exports last modified timestamp for the files that are compiled into the module in the format static_mtime_<type>_<name>. modules can now look into the request headers using http_request_header_get().
2013-05-01 21:16:09 +02:00
char *kore_time_to_date(time_t);
first commit
2013-04-17 22:34:27 +02:00
char *kore_strdup(const char *);
use syslog() for informative messages from all parts of kore.
2013-06-04 23:24:47 +02:00
void kore_log(int, const char *, ...);
Properly convert 64bit parameters. Introduces kore_strtonum64() for just this purpose as to not taint kore_strtonum() too much.
2014-07-21 01:16:03 +02:00
u_int64_t kore_strtonum64(const char *, int, int *);
Improve kore_strlcpy(). Make it return the original length of the input string so the caller can check for truncation. Also guard against len being 0 as this would not do anything with the destination string (not even NUL terminate it).
2016-07-04 11:41:37 +02:00
size_t kore_strlcpy(char *, const char *, const size_t);
lots of new stuff, including processing of http requests and an attempt to build an initial spdy response (SYN frame + header block content).
2013-05-01 08:09:04 +02:00
void kore_server_disconnect(struct connection *);
introduce kore_split_string() to properly split strings apart. introduce kore_date_to_time() to conver http-date formatted strings to time_t.
2013-05-01 20:10:45 +02:00
int kore_split_string(char *, char *, char **, size_t);
constify
2016-02-13 14:33:06 +01:00
void kore_strip_chars(char *, const char, char **);
Add kore_snprintf() as a wrapper around snprintf().
2014-08-11 10:45:10 +02:00
int kore_snprintf(char *, size_t, int *, const char *, ...);
Rework the way validation and param extraction works. - Parameter validation is now done only when http_process_*() is called and upon http_argument_add(). - You MUST have defined your params in a param block or they will be filtered out. - http_argument_lookup() is dead, welcome http_argument_get() and its brothers and sisters: http_argument_get_string() http_argument_get_uint16() http_argument_get_int16() http_argument_get_uint32() http_argument_get_int32() They will automatically do bounds checking on integers for you and return proper integers or a NUL-terminated string. - The http_argument_get* functions no longer create an additional copy of the string which you need to free. Easier going. - http_multiple_args() is dead, byebye - Make some stuff we don't want to share with the modules static.
2013-12-12 00:58:32 +01:00
long long kore_strtonum(const char *, int, long long, long long, int *);
add base64 encoder/decoder in prep for http/2.0 and websockets
2013-07-10 15:00:53 +02:00
int kore_base64_encode(u_int8_t *, u_int32_t, char **);
int kore_base64_decode(char *, u_int8_t **, u_int32_t *);
changed needle length type to size_t in kore_mem_find(). updated function prototype in kore.h. Use tab instead of space indentation
2016-04-01 19:54:10 +02:00
void *kore_mem_find(void *, size_t, void *, size_t);
Redo config parsing a bit. No longer just call kore_string_split() on the line but separate out the configuration directive and let the appropriate callbacks parse things on their own.
2016-02-01 21:33:51 +01:00
char *kore_text_trim(char *, size_t);
Move shared file reading code to kore_read_line()
2016-02-01 22:10:10 +01:00
char *kore_read_line(FILE *, char *, size_t);
initial module support
2013-05-01 16:03:48 +02:00
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
void kore_websocket_handshake(struct http_request *,
struct kore_wscbs *);
void kore_websocket_send(struct connection *,
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
u_int8_t, const void *, size_t);
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
void kore_websocket_broadcast(struct connection *,
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
u_int8_t, const void *, size_t, int);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
void kore_msg_init(void);
void kore_msg_worker_init(void);
void kore_msg_parent_init(void);
void kore_msg_parent_add(struct kore_worker *);
void kore_msg_parent_remove(struct kore_worker *);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
void kore_msg_send(u_int16_t, u_int8_t, const void *, u_int32_t);
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
int kore_msg_register(u_int8_t,
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
void (*cb)(struct kore_msg *, const void *));
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
void kore_domain_init(void);
Added cleanup for the module handlers and validators
2016-02-01 12:30:20 +01:00
void kore_domain_cleanup(void);
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
int kore_domain_new(char *);
Added cleanup for the module handlers and validators
2016-02-01 12:30:20 +01:00
void kore_domain_free(struct kore_domain *);
Allow Kore to load multiple modules at once.
2013-12-15 01:11:56 +01:00
void kore_module_init(void);
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
void kore_module_cleanup(void);
Don't call module onloads until after everything is initialized. This means the onload functions for a module are now called after a worker has started and never from the parent ever again.
2014-06-29 21:15:23 +02:00
void kore_module_reload(int);
void kore_module_onload(void);
initial module support
2013-05-01 16:03:48 +02:00
int kore_module_loaded(void);
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
void kore_domain_closelogs(void);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
void *kore_module_getsym(const char *);
Add CRL support. Allow Kore to use per domain CRLs when requiring client certificates. The require_client_cert configuration option has been renamed to a more sane client_certificates and can optionally take a second argument which is the CRL in pem format. You'll need a restart in case the CRLs get updated.
2014-10-18 02:32:05 +02:00
void kore_domain_load_crl(void);
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
void kore_domain_keymgr_init(void);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
void kore_module_load(const char *, const char *);
add SNI support, and change domain configuration a bit.
2013-06-24 11:32:45 +02:00
void kore_domain_sslstart(struct kore_domain *);
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
void kore_domain_callback(void (*cb)(struct kore_domain *));
sprinkle more const chars
2014-08-04 12:40:21 +02:00
int kore_module_handler_new(const char *, const char *,
const char *, const char *, int);
Added cleanup for the module handlers and validators
2016-02-01 12:30:20 +01:00
void kore_module_handler_free(struct kore_module_handle *);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
struct kore_domain *kore_domain_lookup(const char *);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
struct kore_module_handle *kore_module_handler_find(const char *,
const char *);
first commit
2013-04-17 22:34:27 +02:00
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
Add validators to kore, specified in the configuration using 'validator' keyword. Example: validator v_id function v_id_function validator v_url regex ^/url/path/[a-z]*$ You can then call these using kore_validator_run(char *, char *), example: if (!kore_validator_run("v_url", req->path)) [req->path is bad];
2013-11-09 16:21:52 +01:00
void kore_validator_init(void);
void kore_validator_reload(void);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
int kore_validator_add(const char *, u_int8_t, const char *);
int kore_validator_run(struct http_request *, const char *, char *);
Pass the http_request responsible for calling the validator.
2014-02-01 17:47:58 +01:00
int kore_validator_check(struct http_request *,
Add request as an authentication_type. request can be used for when you want to validate something for authentication bmanually. Youur validator will receive the http_reques passed down. A practical use of this is doing IP based ACL's.
2014-03-01 19:18:30 +01:00
struct kore_validator *, void *);
sprinkle more const chars
2014-08-04 12:40:21 +02:00
struct kore_validator *kore_validator_lookup(const char *);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
Add validators to kore, specified in the configuration using 'validator' keyword. Example: validator v_id function v_id_function validator v_url regex ^/url/path/[a-z]*$ You can then call these using kore_validator_run(char *, char *), example: if (!kore_validator_run("v_url", req->path)) [req->path is bad];
2013-11-09 16:21:52 +01:00
add noreturn attribute fix false-positive warning
2015-05-19 09:45:04 +02:00
void fatal(const char *, ...) __attribute__((noreturn));
rename kore_log to kore_debug, and allow one to turn it off.
2013-06-04 16:30:53 +02:00
void kore_debug_internal(char *, int, const char *, ...);
first commit
2013-04-17 22:34:27 +02:00
properly deflate/inflate of name/value blocks. and add ability to get headers from them.
2013-05-01 00:35:33 +02:00
u_int16_t net_read16(u_int8_t *);
u_int32_t net_read32(u_int8_t *);
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
u_int64_t net_read64(u_int8_t *);
lots of new stuff, including processing of http requests and an attempt to build an initial spdy response (SYN frame + header block content).
2013-05-01 08:09:04 +02:00
void net_write16(u_int8_t *, u_int16_t);
void net_write32(u_int8_t *, u_int32_t);
Add websocket support to Kore. Introduces a few new api functions: - kore_websocket_handshake(struct http_request *): Performs the handshake on an HTTP request (coming from page handler) - kore_websocket_send(struct connection *, u_int8_t, void *, size_t): Sends data to a websocket connection. - kore_websocket_broadcast(struct connection *, u_int8_t, void *, size_t, int): Broadcast the given websocket op and data to all connected websocket clients on the worker. Note that as of right now the WEBSOCKET_BROADCAST_GLOBAL scope option does not work yet and messages broadcasted will be restricted to workers only. - kore_worker_websocket_broadcast(struct connection *, void *, void *): Backend function used by kore_websocket_broadcast(). Could prove useful for developers to have access to. A simple example is given under examples/websocket. Known issues: Kore does not support PING or CONT frames just yet.
2014-11-24 11:01:12 +01:00
void net_write64(u_int8_t *, u_int64_t);
Add fixed size memory pools and use them throughout Kore.
2013-07-15 10:13:36 +02:00
void net_init(void);
rename *_fini to *_cleanup
2016-01-04 22:40:14 +01:00
void net_cleanup(void);
begin with the ability to read control frames. something feels fishy with epoll() and its triggering of events. I probably got it wrong.
2013-04-28 19:11:44 +02:00
int net_send(struct connection *);
rework the way events trigger and the way we handle them. follow the manual for once, as edge trigger does not refire until we got an EAGAIN. introduce net_send_flush() and net_recv_flush() for this purpose, we attempt to make as much headway as possible until we reach EAGAIN or until we can simply pickup again later. should merge all the stuff in http_response() into a single send buffer, **out is in place in net_send_queue() for that purpose.
2013-05-01 13:43:47 +02:00
int net_send_flush(struct connection *);
int net_recv_flush(struct connection *);
Allow applications to create new connections in our event loop.
2014-09-17 08:25:45 +02:00
int net_read(struct connection *, int *);
int net_read_ssl(struct connection *, int *);
int net_write(struct connection *, int, int *);
int net_write_ssl(struct connection *, int, int *);
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
void net_recv_reset(struct connection *, u_int32_t,
int (*cb)(struct netbuf *));
Implement some great SPDY improvements. * Bump spdy announcement to the correct draft version * When receiving a RST, clean out the netbuf chain of that stream
2014-08-07 10:22:54 +02:00
void net_remove_netbuf(struct netbuf_head *, struct netbuf *);
Add a callback to http_response_stream(). This way we can get our code called whenever a stream is completed. This cb handler does stand alone from an http_request and is passed a netbuf data structure.
2014-08-10 18:46:44 +02:00
void net_recv_queue(struct connection *, u_int32_t, int,
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
int (*cb)(struct netbuf *));
Bring net_recv_expand() in line with others.
2015-03-16 16:52:40 +01:00
void net_recv_expand(struct connection *c, u_int32_t,
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
int (*cb)(struct netbuf *));
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
void net_send_queue(struct connection *, const void *, u_int32_t);
Add http_response_stream() which can stream data from a buffer to the client.
2014-08-07 14:23:26 +02:00
void net_send_stream(struct connection *, void *,
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
u_int32_t, int (*cb)(struct netbuf *), struct netbuf **);
begin with the ability to read control frames. something feels fishy with epoll() and its triggering of events. I probably got it wrong.
2013-04-28 19:11:44 +02:00
add base64 encoder/decoder in prep for http/2.0 and websockets
2013-07-10 15:00:53 +02:00
void kore_buf_free(struct kore_buf *);
move buf stuff back into kore.h, wont need it separately
2013-05-02 15:14:00 +02:00
struct kore_buf *kore_buf_create(u_int32_t);
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
void kore_buf_append(struct kore_buf *, const void *, u_int32_t);
move buf stuff back into kore.h, wont need it separately
2013-05-02 15:14:00 +02:00
u_int8_t *kore_buf_release(struct kore_buf *, u_int32_t *);
Massive rework of HTTP layer. This commit is a flag day, your old modules will almost certainly need to be updated in order to build properly with these changes. Summary of changes: - Offload HTTP bodies to disk if they are large (inspired by #100). (disabled by default) - The http_argument_get* macros now takes an explicit http_request parameter. - Kore will now throw 404 errors almost immediately after an HTTP request has come in instead of waiting until all data has arrived. API changes: - http_argument_get* macros now require an explicit http_request parameter. (no more magic invokations). - http_generic_404() is gone - http_populate_arguments() is gone - http_body_bytes() is gone - http_body_text() is gone - http_body_read() has been added - http_populate_post() has been added - http_populate_get() has been added - http_file_read() has been added - http_file_rewind() has been added - http_file_lookup() no longer takes name, fname, data and len parameters. - http_file_lookup() now returns a struct http_file pointer. - http_populate_multipart_form() no longer takes an secondary parameter. New configuration options: - http_body_disk_offload: Number of bytes after which Kore will offload the HTTP body to disk instead of retaining it in memory. If 0 this feature is disabled. (Default: 0) - http_body_disk_path: The path where Kore will store temporary HTTP body files. (this directory does not get created if http_body_disk_offload is 0). New example: The upload example has been added, demonstrating how to deal with file uploads from a multipart form.
2016-01-18 11:30:22 +01:00
void kore_buf_reset(struct kore_buf *);
Improvements to our message framework. Change the callback prototypes to: void callback(struct kore_msg *msg, const void *data); This allows the callbacks to receive the full kore_msg data structure as sent over the wire (including length and id). Useful for future additions to the kore_msg structure (such as worker origin). Several other improvements: * Accesslog now uses the msg framework as well. * Websocket WEBSOCKET_BROADCAST_GLOBAL now works. Small websocket improvement in this commit: * Build the frame to be sent only once when broadcasting instead of per connection we are broadcasting towards.
2015-06-23 18:17:14 +02:00
Change kore_buf_stringify() a bit. Takes a size_t pointer as its second argument now, if not NULL this will be populated with the length of the string that is being returned.
2016-06-02 07:08:19 +02:00
char *kore_buf_stringify(struct kore_buf *, size_t *);
q->value should be set to NULL if a query parameter is present but no value was set for it. introduce kore_buf_appendv() (much like readv()). introduce kore_buf_appendf() (printf into buffers).
2013-05-04 19:09:07 +02:00
void kore_buf_appendf(struct kore_buf *, const char *, ...);
Change kore_buf_appendv() to take a va_list
2014-04-22 12:16:21 +02:00
void kore_buf_appendv(struct kore_buf *, const char *, va_list);
add kore_buf_appendb() which allows us to append a kore_buf to another one. (releases the kore_buf that is being appended while at it).
2013-06-05 11:27:03 +02:00
void kore_buf_appendb(struct kore_buf *, struct kore_buf *);
Use kore_mem_find() in kore_buf_replace_string(), fixes certain crashes that could occur when calling kore_buf_replace_string() with patterns that would be found at the end or start of the buffer.
2013-09-26 16:49:44 +02:00
void kore_buf_replace_string(struct kore_buf *, char *, void *, size_t);
move buf stuff back into kore.h, wont need it separately
2013-05-02 15:14:00 +02:00
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
void kore_keymgr_run(void);
void kore_keymgr_cleanup(void);
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
#if defined(KORE_SINGLE_BINARY)
Add prototype for kore_onload().
2016-07-07 13:18:41 +02:00
void kore_onload(void);
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
#endif
Add __cplusplus macros to headers.
2015-04-01 13:25:10 +02:00
#if defined(__cplusplus)
}
#endif
first commit
2013-04-17 22:34:27 +02:00
#endif /* !__H_KORE_H */