mirrors
/
kore
mirror of https://git.kore.io/kore.git
1
0
Fork 1
Code Issues Projects Releases Wiki Activity
kore/src/worker.c

769 lines
16 KiB
C
Raw Normal View History

refactor code quite a bit.
2013-06-26 11:18:32 +02:00
/*
bump copyright to 2019
2019-02-22 16:57:28 +01:00
* Copyright (c) 2013-2019 Joris Vink <joris@coders.se>
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
NetBSD fixes, from David Carlier.
2015-06-14 16:44:37 +02:00
#include <sys/param.h>
explicitly include sys/types.h some smaller libc variants do not include this from sys/param.h.
2019-03-06 09:29:46 +01:00
#include <sys/types.h>
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
#include <sys/shm.h>
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
#include <sys/wait.h>
Add worker_rlimit_nofiles as a configurable option.
2014-07-31 09:14:03 +02:00
#include <sys/time.h>
#include <sys/resource.h>
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
#include <sys/socket.h>
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#if !defined(KORE_NO_TLS)
#include <openssl/rand.h>
#endif
Properly deal with accept() failing. In cases where accept() failed Kore would not relinquish the lock towards other worker processes. This becomes evident when dealing with a high number of concurrent connections to the point the fd table gets full. In this scenario the worker with the full fd table will spin on attempt to accept newer connections. As a bonus, Kore now has allows exactly up to worker_max_connections of connections per worker before no longer attempting to grab the accept lock.
2015-04-20 15:17:42 +02:00
#include <fcntl.h>
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
#include <grp.h>
Clean up header includes, based on a diff from Ewan Higgs via github. And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
#include <pwd.h>
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
#include <signal.h>
#include "kore.h"
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
#include "http.h"
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
#if defined(KORE_USE_PGSQL)
Normalize kore_*.h headers
2014-07-03 22:14:46 +02:00
#include "pgsql.h"
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
#endif
Add initial stab at asynchronous background tasks. More to follow.
2014-06-28 16:17:18 +02:00
#if defined(KORE_USE_TASKS)
Normalize kore_*.h headers
2014-07-03 22:14:46 +02:00
#include "tasks.h"
Add initial stab at asynchronous background tasks. More to follow.
2014-06-28 16:17:18 +02:00
#endif
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#if defined(KORE_USE_PYTHON)
#include "python_api.h"
#endif
Define WAIT_ANY if not defined previously.
2016-01-31 16:47:00 +01:00
#if !defined(WAIT_ANY)
#define WAIT_ANY (-1)
#endif
Reduce accept lock accept lockout time to 100ms.
2018-11-16 11:35:53 +01:00
#define WORKER_LOCK_TIMEOUT 100
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
#if !defined(KORE_NO_TLS)
#define WORKER_SOLO_COUNT 2
#else
#define WORKER_SOLO_COUNT 1
#endif
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
Properly calculate worker offset, otherwise we'll eventually run into trouble.
2013-07-15 11:24:49 +02:00
#define WORKER(id) \
(struct kore_worker *)((u_int8_t *)kore_workers + \
(sizeof(struct kore_worker) * id))
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
struct wlock {
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
volatile int lock;
pid_t current;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
};
static int worker_trylock(void);
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
static void worker_unlock(void);
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
static inline int worker_acceptlock_obtain(void);
static inline int worker_acceptlock_release(void);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#if !defined(KORE_NO_TLS)
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
static void worker_entropy_recv(struct kore_msg *, const void *);
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
static void worker_keymgr_response(struct kore_msg *, const void *);
static int worker_keymgr_response_verify(struct kore_msg *, const void *,
struct kore_domain **);
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#endif
Let kore_worker_make_busy() set next_lock.
2018-10-22 09:00:55 +02:00
static u_int64_t next_lock;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
static struct kore_worker *kore_workers;
Allow workers to listen on different ports. This commit allows worker processes to have individual listeners not configured by the kore configuration. If you do the following: - do not configure listeners in your .conf file - call kore_server_bind() in kore_worker_configure() the workers will no longer fight over accept locks as the configured ports no longer conflict with each other. This allows me to create X amount of instances of a worker process that are each individually accessible via unique ports.
2017-03-13 15:32:04 +01:00
static int worker_no_lock;
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
static int shm_accept_key;
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
static struct wlock *accept_lock;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
struct kore_worker *worker = NULL;
Add new configuration option: worker_set_affinity. Controls wether or not worker processes sets its affinity mask to bind themselves to a single CPU. Defaults to on (1).
2015-04-27 10:36:33 +02:00
u_int8_t worker_set_affinity = 1;
change accept threshold default to 16.
2018-11-16 11:37:09 +01:00
u_int32_t worker_accept_threshold = 16;
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
u_int32_t worker_rlimit_nofiles = 768;
u_int32_t worker_max_connections = 512;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
u_int32_t worker_active_connections = 0;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
void
kore_worker_init(void)
{
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
size_t len;
Rework accesslog handling. Move away from the parent constantly hitting the disk for every accesslog the workers are sending. The workers will now write their own accesslogs to shared memory before the parent will pick those up. The parent will flush them to disk once every second or if they grow larger then 1MB. This removes the heavy penalty for having access logs turned on when you are dealing with a large volume of requests.
2018-12-22 09:25:00 +01:00
struct kore_worker *kw;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
u_int16_t i, cpu;
Allow workers to listen on different ports. This commit allows worker processes to have individual listeners not configured by the kore configuration. If you do the following: - do not configure listeners in your .conf file - call kore_server_bind() in kore_worker_configure() the workers will no longer fight over accept locks as the configured ports no longer conflict with each other. This allows me to create X amount of instances of a worker process that are each individually accessible via unique ports.
2017-03-13 15:32:04 +01:00
worker_no_lock = 0;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
if (worker_count == 0)
Use number of cpu cores if no worker count is set.
2018-10-22 21:27:17 +02:00
worker_count = cpu_count;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#if !defined(KORE_NO_TLS)
/* account for the key manager. */
worker_count += 1;
#endif
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
len = sizeof(*accept_lock) +
(sizeof(struct kore_worker) * worker_count);
Long overdue, let Kore pick its own shm key if the default KORE_SHM_KEY is already taken.
2013-10-14 11:26:11 +02:00
Use IPC_PRIVATE when creating our shm. Instead of a homegrown mechanism of finding empty SHM keys that may or may not be available. Should fix #44
2015-04-28 10:20:36 +02:00
shm_accept_key = shmget(IPC_PRIVATE, len, IPC_CREAT | IPC_EXCL | 0700);
if (shm_accept_key == -1)
fatal("kore_worker_init(): shmget() %s", errno_s);
shmat() returns (void *)-1 upon error.
2015-03-25 19:42:24 +01:00
if ((accept_lock = shmat(shm_accept_key, NULL, 0)) == (void *)-1)
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
fatal("kore_worker_init(): shmat() %s", errno_s);
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
accept_lock->lock = 0;
accept_lock->current = 0;
Properly calculate worker offset, otherwise we'll eventually run into trouble.
2013-07-15 11:24:49 +02:00
kore_workers = (struct kore_worker *)((u_int8_t *)accept_lock +
sizeof(*accept_lock));
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
memset(kore_workers, 0, sizeof(struct kore_worker) * worker_count);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
kore_debug("kore_worker_init(): system has %d cpu's", cpu_count);
kore_debug("kore_worker_init(): starting %d workers", worker_count);
Brackets would be great here.
2015-01-19 15:26:53 +01:00
if (worker_count > cpu_count) {
Fix typo in log message
2016-02-12 08:57:58 +01:00
kore_debug("kore_worker_init(): more workers than cpu's");
Brackets would be great here.
2015-01-19 15:26:53 +01:00
}
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Rework accesslog handling. Move away from the parent constantly hitting the disk for every accesslog the workers are sending. The workers will now write their own accesslogs to shared memory before the parent will pick those up. The parent will flush them to disk once every second or if they grow larger then 1MB. This removes the heavy penalty for having access logs turned on when you are dealing with a large volume of requests.
2018-12-22 09:25:00 +01:00
/* Setup log buffers. */
for (i = 0; i < worker_count; i++) {
kw = WORKER(i);
kw->lb.offset = 0;
}
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
cpu = 0;
for (i = 0; i < worker_count; i++) {
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
kore_worker_spawn(i, cpu++);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
if (cpu == cpu_count)
cpu = 0;
}
}
void
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
kore_worker_spawn(u_int16_t id, u_int16_t cpu)
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
{
struct kore_worker *kw;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
kw = WORKER(id);
kw->id = id;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
kw->cpu = cpu;
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
kw->has_lock = 0;
kw->active_hdlr = NULL;
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
if (socketpair(AF_UNIX, SOCK_STREAM, 0, kw->pipe) == -1)
fatal("socketpair(): %s", errno_s);
Kill TCP_NODELAY warnings for socketpair() fds.
2015-06-22 22:11:03 +02:00
if (!kore_connection_nonblock(kw->pipe[0], 0) ||
!kore_connection_nonblock(kw->pipe[1], 0))
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
fatal("could not set pipe fds to nonblocking: %s", errno_s);
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
kw->pid = fork();
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
if (kw->pid == -1)
fatal("could not spawn worker child: %s", errno_s);
if (kw->pid == 0) {
kw->pid = getpid();
kore_worker_entry(kw);
/* NOTREACHED */
}
}
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
struct kore_worker *
kore_worker_data(u_int8_t id)
{
if (id >= worker_count)
fatal("id %u too large for worker count", id);
return (WORKER(id));
}
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
void
kore_worker_shutdown(void)
{
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
struct kore_worker *kw;
u_int16_t id, done;
Add -q flag. If specified Kore will run quietly and only log important messages.
2018-11-15 16:01:37 +01:00
if (!kore_quiet) {
kore_log(LOG_NOTICE,
"waiting for workers to drain and shutdown");
}
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
for (;;) {
done = 0;
for (id = 0; id < worker_count; id++) {
kw = WORKER(id);
if (kw->pid != 0)
kore_worker_wait(1);
else
done++;
}
if (done == worker_count)
break;
}
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
if (shmctl(shm_accept_key, IPC_RMID, NULL) == -1) {
kore_log(LOG_NOTICE,
"failed to deleted shm segment: %s", errno_s);
}
}
void
kore_worker_dispatch_signal(int sig)
{
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
u_int16_t id;
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
struct kore_worker *kw;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
for (id = 0; id < worker_count; id++) {
kw = WORKER(id);
Brackets would be great here.
2015-01-19 15:26:53 +01:00
if (kill(kw->pid, sig) == -1) {
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
kore_debug("kill(%d, %d): %s", kw->pid, sig, errno_s);
Brackets would be great here.
2015-01-19 15:26:53 +01:00
}
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
}
}
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
void
Let kore_worker_privdrop() take user and chroot. This will make it easier when the keymgr gets its own user/chroot settings.
2018-07-11 06:53:51 +02:00
kore_worker_privdrop(const char *runas, const char *root)
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
{
If we're iterating over rlimits, we should use rlim_t. > error: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'rlim_t' (aka 'long') [-Werror,-Wsign-compare]
2016-04-28 03:18:21 +02:00
rlim_t fd;
Add worker_rlimit_nofiles as a configurable option.
2014-07-31 09:14:03 +02:00
struct rlimit rl;
Make runas behave similarly to chroot. Add new command line knob '-r', that disables runas similar to '-n', it's implied as well for kore command runs. Add default runas (nobody) user and chroot (/var/empty) path, if none are specified, fallback to these.
2015-05-18 21:34:39 +02:00
struct passwd *pw = NULL;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
if (root == NULL)
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
fatalx("no root directory for kore_worker_privdrop");
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
Make runas behave similarly to chroot. Add new command line knob '-r', that disables runas similar to '-n', it's implied as well for kore command runs. Add default runas (nobody) user and chroot (/var/empty) path, if none are specified, fallback to these.
2015-05-18 21:34:39 +02:00
/* Must happen before chroot. */
if (skip_runas == 0) {
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
if (runas == NULL)
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
fatalx("no runas user given and -r not specified");
Let kore_worker_privdrop() take user and chroot. This will make it easier when the keymgr gets its own user/chroot settings.
2018-07-11 06:53:51 +02:00
pw = getpwnam(runas);
Make runas behave similarly to chroot. Add new command line knob '-r', that disables runas similar to '-n', it's implied as well for kore command runs. Add default runas (nobody) user and chroot (/var/empty) path, if none are specified, fallback to these.
2015-05-18 21:34:39 +02:00
if (pw == NULL) {
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
fatalx("cannot getpwnam(\"%s\") for user: %s",
Let kore_worker_privdrop() take user and chroot. This will make it easier when the keymgr gets its own user/chroot settings.
2018-07-11 06:53:51 +02:00
runas, errno_s);
Make runas behave similarly to chroot. Add new command line knob '-r', that disables runas similar to '-n', it's implied as well for kore command runs. Add default runas (nobody) user and chroot (/var/empty) path, if none are specified, fallback to these.
2015-05-18 21:34:39 +02:00
}
}
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
if (skip_chroot == 0) {
Let kore_worker_privdrop() take user and chroot. This will make it easier when the keymgr gets its own user/chroot settings.
2018-07-11 06:53:51 +02:00
if (chroot(root) == -1) {
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
fatalx("cannot chroot(\"%s\"): %s",
Let kore_worker_privdrop() take user and chroot. This will make it easier when the keymgr gets its own user/chroot settings.
2018-07-11 06:53:51 +02:00
root, errno_s);
small style fix.
2015-05-20 11:36:02 +02:00
}
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
if (chdir("/") == -1)
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
fatalx("cannot chdir(\"/\"): %s", errno_s);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
} else {
if (chdir(root) == -1)
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
fatalx("cannot chdir(\"%s\"): %s", root, errno_s);
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
}
Properly deal with accept() failing. In cases where accept() failed Kore would not relinquish the lock towards other worker processes. This becomes evident when dealing with a high number of concurrent connections to the point the fd table gets full. In this scenario the worker with the full fd table will spin on attempt to accept newer connections. As a bonus, Kore now has allows exactly up to worker_max_connections of connections per worker before no longer attempting to grab the accept lock.
2015-04-20 15:17:42 +02:00
if (getrlimit(RLIMIT_NOFILE, &rl) == -1) {
kore_log(LOG_WARNING, "getrlimit(RLIMIT_NOFILE): %s", errno_s);
} else {
for (fd = 0; fd < rl.rlim_cur; fd++) {
if (fcntl(fd, F_GETFD, NULL) != -1) {
worker_rlimit_nofiles++;
}
}
}
call setrlimit() before we drop privs.
2015-04-09 10:06:18 +02:00
rl.rlim_cur = worker_rlimit_nofiles;
rl.rlim_max = worker_rlimit_nofiles;
if (setrlimit(RLIMIT_NOFILE, &rl) == -1) {
use %u when logging worker_rlimit_nofiles
2019-02-11 14:00:28 +01:00
kore_log(LOG_ERR, "setrlimit(RLIMIT_NOFILE, %u): %s",
call setrlimit() before we drop privs.
2015-04-09 10:06:18 +02:00
worker_rlimit_nofiles, errno_s);
}
Make runas behave similarly to chroot. Add new command line knob '-r', that disables runas similar to '-n', it's implied as well for kore command runs. Add default runas (nobody) user and chroot (/var/empty) path, if none are specified, fallback to these.
2015-05-18 21:34:39 +02:00
if (skip_runas == 0) {
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
if (setgroups(1, &pw->pw_gid) ||
NetBSD fixes, from David Carlier.
2015-06-14 16:44:37 +02:00
#if defined(__MACH__) || defined(NetBSD)
Fix typo.
2015-06-14 16:50:10 +02:00
setgid(pw->pw_gid) || setegid(pw->pw_gid) ||
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
setuid(pw->pw_uid) || seteuid(pw->pw_uid))
Add OSX support, buildable via "make osx". Make sure you have OpenSSL 1.0.0d+ (available from Macports) installed. Based on diff from Vaibhav Bhembre via github
2013-07-17 20:17:00 +02:00
#else
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
setresgid(pw->pw_gid, pw->pw_gid, pw->pw_gid) ||
setresuid(pw->pw_uid, pw->pw_uid, pw->pw_uid))
Add OSX support, buildable via "make osx". Make sure you have OpenSSL 1.0.0d+ (available from Macports) installed. Based on diff from Vaibhav Bhembre via github
2013-07-17 20:17:00 +02:00
#endif
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
fatalx("cannot drop privileges");
Kore no longer requires root to be started.
2013-11-18 00:42:57 +01:00
}
Add pledge support under OpenBSD. All worker processes will now call pledge(2) after dropping privileges (even if -rn was specified). By default Kore will use the following promises: "stdio rpath inet error" If your application requires more privileges, you can add more pledges by setting them in your configuration using the 'pledge' directive: pledge dns wpath
2018-07-31 06:51:34 +02:00
#if defined(KORE_USE_PLATFORM_PLEDGE)
kore_platform_pledge();
#endif
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
}
void
kore_worker_entry(struct kore_worker *kw)
{
Change kore_preload() and kore_onload(). Renamed both of them: kore_preload -> kore_parent_configure kore_onload -> kore_worker_configure These functions will now always be called if they are defined in any module regardless of your application being built as a single binary or not.
2017-02-01 17:12:11 +01:00
struct kore_runtime_call *rcall;
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
char buf[16];
Rework accesslog handling. Move away from the parent constantly hitting the disk for every accesslog the workers are sending. The workers will now write their own accesslogs to shared memory before the parent will pick those up. The parent will flush them to disk once every second or if they grow larger then 1MB. This removes the heavy penalty for having access logs turned on when you are dealing with a large volume of requests.
2018-12-22 09:25:00 +01:00
int quit, had_lock;
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
u_int64_t netwait, now, next_prune;
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#if !defined(KORE_NO_TLS)
u_int64_t last_seed;
#endif
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
worker = kw;
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Correct snprintf() usage
2014-04-23 16:29:58 +02:00
(void)snprintf(buf, sizeof(buf), "kore [wrk %d]", kw->id);
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#if !defined(KORE_NO_TLS)
if (kw->id == KORE_WORKER_KEYMGR)
(void)snprintf(buf, sizeof(buf), "kore [keymgr]");
#endif
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
kore_platform_proctitle(buf);
Add new configuration option: worker_set_affinity. Controls wether or not worker processes sets its affinity mask to bind themselves to a single CPU. Defaults to on (1).
2015-04-27 10:36:33 +02:00
if (worker_set_affinity == 1)
kore_platform_worker_setcpu(kw);
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
kore_pid = kw->pid;
Use sigaction() for signals. Don't duplicate signal setup code between parent and worker processes.
2018-05-25 20:49:02 +02:00
kore_signal_setup();
Allow Kore to be started in foreground mode (-f) Sending SIGINT or Ctrl-c will reap all workers and exit again. Useful for development.
2014-07-31 13:27:04 +02:00
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#if !defined(KORE_NO_TLS)
if (kw->id == KORE_WORKER_KEYMGR) {
kore_keymgr_run();
exit(0);
}
#endif
shuffle stuff around so we're not hitting bad juju.
2018-08-13 13:07:32 +02:00
net_init();
kore_connection_init();
call fatalx() instead of fatal() in certain cases.
2018-08-13 13:01:27 +02:00
kore_platform_event_init();
kore_msg_worker_init();
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
kore_worker_privdrop(kore_runas_user, kore_root_path);
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
http_init();
resolve filemap paths after workers chrooted. otherwise the paths inside chrooted workers are incorrect.
2018-07-08 17:51:35 +02:00
kore_filemap_resolve_paths();
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
kore_accesslog_worker_init();
#endif
Add rudimentary timers to Kore. Timers are in ms resolution and are added using kore_timer_add(cb, interval, flags). Both oneshot timers and continious timers are supported.
2015-04-06 18:54:35 +02:00
kore_timer_init();
Add filemaps. A filemap is a way of telling Kore to serve files from a directory much like a traditional webserver can do. Kore filemaps only handles files. Kore does not generate directory indexes or deal with non-regular files. The way files are sent to a client differs a bit per platform and build options: default: - mmap() backed file transfer due to TLS. NOTLS=1 - sendfile() under FreeBSD, macOS and Linux. - mmap() backed file for OpenBSD. The opened file descriptors/mmap'd regions are cached and reused when appropriate. If a file is no longer in use it will be closed and evicted from the cache after 30 seconds. New API's are available allowing developers to use these facilities via: void net_send_fileref(struct connection *, struct kore_fileref *); void http_response_fileref(struct http_request *, struct kore_fileref *); Kore will attempt to match media types based on file extensions. A few default types are built-in. Others can be added via the new "http_media_type" configuration directive.
2018-06-28 13:27:44 +02:00
kore_fileref_init();
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
kore_domain_keymgr_init();
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
quit = 0;
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
had_lock = 0;
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
next_lock = 0;
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
next_prune = 0;
set worker_active_connections to 0 always. fixes a bug where if a worker died it contained garbled data...
2016-12-04 16:49:42 +01:00
worker_active_connections = 0;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
#if defined(KORE_USE_PGSQL)
Decouple pgsql from the http layer. When the pgsql layer was introduced it was tightly coupled with the http layer in order to make async work fluently. The time has come to split these up and follow the same method we used for tasks, allowing either http requests to be tied to a pgsql data structure or a simple callback function. This also reworks the internal queueing of pgsql requests until connections to the db are available again. The following API functions were changes: - kore_pgsql_query_init() -> kore_pgsql_setup() no longer takes an http_request parameter. - NEW kore_pgsql_init() must be called before operating on an kore_pgsql structure. - NEW kore_pgsql_bind_request() binds an http_request to a kore_pgsql data structure. - NEW kore_pgsql_bind_callback() binds a callback to a kore_pgsql data structure. With all of this you can now build kore with PGSQL=1 NOHTTP=1. The pgsql/ example has been updated to reflect these changes and new features.
2017-03-24 12:53:07 +01:00
kore_pgsql_sys_init();
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
#endif
Add initial stab at asynchronous background tasks. More to follow.
2014-06-28 16:17:18 +02:00
#if defined(KORE_USE_TASKS)
kore_task_init();
#endif
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#if !defined(KORE_NO_TLS)
last_seed = 0;
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
kore_msg_register(KORE_MSG_CRL, worker_keymgr_response);
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
kore_msg_register(KORE_MSG_ENTROPY_RESP, worker_entropy_recv);
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
kore_msg_register(KORE_MSG_CERTIFICATE, worker_keymgr_response);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
if (worker->restarted) {
kore_msg_send(KORE_WORKER_KEYMGR,
KORE_MSG_CERTIFICATE_REQ, NULL, 0);
}
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#endif
Allow workers to listen on different ports. This commit allows worker processes to have individual listeners not configured by the kore configuration. If you do the following: - do not configure listeners in your .conf file - call kore_server_bind() in kore_worker_configure() the workers will no longer fight over accept locks as the configured ports no longer conflict with each other. This allows me to create X amount of instances of a worker process that are each individually accessible via unique ports.
2017-03-13 15:32:04 +01:00
if (nlisteners == 0)
worker_no_lock = 1;
Add -q flag. If specified Kore will run quietly and only log important messages.
2018-11-15 16:01:37 +01:00
if (!kore_quiet) {
kore_log(LOG_NOTICE,
log pid number of worker upon start.
2019-03-19 13:42:22 +01:00
"worker %d started (cpu#%d, pid#%d)",
kw->id, kw->cpu, kw->pid);
Add -q flag. If specified Kore will run quietly and only log important messages.
2018-11-15 16:01:37 +01:00
}
Make kore_onload() to be done per worker. Mimics the behaviour of dso builds.
2016-07-06 21:59:17 +02:00
Change kore_preload() and kore_onload(). Renamed both of them: kore_preload -> kore_parent_configure kore_onload -> kore_worker_configure These functions will now always be called if they are defined in any module regardless of your application being built as a single binary or not.
2017-02-01 17:12:11 +01:00
rcall = kore_runtime_getcall("kore_worker_configure");
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
if (rcall != NULL) {
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
kore_runtime_execute(rcall);
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
kore_free(rcall);
}
Change kore_preload() and kore_onload(). Renamed both of them: kore_preload -> kore_parent_configure kore_onload -> kore_worker_configure These functions will now always be called if they are defined in any module regardless of your application being built as a single binary or not.
2017-02-01 17:12:11 +01:00
Teach single binaries about SIGHUP. Make sure kore_onload() and kore_preload() can be called either in native or python runtime.
2017-01-26 13:26:55 +01:00
kore_module_onload();
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
worker->restarted = 0;
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
for (;;) {
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
now = kore_time_ms();
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#if !defined(KORE_NO_TLS)
if ((now - last_seed) > KORE_RESEED_TIME) {
kore_msg_send(KORE_WORKER_KEYMGR,
KORE_MSG_ENTROPY_REQ, NULL, 0);
last_seed = now;
}
#endif
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
if (!worker->has_lock && next_lock <= now) {
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
if (worker_acceptlock_obtain()) {
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
if (had_lock == 0) {
kore_platform_enable_accept();
had_lock = 1;
}
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
} else {
next_lock = now + WORKER_LOCK_TIMEOUT / 2;
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
}
}
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
netwait = kore_timer_next_run(now);
Rework accesslog handling. Move away from the parent constantly hitting the disk for every accesslog the workers are sending. The workers will now write their own accesslogs to shared memory before the parent will pick those up. The parent will flush them to disk once every second or if they grow larger then 1MB. This removes the heavy penalty for having access logs turned on when you are dealing with a large volume of requests.
2018-12-22 09:25:00 +01:00
kore_platform_event_wait(netwait);
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
now = kore_time_ms();
Rework accesslog handling. Move away from the parent constantly hitting the disk for every accesslog the workers are sending. The workers will now write their own accesslogs to shared memory before the parent will pick those up. The parent will flush them to disk once every second or if they grow larger then 1MB. This removes the heavy penalty for having access logs turned on when you are dealing with a large volume of requests.
2018-12-22 09:25:00 +01:00
if (worker->has_lock) {
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
if (worker_acceptlock_release())
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
next_lock = now + WORKER_LOCK_TIMEOUT;
}
if (!worker->has_lock) {
if (had_lock == 1) {
had_lock = 0;
kore_platform_disable_accept();
}
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
}
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
Add asynchronous subprocess support. This adds kore.proc to the python runtime allowing async processing handling: The kore.proc method takes the command to run and an optional timeout parameter in milliseconds. If the process did not exit normally after that amount of time a TimeoutError exception is raised. For instance: async def run(cmd): proc = kore.proc(cmd, 1000) try: await proc.send("hello") proc.close_stdin() except TimeoutError: proc.kill() retcode = await proc.reap() return retcode
2018-10-26 19:19:47 +02:00
if (sig_recv != 0) {
switch (sig_recv) {
case SIGHUP:
kore_module_reload(1);
break;
case SIGQUIT:
case SIGINT:
case SIGTERM:
quit = 1;
break;
case SIGCHLD:
#if defined(KORE_USE_PYTHON)
kore_python_proc_reap();
#endif
break;
default:
break;
}
sig_recv = 0;
}
Rework accesslog handling. Move away from the parent constantly hitting the disk for every accesslog the workers are sending. The workers will now write their own accesslogs to shared memory before the parent will pick those up. The parent will flush them to disk once every second or if they grow larger then 1MB. This removes the heavy penalty for having access logs turned on when you are dealing with a large volume of requests.
2018-12-22 09:25:00 +01:00
if (quit)
break;
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
kore_timer_run(now);
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#if !defined(KORE_NO_HTTP)
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
http_process();
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
#endif
Add async/await support for socket i/o in python. This means you can now do things like: resp = await koresock.recv(1024) await koresock.send(resp) directly from page handlers if they are defined as async. Adds lots more to the python goo such as fatalx(), bind_unix(), task_create() and socket_wrap().
2018-10-15 20:18:54 +02:00
#if defined(KORE_USE_PYTHON)
kore_python_coro_run();
#endif
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
if (next_prune <= now) {
kore_connection_check_timeout(now);
kore_connection_prune(KORE_CONNECTION_PRUNE_DISCONNECT);
next_prune = now + 500;
}
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
}
Add kore_[parent|worker]_teardown(). If exists these functions are called when the worker is exiting and when right before the parent exists. Allows for cleanup code for applications if need to do cleanup on exit.
2018-10-23 21:46:34 +02:00
rcall = kore_runtime_getcall("kore_worker_teardown");
if (rcall != NULL) {
kore_runtime_execute(rcall);
kore_free(rcall);
}
rename *_fini to *_cleanup
2016-01-04 22:40:14 +01:00
kore_platform_event_cleanup();
kore_connection_cleanup();
Added cleanup for the module handlers and validators
2016-02-01 12:30:20 +01:00
kore_domain_cleanup();
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
kore_module_cleanup();
Add resource management as part of the kore shutdown process.
2015-12-29 20:39:39 +01:00
#if !defined(KORE_NO_HTTP)
rename *_fini to *_cleanup
2016-01-04 22:40:14 +01:00
http_cleanup();
Add resource management as part of the kore shutdown process.
2015-12-29 20:39:39 +01:00
#endif
rename *_fini to *_cleanup
2016-01-04 22:40:14 +01:00
net_cleanup();
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
#if defined(KORE_USE_PYTHON)
kore_python_cleanup();
#endif
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
#if defined(KORE_USE_PGSQL)
kore_pgsql_sys_cleanup();
#endif
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
kore_debug("worker %d shutting down", kw->id);
Add asynchronous pgsql query support to python. This commit adds the ability to use python "await" to suspend execution of your page handler until the query sent to postgresql has returned a result. This is built upon the existing asynchrous query framework Kore had. With this you can now write stuff like: async def page(req): result = await req.pgsql("db", "SELECT name FROM table"); req.response(200, json.dumps(result).encode("utf-8")) The above code will fire off a query and suspend itself so Kore can take care of business as usual until the query is successful at which point Kore will jump back into the handler and resume. This does not use threading, it's purely based on Python's excellent coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
kore_mem_cleanup();
refactor code quite a bit.
2013-06-26 11:18:32 +02:00
exit(0);
}
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
void
kore_worker_wait(int final)
{
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
u_int16_t id;
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
pid_t pid;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
struct kore_worker *kw;
remove kore_module_handle for NOHTTP=1 builds.
2018-07-17 14:28:43 +02:00
const char *func;
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
int status;
if (final)
pid = waitpid(WAIT_ANY, &status, 0);
else
pid = waitpid(WAIT_ANY, &status, WNOHANG);
if (pid == -1) {
kore_debug("waitpid(): %s", errno_s);
return;
}
if (pid == 0)
return;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
for (id = 0; id < worker_count; id++) {
kw = WORKER(id);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
if (kw->pid != pid)
continue;
Add -q flag. If specified Kore will run quietly and only log important messages.
2018-11-15 16:01:37 +01:00
if (final == 0 || (final == 1 && !kore_quiet)) {
kore_log(LOG_NOTICE, "worker %d (%d)-> status %d",
kw->id, pid, status);
}
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
if (final) {
kw->pid = 0;
break;
}
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
if (WEXITSTATUS(status) || WTERMSIG(status) ||
WCOREDUMP(status)) {
remove kore_module_handle for NOHTTP=1 builds.
2018-07-17 14:28:43 +02:00
func = "none";
#if !defined(KORE_NO_HTTP)
if (kw->active_hdlr != NULL)
func = kw->active_hdlr->func;
#endif
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
kore_log(LOG_NOTICE,
use handler function name if available when logging a worker process that has died.
2013-07-07 14:56:50 +02:00
"worker %d (pid: %d) (hdlr: %s) gone",
remove kore_module_handle for NOHTTP=1 builds.
2018-07-17 14:28:43 +02:00
kw->id, kw->pid, func);
If the worker that owns the accept lock dies, make sure the next one in line can grab it. Otherwise kore comes to a halt (by not accepting new connections).
2013-07-05 20:19:50 +02:00
Separate private keys from worker processes. Kore will now isolate RSA private keys to a separate process (keymgr). Worker processes that require RSA signing for TLS connections will communicate with this keymgr process in order to do so. This behaviour cannot be disabled and is always turned on.
2016-06-08 13:55:14 +02:00
#if !defined(KORE_NO_TLS)
if (id == KORE_WORKER_KEYMGR) {
kore_log(LOG_CRIT, "keymgr gone, stopping");
kw->pid = 0;
if (raise(SIGTERM) != 0) {
kore_log(LOG_WARNING,
"failed to raise SIGTERM signal");
}
break;
}
#endif
Allow workers to listen on different ports. This commit allows worker processes to have individual listeners not configured by the kore configuration. If you do the following: - do not configure listeners in your .conf file - call kore_server_bind() in kore_worker_configure() the workers will no longer fight over accept locks as the configured ports no longer conflict with each other. This allows me to create X amount of instances of a worker process that are each individually accessible via unique ports.
2017-03-13 15:32:04 +01:00
if (kw->pid == accept_lock->current &&
worker_no_lock == 0)
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
worker_unlock();
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
remove kore_module_handle for NOHTTP=1 builds.
2018-07-17 14:28:43 +02:00
#if !defined(KORE_NO_HTTP)
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
if (kw->active_hdlr != NULL) {
kw->active_hdlr->errors++;
kore_log(LOG_NOTICE,
"hdlr %s has caused %d error(s)",
kw->active_hdlr->func,
kw->active_hdlr->errors);
If the worker that owns the accept lock dies, make sure the next one in line can grab it. Otherwise kore comes to a halt (by not accepting new connections).
2013-07-05 20:19:50 +02:00
}
remove kore_module_handle for NOHTTP=1 builds.
2018-07-17 14:28:43 +02:00
#endif
If the worker that owns the accept lock dies, make sure the next one in line can grab it. Otherwise kore comes to a halt (by not accepting new connections).
2013-07-05 20:19:50 +02:00
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
kore_log(LOG_NOTICE, "restarting worker %d", kw->id);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
kw->restarted = 1;
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
kore_msg_parent_remove(kw);
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
kore_worker_spawn(kw->id, kw->cpu);
Add our messaging framework. With this framework apps can now send messages between worker processes. A new API function exists: int kore_msg_register(u_int8_t id, void (*cb)(const void *, u_int32_t); This API call allows your app to register a new message callback for a given ID. You can then send messages on this ID to other workers using: void kore_msg_send(u_int8_t id, void *data, u_int32_t length); This framework will interally be used for a few things such as allowing websocket data to broadcasted between all workers, adding unified caching and hopefully eventually moving the access log to this as well. Some internals have changed with this commit: * worker_clients has been called connections. * the parent now initializes the net, and event subsystems. * kore_worker_websocket_broadcast() is dead.
2015-06-22 21:13:32 +02:00
kore_msg_parent_add(kw);
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
} else {
kore_log(LOG_NOTICE,
Kore will now keep track of page handlers that cause workers to die. This is useful to track down any issues you might have in your module. A log entry with a page handler causing issues looks like: Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11 Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
"worker %d (pid: %d) signaled us (%d)",
kw->id, kw->pid, status);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
}
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
break;
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
}
}
Add kore_worker_make_busy(). Calling this from your page handler will cause your current worker to give up the acceptlock (if it holds it). This is particularly useful if you are about to run code that may block a bit longer then you are comfortable with. Calling this will cause the acceptlock to shuffle to another free worker which in turn makes sure your application can keep accepting requests.
2018-07-11 18:00:16 +02:00
void
kore_worker_make_busy(void)
{
Let kore_worker_make_busy() not unlock solo workers
2018-07-13 21:04:59 +02:00
if (worker_count == WORKER_SOLO_COUNT || worker_no_lock == 1)
return;
Add kore_worker_make_busy(). Calling this from your page handler will cause your current worker to give up the acceptlock (if it holds it). This is particularly useful if you are about to run code that may block a bit longer then you are comfortable with. Calling this will cause the acceptlock to shuffle to another free worker which in turn makes sure your application can keep accepting requests.
2018-07-11 18:00:16 +02:00
if (worker->has_lock) {
worker_unlock();
worker->has_lock = 0;
Let kore_worker_make_busy() set next_lock.
2018-10-22 09:00:55 +02:00
next_lock = kore_time_ms() + WORKER_LOCK_TIMEOUT;
Add kore_worker_make_busy(). Calling this from your page handler will cause your current worker to give up the acceptlock (if it holds it). This is particularly useful if you are about to run code that may block a bit longer then you are comfortable with. Calling this will cause the acceptlock to shuffle to another free worker which in turn makes sure your application can keep accepting requests.
2018-07-11 18:00:16 +02:00
}
}
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
static inline int
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
worker_acceptlock_release(void)
Make kore_worker_acceptlock_release() available to loaded modules. This allows us to move the accept lock manually to another Kore worker in case we own it and are about to do some heavy lifting on the current Kore worker.
2013-10-24 09:05:46 +02:00
{
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
if (worker_count == WORKER_SOLO_COUNT || worker_no_lock == 1)
return (0);
Make kore_worker_acceptlock_release() available to loaded modules. This allows us to move the accept lock manually to another Kore worker in case we own it and are about to do some heavy lifting on the current Kore worker.
2013-10-24 09:05:46 +02:00
if (worker->has_lock != 1)
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
return (0);
if (worker_active_connections < worker_max_connections) {
#if !defined(KORE_NO_HTTP)
if (http_request_count < http_request_limit)
return (0);
#else
return (0);
#endif
}
#if defined(WORKER_DEBUG)
kore_log(LOG_DEBUG, "worker busy, releasing lock");
#endif
Make kore_worker_acceptlock_release() available to loaded modules. This allows us to move the accept lock manually to another Kore worker in case we own it and are about to do some heavy lifting on the current Kore worker.
2013-10-24 09:05:46 +02:00
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
worker_unlock();
worker->has_lock = 0;
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
return (1);
Make kore_worker_acceptlock_release() available to loaded modules. This allows us to move the accept lock manually to another Kore worker in case we own it and are about to do some heavy lifting on the current Kore worker.
2013-10-24 09:05:46 +02:00
}
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
static inline int
rework timers so they fire more predictably. this change also stops python coroutines from waking up very late after their timeout has expired. in filerefs, don't prime the timer until we actually have something to expire, and kill the timer when the last ref drops.
2019-03-21 10:17:08 +01:00
worker_acceptlock_obtain(void)
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
{
If we do not own the accept lock, reduce timeout for platform wait. This way multiple workers won't stall when passing the accept lock around for at maximum 100 ms (time spent in epoll/kqueue waits).
2014-07-28 23:27:58 +02:00
int r;
Rework net, worker and some http internals. - The net code no longer has a recv_queue, instead reuse same recv buffer. - Introduce net_recv_reset() to reset the recv buffer when needed. - Have the workers spread the load better between them by slightly delaying their next accept lock and giving them an accept treshold so they don't go ahead and keep accepting connections if they end up winning the race constantly between the workers. - The kore_worker_acceptlock_release() is no longer available. - Prepopulate the HTTP server response header that is added to each response in both normal HTTP and SPDY modes. - The path and host members of http_request are now allocated on the heap. These changes overall result better performance on a multicore machine, especially the worker load changes shine through.
2014-10-22 21:16:49 +02:00
if (worker->has_lock == 1)
return (1);
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
if (worker_count == WORKER_SOLO_COUNT || worker_no_lock == 1) {
- Better spread load between all worker processes. - Introduce own memory management system on top of malloc to keep track of all our allocations and free's. Later we should introduce a pooling mechanism for fixed size allocations (http_request comes to mind). - Introduce ssl_cipher in configuration. Memory usage is kind of high right now, but it seems its OpenSSL doing it rather then Kore.
2013-06-27 08:43:07 +02:00
worker->has_lock = 1;
If we do not own the accept lock, reduce timeout for platform wait. This way multiple workers won't stall when passing the accept lock around for at maximum 100 ms (time spent in epoll/kqueue waits).
2014-07-28 23:27:58 +02:00
return (1);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
}
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
if (worker_active_connections >= worker_max_connections)
return (0);
If we do not own the accept lock, reduce timeout for platform wait. This way multiple workers won't stall when passing the accept lock around for at maximum 100 ms (time spent in epoll/kqueue waits).
2014-07-28 23:27:58 +02:00
Let http_request_limit matter. Before http_request_limit just constrained the number of HTTP requests we'd deal with in a single http_process_requests() call. But it should really mean how many maximum HTTP requests are allowed to be alive in the worker process before we start sending 503s back. While here, drop the lock timeout for a worker to 100ms down from 500ms and do not allow a worker to grab the accept lock if their HTTP request queue is full. This makes things much more pleasant memory wise as the http_request_pool won't just grow over time.
2018-02-13 11:56:51 +01:00
#if !defined(KORE_NO_HTTP)
if (http_request_count >= http_request_limit)
return (0);
#endif
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
r = 0;
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
if (worker_trylock()) {
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
r = 1;
worker->has_lock = 1;
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
#if defined(WORKER_DEBUG)
kore_log(LOG_DEBUG, "got lock");
#endif
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
}
If we do not own the accept lock, reduce timeout for platform wait. This way multiple workers won't stall when passing the accept lock around for at maximum 100 ms (time spent in epoll/kqueue waits).
2014-07-28 23:27:58 +02:00
return (r);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
}
static int
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
worker_trylock(void)
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
{
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
if (!__sync_bool_compare_and_swap(&(accept_lock->lock), 0, 1))
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
return (0);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
accept_lock->current = worker->pid;
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
return (1);
Add a form of synchronization between what worker will be accepting new connections and which ones will not be notified for it. Fixes the thundering herd problem, and nicely spreads out load between all the workers equally. A configuration option (workers_max_connections) is available to tweak how many connections a worker will have before giving up the accept lock. Two ways are added to this commit for access locking: - Locking via semaphores. - Locking via GCC's builtin atomic methods. The default is running with semaphores disabled (OpenBSD cannot do sem_init() with pshared set to 1, which is required). If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS, and -lpthread to LDFLAGS in the Makefile. Other fixes: - BSD: add a timeout to kevent(). - Merge kore_worker_wait together, linux knows waitpid() as well. - Send the correct SIGQUIT signal to workers instead of SIGINT. - Fix kore_time_ms(). - Log fatal worker messages in syslog. - Refactor code even more. - Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
}
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
static void
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
worker_unlock(void)
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
{
Rework the accept lock. Kore no longer passes the accept lock to the "next in line" worker but instead all workers will attempt to grab the lock if they can. Also remember if we had the lock previous iteration of the event loop and don't constantly disable/enable the accepting sockets. Makes Kore scale even better across multiple cpu's.
2014-07-30 15:20:09 +02:00
accept_lock->current = 0;
if (!__sync_bool_compare_and_swap(&(accept_lock->lock), 1, 0))
kore_log(LOG_NOTICE, "worker_unlock(): wasnt locked");
Rework the way worker processes give each other the accept lock. Instead of waiting until one worker is filled up on connections the workers find the next lowest loaded worker and will hand over the lock to them instead. This will cause a nicer spread of load. Instead of running one accept per event loop, we attempt to accept as many as worker_max_connections allows. Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
}
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#if !defined(KORE_NO_TLS)
static void
worker_entropy_recv(struct kore_msg *msg, const void *data)
{
add warning + RAND_poll() to callback.
2017-02-28 06:31:09 +01:00
if (msg->length != 1024) {
kore_log(LOG_WARNING,
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
"invalid entropy response (got:%zu - wanted:1024)",
add warning + RAND_poll() to callback.
2017-02-28 06:31:09 +01:00
msg->length);
}
RAND_poll();
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
RAND_seed(data, msg->length);
}
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
static void
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
worker_keymgr_response(struct kore_msg *msg, const void *data)
{
struct kore_domain *dom;
const struct kore_x509_msg *req;
if (!worker_keymgr_response_verify(msg, data, &dom))
return;
req = (const struct kore_x509_msg *)data;
switch (msg->id) {
case KORE_MSG_CERTIFICATE:
kore_domain_tlsinit(dom, req->data, req->data_len);
break;
case KORE_MSG_CRL:
kore_domain_crl_add(dom, req->data, req->data_len);
break;
default:
kore_log(LOG_WARNING, "unknown keymgr request %u", msg->id);
break;
}
}
static int
worker_keymgr_response_verify(struct kore_msg *msg, const void *data,
struct kore_domain **out)
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
{
struct kore_domain *dom;
const struct kore_x509_msg *req;
if (msg->length < sizeof(*req)) {
kore_log(LOG_WARNING,
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
"short keymgr message (%zu)", msg->length);
return (KORE_RESULT_ERROR);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
}
req = (const struct kore_x509_msg *)data;
if (msg->length != (sizeof(*req) + req->data_len)) {
kore_log(LOG_WARNING,
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
"invalid keymgr payload (%zu)", msg->length);
return (KORE_RESULT_ERROR);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
}
if (req->domain_len > KORE_DOMAINNAME_LEN) {
kore_log(LOG_WARNING,
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
"invalid keymgr domain (%u)",
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
req->domain_len);
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
return (KORE_RESULT_ERROR);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
}
dom = NULL;
TAILQ_FOREACH(dom, &domains, list) {
if (!strncmp(dom->domain, req->domain, req->domain_len))
break;
}
if (dom == NULL) {
kore_log(LOG_WARNING,
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
"got keymgr response for domain that does not exist");
return (KORE_RESULT_ERROR);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
}
Let CRLs be reloadable via keymgr. With these changes CRLs can be reloaded like certificates by sending a SIGUSR1 to the parent process. Track mtime on both certificate files and CRL files as well and only submit them to the workers if this has changed.
2019-01-14 11:41:50 +01:00
*out = dom;
return (KORE_RESULT_OK);
Allow on-the-fly reloading of certificates/keys. This commit introduces the ability for the keymgr process to reload the certificates/keys for domains when receiving a SIGUSR1 signal. The keymgr receives 2 new configuration options: - keymgr_root_path The root path where the keymgr will live. If -n is not specified when the application starts the keymgr process will chroot into here. - keymgr_runas_user The user the keymgr will drop privileges towards if -r was not specified. All certfile and certkey configuration options are now relative to the keymgr_root_path configuration setting. The keymgr process will now also load the certificate for the domain (rather then the workers) and submit these to the worker processes so they can be reloaded when required. Worker processes will refuse connections until the TLS configuration for a given domain is completed (aka: the workers receive the certificate for that domain). Other changes: - client_certificates renamed to client_verify. - the chroot configuration option is now called root. - kore is a little more verbose if privsep options are missing. - filemaps are now relative to the root configuration option.
2018-07-11 09:44:29 +02:00
}
Let workers fetch entropy from keymgr. At bootup and every 1800 seconds after that the worker processes will ask the keymgr for new entropy that they will seed into their PRNG. Additionally once received the worker calls RAND_poll() to grab more entropy from the system to be mixed in.
2017-02-28 06:28:35 +01:00
#endif