kore/README.md

134 lines
3.6 KiB
Markdown
Raw Permalink Normal View History

2014-03-02 18:16:32 +01:00
About
-----
2015-05-13 09:59:02 +02:00
2022-08-08 12:49:55 +02:00
Kore (https://kore.io) is a web application platform for writing scalable,
concurrent web based processes in C or Python.
2013-05-01 21:36:00 +02:00
2022-08-08 12:49:55 +02:00
It is built with a "secure by default" approach. It is fully privilege
separated while using strong security features at the operating system level
such as seccomp, pledge, unveil and more.
Today Kore is used in a variety of applications ranging from high assurance
cryptographic devices, machine-learning stacks and even in the aerospace
industry.
From embedded platforms all the way to high performance servers. *Kore scales.*
2013-05-01 21:36:00 +02:00
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
Key Features
------------
2014-03-02 18:16:32 +01:00
* Supports SNI
* Supports HTTP/1.1
2014-11-24 11:11:01 +01:00
* Websocket support
2016-08-01 09:41:12 +02:00
* Privseps by default
2017-06-28 10:20:14 +02:00
* TLS enabled by default
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
* Optional background tasks
2014-03-02 18:16:32 +01:00
* Built-in parameter validation
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
* Optional asynchronous PostgreSQL support
2018-02-05 16:21:28 +01:00
* Optional support for page handlers in Python
2018-07-17 15:16:27 +02:00
* Reload private keys and certificates on-the-fly
2019-11-07 12:25:14 +01:00
* Automatic X509 certificates via ACME (with privsep)
2016-08-01 09:41:12 +02:00
* Private keys isolated in separate process (RSA and ECDSA)
2014-11-07 17:19:41 +01:00
* Default sane TLS ciphersuites (PFS in all major browsers)
2014-03-02 18:16:32 +01:00
* Modules can be reloaded on-the-fly, even while serving content
2019-09-27 23:53:15 +02:00
* Worker processes sandboxed on OpenBSD (pledge) and Linux (seccomp)
2017-02-06 23:38:21 +01:00
* Event driven (epoll/kqueue) architecture with per CPU worker processes
2016-08-01 09:41:12 +02:00
* Build your web application as a precompiled dynamic library or single binary
2013-06-05 14:10:29 +02:00
2019-11-13 15:57:07 +01:00
And lots more.
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
2013-06-24 12:05:22 +02:00
License
2014-03-02 18:16:32 +01:00
-------
* Kore is licensed under the ISC license
2013-06-24 12:05:22 +02:00
2016-08-01 15:03:22 +02:00
Documentation
--------------
2022-03-21 12:44:01 +01:00
[Read the documentation](https://docs.kore.io/4.2.0/)
2016-08-01 15:03:22 +02:00
2018-11-22 13:05:02 +01:00
Performance
-----------
Read the [benchmarks](https://blog.kore.io/posts/benchmarks) blog post.
2013-07-28 19:25:46 +02:00
Platforms supported
2014-03-02 18:16:32 +01:00
-------------------
* Linux
* OpenBSD
* FreeBSD
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
* MacOS
2014-03-02 18:16:32 +01:00
Kore only supports x64, arm and aarch64 architectures.
2014-07-03 22:40:12 +02:00
Building Kore
-------------
2022-09-08 13:08:31 +02:00
Clone this repository or get the latest release at [https://kore.io/releases/4.2.3](https://kore.io/releases/4.2.3).
2014-07-03 22:40:12 +02:00
Requirements
2022-12-28 15:51:14 +01:00
* openssl 1.1.1, libressl 3.x or openssl 3.
2014-07-03 22:40:12 +02:00
2019-05-01 22:40:27 +02:00
Requirement for asynchronous curl (optional)
* libcurl (7.64.0 or higher)
2019-05-01 22:40:27 +02:00
Requirements for background tasks (optional)
2014-07-03 22:40:12 +02:00
* pthreads
Requirements for pgsql (optional)
2014-07-03 22:40:12 +02:00
* libpq
Requirements for python (optional)
2017-03-30 09:40:13 +02:00
* Python 3.6+
2023-01-23 21:34:22 +01:00
Requirements for lua support (optional)
* Lua 5.4+
2014-07-03 22:40:12 +02:00
Normal compilation and installation:
```
2017-03-06 14:28:06 +01:00
$ cd kore
$ make
2014-07-03 22:40:12 +02:00
# make install
```
If you would like to build a specific flavor, you can enable
those by setting a shell environment variable before running **_make_**.
2023-01-23 21:34:22 +01:00
* LUA=1 (compiles in LUA support)
2019-11-07 12:25:14 +01:00
* ACME=1 (compiles in ACME support)
2019-05-01 22:40:27 +02:00
* CURL=1 (compiles in asynchronous curl support)
2014-07-03 22:40:12 +02:00
* TASKS=1 (compiles in task support)
* PGSQL=1 (compiles in pgsql support)
* DEBUG=1 (enables use of -d for debug)
* NOHTTP=1 (compiles Kore without HTTP support)
2016-02-01 22:19:16 +01:00
* NOOPT=1 (disable compiler optimizations)
2016-08-01 09:41:12 +02:00
* JSONRPC=1 (compiles in JSONRPC support)
* PYTHON=1 (compiles in the Python support)
2022-02-18 15:49:56 +01:00
* TLS_BACKEND=none (compiles Kore without any TLS backend)
2014-07-03 22:40:12 +02:00
2017-02-07 23:18:05 +01:00
Note that certain build flavors cannot be mixed together and you will just
be met with compilation errors.
2016-08-01 09:41:12 +02:00
Example applications
2014-07-03 22:40:12 +02:00
-----------------
2016-08-01 09:41:12 +02:00
You can find example applications under **_examples/_**.
2014-07-03 22:40:12 +02:00
2014-08-03 21:44:14 +02:00
The examples contain a README file with instructions on how
to build or use them.
2014-07-03 22:40:12 +02:00
Mailing lists
-------------
**patches@kore.io** - Send patches here, preferably inline.
2018-07-09 14:17:10 +02:00
**users@kore.io** - Questions regarding kore.
2018-07-09 14:17:10 +02:00
If you want to signup to those mailing lists send an empty email to
listname+subscribe@kore.io
Other mailboxes (these are **not** mailing lists):
2018-07-09 14:17:39 +02:00
**security@kore.io** - Mail this email if you think you found a security problem.
2018-07-09 14:17:10 +02:00
**sponsor@kore.io** - If your company would like to sponsor part of Kore development.
2013-06-05 14:10:29 +02:00
More information can be found on https://kore.io/