kore/README.md

123 lines
3.3 KiB
Markdown
Raw Normal View History

2014-03-02 18:16:32 +01:00
About
-----
2015-05-13 09:59:02 +02:00
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
Kore (https://kore.io) is an easy to use web application platform for
2014-11-07 09:14:53 +01:00
writing scalable web APIs in C. Its main goals are security, scalability
and allowing rapid development and deployment of such APIs.
2013-05-01 21:36:00 +02:00
2014-11-07 09:14:53 +01:00
Because of this Kore is an ideal candidate for building robust, scalable and secure web things.
2013-05-01 21:36:00 +02:00
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
Key Features
------------
2014-03-02 18:16:32 +01:00
* Supports SNI
* Supports HTTP/1.1
2014-11-24 11:11:01 +01:00
* Websocket support
2016-08-01 09:41:12 +02:00
* Privseps by default
2017-06-28 10:20:14 +02:00
* TLS enabled by default
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
* Optional background tasks
2014-03-02 18:16:32 +01:00
* Built-in parameter validation
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
* Optional asynchronous PostgreSQL support
2018-02-05 16:21:28 +01:00
* Optional support for page handlers in Python
2018-07-17 15:16:27 +02:00
* Reload private keys and certificates on-the-fly
2019-11-07 12:25:14 +01:00
* Automatic X509 certificates via ACME (with privsep)
2016-08-01 09:41:12 +02:00
* Private keys isolated in separate process (RSA and ECDSA)
2014-11-07 17:19:41 +01:00
* Default sane TLS ciphersuites (PFS in all major browsers)
2014-03-02 18:16:32 +01:00
* Modules can be reloaded on-the-fly, even while serving content
2019-09-27 23:53:15 +02:00
* Worker processes sandboxed on OpenBSD (pledge) and Linux (seccomp)
2017-02-06 23:38:21 +01:00
* Event driven (epoll/kqueue) architecture with per CPU worker processes
2016-08-01 09:41:12 +02:00
* Build your web application as a precompiled dynamic library or single binary
2013-06-05 14:10:29 +02:00
2019-11-13 15:57:07 +01:00
And lots more.
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
2013-06-24 12:05:22 +02:00
License
2014-03-02 18:16:32 +01:00
-------
* Kore is licensed under the ISC license
2013-06-24 12:05:22 +02:00
2016-08-01 15:03:22 +02:00
Documentation
--------------
[Read the documentation](https://docs.kore.io/4.1.0/)
2016-08-01 15:03:22 +02:00
2018-11-22 13:05:02 +01:00
Performance
-----------
Read the [benchmarks](https://blog.kore.io/posts/benchmarks) blog post.
2013-07-28 19:25:46 +02:00
Platforms supported
2014-03-02 18:16:32 +01:00
-------------------
* Linux
* OpenBSD
* FreeBSD
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
* MacOS
2014-03-02 18:16:32 +01:00
Kore only supports x64, arm and aarch64 architectures.
2014-07-03 22:40:12 +02:00
Building Kore
-------------
2021-01-25 23:33:05 +01:00
Clone this repository or get the latest release at [https://kore.io/releases/4.1.0](https://kore.io/releases/4.1.0).
2014-07-03 22:40:12 +02:00
Requirements
* openssl 1.1.1 or libressl 3.x
2021-12-06 21:27:11 +01:00
(note: openssl 3.0.0 is currently *not* supported)
2014-07-03 22:40:12 +02:00
2019-05-01 22:40:27 +02:00
Requirement for asynchronous curl (optional)
* libcurl (7.64.0 or higher)
2019-05-01 22:40:27 +02:00
Requirements for background tasks (optional)
2014-07-03 22:40:12 +02:00
* pthreads
Requirements for pgsql (optional)
2014-07-03 22:40:12 +02:00
* libpq
Requirements for python (optional)
2017-03-30 09:40:13 +02:00
* Python 3.6+
2014-07-03 22:40:12 +02:00
Normal compilation and installation:
```
2017-03-06 14:28:06 +01:00
$ cd kore
$ make
2014-07-03 22:40:12 +02:00
# make install
```
If you would like to build a specific flavor, you can enable
those by setting a shell environment variable before running **_make_**.
2019-11-07 12:25:14 +01:00
* ACME=1 (compiles in ACME support)
2019-05-01 22:40:27 +02:00
* CURL=1 (compiles in asynchronous curl support)
2014-07-03 22:40:12 +02:00
* TASKS=1 (compiles in task support)
* PGSQL=1 (compiles in pgsql support)
* DEBUG=1 (enables use of -d for debug)
* NOHTTP=1 (compiles Kore without HTTP support)
2016-02-01 22:19:16 +01:00
* NOOPT=1 (disable compiler optimizations)
2016-08-01 09:41:12 +02:00
* JSONRPC=1 (compiles in JSONRPC support)
* PYTHON=1 (compiles in the Python support)
2014-07-03 22:40:12 +02:00
2017-02-07 23:18:05 +01:00
Note that certain build flavors cannot be mixed together and you will just
be met with compilation errors.
2016-08-01 09:41:12 +02:00
Example applications
2014-07-03 22:40:12 +02:00
-----------------
2016-08-01 09:41:12 +02:00
You can find example applications under **_examples/_**.
2014-07-03 22:40:12 +02:00
2014-08-03 21:44:14 +02:00
The examples contain a README file with instructions on how
to build or use them.
2014-07-03 22:40:12 +02:00
Mailing lists
-------------
**patches@kore.io** - Send patches here, preferably inline.
2018-07-09 14:17:10 +02:00
**users@kore.io** - Questions regarding kore.
2018-07-09 14:17:10 +02:00
If you want to signup to those mailing lists send an empty email to
listname+subscribe@kore.io
Other mailboxes (these are **not** mailing lists):
2018-07-09 14:17:39 +02:00
**security@kore.io** - Mail this email if you think you found a security problem.
2018-07-09 14:17:10 +02:00
**sponsor@kore.io** - If your company would like to sponsor part of Kore development.
2013-06-05 14:10:29 +02:00
More information can be found on https://kore.io/