kore/README.md

About
-----

Kore (https://kore.io) is an easy to use web application platform for
writing scalable web APIs in C. Its main goals are security, scalability
and allowing rapid development and deployment of such APIs.

Because of this Kore is an ideal candidate for building robust, scalable and secure web things.

Key Features
------------
* Supports SNI
* Supports HTTP/1.1
* Websocket support
* Privseps by default
* TLS enabled by default
* Optional background tasks
* Built-in parameter validation
* Optional asynchronous PostgreSQL support
* Optional support for page handlers in Python
* Reload private keys and certificates on-the-fly
* Automatic X509 certificates via ACME (with privsep)
* Private keys isolated in separate process (RSA and ECDSA)
* Default sane TLS ciphersuites (PFS in all major browsers)
* Modules can be reloaded on-the-fly, even while serving content
* Worker processes sandboxed on OpenBSD (pledge) and Linux (seccomp)
* Event driven (epoll/kqueue) architecture with per CPU worker processes
* Build your web application as a precompiled dynamic library or single binary

And lots more.

License
-------
* Kore is licensed under the ISC license

Documentation
--------------
[Read the documentation](https://docs.kore.io/3.3.0/)

Performance
-----------
Read the [benchmarks](https://blog.kore.io/posts/benchmarks) blog post.

Platforms supported
-------------------
* Linux
* OpenBSD
* FreeBSD
* MacOS

Building Kore
-------------
Clone this repository or get the latest release at [https://kore.io/releases/3.2.0](https://kore.io/releases/3.2.0).

Requirements
* openssl (1.0.2, 1.1.0 or 1.1.1)
  (note: libressl 3.0.0+ works as a replacement)

Requirement for asynchronous curl (optional)
* libcurl (7.64.0 or higher)

Requirements for background tasks (optional)
* pthreads

Requirements for pgsql (optional)
* libpq

Requirements for python (optional)
* Python 3.6+

Normal compilation and installation:

```
$ cd kore
$ make
# make install
```

If you would like to build a specific flavor, you can enable
those by setting a shell environment variable before running **_make_**.

* ACME=1 (compiles in ACME support)
* CURL=1 (compiles in asynchronous curl support)
* TASKS=1 (compiles in task support)
* PGSQL=1 (compiles in pgsql support)
* DEBUG=1 (enables use of -d for debug)
* NOHTTP=1 (compiles Kore without HTTP support)
* NOOPT=1 (disable compiler optimizations)
* JSONRPC=1 (compiles in JSONRPC support)
* PYTHON=1 (compiles in the Python support)

Note that certain build flavors cannot be mixed together and you will just
be met with compilation errors.

Example applications
-----------------
You can find example applications under **_examples/_**.

The examples contain a README file with instructions on how
to build or use them.

Mailing lists
-------------

**patches@kore.io** - Send patches here, preferably inline.

**users@kore.io** - Questions regarding kore.


If you want to signup to those mailing lists send an empty email to
	listname+subscribe@kore.io


Other mailboxes (these are **not** mailing lists):

**security@kore.io** - Mail this email if you think you found a security problem.

**sponsor@kore.io** - If your company would like to sponsor part of Kore development.

More information can be found on https://kore.io/
Update README with some flair 2014-03-02 18:16:32 +01:00			`About`
			`-----`
travis build badge 2015-05-13 09:59:02 +02:00
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char rather than a char out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up. 2018-02-14 13:48:49 +01:00			`Kore (https://kore.io) is an easy to use web application platform for`
Bump readme 2014-11-07 09:14:53 +01:00			`writing scalable web APIs in C. Its main goals are security, scalability`
			`and allowing rapid development and deployment of such APIs.`
add a little readme 2013-05-01 21:36:00 +02:00
Bump readme 2014-11-07 09:14:53 +01:00			`Because of this Kore is an ideal candidate for building robust, scalable and secure web things.`
add a little readme 2013-05-01 21:36:00 +02:00
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char rather than a char out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up. 2018-02-14 13:48:49 +01:00			`Key Features`
			`------------`
Update README with some flair 2014-03-02 18:16:32 +01:00			`* Supports SNI`
			`* Supports HTTP/1.1`
Add websocket mention 2014-11-24 11:11:01 +01:00			`* Websocket support`
update with latest 2016-08-01 09:41:12 +02:00			`* Privseps by default`
slight change in wording 2017-06-28 10:20:14 +02:00			`* TLS enabled by default`
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char rather than a char out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up. 2018-02-14 13:48:49 +01:00			`* Optional background tasks`
Update README with some flair 2014-03-02 18:16:32 +01:00			`* Built-in parameter validation`
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char rather than a char out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up. 2018-02-14 13:48:49 +01:00			`* Optional asynchronous PostgreSQL support`
reword 2018-02-05 16:21:28 +01:00			`* Optional support for page handlers in Python`
toot toot 2018-07-17 15:16:27 +02:00			`* Reload private keys and certificates on-the-fly`
add ACME mention 2019-11-07 12:25:14 +01:00			`* Automatic X509 certificates via ACME (with privsep)`
update with latest 2016-08-01 09:41:12 +02:00			`* Private keys isolated in separate process (RSA and ECDSA)`
bump README 2014-11-07 17:19:41 +01:00			`* Default sane TLS ciphersuites (PFS in all major browsers)`
Update README with some flair 2014-03-02 18:16:32 +01:00			`* Modules can be reloaded on-the-fly, even while serving content`
mention sandboxing 2019-09-27 23:53:15 +02:00			`* Worker processes sandboxed on OpenBSD (pledge) and Linux (seccomp)`
reword 2017-02-06 23:38:21 +01:00			`* Event driven (epoll/kqueue) architecture with per CPU worker processes`
update with latest 2016-08-01 09:41:12 +02:00			`* Build your web application as a precompiled dynamic library or single binary`
update README 2013-06-05 14:10:29 +02:00
wording 2019-11-13 15:57:07 +01:00			`And lots more.`
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char rather than a char out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up. 2018-02-14 13:48:49 +01:00
properly mention what license kore is 2013-06-24 12:05:22 +02:00			`License`
Update README with some flair 2014-03-02 18:16:32 +01:00			`-------`
			`* Kore is licensed under the ISC license`
properly mention what license kore is 2013-06-24 12:05:22 +02:00
Update README 2016-08-01 15:03:22 +02:00			`Documentation`
			`--------------`
fix doc link 2019-09-27 23:55:42 +02:00			`[Read the documentation](https://docs.kore.io/3.3.0/)`
Update README 2016-08-01 15:03:22 +02:00
update 2018-11-22 13:05:02 +01:00			`Performance`
			`-----------`
			`Read the [benchmarks](https://blog.kore.io/posts/benchmarks) blog post.`

s/support/supported 2013-07-28 19:25:46 +02:00			`Platforms supported`
Update README with some flair 2014-03-02 18:16:32 +01:00			`-------------------`
			`* Linux`
			`* OpenBSD`
			`* FreeBSD`
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char rather than a char out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up. 2018-02-14 13:48:49 +01:00			`* MacOS`
Update README with some flair 2014-03-02 18:16:32 +01:00
Update with more instructions 2014-07-03 22:40:12 +02:00			`Building Kore`
			`-------------`
3.2.0 2018-11-30 21:36:00 +01:00			`Clone this repository or get the latest release at [https://kore.io/releases/3.2.0](https://kore.io/releases/3.2.0).`
Update with more instructions 2014-07-03 22:40:12 +02:00
			`Requirements`
specify what openssl versions are expected to work. 2018-10-30 08:45:11 +01:00			`* openssl (1.0.2, 1.1.0 or 1.1.1)`
nope, changed my mind, drop libressl 2.7.5. 2019-10-28 12:47:51 +01:00			`(note: libressl 3.0.0+ works as a replacement)`
Update with more instructions 2014-07-03 22:40:12 +02:00
mention libcurl support in README 2019-05-01 22:40:27 +02:00			`Requirement for asynchronous curl (optional)`
Python: Several fixes for our async curl support. - Fix the curl-extract-opt.sh generation script to work on newer curl releases as the header changed slightly. - Use the correct handles when calling curl_easy_setopt() inside of our setopt functions exported via Python. - Add a curl.setbody() method, allowing a body to be sent to be set. (eg when sending mail via SMTP). - Regen of our python_curlopt.h from 7.71.1 2020-07-02 08:41:17 +02:00			`* libcurl (7.64.0 or higher)`
mention libcurl support in README 2019-05-01 22:40:27 +02:00
I'm such a slacker. Forgot to mention 1.2.2 2015-04-27 10:51:37 +02:00			`Requirements for background tasks (optional)`
Update with more instructions 2014-07-03 22:40:12 +02:00			`* pthreads`

I'm such a slacker. Forgot to mention 1.2.2 2015-04-27 10:51:37 +02:00			`Requirements for pgsql (optional)`
Update with more instructions 2014-07-03 22:40:12 +02:00			`* libpq`

add first mentions of python support to README. 2017-01-25 22:22:05 +01:00			`Requirements for python (optional)`
be more clear about openssl releases. 2017-03-30 09:40:13 +02:00			`* Python 3.6+`
add first mentions of python support to README. 2017-01-25 22:22:05 +01:00
Update with more instructions 2014-07-03 22:40:12 +02:00			`Normal compilation and installation:`

			```
update README 2017-03-06 14:28:06 +01:00			`$ cd kore`
			`$ make`
Update with more instructions 2014-07-03 22:40:12 +02:00			`# make install`
			```

			`If you would like to build a specific flavor, you can enable`
			`those by setting a shell environment variable before running _make_.`

add ACME mention 2019-11-07 12:25:14 +01:00			`* ACME=1 (compiles in ACME support)`
mention libcurl support in README 2019-05-01 22:40:27 +02:00			`* CURL=1 (compiles in asynchronous curl support)`
Update with more instructions 2014-07-03 22:40:12 +02:00			`* TASKS=1 (compiles in task support)`
			`* PGSQL=1 (compiles in pgsql support)`
			`* DEBUG=1 (enables use of -d for debug)`
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application. 2015-11-27 16:22:50 +01:00			`* NOHTTP=1 (compiles Kore without HTTP support)`
Mention NOOPT. 2016-02-01 22:19:16 +01:00			`* NOOPT=1 (disable compiler optimizations)`
update with latest 2016-08-01 09:41:12 +02:00			`* JSONRPC=1 (compiles in JSONRPC support)`
add first mentions of python support to README. 2017-01-25 22:22:05 +01:00			`* PYTHON=1 (compiles in the Python support)`
Update with more instructions 2014-07-03 22:40:12 +02:00
add little note on non mixables. 2017-02-07 23:18:05 +01:00			`Note that certain build flavors cannot be mixed together and you will just`
			`be met with compilation errors.`

update with latest 2016-08-01 09:41:12 +02:00			`Example applications`
Update with more instructions 2014-07-03 22:40:12 +02:00			`-----------------`
update with latest 2016-08-01 09:41:12 +02:00			`You can find example applications under _examples/_.`
Update with more instructions 2014-07-03 22:40:12 +02:00
Add more READMEs 2014-08-03 21:44:14 +02:00			`The examples contain a README file with instructions on how`
			`to build or use them.`
Update with more instructions 2014-07-03 22:40:12 +02:00
better README now that we're moving from GH 2018-07-09 09:10:57 +02:00			`Mailing lists`
			`-------------`

			`patches@kore.io - Send patches here, preferably inline.`
spacing 2018-07-09 14:17:10 +02:00
better README now that we're moving from GH 2018-07-09 09:10:57 +02:00			`users@kore.io - Questions regarding kore.`

spacing 2018-07-09 14:17:10 +02:00
better README now that we're moving from GH 2018-07-09 09:10:57 +02:00			`If you want to signup to those mailing lists send an empty email to`
			`listname+subscribe@kore.io`


people keep subbing to non-mailing-list mails. 2018-11-29 15:52:40 +01:00			`Other mailboxes (these are not mailing lists):`
even better 2018-07-09 14:17:39 +02:00
better README now that we're moving from GH 2018-07-09 09:10:57 +02:00			`security@kore.io - Mail this email if you think you found a security problem.`
spacing 2018-07-09 14:17:10 +02:00
better README now that we're moving from GH 2018-07-09 09:10:57 +02:00			`sponsor@kore.io - If your company would like to sponsor part of Kore development.`
update README 2013-06-05 14:10:29 +02:00
update README for github and split it up into docs/ 2013-06-05 16:41:42 +02:00			`More information can be found on https://kore.io/`