mirrors/kore
1
0
Fork 1
mirror of https://git.kore.io/kore.git synced 2024-11-15 22:46:21 +01:00
Code Issues Projects Releases Wiki Activity
21d1e5156b
kore/Makefile

268 lines
6.5 KiB
Makefile
Raw Normal View History

first commit
2013-04-17 22:34:27 +02:00
# Kore Makefile
Alter where the version number comes from. Now if we are a git repo we fetch the branch name and commitid to build the version string. If there is no git repo we'll look at the RELEASE file.
2018-06-19 19:05:55 +02:00
CC?=cc
Add DESTDIR support in both kore and kodev Makefiles.
2020-09-08 20:54:34 +02:00
DESTDIR?=
Added ability to set path prefix during make.
2014-12-11 23:58:15 +01:00
PREFIX?=/usr/local
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
OBJDIR?=obj
Hook orbit into the build
2014-08-01 10:46:50 +02:00
KORE=kore
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
KODEV=kodev/kodev
Add tools directory. Includes the kore-serve utility that spins up a Kore instance on the local directory and serves the contents via a filemap on localhost port 8888. Used by myself when hacking on the kore website.
2020-09-09 22:14:29 +02:00
KOREPATH?=$(shell pwd)
Makefile improvements. Add KORE_PYTHON_LIB and KORE_PYTHON_INC which can be set by a caller in case the libraries exist somewhere else. Add KORE_CRYPTO to be able to override the name of the default crypto library Kore would link with.
2018-10-17 11:31:36 +02:00
KORE_CRYPTO?=crypto
Added ability to set path prefix during make.
2014-12-11 23:58:15 +01:00
INSTALL_DIR=$(PREFIX)/bin
Set default MAN_DIR value only if it's not already set.
2019-09-02 10:33:56 +02:00
MAN_DIR?=$(PREFIX)/share/man
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
SHARE_DIR=$(PREFIX)/share/kore
Added ability to set path prefix during make.
2014-12-11 23:58:15 +01:00
INCLUDE_DIR=$(PREFIX)/include/kore
first commit
2013-04-17 22:34:27 +02:00
Add tools directory. Includes the kore-serve utility that spins up a Kore instance on the local directory and serves the contents via a filemap on localhost port 8888. Used by myself when hacking on the kore website.
2020-09-09 22:14:29 +02:00
TOOLS= kore-serve
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
GENERATED=
Generate platform.h into OBJDIR if need be.
2019-10-07 16:23:21 +02:00
PLATFORM=platform.h
Add an install-sources target. This will place the required sources for building single binary builds under $PREFIX/share/kore. The kodev utility will now pickup this KORE_SOURCE path automatically unless another one is given via the conf/build.conf file or the KORE_SOURCE environment path.
2020-09-09 21:09:40 +02:00
VERSION=$(OBJDIR)/version.c
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
PYTHON_CURLOPT=misc/curl/python_curlopt.h
Alter where the version number comes from. Now if we are a git repo we fetch the branch name and commitid to build the version string. If there is no git repo we'll look at the RELEASE file.
2018-06-19 19:05:55 +02:00
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
S_SRC= src/kore.c src/buf.c src/config.c src/connection.c \
Add a full native JSON parser to Kore. Mostly compliant, ignores \uXXXX in strings for now. New API functions: void kore_json_init(struct kore_json *json, const u_int8_t *data, size_t len); - Prepares JSON data for parsing. int kore_json_parse(struct kore_json *json) - Parses the JSON data prepared via kore_json_init. Returns KORE_RESULT_ERROR if parsing failed or KORE_RESULT_OK if it succeeded. struct kore_json_item *kore_json_get(struct kore_json *json, const char *path, int type); - Try to find the object matching a given search patch and type. eg, given a JSON structure of: { "reasons": { "strings": [ "first reason", "second" ] } } one can obtain the second element in the reasons.strings array via: item = kore_json_get(json, "reasons/strings[0]", KORE_JSON_TYPE_STRING); Returns NULL if the item was not found or a type mismatch was hit, otherwise will return the item of that type. The kore_json_item data structure has a data member that contains the relevant bits depending on the type: KORE_JSON_TYPE_ARRAY, KORE_JSON_TYPE_OBJECT: the data.items member is valid. KORE_JSON_TYPE_STRING: the data.string member is valid. KORE_JSON_TYPE_NUMBER: the data.number member is valid. KORE_JSON_TYPE_LITERAL: the data.literal member is valid. void kore_json_cleanup(struct kore_json *json); - Cleanup any resources const char *kore_json_strerror(struct kore_json *json); - Return pointer to human readable error string.
2019-10-20 23:22:11 +02:00
src/domain.c src/filemap.c src/fileref.c src/json.c src/mem.c \
src/msg.c src/module.c src/net.c src/pool.c src/runtime.c src/timer.c \
Add an install-sources target. This will place the required sources for building single binary builds under $PREFIX/share/kore. The kodev utility will now pickup this KORE_SOURCE path automatically unless another one is given via the conf/build.conf file or the KORE_SOURCE environment path.
2020-09-09 21:09:40 +02:00
src/utils.c src/worker.c src/keymgr.c
first commit
2013-04-17 22:34:27 +02:00
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
FEATURES=
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
FEATURES_INC=
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
Build option changes. - Build with -O2 unless NOOPT is set to 1. - Hide -g behind DEBUG instead of always building with it. - Explicitely set the standard used to c99, use pedantic.
2016-02-01 15:33:40 +01:00
CFLAGS+=-Wall -Werror -Wstrict-prototypes -Wmissing-prototypes
first commit
2013-04-17 22:34:27 +02:00
CFLAGS+=-Wmissing-declarations -Wshadow -Wpointer-arith -Wcast-qual
Generate platform.h into OBJDIR if need be.
2019-10-07 16:23:21 +02:00
CFLAGS+=-Wsign-compare -Iinclude/kore -I$(OBJDIR) -std=c99 -pedantic
Enable type-limits warning. Remove unneeded comparison in the JSON code. via https://marc.info/?l=openbsd-ports&m=159958572325174&w=2
2020-09-08 19:29:15 +02:00
CFLAGS+=-Wtype-limits
Rework HTTP and worker processes. The HTTP layer used to make a copy of each incoming header and its value for a request. Stop doing that and make HTTP headers zero-copy all across the board. This change comes with some api function changes, notably the http_request_header() function which now takes a const char ** rather than a char ** out pointer. This commit also constifies several members of http_request, beware. Additional rework how the worker processes deal with the accept lock. Before: if a worker held the accept lock and it accepted a new connection it would release the lock for others and back off for 500ms before attempting to grab the lock again. This approach worked but under high load this starts becoming obvious. Now: - workers not holding the accept lock and not having any connections will wait less long before returning from kore_platform_event_wait(). - workers not holding the accept lock will no longer blindly wait an arbitrary amount in kore_platform_event_wait() but will look at how long until the next lock grab is and base their timeout on that. - if a worker its next_lock timeout is up and failed to grab the lock it will try again in half the time again. - the worker process holding the lock will when releasing the lock double check if it still has space for newer connections, if it does it will keep the lock until it is full. This prevents the lock from bouncing between several non busy worker processes all the time. Additional fixes: - Reduce the number of times we check the timeout list, only do it twice per second rather then every event tick. - Fix solo worker count for TLS (we actually hold two processes, not one). - Make sure we don't accidentally miscalculate the idle time causing new connections under heavy load to instantly drop. - Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
2018-02-14 13:48:49 +01:00
CFLAGS+=-DPREFIX='"$(PREFIX)"' -fstack-protector-all
Add OPENSSL_PATH environment variable.
2018-11-28 13:54:38 +01:00
ifneq ("$(OPENSSL_PATH)", "")
CFLAGS+=-I$(OPENSSL_PATH)/include
its about time LDFLAGS are appended instead of set.
2019-11-14 23:48:27 +01:00
LDFLAGS+=-rdynamic -L$(OPENSSL_PATH)/lib -lssl -l$(KORE_CRYPTO)
Add OPENSSL_PATH environment variable.
2018-11-28 13:54:38 +01:00
else
its about time LDFLAGS are appended instead of set.
2019-11-14 23:48:27 +01:00
LDFLAGS+=-rdynamic -lssl -l$(KORE_CRYPTO)
Add OPENSSL_PATH environment variable.
2018-11-28 13:54:38 +01:00
endif
Add BSD kqueue(2) support. Compile with make bsd (or make linux for linux)
2013-06-17 23:39:17 +02:00
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
ifneq ("$(KORE_SINGLE_BINARY)", "")
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
CFLAGS+=-DKORE_SINGLE_BINARY
FEATURES+=-DKORE_SINGLE_BINARY
endif
fix brocken DEBUG define
2013-11-21 12:00:07 +01:00
ifneq ("$(DEBUG)", "")
Build option changes. - Build with -O2 unless NOOPT is set to 1. - Hide -g behind DEBUG instead of always building with it. - Explicitely set the standard used to c99, use pedantic.
2016-02-01 15:33:40 +01:00
CFLAGS+=-DKORE_DEBUG -g
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
FEATURES+=-DKORE_DEBUG
Build option changes. - Build with -O2 unless NOOPT is set to 1. - Hide -g behind DEBUG instead of always building with it. - Explicitely set the standard used to c99, use pedantic.
2016-02-01 15:33:40 +01:00
endif
ifneq ("$(NOOPT)", "")
CFLAGS+=-O0
else
move optimzations back to -O2, not -O3.
2018-07-09 09:45:10 +02:00
CFLAGS+=-O2
If DEBUG is set in the environment, define KORE_DEBUG
2013-11-16 13:30:47 +01:00
endif
Add filemaps. A filemap is a way of telling Kore to serve files from a directory much like a traditional webserver can do. Kore filemaps only handles files. Kore does not generate directory indexes or deal with non-regular files. The way files are sent to a client differs a bit per platform and build options: default: - mmap() backed file transfer due to TLS. NOTLS=1 - sendfile() under FreeBSD, macOS and Linux. - mmap() backed file for OpenBSD. The opened file descriptors/mmap'd regions are cached and reused when appropriate. If a file is no longer in use it will be closed and evicted from the cache after 30 seconds. New API's are available allowing developers to use these facilities via: void net_send_fileref(struct connection *, struct kore_fileref *); void http_response_fileref(struct http_request *, struct kore_fileref *); Kore will attempt to match media types based on file extensions. A few default types are built-in. Others can be added via the new "http_media_type" configuration directive.
2018-06-28 13:27:44 +02:00
ifneq ("$(NOSENDFILE)", "")
CFLAGS+=-DKORE_NO_SENDFILE
endif
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
ifneq ("$(NOHTTP)", "")
CFLAGS+=-DKORE_NO_HTTP
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
FEATURES+=-DKORE_NO_HTTP
Introduce NOHTTP=1 build option. This basically turns off the HTTP layer for Kore. It does not compile in anything for HTTP. This allows Kore to be used as a network application platform as well. Added an example for this called nohttp. Other changes that sneaked in while hacking on this: * Use calloc(), kill pendantic malloc option. * Killed off SPDY/3.1 support completely, will be superseded by http2 Note that comes with massive changes to a lot of the core API functions provided by Kore, these might break your application.
2015-11-27 16:22:50 +01:00
else
S_SRC+= src/auth.c src/accesslog.c src/http.c \
Kill off zlib_dict.c
2015-12-04 14:11:37 +01:00
src/validator.c src/websocket.c
Add KORE_PENDANTIC_MALLOC option. This option tells Kore to zero out memory when allocated, freed or when get/put from the pools.
2014-04-22 12:46:23 +02:00
endif
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
ifneq ("$(PGSQL)", "")
Move actual code out of contrib into src/.
2014-07-02 12:19:38 +02:00
S_SRC+=src/pgsql.c
Support postgresl under the bsds
2014-04-02 00:06:24 +02:00
LDFLAGS+=-L$(shell pg_config --libdir) -lpq
When compiling with PGSQL, note the include dir for "kore build"
2014-08-03 17:33:40 +02:00
CFLAGS+=-I$(shell pg_config --includedir) -DKORE_USE_PGSQL \
-DPGSQL_INCLUDE_PATH="\"$(shell pg_config --includedir)\""
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
FEATURES+=-DKORE_USE_PGSQL
FEATURES_INC+=-I$(shell pg_config --includedir)
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
endif
Add initial stab at asynchronous background tasks. More to follow.
2014-06-28 16:17:18 +02:00
ifneq ("$(TASKS)", "")
Move actual code out of contrib into src/.
2014-07-02 12:19:38 +02:00
S_SRC+=src/tasks.c
Add initial stab at asynchronous background tasks. More to follow.
2014-06-28 16:17:18 +02:00
LDFLAGS+=-lpthread
CFLAGS+=-DKORE_USE_TASKS
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
FEATURES+=-DKORE_USE_TASKS
Add initial stab at asynchronous background tasks. More to follow.
2014-06-28 16:17:18 +02:00
endif
JSONRPC Changed Makefile option location
2016-07-12 13:33:21 +02:00
ifneq ("$(JSONRPC)", "")
S_SRC+=src/jsonrpc.c
LDFLAGS+=-lyajl
CFLAGS+=-DKORE_USE_JSONRPC
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
FEATURES+=-DKORE_USE_JSONRPC
JSONRPC Changed Makefile option location
2016-07-12 13:33:21 +02:00
endif
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
ifneq ("$(PYTHON)", "")
S_SRC+=src/python.c
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
GENERATED+=$(PYTHON_CURLOPT)
Allow building with python3.8
2019-10-15 10:16:53 +02:00
KORE_PYTHON_LIB?=$(shell ./misc/python3-config.sh --ldflags)
KORE_PYTHON_INC?=$(shell ./misc/python3-config.sh --includes)
Makefile improvements. Add KORE_PYTHON_LIB and KORE_PYTHON_INC which can be set by a caller in case the libraries exist somewhere else. Add KORE_CRYPTO to be able to override the name of the default crypto library Kore would link with.
2018-10-17 11:31:36 +02:00
LDFLAGS+=$(KORE_PYTHON_LIB)
CFLAGS+=$(KORE_PYTHON_INC) -DKORE_USE_PYTHON
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
FEATURES+=-DKORE_USE_PYTHON
Makefile improvements. Add KORE_PYTHON_LIB and KORE_PYTHON_INC which can be set by a caller in case the libraries exist somewhere else. Add KORE_CRYPTO to be able to override the name of the default crypto library Kore would link with.
2018-10-17 11:31:36 +02:00
FEATURES_INC+=$(KORE_PYTHON_INC)
Add initial python support. Based on work done by Stanislav Yudin.
2017-01-12 23:38:51 +01:00
endif
avoid using pkg-config on FreeBSD for CURL=1.
2019-05-14 20:53:27 +02:00
OSNAME=$(shell uname -s | sed -e 's/[-_].*//g' | tr A-Z a-z)
ifeq ("$(OSNAME)", "freebsd")
KORE_CURL_LIB=-L/usr/local/lib -lcurl
KORE_CURL_INC=-I/usr/local/include
endif
Add acmev2 (RFC8555) support to Kore. A new acme process is created that communicates with the acme servers. This process does not hold any of your private keys (no account keys, no domain keys etc). Whenever the acme process requires a signed payload it will ask the keymgr process to do the signing with the relevant keys. This process is also sandboxed with pledge+unveil on OpenBSD and seccomp syscall filtering on Linux. The implementation only supports the tls-alpn-01 challenge. This means that you do not need to open additional ports on your machine. http-01 and dns-01 are currently not supported (no wildcard support). A new configuration option "acme_provider" is available and can be set to the acme server its directory. By default this will point to the live letsencrypt environment: https://acme-v02.api.letsencrypt.org/directory The acme process can be controlled via the following config options: - acme_root (where the acme process will chroot/chdir into). - acme_runas (the user the acme process will run as). If none are set, the values from 'root' and 'runas' are taken. If you want to turn on acme for domains you do it as follows: domain kore.io { acme yes } You do not need to specify certkey/certfile anymore, if they are present still they will be overwritten by the acme system. The keymgr will store all certificates and keys under its root (keymgr_root), the account key is stored as "/account-key.pem" and all obtained certificates go under "certificates/<domain>/fullchain.pem" while keys go under "certificates/<domain>/key.pem". Kore will automatically renew certificates if they will expire in 7 days or less.
2019-11-06 19:33:53 +01:00
ifneq ("$(ACME)", "")
S_SRC+=src/acme.c
CURL=1
CFLAGS+=-DKORE_USE_ACME
FEATURES+=-DKORE_USE_ACME
endif
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
ifneq ("$(CURL)", "")
S_SRC+=src/curl.c
use curl-config instead of pkg-config.
2019-10-28 12:18:15 +01:00
KORE_CURL_LIB?=$(shell curl-config --libs)
KORE_CURL_INC?=$(shell curl-config --cflags)
Add asynchronous libcurl support. This commit adds the CURL=1 build option. When enabled allows you to schedule CURL easy handles onto the Kore event loop. It also adds an easy to use HTTP client API that abstracts away the settings required from libcurl to make HTTP requests. Tied together with HTTP request state machines this means you can write fully asynchronous HTTP client requests in an easy way. Additionally this exposes that API to the Python code as well allowing you do to things like: client = kore.httpclient("https://kore.io") status, body = await client.get() Introduces 2 configuration options: - curl_recv_max Max incoming bytes for a response. - curl_timeout Timeout in seconds before a transfer is cancelled. This API also allows you to take the CURL easy handle and send emails with it, run FTP, etc. All asynchronously.
2019-04-24 00:10:47 +02:00
LDFLAGS+=$(KORE_CURL_LIB)
CFLAGS+=$(KORE_CURL_INC) -DKORE_USE_CURL
FEATURES+=-DKORE_USE_CURL
FEATURES_INC+=$(KORE_CURL_INC)
endif
add shorthand for building with fsanitize.
2018-06-12 19:43:10 +02:00
ifneq ("$(SANITIZE)", "")
CFLAGS+=-fsanitize=$(SANITIZE)
LDFLAGS+=-fsanitize=$(SANITIZE)
endif
Detect OS rather then supplying a build target
2013-11-16 12:41:58 +01:00
ifeq ("$(OSNAME)", "darwin")
Allow kore to build with openssl from osx brew.
2014-10-13 11:53:56 +02:00
CFLAGS+=-I/opt/local/include/ -I/usr/local/opt/openssl/include
LDFLAGS+=-L/opt/local/lib -L/usr/local/opt/openssl/lib
Detect OS rather then supplying a build target
2013-11-16 12:41:58 +01:00
S_SRC+=src/bsd.c
else ifeq ("$(OSNAME)", "linux")
undefine _FORTIFY_SOURCE before defining it. Fixes building with compilers/distributions that set it by default. From Bryan Baldwin via patches@
2019-04-25 20:09:11 +02:00
CFLAGS+=-D_GNU_SOURCE=1 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
Detect OS rather then supplying a build target
2013-11-16 12:41:58 +01:00
LDFLAGS+=-ldl
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
S_SRC+=src/linux.c src/seccomp.c
Detect OS rather then supplying a build target
2013-11-16 12:41:58 +01:00
else
S_SRC+=src/bsd.c
Add /usr/local/[include|lib] for BSD / JSONRPC.
2016-07-15 22:33:58 +02:00
ifneq ("$(JSONRPC)", "")
Typo.
2016-07-15 22:34:21 +02:00
CFLAGS+=-I/usr/local/include
Add /usr/local/[include|lib] for BSD / JSONRPC.
2016-07-15 22:33:58 +02:00
LDFLAGS+=-L/usr/local/lib
endif
Detect OS rather then supplying a build target
2013-11-16 12:41:58 +01:00
endif
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
S_OBJS= $(S_SRC:src/%.c=$(OBJDIR)/%.o)
Add an install-sources target. This will place the required sources for building single binary builds under $PREFIX/share/kore. The kodev utility will now pickup this KORE_SOURCE path automatically unless another one is given via the conf/build.conf file or the KORE_SOURCE environment path.
2020-09-09 21:09:40 +02:00
S_OBJS+=$(OBJDIR)/version.o
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
all: $(PLATFORM) $(GENERATED) $(VERSION) $(KORE) $(KODEV)
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
Generate platform.h into OBJDIR if need be.
2019-10-07 16:23:21 +02:00
$(PLATFORM): $(OBJDIR) force
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
@if [ -f misc/$(OSNAME)-platform.sh ]; then \
Generate platform.h into OBJDIR if need be.
2019-10-07 16:23:21 +02:00
misc/$(OSNAME)-platform.sh > $(OBJDIR)/$(PLATFORM) ; \
Add seccomp syscall filtering to kore. With this commit all Kore processes (minus the parent) are running under seccomp. The worker processes get the bare minimum allowed syscalls while each module like curl, pgsql, etc will add their own filters to allow what they require. New API functions: int kore_seccomp_filter(const char *name, void *filter, size_t len); Adds a filter into the seccomp system (must be called before seccomp is enabled). New helpful macro: define KORE_SYSCALL_ALLOW(name) Allow the syscall with a given name, should be used in a sock_filter data structure. New hooks: void kore_seccomp_hook(void); Called before seccomp is enabled, allows developers to add their own BPF filters into seccomp.
2019-09-25 14:25:49 +02:00
fi
Alter where the version number comes from. Now if we are a git repo we fetch the branch name and commitid to build the version string. If there is no git repo we'll look at the RELEASE file.
2018-06-19 19:05:55 +02:00
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
$(PYTHON_CURLOPT): $(OBJDIR) force
@cp $(PYTHON_CURLOPT) $(OBJDIR)
Alter where the version number comes from. Now if we are a git repo we fetch the branch name and commitid to build the version string. If there is no git repo we'll look at the RELEASE file.
2018-06-19 19:05:55 +02:00
$(VERSION): force
@if [ -d .git ]; then \
GIT_REVISION=`git rev-parse --short=8 HEAD`; \
GIT_BRANCH=`git rev-parse --abbrev-ref HEAD`; \
rm -f $(VERSION); \
printf "const char *kore_version = \"%s-%s\";\n" \
$$GIT_BRANCH $$GIT_REVISION > $(VERSION); \
elif [ -f RELEASE ]; then \
printf "const char *kore_version = \"%s\";\n" \
`cat RELEASE` > $(VERSION); \
else \
echo "No version information found (no .git or RELEASE)"; \
exit 1; \
fi
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
$(KODEV):
$(MAKE) -C kodev
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
$(KORE): $(OBJDIR) $(S_OBJS)
Correct order for linking
2014-08-02 13:01:58 +02:00
$(CC) $(S_OBJS) $(LDFLAGS) -o $(KORE)
make sure FEATURES_INC hits kore.features.
2017-08-30 12:03:58 +02:00
@echo $(FEATURES) $(FEATURES_INC) > kore.features
first commit
2013-04-17 22:34:27 +02:00
Expose our async libcurl support to the Python api. Kore already exposed parts of this via the kore.httpclient() method but this commit takes it a bit further and exposes the libcurl interface completely (including the setopt options). tldr: handle = kore.curl("ftp://ftp.eu.openbsd.org/pub/OpenBSD/README") handle.setopt(kore.CURLOPT_TIMEOUT, 5) data = await handle.run() print("%s" % data.decode())
2020-01-18 19:43:38 +01:00
objects: $(OBJDIR) $(PLATFORM) $(GENERATED) $(S_OBJS)
Improve single binary builds. Allow kore build to pickup the required libraries for kore when building single binaries so that you no longer have to manually add them to ldflags.
2017-02-09 11:30:44 +01:00
@echo $(LDFLAGS) > $(OBJDIR)/ldflags
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
@echo "$(FEATURES) $(FEATURES_INC)" > $(OBJDIR)/features
Don't link object files if not needed
2015-07-25 19:10:48 +02:00
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
$(OBJDIR):
@mkdir -p $(OBJDIR)
Add filemaps. A filemap is a way of telling Kore to serve files from a directory much like a traditional webserver can do. Kore filemaps only handles files. Kore does not generate directory indexes or deal with non-regular files. The way files are sent to a client differs a bit per platform and build options: default: - mmap() backed file transfer due to TLS. NOTLS=1 - sendfile() under FreeBSD, macOS and Linux. - mmap() backed file for OpenBSD. The opened file descriptors/mmap'd regions are cached and reused when appropriate. If a file is no longer in use it will be closed and evicted from the cache after 30 seconds. New API's are available allowing developers to use these facilities via: void net_send_fileref(struct connection *, struct kore_fileref *); void http_response_fileref(struct http_request *, struct kore_fileref *); Kore will attempt to match media types based on file extensions. A few default types are built-in. Others can be added via the new "http_media_type" configuration directive.
2018-06-28 13:27:44 +02:00
install:
Add DESTDIR support in both kore and kodev Makefiles.
2020-09-08 20:54:34 +02:00
mkdir -p $(DESTDIR)$(SHARE_DIR)
mkdir -p $(DESTDIR)$(INCLUDE_DIR)
mkdir -p $(DESTDIR)$(INSTALL_DIR)
mkdir -p $(DESTDIR)$(MAN_DIR)/man1
install -m 644 share/man/kodev.1 $(DESTDIR)$(MAN_DIR)/man1/kodev.1
install -m 555 $(KORE) $(DESTDIR)$(INSTALL_DIR)/$(KORE)
install -m 644 kore.features $(DESTDIR)$(SHARE_DIR)/features
install -m 644 include/kore/*.h $(DESTDIR)$(INCLUDE_DIR)
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
$(MAKE) -C kodev install
Add an install-sources target. This will place the required sources for building single binary builds under $PREFIX/share/kore. The kodev utility will now pickup this KORE_SOURCE path automatically unless another one is given via the conf/build.conf file or the KORE_SOURCE environment path.
2020-09-09 21:09:40 +02:00
$(MAKE) install-sources
install-sources:
@mkdir -p $(DESTDIR)$(SHARE_DIR)
@cp Makefile $(DESTDIR)$(SHARE_DIR)
@cp -R src $(DESTDIR)$(SHARE_DIR)
@cp -R include $(DESTDIR)$(SHARE_DIR)
@cp -R misc $(DESTDIR)$(SHARE_DIR)
@if [ -d .git ]; then \
GIT_REVISION=`git rev-parse --short=8 HEAD`; \
GIT_BRANCH=`git rev-parse --abbrev-ref HEAD`; \
rm -f $(VERSION); \
echo "$$GIT_BRANCH-$$GIT_REVISION" > \
$(DESTDIR)$(SHARE_DIR)/RELEASE; \
elif [ -f RELEASE ]; then \
cp RELEASE $(DESTDIR)$(SHARE_DIR); \
else \
echo "No version information found (no .git or RELEASE)"; \
exit 1; \
fi
Add install/uninstall targets.
2014-07-03 21:38:16 +02:00
uninstall:
Hook orbit into the build
2014-08-01 10:46:50 +02:00
rm -f $(INSTALL_DIR)/$(KORE)
Install kore headers under /usr/local/include/kore
2014-07-03 22:39:16 +02:00
rm -rf $(INCLUDE_DIR)
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
rm -rf $(SHARE_DIR)
$(MAKE) -C kodev uninstall
Add install/uninstall targets.
2014-07-03 21:38:16 +02:00
Add tools directory. Includes the kore-serve utility that spins up a Kore instance on the local directory and serves the contents via a filemap on localhost port 8888. Used by myself when hacking on the kore website.
2020-09-09 22:14:29 +02:00
tools-build: $(KODEV)
for t in $(TOOLS); do \
pushd tools/$$t; \
env KODEV_OUTPUT=$(KOREPATH) $(KOREPATH)/$(KODEV) build; \
popd; \
done
tools-clean: $(KODEV)
for t in $(TOOLS); do \
pushd tools/$$t; \
$(KOREPATH)/$(KODEV) clean; \
popd; \
done
tools-install:
mkdir -p $(DESTDIR)$(INSTALL_DIR)
for t in $(TOOLS); do \
install -m 555 $$t $(DESTDIR)$(INSTALL_DIR)/$$t; \
done
Allow "kore build" to produce single binaries. Producing single binaries can now be done with building with "kore build". To get started edit your build.conf and add the following directives: single_binary = yes kore_source = /path/to/kore optionally you can add kore_flavor to instruct how kore should be built: kore_flavor = NOTLS=1 When doing this your build.conf must also include the correct linking options as the linking is now done fully by kore build. The binary produced will include your configuration and takes over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
$(OBJDIR)/%.o: src/%.c
first commit
2013-04-17 22:34:27 +02:00
$(CC) $(CFLAGS) -c $< -o $@
Unbreak build with multiple make jobs Without it python_curlopt.h might not be available at the right time when using something like make -j4: src/python.c:50:10: fatal error: 'python_curlopt.h' file not found ^~~~~~~~~~~~~~~~~~ 1 error generated. Signed-off-by: Tobias Kortkamp <t@tobik.me>
2020-09-03 11:32:55 +02:00
src/python.c: $(PYTHON_CURLOPT)
first commit
2013-04-17 22:34:27 +02:00
clean:
remove lingering character
2018-06-22 23:02:57 +02:00
rm -f $(VERSION)
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
find . -type f -name \*.o -exec rm {} \;
Split up kore cli tools into new binary. Having the create, build, run tools baked into the kore binary made things harder then they had to be for multiple projects with each different build flavors. So move away this functionality into a new "kodev" (name may change) binary that is installed next to kore. The new build tools will automatically pick up the correct flavors the kore binary it points to is installed with. Or for single builds what flavors where enabled. The new tool also will honor looking into PREFIX for the kore binary when doing a `kodev run`. Additionally add a new command "info" that shows some basic info about your project and how it will be built. For example it will show you the flavors of the kore binary installed on the system or the flavors you configured for a single binary build. Obligitory, hacking on a plane comment.
2017-02-19 06:52:29 +01:00
rm -rf $(KORE) $(OBJDIR) kore.features
$(MAKE) -C kodev clean
add releng-build-examples target for myself
2018-11-29 21:13:34 +01:00
releng-build-examples:
rm -rf /tmp/kore_releng
$(MAKE) clean
$(MAKE) PYTHON=1 PGSQL=1 TASKS=1 PREFIX=/tmp/kore_releng
$(MAKE) install PREFIX=/tmp/kore_releng
$(MAKE) -C examples
Detect OS rather then supplying a build target
2013-11-16 12:41:58 +01:00
Alter where the version number comes from. Now if we are a git repo we fetch the branch name and commitid to build the version string. If there is no git repo we'll look at the RELEASE file.
2018-06-19 19:05:55 +02:00
.PHONY: all clean force
Reference in New Issue Copy Permalink