1
0
mirror of https://git.kore.io/kore.git synced 2024-11-11 04:39:00 +01:00
Commit Graph

213 Commits

Author SHA1 Message Date
Joris Vink
bb2d4903f2 remove NETBUF_RETAIN, no longer needed 2013-07-10 13:39:35 +02:00
Joris Vink
fa78d24948 remove WILLCLOSE, no longer needed 2013-07-09 15:23:33 +02:00
Joris Vink
a17f93e896 Include sys/types.h and unistd.h for close(), getopt(), getpid(),
getuid(), setproctitle(), unlink() and declaration of optarg.

Makes Kore compatible with FreeBSD too.

From Ayan George <ayan@ayan.net>
2013-07-09 09:45:16 +02:00
Joris Vink
95bacb5690 Kore will now keep track of page handlers that cause workers to die.
This is useful to track down any issues you might have in your module.

A log entry with a page handler causing issues looks like:
Jul  7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11
Jul  7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone
Jul  7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
Joris Vink
c1723f2db5 Clean up header includes, based on a diff from Ewan Higgs via github.
And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
Joris Vink
0c08b57d3e - add idle timer for normal connections (ie: !SPDY), max 20 seconds.
- use idle timer to make sure we don't block SPDY connections indefinately
  when our window size has reached 0.
2013-07-01 12:08:51 +02:00
Joris Vink
51a9e4db9d Implement SPDY WINDOW_UPDATE and SETTINGS. 2013-07-01 11:30:18 +02:00
Joris Vink
4b2420097b - Better spread load between all worker processes.
- Introduce own memory management system on top of malloc to keep track
  of all our allocations and free's. Later we should introduce a pooling
  mechanism for fixed size allocations (http_request comes to mind).
- Introduce ssl_cipher in configuration.

Memory usage is kind of high right now, but it seems its OpenSSL
doing it rather then Kore.
2013-06-27 08:46:18 +02:00
Joris Vink
b4a0330a96 - Better spread load between all worker processes.
- Introduce own memory management system on top of malloc to keep track
  of all our allocations and free's. Later we should introduce a pooling
  mechanism for fixed size allocations (http_request comes to mind).
- Introduce ssl_cipher in configuration.

Memory usage is kind of high right now, but it seems its OpenSSL
doing it rather then Kore.
2013-06-27 08:43:07 +02:00
Joris Vink
2fc5233358 Rework the way worker processes give each other the accept lock.
Instead of waiting until one worker is filled up on connections
the workers find the next lowest loaded worker and will hand
over the lock to them instead. This will cause a nicer spread of load.

Instead of running one accept per event loop, we attempt to accept
as many as worker_max_connections allows.

Refactor net sending/recv code a bit.
2013-06-27 00:22:48 +02:00
Joris Vink
bf1940225a everybody loves to tout their own horn.. so introduce a server response header 2013-06-26 16:58:01 +02:00
Joris Vink
0dda6f996f Add a form of synchronization between what worker will be accepting
new connections and which ones will not be notified for it.

Fixes the thundering herd problem, and nicely spreads out load between
all the workers equally. A configuration option (workers_max_connections)
is available to tweak how many connections a worker will have before
giving up the accept lock.

Two ways are added to this commit for access locking:
- Locking via semaphores.
- Locking via GCC's builtin atomic methods.

The default is running with semaphores disabled (OpenBSD cannot do
sem_init() with pshared set to 1, which is required).

If you want to use semaphores add KORE_USE_SEMAPHORES to CFLAGS,
and -lpthread to LDFLAGS in the Makefile.

Other fixes:
- BSD: add a timeout to kevent().
- Merge kore_worker_wait together, linux knows waitpid() as well.
- Send the correct SIGQUIT signal to workers instead of SIGINT.
- Fix kore_time_ms().
- Log fatal worker messages in syslog.
- Refactor code even more.
- Do not free our own kore_worker structure.
2013-06-26 16:37:22 +02:00
Joris Vink
3e5c17b8a3 refactor code quite a bit. 2013-06-26 11:18:32 +02:00
Joris Vink
6026a6d4ee add SNI support, and change domain configuration a bit. 2013-06-24 11:32:45 +02:00
Joris Vink
a1b400c400 Add access logging to Kore. 2013-06-24 09:36:40 +02:00
Joris Vink
25f1ab9865 Add BSD kqueue(2) support. Compile with make bsd (or make linux for linux) 2013-06-17 23:39:17 +02:00
Joris Vink
b4deea82b4 add kore_buf_appendb() which allows us to append a kore_buf to another one.
(releases the kore_buf that is being appended while at it).
2013-06-05 11:27:03 +02:00
Joris Vink
a74fffe40c Introduce certfile and certkey in the configuration to specify where the certificate file and keys are located on a system.
Free unused vars in the main process after starting.
2013-06-05 09:47:08 +02:00
Joris Vink
b9f47b0f86 configuration files are now passed using the -c option.
allow debug output when the -d flag is specified.
2013-06-05 08:55:07 +02:00
Joris Vink
90e1b3a7da use syslog() for informative messages from all parts of kore. 2013-06-04 23:24:47 +02:00
Joris Vink
9ef669ff6f write main process pid to /var/run/kore.pid (changable in configuration) 2013-06-04 16:53:30 +02:00
Joris Vink
e7db5ee6b1 rename kore_log to kore_debug, and allow one to turn it off. 2013-06-04 16:30:53 +02:00
Joris Vink
ab0dc25c61 use sched_setaffinity() to set what CPU each worker process should run on. 2013-06-04 13:54:16 +02:00
Joris Vink
8f8ab92521 upon quit time, workers will not accept new connections but will
handle all outstanding http requests.
2013-06-04 13:43:11 +02:00
Joris Vink
0de28488a6 move from multithreads to single threaded worker processes. 2013-06-04 11:55:38 +02:00
Joris Vink
b65cc93426 allow handlers to return KORE_RESULT_RETRY. This will tell the worker to reschedule the page request again at the end of its list. (Allows module creators to write truely nonblocking modules). 2013-05-31 00:40:06 +02:00
Joris Vink
cf6a6550f0 allow onload to be given in the config file.
onload specifies what function in your module to call when the module has been loaded or reloaded.
2013-05-30 21:26:39 +02:00
Joris Vink
9ad263e287 do not remove disconnected connections until we actually are ready to disconnect them. 2013-05-30 20:07:06 +02:00
Joris Vink
9243f409cc move to a worker based threading approach where we delegate http requests to workers in a round robin basis (later this should be swapped to find the laziest worker and assign the request to that instead). 2013-05-30 19:36:42 +02:00
Joris Vink
8478d8df54 add chroot and runas directives so we can chroot and drop privilegs properly 2013-05-04 22:18:27 +02:00
Joris Vink
45adae62f7 q->value should be set to NULL if a query parameter is present but no value was set for it.
introduce kore_buf_appendv() (much like readv()).
introduce kore_buf_appendf() (printf into buffers).
2013-05-04 19:09:07 +02:00
Joris Vink
088d877715 switch dynamic handlers to regex based patterns, much easier. 2013-05-29 14:29:46 +02:00
Joris Vink
a88b8de6a2 add proper query argument parsing, modules must call
http_populate_arguments() before using http_argument_lookup() to lookup a given argument.

population uses the appropriate query string (from GET or POST) automatically.
2013-05-03 09:09:27 +02:00
Joris Vink
b3cdccbc87 add http_post_data_text() which returns the post data NUL-terminated to the caller. 2013-05-03 07:49:40 +02:00
Joris Vink
2401bd1a9f support for POST in both spdy and normal connections 2013-05-03 07:35:11 +02:00
Joris Vink
90588913d4 allow sending SIGHUP to kore, which will then reload its content module 2013-05-03 00:04:06 +02:00
Joris Vink
e7feaaba76 properly close a spdy stream once both sides have FLAG_FIN. 2013-05-02 17:30:06 +02:00
Joris Vink
4ed4f76e44 allow POST to work with spdy, all POST data is stored in the httpreq->post_data field. (as a kore_buf) 2013-05-02 17:14:07 +02:00
Joris Vink
f61bbe8ff4 move buf stuff back into kore.h, wont need it separately 2013-05-02 15:14:00 +02:00
Joris Vink
a025f0fa8e save the http request for spdy later on 2013-05-02 15:06:26 +02:00
Joris Vink
4fb8a13f1b mmmm 2013-05-02 15:00:03 +02:00
Joris Vink
84428f7133 move buf stuff into its own header, so we can use it in spdy.h 2013-05-02 14:55:57 +02:00
Joris Vink
60ed3e0fd2 the net qeueuing functions don't need to return status codes, simplifies a lot of other code that revolves around those.
prepare for POST support
2013-05-02 14:47:02 +02:00
Joris Vink
807764614b from now on configuration files must specify a domain for the handlers that follow. This allows for easy subdomain configuration.
example:

domain joris.local
static / serve_index

domain .joris.local
static / serve_another_index
2013-05-02 13:30:13 +02:00
Joris Vink
f1fa108f98 begin processing window_update messages 2013-05-02 10:10:03 +02:00
Joris Vink
364dc58219 rework disconnecting clients and fix bug where spdy ocnnections could segfault after disconnecting 2013-05-02 09:10:35 +02:00
Joris Vink
f46bba50ef add very basic support for HTTP/1.1. 2013-05-02 03:51:04 +02:00
Joris Vink
5f983d575b add flags to net_recv_queue() and net_send_queue(),.
NETBUF_CALL_CB_ALWAYS flag, calls the registered cb everytime data has
been received or sent.
2013-05-02 00:28:49 +02:00
Joris Vink
a9ebf37cae allow modules to add headers to the response before calling http_response()
by calling http_response_header_add().

fix wrong overflow check in spdy_stream_get_header().

html_inject now exports last modified timestamp for the files that are
compiled into the module in the format static_mtime_<type>_<name>.

modules can now look into the request headers using http_request_header_get().
2013-05-01 21:16:09 +02:00
Joris Vink
a228cdba0e introduce kore_split_string() to properly split strings apart.
introduce kore_date_to_time() to conver http-date formatted strings to time_t.
2013-05-01 20:10:45 +02:00
Joris Vink
4ecfd81e6e allow us to pass content-type to http_response() 2013-05-01 16:51:34 +02:00
Joris Vink
d9bbb0b1f1 initial module support 2013-05-01 16:03:48 +02:00
Joris Vink
a074258dcc rework the way events trigger and the way we handle them. follow the manual for once, as edge trigger does not refire until we got an EAGAIN.
introduce net_send_flush() and net_recv_flush() for this purpose, we attempt to make as much headway as possible until we reach EAGAIN or until we can simply pickup again later.

should merge all the stuff in http_response() into a single send buffer, **out is in place in net_send_queue() for that purpose.
2013-05-01 13:43:47 +02:00
Joris Vink
d59847c448 SPDY works, we can serve content. Now for dynamic modules so we can
get stuff on the road.

TODO: add plain HTTP support if SPDY isnt working in the browser.
2013-05-01 12:23:21 +02:00
Joris Vink
848704f74b lots of new stuff, including processing of http requests and an attempt
to build an initial spdy response (SYN frame + header block content).
2013-05-01 08:09:04 +02:00
Joris Vink
05156c9a3a properly deflate/inflate of name/value blocks. and add ability
to get headers from them.
2013-05-01 00:35:33 +02:00
Joris Vink
bb3b804222 add beginnings of zlib inflation, not really working. nap time. 2013-04-29 23:35:36 +02:00
Joris Vink
c8b422d29d allow us to expand receive buffers automatically so we can keep chaining data into the same netbuf. This gives us the possibility to retain the ctrl_frame by the time we reach the proper cb for the actual frame message. 2013-04-28 23:42:13 +02:00
Joris Vink
23c0ec67c6 begin with the ability to read control frames. something feels fishy with epoll() and its triggering of events. I probably got it wrong. 2013-04-28 19:11:44 +02:00
Joris Vink
c6c6b1823f read next protocol properly and remember it for later. 2013-04-21 20:35:47 +02:00
Joris Vink
68d0523817 Add initial SSL stuff + bare bones NPN advertisement. So far chrome connects and
shows our connection as SPDY/3, good start.
2013-04-21 20:21:46 +02:00
Joris Vink
cba9628f16 oops 2013-04-17 22:34:42 +02:00
Joris Vink
857c3f9121 first commit 2013-04-17 22:34:27 +02:00