mirrors
/
kore
mirror of https://git.kore.io/kore.git
1
0
Fork 1
Code Issues Projects Releases Wiki Activity
kore/src/pgsql.c

598 lines
13 KiB
C
Raw Normal View History

Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
/*
Bump copyright to 2016.
2016-01-04 12:58:51 +01:00
* Copyright (c) 2014-2016 Joris Vink <joris@coders.se>
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <sys/param.h>
#include <sys/queue.h>
#include <libpq-fe.h>
Pull in pg_config.h correctly, fixes compilation for pgsql under OpenBSD
2014-07-18 15:59:07 +02:00
#include <pg_config.h>
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
#include "kore.h"
#include "http.h"
Normalize kore_*.h headers
2014-07-03 22:14:46 +02:00
#include "pgsql.h"
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
struct pgsql_job {
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
struct http_request *req;
struct kore_pgsql *pgsql;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
TAILQ_ENTRY(pgsql_job) list;
};
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
struct pgsql_wait {
struct http_request *req;
TAILQ_ENTRY(pgsql_wait) list;
};
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
#define PGSQL_CONN_MAX 2
#define PGSQL_CONN_FREE 0x01
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
#define PGSQL_LIST_INSERTED 0x0100
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
static void pgsql_queue_wakeup(void);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
static void pgsql_set_error(struct kore_pgsql *, const char *);
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
static void pgsql_queue_add(struct http_request *);
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
static void pgsql_conn_release(struct kore_pgsql *);
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
static void pgsql_conn_cleanup(struct pgsql_conn *);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
static void pgsql_read_result(struct kore_pgsql *);
static void pgsql_schedule(struct kore_pgsql *);
static struct pgsql_conn *pgsql_conn_create(struct kore_pgsql *,
struct pgsql_db *);
static struct pgsql_conn *pgsql_conn_next(struct kore_pgsql *,
struct pgsql_db *,
struct http_request *);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
static struct kore_pool pgsql_job_pool;
static struct kore_pool pgsql_wait_pool;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
static TAILQ_HEAD(, pgsql_conn) pgsql_conn_free;
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
static TAILQ_HEAD(, pgsql_wait) pgsql_wait_queue;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
static LIST_HEAD(, pgsql_db) pgsql_db_conn_strings;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
static u_int16_t pgsql_conn_count;
Add pgsql_conn_max configuration parameter. Allows you to tune how many pgsql connections kore will make at one time.
2014-07-04 09:14:05 +02:00
u_int16_t pgsql_conn_max = PGSQL_CONN_MAX;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
void
kore_pgsql_init(void)
{
pgsql_conn_count = 0;
TAILQ_INIT(&pgsql_conn_free);
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
TAILQ_INIT(&pgsql_wait_queue);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
LIST_INIT(&pgsql_db_conn_strings);
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
kore_pool_init(&pgsql_job_pool, "pgsql_job_pool",
sizeof(struct pgsql_job), 100);
kore_pool_init(&pgsql_wait_pool, "pgsql_wait_pool",
sizeof(struct pgsql_wait), 100);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
int
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
kore_pgsql_query_init(struct kore_pgsql *pgsql, struct http_request *req,
const char *dbname, int flags)
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
{
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
struct pgsql_db *db;
memset(pgsql, 0, sizeof(*pgsql));
pgsql->flags = flags;
pgsql->state = KORE_PGSQL_STATE_INIT;
if ((req == NULL && (flags & KORE_PGSQL_ASYNC)) ||
((flags & KORE_PGSQL_ASYNC) && (flags & KORE_PGSQL_SYNC))) {
pgsql_set_error(pgsql, "invalid query init parameters");
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
return (KORE_RESULT_ERROR);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
}
db = NULL;
LIST_FOREACH(db, &pgsql_db_conn_strings, rlist) {
if (!strcmp(db->name, dbname))
break;
}
if (db == NULL) {
pgsql_set_error(pgsql, "no database found");
return (KORE_RESULT_ERROR);
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if ((pgsql->conn = pgsql_conn_next(pgsql, db, req)) == NULL)
return (KORE_RESULT_ERROR);
if (pgsql->flags & KORE_PGSQL_ASYNC) {
pgsql->conn->job = kore_pool_get(&pgsql_job_pool);
pgsql->conn->job->req = req;
pgsql->conn->job->pgsql = pgsql;
http_request_sleep(req);
pgsql->flags |= PGSQL_LIST_INSERTED;
LIST_INSERT_HEAD(&(req->pgsqls), pgsql, rlist);
}
return (KORE_RESULT_OK);
}
int
kore_pgsql_query(struct kore_pgsql *pgsql, const char *query)
{
if (pgsql->conn == NULL) {
pgsql_set_error(pgsql, "no connection was set before query");
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
return (KORE_RESULT_ERROR);
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if (pgsql->flags & KORE_PGSQL_SYNC) {
pgsql->result = PQexec(pgsql->conn->db, query);
if ((PQresultStatus(pgsql->result) != PGRES_TUPLES_OK) &&
(PQresultStatus(pgsql->result) != PGRES_COMMAND_OK)) {
pgsql_set_error(pgsql, PQerrorMessage(pgsql->conn->db));
return (KORE_RESULT_ERROR);
}
pgsql->state = KORE_PGSQL_STATE_DONE;
} else {
if (!PQsendQuery(pgsql->conn->db, query)) {
pgsql_set_error(pgsql, PQerrorMessage(pgsql->conn->db));
return (KORE_RESULT_ERROR);
}
pgsql_schedule(pgsql);
}
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
return (KORE_RESULT_OK);
}
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
int
Added new function kore_pgsql_v_query_params(). Same as kore_pgsql_query_params but takes a va_list as last parameter (non-v version takes a variable list of parameters). Lets people write easier to call wrappers around the query calls. I use it in a wrapper that takes next states (error, current, continue) as arguments in a handler with multiple async queries.
2016-06-07 13:12:31 +02:00
kore_pgsql_v_query_params(struct kore_pgsql *pgsql,
const char *query, int result, u_int8_t count, va_list args)
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
{
u_int8_t i;
char **values;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
int *lengths, *formats, ret;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if (pgsql->conn == NULL) {
pgsql_set_error(pgsql, "no connection was set before query");
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
return (KORE_RESULT_ERROR);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Allow 0 parameters for kore_pgsql_query_params()
2014-10-12 01:17:35 +02:00
if (count > 0) {
lengths = kore_calloc(count, sizeof(int));
formats = kore_calloc(count, sizeof(int));
values = kore_calloc(count, sizeof(char *));
for (i = 0; i < count; i++) {
values[i] = va_arg(args, void *);
lengths[i] = va_arg(args, u_int32_t);
formats[i] = va_arg(args, int);
}
} else {
lengths = NULL;
formats = NULL;
values = NULL;
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
}
Attach tasks/pgsqls to http_requests once more. This way if an http request is removed while tasks or pgsqls are still active they are free'd out and cancelled properly.
2014-08-14 22:05:34 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
ret = KORE_RESULT_ERROR;
if (pgsql->flags & KORE_PGSQL_SYNC) {
pgsql->result = PQexecParams(pgsql->conn->db, query, count,
NULL, (const char * const *)values, lengths, formats,
result);
if ((PQresultStatus(pgsql->result) != PGRES_TUPLES_OK) &&
(PQresultStatus(pgsql->result) != PGRES_COMMAND_OK)) {
pgsql_set_error(pgsql, PQerrorMessage(pgsql->conn->db));
goto cleanup;
}
pgsql->state = KORE_PGSQL_STATE_DONE;
} else {
if (!PQsendQueryParams(pgsql->conn->db, query, count, NULL,
(const char * const *)values, lengths, formats, result)) {
pgsql_set_error(pgsql, PQerrorMessage(pgsql->conn->db));
goto cleanup;
}
pgsql_schedule(pgsql);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
ret = KORE_RESULT_OK;
cleanup:
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
kore_mem_free(values);
kore_mem_free(lengths);
kore_mem_free(formats);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
return (ret);
}
Added new function kore_pgsql_v_query_params(). Same as kore_pgsql_query_params but takes a va_list as last parameter (non-v version takes a variable list of parameters). Lets people write easier to call wrappers around the query calls. I use it in a wrapper that takes next states (error, current, continue) as arguments in a handler with multiple async queries.
2016-06-07 13:12:31 +02:00
int
kore_pgsql_query_params(struct kore_pgsql *pgsql,
const char *query, int result, u_int8_t count, ...)
{
int ret;
va_list args;
if (count > 0)
va_start(args, count);
ret = kore_pgsql_v_query_params(pgsql, query, result, count, args);
if (count > 0)
va_end(args);
return (ret);
}
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
int
kore_pgsql_register(const char *dbname, const char *connstring)
{
struct pgsql_db *pgsqldb;
LIST_FOREACH(pgsqldb, &pgsql_db_conn_strings, rlist) {
if (!strcmp(pgsqldb->name, dbname))
return (KORE_RESULT_ERROR);
}
pgsqldb = kore_malloc(sizeof(*pgsqldb));
pgsqldb->name = kore_strdup(dbname);
pgsqldb->conn_string = kore_strdup(connstring);
LIST_INSERT_HEAD(&pgsql_db_conn_strings, pgsqldb, rlist);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
return (KORE_RESULT_OK);
}
void
kore_pgsql_handle(void *c, int err)
{
struct http_request *req;
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
struct kore_pgsql *pgsql;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
struct pgsql_conn *conn = (struct pgsql_conn *)c;
Improvements to pgsql contrib code. Including but not limited to: - Correctly use PQerrorMessage() in case we cleanup with PQfinish - If we get a network error, cleanup the connection - No longer call the page handler from inside kore_pgsql_handle() but instead just put it to sleep in case we don't need it. This does grow the http_requests list quite a bit with sleeping connections and can perhaps be improved later on. - Allow us to on error return OK from a page handler from inside the completetion block for KORE_PGSQL(). - Count the cummulative time for a request to finish instead of the latest run time for the handler.
2014-03-31 00:57:00 +02:00
if (err) {
pgsql_conn_cleanup(conn);
return;
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
req = conn->job->req;
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql = conn->job->pgsql;
kore_debug("kore_pgsql_handle: %p (%d)", req, pgsql->state);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
if (!PQconsumeInput(conn->db)) {
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql->state = KORE_PGSQL_STATE_ERROR;
pgsql->error = kore_strdup(PQerrorMessage(conn->db));
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
} else {
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
pgsql_read_result(pgsql);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
if (pgsql->state == KORE_PGSQL_STATE_WAIT) {
Introduce http_request_sleep() and http_request_wakeup(). These 2 functions can be used to move an HTTP request from/to the active http_requests list. Effectively putting them to "sleep" or "waking them up". Sprinkle this through the pgsql and task code. If used correctly greatly reduces overhead for managing sleeping tasks.
2014-07-04 16:51:19 +02:00
http_request_sleep(req);
Kore pgsql improvements. Don't wait for a full event loop until we call the page handler for a received pgsql result. This speeds up page loads using KORE_PGSQL by quite a lot, especially on a non busy server.
2014-04-17 10:49:48 +02:00
} else {
Introduce http_request_sleep() and http_request_wakeup(). These 2 functions can be used to move an HTTP request from/to the active http_requests list. Effectively putting them to "sleep" or "waking them up". Sprinkle this through the pgsql and task code. If used correctly greatly reduces overhead for managing sleeping tasks.
2014-07-04 16:51:19 +02:00
http_request_wakeup(req);
Kore pgsql improvements. Don't wait for a full event loop until we call the page handler for a received pgsql result. This speeds up page loads using KORE_PGSQL by quite a lot, especially on a non busy server.
2014-04-17 10:49:48 +02:00
}
Improvements to pgsql contrib code. Including but not limited to: - Correctly use PQerrorMessage() in case we cleanup with PQfinish - If we get a network error, cleanup the connection - No longer call the page handler from inside kore_pgsql_handle() but instead just put it to sleep in case we don't need it. This does grow the http_requests list quite a bit with sleeping connections and can perhaps be improved later on. - Allow us to on error return OK from a page handler from inside the completetion block for KORE_PGSQL(). - Count the cummulative time for a request to finish instead of the latest run time for the handler.
2014-03-31 00:57:00 +02:00
}
void
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
kore_pgsql_continue(struct http_request *req, struct kore_pgsql *pgsql)
Improvements to pgsql contrib code. Including but not limited to: - Correctly use PQerrorMessage() in case we cleanup with PQfinish - If we get a network error, cleanup the connection - No longer call the page handler from inside kore_pgsql_handle() but instead just put it to sleep in case we don't need it. This does grow the http_requests list quite a bit with sleeping connections and can perhaps be improved later on. - Allow us to on error return OK from a page handler from inside the completetion block for KORE_PGSQL(). - Count the cummulative time for a request to finish instead of the latest run time for the handler.
2014-03-31 00:57:00 +02:00
{
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
kore_debug("kore_pgsql_continue: %p->%p (%d)",
req->owner, req, pgsql->state);
Kore pgsql improvements. Don't wait for a full event loop until we call the page handler for a received pgsql result. This speeds up page loads using KORE_PGSQL by quite a lot, especially on a non busy server.
2014-04-17 10:49:48 +02:00
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
if (pgsql->error) {
kore_mem_free(pgsql->error);
pgsql->error = NULL;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
pgsql improvements * set pgsql->result to NULL when we freed it * drain the connection pipe just in case in pgsql_conn_release()
2014-08-14 16:43:40 +02:00
if (pgsql->result) {
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
PQclear(pgsql->result);
pgsql improvements * set pgsql->result to NULL when we freed it * drain the connection pipe just in case in pgsql_conn_release()
2014-08-14 16:43:40 +02:00
pgsql->result = NULL;
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
switch (pgsql->state) {
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
case KORE_PGSQL_STATE_INIT:
case KORE_PGSQL_STATE_WAIT:
break;
case KORE_PGSQL_STATE_DONE:
Introduce http_request_sleep() and http_request_wakeup(). These 2 functions can be used to move an HTTP request from/to the active http_requests list. Effectively putting them to "sleep" or "waking them up". Sprinkle this through the pgsql and task code. If used correctly greatly reduces overhead for managing sleeping tasks.
2014-07-04 16:51:19 +02:00
http_request_wakeup(req);
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
pgsql_conn_release(pgsql);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
break;
case KORE_PGSQL_STATE_ERROR:
case KORE_PGSQL_STATE_RESULT:
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
kore_pgsql_handle(pgsql->conn, 0);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
break;
default:
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
fatal("unknown pgsql state %d", pgsql->state);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
}
void
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
kore_pgsql_cleanup(struct kore_pgsql *pgsql)
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
{
Attach tasks/pgsqls to http_requests once more. This way if an http request is removed while tasks or pgsqls are still active they are free'd out and cancelled properly.
2014-08-14 22:05:34 +02:00
kore_debug("kore_pgsql_cleanup(%p)", pgsql);
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
if (pgsql->result != NULL)
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
PQclear(pgsql->result);
Improvements to pgsql contrib code. Including but not limited to: - Correctly use PQerrorMessage() in case we cleanup with PQfinish - If we get a network error, cleanup the connection - No longer call the page handler from inside kore_pgsql_handle() but instead just put it to sleep in case we don't need it. This does grow the http_requests list quite a bit with sleeping connections and can perhaps be improved later on. - Allow us to on error return OK from a page handler from inside the completetion block for KORE_PGSQL(). - Count the cummulative time for a request to finish instead of the latest run time for the handler.
2014-03-31 00:57:00 +02:00
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
if (pgsql->error != NULL)
kore_mem_free(pgsql->error);
pgsql_conn improvements. set conn inside kore_pgsql to NULL when we've completed. if at kore_pgsql_cleanup() time the conn is still there we'll drain it as per documentation.
2014-03-31 01:09:08 +02:00
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
if (pgsql->conn != NULL)
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
pgsql_conn_release(pgsql);
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql->result = NULL;
pgsql->error = NULL;
pgsql->conn = NULL;
Attach tasks/pgsqls to http_requests once more. This way if an http request is removed while tasks or pgsqls are still active they are free'd out and cancelled properly.
2014-08-14 22:05:34 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if (pgsql->flags & PGSQL_LIST_INSERTED) {
LIST_REMOVE(pgsql, rlist);
pgsql->flags &= ~PGSQL_LIST_INSERTED;
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
amend example with additional query + kore_pgsql_logerror()
2014-04-02 23:01:47 +02:00
void
kore_pgsql_logerror(struct kore_pgsql *pgsql)
{
kore_log(LOG_NOTICE, "pgsql error: %s",
(pgsql->error) ? pgsql->error : "unknown");
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
int
pgsql fixes + new wrappers
2014-04-01 21:43:32 +02:00
kore_pgsql_ntuples(struct kore_pgsql *pgsql)
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
{
pgsql fixes + new wrappers
2014-04-01 21:43:32 +02:00
return (PQntuples(pgsql->result));
}
Add kore_pgsql_getlength() which will return length of a column.
2014-09-28 23:03:49 +02:00
int
kore_pgsql_getlength(struct kore_pgsql *pgsql, int row, int col)
{
return (PQgetlength(pgsql->result, row, col));
}
pgsql fixes + new wrappers
2014-04-01 21:43:32 +02:00
char *
kore_pgsql_getvalue(struct kore_pgsql *pgsql, int row, int col)
{
return (PQgetvalue(pgsql->result, row, col));
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
void
kore_pgsql_queue_remove(struct http_request *req)
{
struct pgsql_wait *pgw, *next;
for (pgw = TAILQ_FIRST(&pgsql_wait_queue); pgw != NULL; pgw = next) {
next = TAILQ_NEXT(pgw, list);
if (pgw->req != req)
continue;
TAILQ_REMOVE(&pgsql_wait_queue, pgw, list);
kore_pool_put(&pgsql_wait_pool, pgw);
return;
}
}
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
static struct pgsql_conn *
pgsql_conn_next(struct kore_pgsql *pgsql, struct pgsql_db *db,
struct http_request *req)
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
{
struct pgsql_conn *conn;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
conn = NULL;
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
TAILQ_FOREACH(conn, &pgsql_conn_free, list) {
if (!(conn->flags & PGSQL_CONN_FREE))
fatal("got a pgsql connection that was not free?");
if (!strcmp(conn->name, db->name))
break;
}
if (conn == NULL) {
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
if (pgsql_conn_count >= pgsql_conn_max) {
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if (pgsql->flags & KORE_PGSQL_ASYNC) {
pgsql_queue_add(req);
} else {
pgsql_set_error(pgsql,
"no available connection");
}
return (NULL);
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
}
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if ((conn = pgsql_conn_create(pgsql, db)) == NULL)
return (NULL);
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
}
conn->flags &= ~PGSQL_CONN_FREE;
TAILQ_REMOVE(&pgsql_conn_free, conn, list);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
return (conn);
}
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
static void
pgsql_set_error(struct kore_pgsql *pgsql, const char *msg)
{
if (pgsql->error != NULL)
kore_mem_free(pgsql->error);
pgsql->error = kore_strdup(msg);
pgsql->state = KORE_PGSQL_STATE_ERROR;
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
}
static void
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
pgsql_schedule(struct kore_pgsql *pgsql)
Add kore_pgsql_query_params(). This function uses PQsendQueryParams() instead of the normal PQsendQuery() allowing you to pass binary data in a cleaner fashion. A basic call would look something like: char *mydata = "Hello"; size_t mydata_len = strlen(mydata); kore_pgsql_query_params(&pgsql, req, "INSERT INTO foo VALUES($1::text)", KORE_PGSQL_FORMAT_TEXT, 1 mydata, mydata_len, KORE_PGSQL_FORMAT_TEXT); kore_pgsql_query_params() is variadic, allowing you to pass any count of parameters where each parameter has the following: data pointer, data length, type of parameter.
2014-09-28 21:39:16 +02:00
{
int fd;
fd = PQsocket(pgsql->conn->db);
if (fd < 0)
fatal("PQsocket returned < 0 fd on open connection");
kore_platform_schedule_read(fd, pgsql->conn);
pgsql->state = KORE_PGSQL_STATE_WAIT;
}
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
static void
pgsql_queue_add(struct http_request *req)
{
struct pgsql_wait *pgw;
http_request_sleep(req);
pgw = kore_pool_get(&pgsql_wait_pool);
pgw->req = req;
pgw->req->flags |= HTTP_REQUEST_PGSQL_QUEUE;
TAILQ_INSERT_TAIL(&pgsql_wait_queue, pgw, list);
}
static void
pgsql_queue_wakeup(void)
{
struct pgsql_wait *pgw, *next;
for (pgw = TAILQ_FIRST(&pgsql_wait_queue); pgw != NULL; pgw = next) {
next = TAILQ_NEXT(pgw, list);
if (pgw->req->flags & HTTP_REQUEST_DELETE)
continue;
http_request_wakeup(pgw->req);
pgw->req->flags &= ~HTTP_REQUEST_PGSQL_QUEUE;
TAILQ_REMOVE(&pgsql_wait_queue, pgw, list);
kore_pool_put(&pgsql_wait_pool, pgw);
return;
}
}
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
static struct pgsql_conn *
pgsql_conn_create(struct kore_pgsql *pgsql, struct pgsql_db *db)
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
{
struct pgsql_conn *conn;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if (db == NULL || db->conn_string == NULL)
Check connection string right before we connect
2014-04-14 08:45:31 +02:00
fatal("pgsql_conn_create: no connection string");
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
pgsql_conn_count++;
conn = kore_malloc(sizeof(*conn));
kore_debug("pgsql_conn_create(): %p", conn);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
conn->db = PQconnectdb(db->conn_string);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
if (conn->db == NULL || (PQstatus(conn->db) != CONNECTION_OK)) {
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
pgsql_set_error(pgsql, PQerrorMessage(conn->db));
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
pgsql_conn_cleanup(conn);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
return (NULL);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
conn->job = NULL;
conn->flags = PGSQL_CONN_FREE;
conn->type = KORE_TYPE_PGSQL_CONN;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
conn->name = kore_strdup(db->name);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
TAILQ_INSERT_TAIL(&pgsql_conn_free, conn, list);
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
return (conn);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
}
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
static void
pgsql_conn_release(struct kore_pgsql *pgsql)
{
int fd;
if (pgsql->conn == NULL)
return;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
/* Async query cleanup */
if (pgsql->flags & KORE_PGSQL_ASYNC) {
if (pgsql->conn != NULL) {
fd = PQsocket(pgsql->conn->db);
kore_platform_disable_read(fd);
kore_pool_put(&pgsql_job_pool, pgsql->conn->job);
}
}
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
pgsql improvements * set pgsql->result to NULL when we freed it * drain the connection pipe just in case in pgsql_conn_release()
2014-08-14 16:43:40 +02:00
/* Drain just in case. */
while (PQgetResult(pgsql->conn->db) != NULL)
;
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
pgsql->conn->job = NULL;
pgsql->conn->flags |= PGSQL_CONN_FREE;
TAILQ_INSERT_TAIL(&pgsql_conn_free, pgsql->conn, list);
pgsql->conn = NULL;
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
pgsql->state = KORE_PGSQL_STATE_COMPLETE;
pgsql_queue_wakeup();
Properly release pgsql connection if needed
2014-08-14 14:46:21 +02:00
}
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
static void
pgsql_conn_cleanup(struct pgsql_conn *conn)
{
struct http_request *req;
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
struct kore_pgsql *pgsql;
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
kore_debug("pgsql_conn_cleanup(): %p", conn);
if (conn->flags & PGSQL_CONN_FREE)
TAILQ_REMOVE(&pgsql_conn_free, conn, list);
if (conn->job) {
req = conn->job->req;
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql = conn->job->pgsql;
Introduce http_request_sleep() and http_request_wakeup(). These 2 functions can be used to move an HTTP request from/to the active http_requests list. Effectively putting them to "sleep" or "waking them up". Sprinkle this through the pgsql and task code. If used correctly greatly reduces overhead for managing sleeping tasks.
2014-07-04 16:51:19 +02:00
http_request_wakeup(req);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql->conn = NULL;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
pgsql_set_error(pgsql, PQerrorMessage(conn->db));
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
Introduce a wait queue for our pgsql code. Instead of letting http_requests spin, if we cannot allocate a connection for the request we will queue them up put them to sleep. When a connection becomes available, we'll wake up a request that was waiting for a connection and let it continue. This completely avoids consuming massive amounts of cpu time when dealing with thousands of requests waiting for a pgsql worker to become ready.
2014-08-21 16:36:12 +02:00
kore_pool_put(&pgsql_job_pool, conn->job);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
conn->job = NULL;
}
if (conn->db != NULL)
PQfinish(conn->db);
pgsql_conn_count--;
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
kore_mem_free(conn->name);
Initial stab at entering postgresql contrib code. Has support for full async pgsql queries. Most of the logic is hidden behind a KORE_PGSQL() macro allowing you to insert these pgsql calls in your page handlers without blocking the kore worker while the query is going off. There is place for improvement here, and perhaps KORE_PGSQL won't stay as I feel this might overcomplicate things instead of making them simpler as I thought it would.
2014-03-30 23:54:35 +02:00
kore_mem_free(conn);
}
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
static void
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
pgsql_read_result(struct kore_pgsql *pgsql)
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
{
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
if (PQisBusy(pgsql->conn->db)) {
pgsql->state = KORE_PGSQL_STATE_WAIT;
return;
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
}
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql->result = PQgetResult(pgsql->conn->db);
if (pgsql->result == NULL) {
pgsql->state = KORE_PGSQL_STATE_DONE;
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
return;
}
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
switch (PQresultStatus(pgsql->result)) {
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
case PGRES_COPY_OUT:
case PGRES_COPY_IN:
case PGRES_NONFATAL_ERROR:
case PGRES_COPY_BOTH:
break;
case PGRES_COMMAND_OK:
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql->state = KORE_PGSQL_STATE_DONE;
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
break;
case PGRES_TUPLES_OK:
Check for pg version >= 90200 before using PGRES_SINGLE_TUPLE
2014-07-03 22:58:04 +02:00
#if PG_VERSION_NUM >= 90200
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
case PGRES_SINGLE_TUPLE:
Check for pg version >= 90200 before using PGRES_SINGLE_TUPLE
2014-07-03 22:58:04 +02:00
#endif
Major pgsql rework. Gone is the ugly KORE_PGSQL macro that hid an overly complex state machine for the pgsql api. Gone is the pgsql array that was attached to http_requests. Gone are the callback hacks inside the pgsql api. Instead, I strongly encourage people to use the new state machine api Kore offers to properly deal with asynchronous queries. The pgsql example in examples/pgsql has been updated to reflect these changes.
2014-08-14 14:34:23 +02:00
pgsql->state = KORE_PGSQL_STATE_RESULT;
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
break;
case PGRES_EMPTY_QUERY:
case PGRES_BAD_RESPONSE:
case PGRES_FATAL_ERROR:
Introduce synchronous pgsql queries. Semantics for using pgsql API have changed quite heavily with this commit. See the examples for more information. Based on Github issue #95 by PauloMelo (paulo.melo@vintageform.pt) with several modifications by me.
2016-01-04 11:12:43 +01:00
pgsql_set_error(pgsql, PQresultErrorMessage(pgsql->result));
Layout that stuff like a 80 column terminal nerd. And mark a request as DONE when we get PGRES_COMMAND_OK.
2014-03-31 01:04:22 +02:00
break;
}
}