Markus Armbruster 62815d85ae json: Redesign the callback to consume JSON values ... The classical way to structure parser and lexer is to have the client call the parser to get an abstract syntax tree, the parser call the lexer to get the next token, and the lexer call some function to get input characters. Another way to structure them would be to have the client feed characters to the lexer, the lexer feed tokens to the parser, and the parser feed abstract syntax trees to some callback provided by the client. This way is more easily integrated into an event loop that dispatches input characters as they arrive. Our JSON parser is kind of between the two. The lexer feeds tokens to a "streamer" instead of a real parser. The streamer accumulates tokens until it got the sequence of tokens that comprise a single JSON value (it counts curly braces and square brackets to decide). It feeds those token sequences to a callback provided by the client. The callback passes each token sequence to the parser, and gets back an abstract syntax tree. I figure it was done that way to make a straightforward recursive descent parser possible. "Get next token" becomes "pop the first token off the token sequence". Drawback: we need to store a complete token sequence. Each token eats 13 + input characters + malloc overhead bytes. Observations: 1. This is not the only way to use recursive descent. If we replaced "get next token" by a coroutine yield, we could do without a streamer. 2. The lexer reports errors by passing a JSON_ERROR token to the streamer. This communicates the offending input characters and their location, but no more. 3. The streamer reports errors by passing a null token sequence to the callback. The (already poor) lexical error information is thrown away. 4. Having the callback receive a token sequence duplicates the code to convert token sequence to abstract syntax tree in every callback. 5. Known bug: the streamer silently drops incomplete token sequences. This commit rectifies 4. by lifting the call of the parser from the callbacks into the streamer. Later commits will address 3. and 5. The lifting removes a bug from qjson.c's parse_json(): it passed a pointer to a non-null Error * in certain cases, as demonstrated by check-qjson.c. json_parser_parse() is now unused. It's a stupid wrapper around json_parser_parse_err(). Drop it, and rename json_parser_parse_err() to json_parser_parse(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-35-armbru@redhat.com>		2018-08-24 20:26:37 +02:00
..
acceptance	…
acpi-test-data	tests/acpi: update tables after memory hotplug changes	2018-08-03 11:35:21 +03:00
data	…
decode	…
docker	tests: Allow overriding archive path with SRC_ARCHIVE	2018-08-15 10:12:35 +08:00
guest-debug	…
hex-loader-check-data	Add QTest testcase for the Intel Hexadecimal	2018-08-16 14:05:28 +01:00
image-fuzzer	…
keys	…
libqos	fw_cfg: import & use linux/qemu_fw_cfg.h	2018-08-23 18:46:25 +02:00
migration	…
multiboot	…
qapi-schema	qapi: Fix some pycodestyle-3 complaints	2018-08-15 07:24:22 +02:00
qemu-iotests	block: iotest to catch abort on forced blockjob cancel	2018-08-21 15:20:37 -04:00
rocker	…
tcg	tests: add check_invalid_maps to test-mmap	2018-07-31 09:57:25 +02:00
vm	tests/vm: Increase timeout waiting for VM to boot to 5 minutes	2018-08-24 11:31:28 +01:00
vmstate-static-checker-data	…
.gitignore	tests/.gitignore: don't ignore docker tests	2018-07-24 11:45:25 +01:00
Makefile.include	qmp-cmd-test: Split off qmp-test	2018-08-24 20:25:48 +02:00
ac97-test.c	…
acpi-utils.c	…
acpi-utils.h	…
ahci-test.c	Testing patches for 2018-08-16	2018-08-16 09:50:54 +01:00
atomic_add-bench.c	tests/atomic_add-bench: add -p to enable sync profiler	2018-08-23 18:46:25 +02:00
benchmark-crypto-cipher.c	tests/crypto: Use the IEC binary prefix definitions	2018-07-02 15:41:17 +02:00
benchmark-crypto-hash.c	tests/crypto: Use the IEC binary prefix definitions	2018-07-02 15:41:17 +02:00
benchmark-crypto-hmac.c	tests/crypto: Use the IEC binary prefix definitions	2018-07-02 15:41:17 +02:00
bios-tables-test.c	…
boot-order-test.c	fw_cfg: import & use linux/qemu_fw_cfg.h	2018-08-23 18:46:25 +02:00
boot-sector.c	…
boot-sector.h	…
boot-serial-test.c	hw/ppc: deprecate the machine type 'prep', replaced by '40p'	2018-08-21 14:28:45 +10:00
cdrom-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
check-block-qdict.c	tests: fix crumple/recursive leak	2018-08-15 08:12:19 +02:00
check-block.sh	…
check-qdict.c	…
check-qjson.c	json: Redesign the callback to consume JSON values	2018-08-24 20:26:37 +02:00
check-qlist.c	…
check-qlit.c	…
check-qnull.c	…
check-qnum.c	…
check-qobject.c	qstring: Move qstring_from_substr()'s @end one to the right	2018-07-28 09:09:58 +02:00
check-qom-interface.c	…
check-qom-proplist.c	…
check-qstring.c	qstring: Move qstring_from_substr()'s @end one to the right	2018-07-28 09:09:58 +02:00
cpu-plug-test.c	tests: Skip old versioned machine types in quick testing mode	2018-08-23 18:46:23 +02:00
crypto-tls-psk-helpers.c	crypto: Implement TLS Pre-Shared Keys (PSK).	2018-07-03 13:04:38 +01:00
crypto-tls-psk-helpers.h	crypto: Implement TLS Pre-Shared Keys (PSK).	2018-07-03 13:04:38 +01:00
crypto-tls-x509-helpers.c	tests: call qcrypto_init instead of gnutls_global_init	2018-07-24 17:33:39 +01:00
crypto-tls-x509-helpers.h	…
device-introspect-test.c	tests/device-introspect: Test with all machines, not only with "none"	2018-08-23 18:46:25 +02:00
display-vga-test.c	…
drive_del-test.c	libqtest: Remove qtest_qmp_discard_response() & friends	2018-08-16 08:42:06 +02:00
ds1338-test.c	…
e1000-test.c	…
e1000e-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
eepro100-test.c	…
endianness-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
es1370-test.c	…
fdc-test.c	libqtest: Remove qtest_qmp_discard_response() & friends	2018-08-16 08:42:06 +02:00
fw_cfg-test.c	fw_cfg: import & use linux/qemu_fw_cfg.h	2018-08-23 18:46:25 +02:00
hd-geo-test.c	block: Remove deprecated -drive geometry options	2018-08-15 12:50:39 +02:00
hexloader-test.c	Add QTest testcase for the Intel Hexadecimal	2018-08-16 14:05:28 +01:00
i440fx-test.c	…
i82801b11-test.c	…
ide-test.c	Testing patches for 2018-08-16	2018-08-16 09:50:54 +01:00
intel-hda-test.c	…
io-channel-helpers.c	…
io-channel-helpers.h	…
ioh3420-test.c	…
iothread.c	…
iothread.h	…
ipmi-bt-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
ipmi-kcs-test.c	…
ipoctal232-test.c	…
ivshmem-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
libqtest.c	json: Redesign the callback to consume JSON values	2018-08-24 20:26:37 +02:00
libqtest.h	test-qga: Clean up how we test QGA synchronization	2018-08-24 20:26:37 +02:00
m25p80-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
m48t59-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
machine-none-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
megasas-test.c	…
migration-test.c	tests/migration-test: Silence the kvm_hv message by default	2018-08-23 13:32:50 +02:00
ne2000-test.c	…
numa-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
nvme-test.c	…
pca9552-test.c	…
pcnet-test.c	…
pkix_asn1_tab.c	…
pnv-xscom-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
prom-env-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
ptimer-test-stubs.c	…
ptimer-test.c	ptimer: Add TRIGGER_ONLY_ON_DECREMENT policy option	2018-07-09 14:51:34 +01:00
ptimer-test.h	…
pvpanic-test.c	…
pxe-test.c	…
q35-test.c	…
qemu-iotests-quick.sh	…
qht-bench.c	…
qmp-cmd-test.c	qmp-cmd-test: Split off qmp-test	2018-08-24 20:25:48 +02:00
qmp-test.c	json: Reject unescaped control characters	2018-08-24 20:26:37 +02:00
qom-test.c	tests: Skip old versioned machine types in quick testing mode	2018-08-23 18:46:23 +02:00
rcutorture.c	…
rtas-test.c	…
rtc-test.c	…
rtl8139-test.c	…
sdhci-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
socket-helpers.c	…
socket-helpers.h	…
spapr-phb-test.c	…
tco-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
test-aio-multithread.c	…
test-aio.c	…
test-arm-mptimer.c	…
test-base64.c	…
test-bdrv-drain.c	test-bdrv-drain: Test bdrv_append() to drained node	2018-07-10 10:36:15 +02:00
test-bitcnt.c	…
test-bitops.c	…
test-block-backend.c	…
test-blockjob-txn.c	…
test-blockjob.c	…
test-bufferiszero.c	…
test-char.c	build-sys: remove glib_subprocess check	2018-08-23 18:46:25 +02:00
test-clone-visitor.c	…
test-coroutine.c	…
test-crypto-afsplit.c	…
test-crypto-block.c	…
test-crypto-cipher.c	…
test-crypto-hash.c	…
test-crypto-hmac.c	…
test-crypto-ivgen.c	…
test-crypto-pbkdf.c	…
test-crypto-secret.c	…
test-crypto-tlscredsx509.c	tests: use error_abort in places expecting errors	2018-07-24 17:35:57 +01:00
test-crypto-tlssession.c	tests: fix TLS handshake failure with TLS 1.3	2018-07-24 17:36:12 +01:00
test-crypto-xts.c	…
test-cutils.c	hw: Use IEC binary prefix definitions from "qemu/units.h"	2018-07-02 15:41:10 +02:00
test-filter-mirror.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
test-filter-redirector.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
test-hbitmap.c	…
test-hmp.c	tests: Skip old versioned machine types in quick testing mode	2018-08-23 18:46:23 +02:00
test-int128.c	…
test-io-channel-buffer.c	…
test-io-channel-command.c	…
test-io-channel-file.c	…
test-io-channel-socket.c	…
test-io-channel-tls.c	tests: use error_abort in places expecting errors	2018-07-24 17:35:57 +01:00
test-io-task.c	…
test-iov.c	…
test-keyval.c	hw: Use IEC binary prefix definitions from "qemu/units.h"	2018-07-02 15:41:10 +02:00
test-logging.c	…
test-mul64.c	…
test-netfilter.c	…
test-opts-visitor.c	…
test-qapi-util.c	…
test-qdev-global-props.c	…
test-qdist.c	…
test-qemu-opts.c	hw: Use IEC binary prefix definitions from "qemu/units.h"	2018-07-02 15:41:10 +02:00
test-qga.c	test-qga: Clean up how we test QGA synchronization	2018-08-24 20:26:37 +02:00
test-qht-par.c	…
test-qht.c	…
test-qmp-cmds.c	tests: change /0.15/* tests to /qmp/*	2018-08-15 07:24:22 +02:00
test-qmp-event.c	glib: bump min required glib library version to 2.40	2018-06-29 12:22:28 +01:00
test-qobject-input-visitor.c	test-qobject-input-visitor: Avoid format string ambiguity	2018-08-16 08:42:06 +02:00
test-qobject-output-visitor.c	…
test-rcu-list.c	tests: add test-rcu-tailq	2018-08-23 18:46:25 +02:00
test-rcu-simpleq.c	tests: add test-list-simpleq	2018-08-23 18:46:25 +02:00
test-rcu-tailq.c	tests: add test-rcu-tailq	2018-08-23 18:46:25 +02:00
test-replication.c	…
test-shift128.c	…
test-string-input-visitor.c	…
test-string-output-visitor.c	…
test-thread-pool.c	…
test-throttle.c	…
test-timed-average.c	…
test-util-sockets.c	monitor: Fix unsafe sharing of @cur_mon among threads	2018-07-23 14:00:03 +02:00
test-uuid.c	…
test-visitor-serialization.c	…
test-vmstate.c	tests: don't silence error reporting for all tests	2018-07-24 17:35:23 +01:00
test-write-threshold.c	…
test-x86-cpuid-compat.c	build-sys: remove glib_subprocess check	2018-08-23 18:46:25 +02:00
test-x86-cpuid.c	…
test-xbzrle.c	…
tmp105-test.c	…
tpci200-test.c	…
tpm-crb-swtpm-test.c	…
tpm-crb-test.c	…
tpm-emu.c	…
tpm-emu.h	glib: bump min required glib library version to 2.40	2018-06-29 12:22:28 +01:00
tpm-tests.c	…
tpm-tests.h	…
tpm-tis-swtpm-test.c	…
tpm-tis-test.c	…
tpm-util.c	tests: New helper qtest_qmp_receive_success()	2018-08-16 08:42:06 +02:00
tpm-util.h	…
usb-hcd-ehci-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
usb-hcd-ohci-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
usb-hcd-uhci-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
usb-hcd-xhci-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
vhost-user-bridge.c	…
vhost-user-test.c	build-sys: remove glib_subprocess check	2018-08-23 18:46:25 +02:00
virtio-9p-test.c	…
virtio-balloon-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
virtio-blk-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
virtio-ccw-test.c	tests: virtio: separate ccw tests from libqos	2018-08-23 13:32:50 +02:00
virtio-console-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
virtio-net-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
virtio-rng-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
virtio-scsi-test.c	tests: Clean up string interpolation around qtest_qmp_device_add()	2018-08-16 08:42:06 +02:00
virtio-serial-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
vmgenid-test.c	libqtest: Replace qtest_startf() by qtest_initf()	2018-08-16 08:42:06 +02:00
vmxnet3-test.c	…
wdt_ib700-test.c	…