qemu-e2k/tests/.gitignore

84 lines
1.5 KiB
Plaintext
Raw Normal View History

check-qdict
check-qfloat
check-qint
check-qjson
check-qlist
check-qnull
check-qstring
check-qom-interface
check-qom-proplist
qht: add qht-bench, a performance benchmark This serves as a performance benchmark as well as a stress test for QHT. We can tweak quite a number of things, including the number of resize threads and how frequently resizes are triggered. A performance comparison of QHT vs CLHT[1] and ck_hs[2] using this same benchmark program can be found here: http://imgur.com/a/0Bms4 The tests are run on a 64-core AMD Opteron 6376, pinning threads to cores favoring same-socket cores. For each run, qht-bench is invoked with: $ tests/qht-bench -d $duration -n $n -u $u -g $range , where $duration is in seconds, $n is the number of threads, $u is the update rate (0.0 to 100.0), and $range is the number of keys. Note that ck_hs's performance drops significantly as writes go up, since it requires an external lock (I used a ck_spinlock) around every write. Also, note that CLHT instead of using a seqlock, relies on an allocator that does not ever return the same address during the same read-critical section. This gives it a slight performance advantage over QHT on read-heavy workloads, since the seqlock writes aren't there. [1] CLHT: https://github.com/LPD-EPFL/CLHT https://infoscience.epfl.ch/record/207109/files/ascy_asplos15.pdf [2] ck_hs: http://concurrencykit.org/ http://backtrace.io/blog/blog/2015/03/13/workload-specialization/ A few of those plots are shown in text here, since that site might not be online forever. Throughput is on Mops/s on the Y axis. 200K keys, 0 % updates 450 ++--+------+------+-------+-------+-------+-------+------+-------+--++ | + + + + + + + + +N+ | 400 ++ ---+E+ ++ | +++---- | 350 ++ 9 ++------+------++ --+E+ -+H+ ++ | | +H+- | -+N+---- ---- +++ | 300 ++ 8 ++ +E+ ++ -----+E+ --+H+ ++ | | +++ | -+N+-----+H+-- | 250 ++ 7 ++------+------++ +++-----+E+---- ++ 200 ++ 1 -+E+-----+H+ ++ | ---- qht +-E--+ | 150 ++ -+E+ clht +-H--+ ++ | ---- ck +-N--+ | 100 ++ +E+ ++ | ---- | 50 ++ -+E+ ++ | +E+E+ + + + + + + + + | 0 ++--E------+------+-------+-------+-------+-------+------+-------+--++ 1 8 16 24 32 40 48 56 64 Number of threads 200K keys, 1 % updates 350 ++--+------+------+-------+-------+-------+-------+------+-------+--++ | + + + + + + + + -+E+ | 300 ++ -----+H+ ++ | +E+-- | | 9 ++------+------++ +++---- | 250 ++ | +E+ -- | -+E+ ++ | 8 ++ -- ++ ---- | 200 ++ | +++- | +++ ---+E+ ++ | 7 ++------N------++ -+E+-- qht +-E--+ | | 1 +++---- clht +-H--+ | 150 ++ -+E+ ck +-N--+ ++ | ---- | 100 ++ +E+ ++ | ---- | | -+E+ | 50 ++ +H+-+N+----+N+-----+N+------ ++ | +E+E+ + + + +N+-----+N+-----+N+----+N+-----+N+ | 0 ++--E------+------+-------+-------+-------+-------+------+-------+--++ 1 8 16 24 32 40 48 56 64 Number of threads 200K keys, 20 % updates 300 ++--+------+------+-------+-------+-------+-------+------+-------+--++ | + + + + + + + + + | | -+H+ | 250 ++ ---- ++ | 9 ++------+------++ --+H+ ---+E+ | | 8 ++ +H+-- ++ -+H+----+E+-- | 200 ++ | +E+ --| -----+E+-- +++ ++ | 7 ++ + ---- ++ ---+H+---- +++ qht +-E--+ | 150 ++ 6 ++------N------++ -+H+-----+E+ clht +-H--+ ++ | 1 -----+E+-- ck +-N--+ | | -+H+---- | 100 ++ -----+E+ ++ | +E+-- | | ----+++ | 50 ++ -+E+ ++ | +E+ +++ | | +E+N+-+N+-----+ + + + + + + | 0 ++--E------+------N-------N-------N-------N-------N------N-------N--++ 1 8 16 24 32 40 48 56 64 Number of threads 200K keys, 100 % updates qht +-E--+ clht +-H--+ 160 ++--+------+------+-------+-------+-------+-------+---ck-+-N-----+--++ | + + + + + + + + ----H | 140 ++ +H+-- -+E+ ++ | +++---- ---- | 120 ++ 8 ++------+------++ -+H+ +E+ ++ | 7 ++ +H+---- ++ ---- +++---- | 100 ++ | +E+ | +++ ---+H+ -+E+ ++ | 6 ++ +++ ++ -+H+-- +++---- | 80 ++ 5 ++------N----------+E+-----+E+ ++ | 1 -+H+---- +++ | | -----+E+ | 60 ++ +H+---- +++ ++ | ----+E+ | 40 ++ +H+---- ++ | --+E+ | 20 ++ +E+ ++ | +EE+ + + + + + + + + | 0 ++--+N-N---N------N-------N-------N-------N-------N------N-------N--++ 1 8 16 24 32 40 48 56 64 Number of threads Signed-off-by: Emilio G. Cota <cota@braap.org> Message-Id: <1465412133-3029-13-git-send-email-cota@braap.org> Signed-off-by: Richard Henderson <rth@twiddle.net>
2016-06-08 20:55:30 +02:00
qht-bench
rcutorture
test-aio
test-base64
test-bitops
test-blockjob-txn
qapi: Add new clone visitor We have a couple places in the code base that want to deep-clone one QAPI object into another, and they were resorting to serializing the struct out to QObject then reparsing it. A much more efficient version can be done by adding a new clone visitor. Since cloning is still relatively uncommon, expose the use of the new visitor via a QAPI_CLONE() macro that takes care of type-punning the underlying function pointer, rather than generating lots of unused functions for types that won't be cloned. And yes, we're relying on the compiler treating all pointers equally, even though a strict C program cannot portably do so - but we're not the first one in the qemu code base to expect it to work (hello, glib!). The choice of adding a fourth visitor type deserves some explanation. On the surface, the clone visitor is mostly an input visitor (it takes arbitrary input - in this case, another QAPI object - and creates a new QAPI object during the course of the visit). But ever since commit da72ab0 consolidated enum visits based on the visitor type, using VISITOR_INPUT would cause us to run visit_type_str(), even though for cloning there is nothing to do (we just copy the enum value across, without regards to its mapping to strings). Also, since our input happens to be a QAPI object, we can also satisfy the internal checks for VISITOR_OUTPUT. So in the end, I settled with a new VISITOR_CLONE, and chose its value such that many internal checks can use 'v->type & mask', sticking to 'v->type == value' where the difference matters. Note that we can only clone objects (including alternates) and lists, not built-ins or enums. The visitor core hides integer width from the actual visitor (since commit 04e070d), and as long as that's the case, we can't clone top-level integers. Then again, those can always be cloned by direct copy, since they are not objects with deep pointers, so it's no real loss. And restricting cloning to just objects and lists is cleaner than restricting it to non-integers. As such, I documented that the clone visitor is for direct use only by code internal to QAPI, and should not be used on incomplete objects (other than a hack to work around the fact that we allow NULL in place of "" in visit_type_str() in other output visitors). Note that as written, the clone visitor will never fail on a complete object. Scalars (including enums) not at the root of the clone copy just fine with no additional effort while visiting the scalar, by virtue of a g_memdup() each time we push another struct onto the stack. Cloning a string requires deduplication of a pointer, which means it can also provide the guarantee of an input visitor of never producing NULL even when still accepting NULL in place of "" the way the QMP output visitor does. Cloning an 'any' type could be possible by incrementing the QObject refcnt, but it's not obvious whether that is better than implementing a QObject deep clone. So for now, we document it as unsupported, and intentionally omit the .type_any() callback to let a developer know their usage needs implementation. Add testsuite coverage for several different clone situations, to ensure that the code is working. I also tested that valgrind was happy with the test. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-14-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-06-09 18:48:44 +02:00
test-clone-visitor
test-coroutine
test-crypto-afsplit
test-crypto-block
test-crypto-cipher
test-crypto-hash
test-crypto-ivgen
test-crypto-pbkdf
crypto: add QCryptoSecret object class for password/key handling Introduce a new QCryptoSecret object class which will be used for providing passwords and keys to other objects which need sensitive credentials. The new object can provide secret values directly as properties, or indirectly via a file. The latter includes support for file descriptor passing syntax on UNIX platforms. Ordinarily passing secret values directly as properties is insecure, since they are visible in process listings, or in log files showing the CLI args / QMP commands. It is possible to use AES-256-CBC to encrypt the secret values though, in which case all that is visible is the ciphertext. For ad hoc developer testing though, it is fine to provide the secrets directly without encryption so this is not explicitly forbidden. The anticipated scenario is that libvirtd will create a random master key per QEMU instance (eg /var/run/libvirt/qemu/$VMNAME.key) and will use that key to encrypt all passwords it provides to QEMU via '-object secret,....'. This avoids the need for libvirt (or other mgmt apps) to worry about file descriptor passing. It also makes life easier for people who are scripting the management of QEMU, for whom FD passing is significantly more complex. Providing data inline (insecure, only for ad hoc dev testing) $QEMU -object secret,id=sec0,data=letmein Providing data indirectly in raw format printf "letmein" > mypasswd.txt $QEMU -object secret,id=sec0,file=mypasswd.txt Providing data indirectly in base64 format $QEMU -object secret,id=sec0,file=mykey.b64,format=base64 Providing data with encryption $QEMU -object secret,id=master0,file=mykey.b64,format=base64 \ -object secret,id=sec0,data=[base64 ciphertext],\ keyid=master0,iv=[base64 IV],format=base64 Note that 'format' here refers to the format of the ciphertext data. The decrypted data must always be in raw byte format. More examples are shown in the updated docs. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-10-14 10:58:38 +02:00
test-crypto-secret
test-crypto-tlscredsx509
test-crypto-tlscredsx509-work/
test-crypto-tlscredsx509-certs/
test-crypto-tlssession
test-crypto-tlssession-work/
test-crypto-tlssession-client/
test-crypto-tlssession-server/
test-crypto-xts
test-cutils
test-hbitmap
test-int128
test-iov
test-io-channel-buffer
test-io-channel-command
test-io-channel-command.fifo
test-io-channel-file
test-io-channel-file.txt
test-io-channel-socket
test-io-channel-tls
test-io-task
test-logging
test-mul64
test-opts-visitor
test-qapi-event.[ch]
test-qapi-types.[ch]
test-qapi-visit.[ch]
test-qdev-global-props
test-qemu-opts
test-qdist
test-qga
test-qht
test-qht-par
test-qmp-commands
test-qmp-commands.h
test-qmp-event
test-qmp-input-strict
test-qmp-input-visitor
qapi: New QMP command query-qmp-schema for QMP introspection qapi/introspect.json defines the introspection schema. It's designed for QMP introspection, but should do for similar uses, such as QGA. The introspection schema does not reflect all the rules and restrictions that apply to QAPI schemata. A valid QAPI schema has an introspection value conforming to the introspection schema, but the converse is not true. Introspection lowers away a number of schema details, and makes implicit things explicit: * The built-in types are declared with their JSON type. All integer types are mapped to 'int', because how many bits we use internally is an implementation detail. It could be pressed into external interface service as very approximate range information, but that's a bad idea. If we need range information, we better do it properly. * Implicit type definitions are made explicit, and given auto-generated names: - Array types, named by appending "List" to the name of their element type, like in generated C. - The enumeration types implicitly defined by simple union types, named by appending "Kind" to the name of their simple union type, like in generated C. - Types that don't occur in generated C. Their names start with ':' so they don't clash with the user's names. * All type references are by name. * The struct and union types are generalized into an object type. * Base types are flattened. * Commands take a single argument and return a single result. Dictionary argument or list result is an implicit type definition. The empty object type is used when a command takes no arguments or produces no results. The argument is always of object type, but the introspection schema doesn't reflect that. The 'gen': false directive is omitted as implementation detail. The 'success-response' directive is omitted as well for now, even though it's not an implementation detail, because it's not used by QMP. * Events carry a single data value. Implicit type definition and empty object type use, just like for commands. The value is of object type, but the introspection schema doesn't reflect that. * Types not used by commands or events are omitted. Indirect use counts as use. * Optional members have a default, which can only be null right now Instead of a mandatory "optional" flag, we have an optional default. No default means mandatory, default null means optional without default value. Non-null is available for optional with default (possible future extension). * Clients should *not* look up types by name, because type names are not ABI. Look up the command or event you're interested in, then follow the references. TODO Should we hide the type names to eliminate the temptation? New generator scripts/qapi-introspect.py computes an introspection value for its input, and generates a C variable holding it. It can generate awfully long lines. Marked TODO. A new test-qmp-input-visitor test case feeds its result for both tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a QmpInputVisitor to verify it actually conforms to the schema. New QMP command query-qmp-schema takes its return value from that variable. Its reply is some 85KiBytes for me right now. If this turns out to be too much, we have a couple of options: * We can use shorter names in the JSON. Not the QMP style. * Optionally return the sub-schema for commands and events given as arguments. Right now qmp_query_schema() sends the string literal computed by qmp-introspect.py. To compute sub-schema at run time, we'd have to duplicate parts of qapi-introspect.py in C. Unattractive. * Let clients cache the output of query-qmp-schema. It changes only on QEMU upgrades, i.e. rarely. Provide a command query-qmp-schema-hash. Clients can have a cache indexed by hash, and re-query the schema only when they don't have it cached. Even simpler: put the hash in the QMP greeting. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:28 +02:00
test-qmp-introspect.[ch]
test-qmp-marshal.c
test-qmp-output-visitor
test-rcu-list
test-replication
test-rfifolock
test-string-input-visitor
test-string-output-visitor
test-thread-pool
test-throttle
test-timed-average
test-uuid
test-visitor-serialization
test-vmstate
test-write-threshold
test-x86-cpuid
test-xbzrle
test-netfilter
test-filter-mirror
tests/test-filter-redirector: Add unit test for filter-redirector In this unit test,we will test the filter redirector function. Case 1, tx traffic flow: qemu side | test side | +---------+ | +-------+ | backend <---------------+ sock0 | +----+----+ | +-------+ | | +----v----+ +-------+ | | rd0 +->+chardev| | +---------+ +---+---+ | | | +---------+ | | | rd1 <------+ | +----+----+ | | | +----v----+ | +-------+ | rd2 +--------------->sock1 | +---------+ | +-------+ + a. we(sock0) inject packet to qemu socket backend b. backend pass packet to filter redirector0(rd0) c. rd0 redirect packet to out_dev(chardev) which is connected with filter redirector1's(rd1) in_dev d. rd1 read this packet from in_dev, and pass to next filter redirector2(rd2) e. rd2 redirect packet to rd2's out_dev which is connected with an opened socketed(sock1) f. we read packet from sock1 and compare to what we inject Start qemu with: "-netdev socket,id=qtest-bn0,fd=%d " "-device rtl8139,netdev=qtest-bn0,id=qtest-e0 " "-chardev socket,id=redirector0,path=%s,server,nowait " "-chardev socket,id=redirector1,path=%s,server,nowait " "-chardev socket,id=redirector2,path=%s,nowait " "-object filter-redirector,id=qtest-f0,netdev=qtest-bn0," "queue=tx,outdev=redirector0 " "-object filter-redirector,id=qtest-f1,netdev=qtest-bn0," "queue=tx,indev=redirector2 " "-object filter-redirector,id=qtest-f2,netdev=qtest-bn0," "queue=tx,outdev=redirector1 " -------------------------------------- Case 2, rx traffic flow qemu side | test side | +---------+ | +-------+ | backend +---------------> sock1 | +----^----+ | +-------+ | | +----+----+ +-------+ | | rd0 +<-+chardev| | +---------+ +---+---+ | ^ | +---------+ | | | rd1 +------+ | +----^----+ | | | +----+----+ | +-------+ | rd2 <---------------+sock0 | +---------+ | +-------+ a. we(sock0) insert packet to filter redirector2(rd2) b. rd2 pass packet to filter redirector1(rd1) c. rd1 redirect packet to out_dev(chardev) which is connected with filter redirector0's(rd0) in_dev d. rd0 read this packet from in_dev, and pass ti to qemu backend which is connected with an opened socketed(sock1) e. we read packet from sock1 and compare to what we inject Start qemu with: "-netdev socket,id=qtest-bn0,fd=%d " "-device rtl8139,netdev=qtest-bn0,id=qtest-e0 " "-chardev socket,id=redirector0,path=%s,server,nowait " "-chardev socket,id=redirector1,path=%s,server,nowait " "-chardev socket,id=redirector2,path=%s,nowait " "-object filter-redirector,id=qtest-f0,netdev=qtest-bn0," "queue=rx,outdev=redirector0 " "-object filter-redirector,id=qtest-f1,netdev=qtest-bn0," "queue=rx,indev=redirector2 " "-object filter-redirector,id=qtest-f2,netdev=qtest-bn0," "queue=rx,outdev=redirector1 " Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2016-03-17 09:16:27 +01:00
test-filter-redirector
*-test
qapi-schema/*.test.*