qemu-e2k/qemu-nbd.c

1228 lines
38 KiB
C
Raw Normal View History

/*
* Copyright (C) 2005 Anthony Liguori <anthony@codemonkey.ws>
*
* Network Block Device
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; under version 2 of the License.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, see <http://www.gnu.org/licenses/>.
*/
#include "qemu/osdep.h"
#include <getopt.h>
#include <libgen.h>
#include <pthread.h>
#include "qemu-common.h"
2016-03-14 09:01:28 +01:00
#include "qapi/error.h"
#include "qemu/cutils.h"
block: New BlockBackend A block device consists of a frontend device model and a backend. A block backend has a tree of block drivers doing the actual work. The tree is managed by the block layer. We currently use a single abstraction BlockDriverState both for tree nodes and the backend as a whole. Drawbacks: * Its API includes both stuff that makes sense only at the block backend level (root of the tree) and stuff that's only for use within the block layer. This makes the API bigger and more complex than necessary. Moreover, it's not obvious which interfaces are meant for device models, and which really aren't. * Since device models keep a reference to their backend, the backend object can't just be destroyed. But for media change, we need to replace the tree. Our solution is to make the BlockDriverState generic, with actual driver state in a separate object, pointed to by member opaque. That lets us replace the tree by deinitializing and reinitializing its root. This special need of the root makes the data structure awkward everywhere in the tree. The general plan is to separate the APIs into "block backend", for use by device models, monitor and whatever other code dealing with block backends, and "block driver", for use by the block layer and whatever other code (if any) dealing with trees and tree nodes. Code dealing with block backends, device models in particular, should become completely oblivious of BlockDriverState. This should let us clean up both APIs, and the tree data structures. This commit is a first step. It creates a minimal "block backend" API: type BlockBackend and functions to create, destroy and find them. BlockBackend objects are created and destroyed exactly when root BlockDriverState objects are created and destroyed. "Root" in the sense of "in bdrv_states". They're not yet used for anything; that'll come shortly. A root BlockDriverState is created with bdrv_new_root(), so where to create a BlockBackend is obvious. Where these roots get destroyed isn't always as obvious. It is obvious in qemu-img.c, qemu-io.c and qemu-nbd.c, and in error paths of blockdev_init(), blk_connect(). That leaves destruction of objects successfully created by blockdev_init() and blk_connect(). blockdev_init() is used only by drive_new() and qmp_blockdev_add(). Objects created by the latter are currently indestructible (see commit 48f364d "blockdev: Refuse to drive_del something added with blockdev-add" and commit 2d246f0 "blockdev: Introduce DriveInfo.enable_auto_del"). Objects created by the former get destroyed by drive_del(). Objects created by blk_connect() get destroyed by blk_disconnect(). BlockBackend is reference-counted. Its reference count never exceeds one so far, but that's going to change. In drive_del(), the BB's reference count is surely one now. The BDS's reference count is greater than one when something else is holding a reference, such as a block job. In this case, the BB is destroyed right away, but the BDS lives on until all extra references get dropped. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2014-10-07 13:59:04 +02:00
#include "sysemu/block-backend.h"
#include "block/block_int.h"
#include "block/nbd.h"
#include "qemu/main-loop.h"
#include "qemu/module.h"
#include "qemu/option.h"
#include "qemu/error-report.h"
#include "qemu/config-file.h"
#include "qemu/bswap.h"
#include "qemu/log.h"
#include "qemu/systemd.h"
#include "block/snapshot.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qstring.h"
#include "qom/object_interfaces.h"
#include "io/channel-socket.h"
#include "io/net-listener.h"
#include "crypto/init.h"
#include "trace/control.h"
#include "qemu-version.h"
#ifdef __linux__
#define HAVE_NBD_DEVICE 1
#else
#define HAVE_NBD_DEVICE 0
#endif
#define SOCKET_PATH "/var/lock/qemu-nbd-%s"
#define QEMU_NBD_OPT_CACHE 256
#define QEMU_NBD_OPT_AIO 257
#define QEMU_NBD_OPT_DISCARD 258
#define QEMU_NBD_OPT_DETECT_ZEROES 259
#define QEMU_NBD_OPT_OBJECT 260
#define QEMU_NBD_OPT_TLSCREDS 261
#define QEMU_NBD_OPT_IMAGE_OPTS 262
#define QEMU_NBD_OPT_FORK 263
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
#define QEMU_NBD_OPT_TLSAUTHZ 264
#define MBR_SIZE 512
static NBDExport *export;
static int verbose;
static char *srcpath;
static SocketAddress *saddr;
static int persistent = 0;
static enum { RUNNING, TERMINATE, TERMINATING, TERMINATED } state;
static int shared = 1;
static int nb_fds;
static QIONetListener *server;
static QCryptoTLSCreds *tlscreds;
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
static const char *tlsauthz;
static void usage(const char *name)
{
(printf) (
"Usage: %s [OPTIONS] FILE\n"
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
" or: %s -L [OPTIONS]\n"
"QEMU Disk Network Block Device Utility\n"
"\n"
" -h, --help display this help and exit\n"
" -V, --version output version information and exit\n"
"\n"
"Connection properties:\n"
" -p, --port=PORT port to listen on (default `%d')\n"
" -b, --bind=IFACE interface to bind to (default `0.0.0.0')\n"
" -k, --socket=PATH path to the unix socket\n"
" (default '"SOCKET_PATH"')\n"
" -e, --shared=NUM device can be shared by NUM clients (default '1')\n"
" -t, --persistent don't exit on the last connection\n"
" -v, --verbose display extra debugging information\n"
" -x, --export-name=NAME expose export by name (default is empty string)\n"
" -D, --description=TEXT export a human-readable description\n"
"\n"
"Exposing part of the image:\n"
" -o, --offset=OFFSET offset into the image\n"
" -P, --partition=NUM only expose partition NUM\n"
" -B, --bitmap=NAME expose a persistent dirty bitmap\n"
"\n"
"General purpose options:\n"
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
" -L, --list list exports available from another NBD server\n"
" --object type,id=ID,... define an object such as 'secret' for providing\n"
" passwords and/or encryption keys\n"
" --tls-creds=ID use id of an earlier --object to provide TLS\n"
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
" --tls-authz=ID use id of an earlier --object to provide\n"
" authorization\n"
" -T, --trace [[enable=]<pattern>][,events=<file>][,file=<file>]\n"
" specify tracing options\n"
" --fork fork off the server process and exit the parent\n"
" once the server is running\n"
#if HAVE_NBD_DEVICE
"\n"
"Kernel NBD client support:\n"
" -c, --connect=DEV connect FILE to the local NBD device DEV\n"
" -d, --disconnect disconnect the specified device\n"
#endif
"\n"
"Block device options:\n"
" -f, --format=FORMAT set image format (raw, qcow2, ...)\n"
" -r, --read-only export read-only\n"
" -s, --snapshot use FILE as an external snapshot, create a temporary\n"
" file with backing_file=FILE, redirect the write to\n"
" the temporary one\n"
" -l, --load-snapshot=SNAPSHOT_PARAM\n"
" load an internal snapshot inside FILE and export it\n"
" as an read-only device, SNAPSHOT_PARAM format is\n"
" 'snapshot.id=[ID],snapshot.name=[NAME]', or\n"
" '[ID_OR_NAME]'\n"
" -n, --nocache disable host cache\n"
" --cache=MODE set cache mode (none, writeback, ...)\n"
" --aio=MODE set AIO mode (native or threads)\n"
" --discard=MODE set discard mode (ignore, unmap)\n"
" --detect-zeroes=MODE set detect-zeroes mode (off, on, unmap)\n"
" --image-opts treat FILE as a full set of image options\n"
"\n"
QEMU_HELP_BOTTOM "\n"
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
, name, name, NBD_DEFAULT_PORT, "DEVICE");
}
static void version(const char *name)
{
printf(
"%s " QEMU_FULL_VERSION "\n"
"Written by Anthony Liguori.\n"
"\n"
QEMU_COPYRIGHT "\n"
"This is free software; see the source for copying conditions. There is NO\n"
"warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n"
, name);
}
struct partition_record
{
uint8_t bootable;
uint8_t start_head;
uint32_t start_cylinder;
uint8_t start_sector;
uint8_t system;
uint8_t end_head;
uint8_t end_cylinder;
uint8_t end_sector;
uint32_t start_sector_abs;
uint32_t nb_sectors_abs;
};
static void read_partition(uint8_t *p, struct partition_record *r)
{
r->bootable = p[0];
r->start_head = p[1];
r->start_cylinder = p[3] | ((p[2] << 2) & 0x0300);
r->start_sector = p[2] & 0x3f;
r->system = p[4];
r->end_head = p[5];
r->end_cylinder = p[7] | ((p[6] << 2) & 0x300);
r->end_sector = p[6] & 0x3f;
r->start_sector_abs = ldl_le_p(p + 8);
r->nb_sectors_abs = ldl_le_p(p + 12);
}
static int find_partition(BlockBackend *blk, int partition,
uint64_t *offset, uint64_t *size)
{
struct partition_record mbr[4];
uint8_t data[MBR_SIZE];
int i;
int ext_partnum = 4;
int ret;
ret = blk_pread(blk, 0, data, sizeof(data));
if (ret < 0) {
error_report("error while reading: %s", strerror(-ret));
exit(EXIT_FAILURE);
}
if (data[510] != 0x55 || data[511] != 0xaa) {
return -EINVAL;
}
for (i = 0; i < 4; i++) {
read_partition(&data[446 + 16 * i], &mbr[i]);
if (!mbr[i].system || !mbr[i].nb_sectors_abs) {
continue;
}
if (mbr[i].system == 0xF || mbr[i].system == 0x5) {
struct partition_record ext[4];
uint8_t data1[MBR_SIZE];
int j;
ret = blk_pread(blk, mbr[i].start_sector_abs * MBR_SIZE,
data1, sizeof(data1));
if (ret < 0) {
error_report("error while reading: %s", strerror(-ret));
exit(EXIT_FAILURE);
}
for (j = 0; j < 4; j++) {
read_partition(&data1[446 + 16 * j], &ext[j]);
if (!ext[j].system || !ext[j].nb_sectors_abs) {
continue;
}
if ((ext_partnum + j + 1) == partition) {
*offset = (uint64_t)ext[j].start_sector_abs << 9;
*size = (uint64_t)ext[j].nb_sectors_abs << 9;
return 0;
}
}
ext_partnum += 4;
} else if ((i + 1) == partition) {
*offset = (uint64_t)mbr[i].start_sector_abs << 9;
*size = (uint64_t)mbr[i].nb_sectors_abs << 9;
return 0;
}
}
return -ENOENT;
}
static void termsig_handler(int signum)
{
atomic_cmpxchg(&state, RUNNING, TERMINATE);
qemu_notify_event();
}
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
const char *hostname)
{
int ret = EXIT_FAILURE;
int rc;
Error *err = NULL;
QIOChannelSocket *sioc;
NBDExportInfo *list;
int i, j;
sioc = qio_channel_socket_new();
if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
error_report_err(err);
return EXIT_FAILURE;
}
rc = nbd_receive_export_list(QIO_CHANNEL(sioc), tls, hostname, &list,
&err);
if (rc < 0) {
if (err) {
error_report_err(err);
}
goto out;
}
printf("exports available: %d\n", rc);
for (i = 0; i < rc; i++) {
printf(" export: '%s'\n", list[i].name);
if (list[i].description && *list[i].description) {
printf(" description: %s\n", list[i].description);
}
if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
static const char *const flag_names[] = {
[NBD_FLAG_READ_ONLY_BIT] = "readonly",
[NBD_FLAG_SEND_FLUSH_BIT] = "flush",
[NBD_FLAG_SEND_FUA_BIT] = "fua",
[NBD_FLAG_ROTATIONAL_BIT] = "rotational",
[NBD_FLAG_SEND_TRIM_BIT] = "trim",
[NBD_FLAG_SEND_WRITE_ZEROES_BIT] = "zeroes",
[NBD_FLAG_SEND_DF_BIT] = "df",
[NBD_FLAG_CAN_MULTI_CONN_BIT] = "multi",
[NBD_FLAG_SEND_RESIZE_BIT] = "resize",
[NBD_FLAG_SEND_CACHE_BIT] = "cache",
};
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
printf(" size: %" PRIu64 "\n", list[i].size);
printf(" flags: 0x%x (", list[i].flags);
for (size_t bit = 0; bit < ARRAY_SIZE(flag_names); bit++) {
if (flag_names[bit] && (list[i].flags & (1 << bit))) {
printf(" %s", flag_names[bit]);
}
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
}
printf(" )\n");
}
if (list[i].min_block) {
printf(" min block: %u\n", list[i].min_block);
printf(" opt block: %u\n", list[i].opt_block);
printf(" max block: %u\n", list[i].max_block);
}
if (list[i].n_contexts) {
printf(" available meta contexts: %d\n", list[i].n_contexts);
for (j = 0; j < list[i].n_contexts; j++) {
printf(" %s\n", list[i].contexts[j]);
}
}
}
nbd_free_export_list(list, rc);
ret = EXIT_SUCCESS;
out:
object_unref(OBJECT(sioc));
return ret;
}
#if HAVE_NBD_DEVICE
static void *show_parts(void *arg)
{
char *device = arg;
int nbd;
/* linux just needs an open() to trigger
* the partition table update
* but remember to load the module with max_part != 0 :
* modprobe nbd max_part=63
*/
nbd = open(device, O_RDWR);
if (nbd >= 0) {
close(nbd);
}
return NULL;
}
static void *nbd_client_thread(void *arg)
{
char *device = arg;
NBDExportInfo info = { .request_sizes = false, .name = g_strdup("") };
QIOChannelSocket *sioc;
int fd;
int ret;
pthread_t show_parts_thread;
Error *local_error = NULL;
sioc = qio_channel_socket_new();
if (qio_channel_socket_connect_sync(sioc,
saddr,
&local_error) < 0) {
error_report_err(local_error);
goto out;
}
ret = nbd_receive_negotiate(QIO_CHANNEL(sioc),
NULL, NULL, NULL, &info, &local_error);
if (ret < 0) {
if (local_error) {
error_report_err(local_error);
}
goto out_socket;
}
fd = open(device, O_RDWR);
if (fd < 0) {
/* Linux-only, we can use %m in printf. */
error_report("Failed to open %s: %m", device);
goto out_socket;
}
ret = nbd_init(fd, sioc, &info, &local_error);
if (ret < 0) {
error_report_err(local_error);
goto out_fd;
}
/* update partition table */
pthread_create(&show_parts_thread, NULL, show_parts, device);
if (verbose) {
fprintf(stderr, "NBD device %s is now connected to %s\n",
device, srcpath);
} else {
/* Close stderr so that the qemu-nbd process exits. */
dup2(STDOUT_FILENO, STDERR_FILENO);
}
ret = nbd_client(fd);
if (ret) {
goto out_fd;
}
close(fd);
object_unref(OBJECT(sioc));
g_free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_SUCCESS;
out_fd:
close(fd);
out_socket:
object_unref(OBJECT(sioc));
out:
g_free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_FAILURE;
}
#endif /* HAVE_NBD_DEVICE */
static int nbd_can_accept(void)
{
return state == RUNNING && nb_fds < shared;
}
static void nbd_export_closed(NBDExport *export)
{
assert(state == TERMINATING);
state = TERMINATED;
}
static void nbd_update_server_watch(void);
nbd: Fix regression on resiliency to port scan Back in qemu 2.5, qemu-nbd was immune to port probes (a transient server would not quit, regardless of how many probe connections came and went, until a connection actually negotiated). But we broke that in commit ee7d7aa when removing the return value to nbd_client_new(), although that patch also introduced a bug causing an assertion failure on a client that fails negotiation. We then made it worse during refactoring in commit 1a6245a (a segfault before we could even assert); the (masked) assertion was cleaned up in d3780c2 (still in 2.6), and just recently we finally fixed the segfault ("nbd: Fully intialize client in case of failed negotiation"). But that still means that ever since we added TLS support to qemu-nbd, we have been vulnerable to an ill-timed port-scan being able to cause a denial of service by taking down qemu-nbd before a real client has a chance to connect. Since negotiation is now handled asynchronously via coroutines, we no longer have a synchronous point of return by re-adding a return value to nbd_client_new(). So this patch instead wires things up to pass the negotiation status through the close_fn callback function. Simple test across two terminals: $ qemu-nbd -f raw -p 30001 file $ nmap 127.0.0.1 -p 30001 && \ qemu-io -c 'r 0 512' -f raw nbd://localhost:30001 Note that this patch does not change what constitutes successful negotiation (thus, a client must enter transmission phase before that client can be considered as a reason to terminate the server when the connection ends). Perhaps we may want to tweak things in a later patch to also treat a client that uses NBD_OPT_ABORT as being a 'successful' negotiation (the client correctly talked the NBD protocol, and informed us it was not going to use our export after all), but that's a discussion for another day. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1451614 Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20170608222617.20376-1-eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-09 00:26:17 +02:00
static void nbd_client_closed(NBDClient *client, bool negotiated)
{
nb_fds--;
nbd: Fix regression on resiliency to port scan Back in qemu 2.5, qemu-nbd was immune to port probes (a transient server would not quit, regardless of how many probe connections came and went, until a connection actually negotiated). But we broke that in commit ee7d7aa when removing the return value to nbd_client_new(), although that patch also introduced a bug causing an assertion failure on a client that fails negotiation. We then made it worse during refactoring in commit 1a6245a (a segfault before we could even assert); the (masked) assertion was cleaned up in d3780c2 (still in 2.6), and just recently we finally fixed the segfault ("nbd: Fully intialize client in case of failed negotiation"). But that still means that ever since we added TLS support to qemu-nbd, we have been vulnerable to an ill-timed port-scan being able to cause a denial of service by taking down qemu-nbd before a real client has a chance to connect. Since negotiation is now handled asynchronously via coroutines, we no longer have a synchronous point of return by re-adding a return value to nbd_client_new(). So this patch instead wires things up to pass the negotiation status through the close_fn callback function. Simple test across two terminals: $ qemu-nbd -f raw -p 30001 file $ nmap 127.0.0.1 -p 30001 && \ qemu-io -c 'r 0 512' -f raw nbd://localhost:30001 Note that this patch does not change what constitutes successful negotiation (thus, a client must enter transmission phase before that client can be considered as a reason to terminate the server when the connection ends). Perhaps we may want to tweak things in a later patch to also treat a client that uses NBD_OPT_ABORT as being a 'successful' negotiation (the client correctly talked the NBD protocol, and informed us it was not going to use our export after all), but that's a discussion for another day. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1451614 Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20170608222617.20376-1-eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-09 00:26:17 +02:00
if (negotiated && nb_fds == 0 && !persistent && state == RUNNING) {
state = TERMINATE;
}
nbd_update_server_watch();
nbd_client_put(client);
}
static void nbd_accept(QIONetListener *listener, QIOChannelSocket *cioc,
gpointer opaque)
{
if (state >= TERMINATE) {
return;
}
nb_fds++;
nbd_update_server_watch();
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
nbd_client_new(cioc, tlscreds, tlsauthz, nbd_client_closed);
}
static void nbd_update_server_watch(void)
{
if (nbd_can_accept()) {
qio_net_listener_set_client_func(server, nbd_accept, NULL, NULL);
} else {
qio_net_listener_set_client_func(server, NULL, NULL, NULL);
}
}
static SocketAddress *nbd_build_socket_address(const char *sockpath,
const char *bindto,
const char *port)
{
SocketAddress *saddr;
saddr = g_new0(SocketAddress, 1);
if (sockpath) {
saddr->type = SOCKET_ADDRESS_TYPE_UNIX;
saddr->u.q_unix.path = g_strdup(sockpath);
} else {
InetSocketAddress *inet;
saddr->type = SOCKET_ADDRESS_TYPE_INET;
inet = &saddr->u.inet;
inet->host = g_strdup(bindto);
if (port) {
inet->port = g_strdup(port);
} else {
inet->port = g_strdup_printf("%d", NBD_DEFAULT_PORT);
}
}
return saddr;
}
static QemuOptsList file_opts = {
.name = "file",
.implied_opt_name = "file",
.head = QTAILQ_HEAD_INITIALIZER(file_opts.head),
.desc = {
/* no elements => accept any params */
{ /* end of list */ }
},
};
static QemuOptsList qemu_object_opts = {
.name = "object",
.implied_opt_name = "qom-type",
.head = QTAILQ_HEAD_INITIALIZER(qemu_object_opts.head),
.desc = {
{ }
},
};
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, bool list,
Error **errp)
{
Object *obj;
QCryptoTLSCreds *creds;
obj = object_resolve_path_component(
object_get_objects_root(), id);
if (!obj) {
error_setg(errp, "No TLS credentials with id '%s'",
id);
return NULL;
}
creds = (QCryptoTLSCreds *)
object_dynamic_cast(obj, TYPE_QCRYPTO_TLS_CREDS);
if (!creds) {
error_setg(errp, "Object with id '%s' is not TLS credentials",
id);
return NULL;
}
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
if (list) {
if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
error_setg(errp,
"Expecting TLS credentials with a client endpoint");
return NULL;
}
} else {
if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
error_setg(errp,
"Expecting TLS credentials with a server endpoint");
return NULL;
}
}
object_ref(obj);
return creds;
}
static void setup_address_and_port(const char **address, const char **port)
{
if (*address == NULL) {
*address = "0.0.0.0";
}
if (*port == NULL) {
*port = stringify(NBD_DEFAULT_PORT);
}
}
/*
* Check socket parameters compatibility when socket activation is used.
*/
static const char *socket_activation_validate_opts(const char *device,
const char *sockpath,
const char *address,
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
const char *port,
bool list)
{
if (device != NULL) {
return "NBD device can't be set when using socket activation";
}
if (sockpath != NULL) {
return "Unix socket can't be set when using socket activation";
}
if (address != NULL) {
return "The interface can't be set when using socket activation";
}
if (port != NULL) {
return "TCP port number can't be set when using socket activation";
}
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
if (list) {
return "List mode is incompatible with socket activation";
}
return NULL;
}
static void qemu_nbd_shutdown(void)
{
job_cancel_sync_all();
bdrv_close_all();
}
int main(int argc, char **argv)
{
block: New BlockBackend A block device consists of a frontend device model and a backend. A block backend has a tree of block drivers doing the actual work. The tree is managed by the block layer. We currently use a single abstraction BlockDriverState both for tree nodes and the backend as a whole. Drawbacks: * Its API includes both stuff that makes sense only at the block backend level (root of the tree) and stuff that's only for use within the block layer. This makes the API bigger and more complex than necessary. Moreover, it's not obvious which interfaces are meant for device models, and which really aren't. * Since device models keep a reference to their backend, the backend object can't just be destroyed. But for media change, we need to replace the tree. Our solution is to make the BlockDriverState generic, with actual driver state in a separate object, pointed to by member opaque. That lets us replace the tree by deinitializing and reinitializing its root. This special need of the root makes the data structure awkward everywhere in the tree. The general plan is to separate the APIs into "block backend", for use by device models, monitor and whatever other code dealing with block backends, and "block driver", for use by the block layer and whatever other code (if any) dealing with trees and tree nodes. Code dealing with block backends, device models in particular, should become completely oblivious of BlockDriverState. This should let us clean up both APIs, and the tree data structures. This commit is a first step. It creates a minimal "block backend" API: type BlockBackend and functions to create, destroy and find them. BlockBackend objects are created and destroyed exactly when root BlockDriverState objects are created and destroyed. "Root" in the sense of "in bdrv_states". They're not yet used for anything; that'll come shortly. A root BlockDriverState is created with bdrv_new_root(), so where to create a BlockBackend is obvious. Where these roots get destroyed isn't always as obvious. It is obvious in qemu-img.c, qemu-io.c and qemu-nbd.c, and in error paths of blockdev_init(), blk_connect(). That leaves destruction of objects successfully created by blockdev_init() and blk_connect(). blockdev_init() is used only by drive_new() and qmp_blockdev_add(). Objects created by the latter are currently indestructible (see commit 48f364d "blockdev: Refuse to drive_del something added with blockdev-add" and commit 2d246f0 "blockdev: Introduce DriveInfo.enable_auto_del"). Objects created by the former get destroyed by drive_del(). Objects created by blk_connect() get destroyed by blk_disconnect(). BlockBackend is reference-counted. Its reference count never exceeds one so far, but that's going to change. In drive_del(), the BB's reference count is surely one now. The BDS's reference count is greater than one when something else is holding a reference, such as a block job. In this case, the BB is destroyed right away, but the BDS lives on until all extra references get dropped. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2014-10-07 13:59:04 +02:00
BlockBackend *blk;
BlockDriverState *bs;
uint64_t dev_offset = 0;
uint16_t nbdflags = 0;
bool disconnect = false;
const char *bindto = NULL;
const char *port = NULL;
char *sockpath = NULL;
char *device = NULL;
int64_t fd_size;
QemuOpts *sn_opts = NULL;
const char *sn_id_or_name = NULL;
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:B:L";
struct option lopt[] = {
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, 'V' },
{ "bind", required_argument, NULL, 'b' },
{ "port", required_argument, NULL, 'p' },
{ "socket", required_argument, NULL, 'k' },
{ "offset", required_argument, NULL, 'o' },
{ "read-only", no_argument, NULL, 'r' },
{ "partition", required_argument, NULL, 'P' },
{ "bitmap", required_argument, NULL, 'B' },
{ "connect", required_argument, NULL, 'c' },
{ "disconnect", no_argument, NULL, 'd' },
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
{ "list", no_argument, NULL, 'L' },
{ "snapshot", no_argument, NULL, 's' },
{ "load-snapshot", required_argument, NULL, 'l' },
{ "nocache", no_argument, NULL, 'n' },
{ "cache", required_argument, NULL, QEMU_NBD_OPT_CACHE },
{ "aio", required_argument, NULL, QEMU_NBD_OPT_AIO },
{ "discard", required_argument, NULL, QEMU_NBD_OPT_DISCARD },
{ "detect-zeroes", required_argument, NULL,
QEMU_NBD_OPT_DETECT_ZEROES },
{ "shared", required_argument, NULL, 'e' },
{ "format", required_argument, NULL, 'f' },
{ "persistent", no_argument, NULL, 't' },
{ "verbose", no_argument, NULL, 'v' },
{ "object", required_argument, NULL, QEMU_NBD_OPT_OBJECT },
{ "export-name", required_argument, NULL, 'x' },
{ "description", required_argument, NULL, 'D' },
{ "tls-creds", required_argument, NULL, QEMU_NBD_OPT_TLSCREDS },
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
{ "tls-authz", required_argument, NULL, QEMU_NBD_OPT_TLSAUTHZ },
{ "image-opts", no_argument, NULL, QEMU_NBD_OPT_IMAGE_OPTS },
{ "trace", required_argument, NULL, 'T' },
{ "fork", no_argument, NULL, QEMU_NBD_OPT_FORK },
{ NULL, 0, NULL, 0 }
};
int ch;
int opt_ind = 0;
int flags = BDRV_O_RDWR;
int partition = 0;
int ret = 0;
bool seen_cache = false;
bool seen_discard = false;
bool seen_aio = false;
pthread_t client_thread;
const char *fmt = NULL;
Error *local_err = NULL;
BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
QDict *options = NULL;
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
const char *export_name = NULL; /* defaults to "" later for server mode */
const char *export_description = NULL;
const char *bitmap = NULL;
const char *tlscredsid = NULL;
bool imageOpts = false;
bool writethrough = true;
char *trace_file = NULL;
bool fork_process = false;
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
bool list = false;
int old_stderr = -1;
unsigned socket_activation;
/* The client thread uses SIGTERM to interrupt the server. A signal
* handler ensures that "qemu-nbd -v -c" exits with a nice status code.
*/
struct sigaction sa_sigterm;
memset(&sa_sigterm, 0, sizeof(sa_sigterm));
sa_sigterm.sa_handler = termsig_handler;
sigaction(SIGTERM, &sa_sigterm, NULL);
#ifdef CONFIG_POSIX
signal(SIGPIPE, SIG_IGN);
#endif
error_init(argv[0]);
module_call_init(MODULE_INIT_TRACE);
qcrypto_init(&error_fatal);
module_call_init(MODULE_INIT_QOM);
qemu_add_opts(&qemu_object_opts);
qemu_add_opts(&qemu_trace_opts);
qemu_init_exec_dir(argv[0]);
while ((ch = getopt_long(argc, argv, sopt, lopt, &opt_ind)) != -1) {
switch (ch) {
case 's':
flags |= BDRV_O_SNAPSHOT;
break;
case 'n':
optarg = (char *) "none";
/* fallthrough */
case QEMU_NBD_OPT_CACHE:
if (seen_cache) {
error_report("-n and --cache can only be specified once");
exit(EXIT_FAILURE);
}
seen_cache = true;
if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) == -1) {
error_report("Invalid cache mode `%s'", optarg);
exit(EXIT_FAILURE);
}
break;
case QEMU_NBD_OPT_AIO:
if (seen_aio) {
error_report("--aio can only be specified once");
exit(EXIT_FAILURE);
}
seen_aio = true;
if (!strcmp(optarg, "native")) {
flags |= BDRV_O_NATIVE_AIO;
} else if (!strcmp(optarg, "threads")) {
/* this is the default */
} else {
error_report("invalid aio mode `%s'", optarg);
exit(EXIT_FAILURE);
}
break;
case QEMU_NBD_OPT_DISCARD:
if (seen_discard) {
error_report("--discard can only be specified once");
exit(EXIT_FAILURE);
}
seen_discard = true;
if (bdrv_parse_discard_flags(optarg, &flags) == -1) {
error_report("Invalid discard mode `%s'", optarg);
exit(EXIT_FAILURE);
}
break;
case QEMU_NBD_OPT_DETECT_ZEROES:
detect_zeroes =
qapi_enum_parse(&BlockdevDetectZeroesOptions_lookup,
optarg,
BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF,
&local_err);
if (local_err) {
error_reportf_err(local_err,
"Failed to parse detect_zeroes mode: ");
exit(EXIT_FAILURE);
}
if (detect_zeroes == BLOCKDEV_DETECT_ZEROES_OPTIONS_UNMAP &&
!(flags & BDRV_O_UNMAP)) {
error_report("setting detect-zeroes to unmap is not allowed "
"without setting discard operation to unmap");
exit(EXIT_FAILURE);
}
break;
case 'b':
bindto = optarg;
break;
case 'p':
port = optarg;
break;
case 'o':
if (qemu_strtou64(optarg, NULL, 0, &dev_offset) < 0) {
error_report("Invalid offset '%s'", optarg);
exit(EXIT_FAILURE);
}
break;
case 'l':
if (strstart(optarg, SNAPSHOT_OPT_BASE, NULL)) {
QemuOpts: Wean off qerror_report_err() qerror_report_err() is a transitional interface to help with converting existing monitor commands to QMP. It should not be used elsewhere. The only remaining user in qemu-option.c is qemu_opts_parse(). Is it used in QMP context? If not, we can simply replace qerror_report_err() by error_report_err(). The uses in qemu-img.c, qemu-io.c, qemu-nbd.c and under tests/ are clearly not in QMP context. The uses in vl.c aren't either, because the only QMP command handlers there are qmp_query_status() and qmp_query_machines(), and they don't call it. Remaining uses: * drive_def(): Command line -drive and such, HMP drive_add and pci_add * hmp_chardev_add(): HMP chardev-add * monitor_parse_command(): HMP core * tmp_config_parse(): Command line -tpmdev * net_host_device_add(): HMP host_net_add * net_client_parse(): Command line -net and -netdev * qemu_global_option(): Command line -global * vnc_parse_func(): Command line -display, -vnc, default display, HMP change, QMP change. Bummer. * qemu_pci_hot_add_nic(): HMP pci_add * usb_net_init(): Command line -usbdevice, HMP usb_add Propagate errors through qemu_opts_parse(). Create a convenience function qemu_opts_parse_noisily() that passes errors to error_report_err(). Switch all non-QMP users outside tests to it. That leaves vnc_parse_func(). Propagate errors through it. Since I'm touching it anyway, rename it to vnc_parse(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
2015-02-13 12:50:26 +01:00
sn_opts = qemu_opts_parse_noisily(&internal_snapshot_opts,
optarg, false);
if (!sn_opts) {
error_report("Failed in parsing snapshot param `%s'",
optarg);
exit(EXIT_FAILURE);
}
} else {
sn_id_or_name = optarg;
}
/* fall through */
case 'r':
nbdflags |= NBD_FLAG_READ_ONLY;
flags &= ~BDRV_O_RDWR;
break;
case 'P':
qemu-nbd: Deprecate qemu-nbd --partition The existing qemu-nbd --partition code claims to handle logical partitions up to 8, since its introduction in 2008 (commit 7a5ca86). However, the implementation is bogus (actual MBR logical partitions form a sort of linked list, with one partition per extended table entry, rather than four logical partitions in a single extended table), making the code unlikely to work for anything beyond -P5 on actual guest images. What's more, the code does not support GPT partitions, which are becoming more popular, and maintaining device subsetting in both NBD and the raw device is unnecessary duplication of effort (even if it is not too difficult). Note that obtaining the offsets of a partition (MBR or GPT) can be learned by using 'qemu-nbd -c /dev/nbd0 file.qcow2 && sfdisk --dump /dev/nbd0', but by the time you've done that, you might as well just mount /dev/nbd0p1 that the kernel creates for you instead of bothering with qemu exporting a subset. Or, keeping to just user-space code, use nbdkit's partition filter, which has already known both GPT and primary MBR partitions for a while, and was just recently enhanced to support arbitrary logical MBR parititions. Start the clock on the deprecation cycle, with examples of how to accomplish device subsetting without using -P. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190125234837.2272-1-eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
2019-01-26 00:48:37 +01:00
warn_report("The '-P' option is deprecated; use --image-opts with "
"a raw device wrapper for subset exports instead");
if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
partition < 1 || partition > 8) {
error_report("Invalid partition '%s'", optarg);
exit(EXIT_FAILURE);
}
break;
case 'B':
bitmap = optarg;
break;
case 'k':
sockpath = optarg;
if (sockpath[0] != '/') {
error_report("socket path must be absolute");
exit(EXIT_FAILURE);
}
break;
case 'd':
disconnect = true;
break;
case 'c':
device = optarg;
break;
case 'e':
if (qemu_strtoi(optarg, NULL, 0, &shared) < 0 ||
shared < 1) {
error_report("Invalid shared device number '%s'", optarg);
exit(EXIT_FAILURE);
}
break;
case 'f':
fmt = optarg;
break;
case 't':
persistent = 1;
break;
case 'x':
export_name = optarg;
break;
case 'D':
export_description = optarg;
break;
case 'v':
verbose = 1;
break;
case 'V':
version(argv[0]);
exit(0);
break;
case 'h':
usage(argv[0]);
exit(0);
break;
case '?':
error_report("Try `%s --help' for more information.", argv[0]);
exit(EXIT_FAILURE);
case QEMU_NBD_OPT_OBJECT: {
QemuOpts *opts;
opts = qemu_opts_parse_noisily(&qemu_object_opts,
optarg, true);
if (!opts) {
exit(EXIT_FAILURE);
}
} break;
case QEMU_NBD_OPT_TLSCREDS:
tlscredsid = optarg;
break;
case QEMU_NBD_OPT_IMAGE_OPTS:
imageOpts = true;
break;
case 'T':
g_free(trace_file);
trace_file = trace_opt_parse(optarg);
break;
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
case QEMU_NBD_OPT_TLSAUTHZ:
tlsauthz = optarg;
break;
case QEMU_NBD_OPT_FORK:
fork_process = true;
break;
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
case 'L':
list = true;
break;
}
}
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
if (list) {
if (argc != optind) {
error_report("List mode is incompatible with a file name");
exit(EXIT_FAILURE);
}
if (export_name || export_description || dev_offset || partition ||
device || disconnect || fmt || sn_id_or_name || bitmap ||
seen_aio || seen_discard || seen_cache) {
error_report("List mode is incompatible with per-device settings");
exit(EXIT_FAILURE);
}
if (fork_process) {
error_report("List mode is incompatible with forking");
exit(EXIT_FAILURE);
}
} else if ((argc - optind) != 1) {
error_report("Invalid number of arguments");
error_printf("Try `%s --help' for more information.\n", argv[0]);
exit(EXIT_FAILURE);
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
} else if (!export_name) {
export_name = "";
}
qemu_opts_foreach(&qemu_object_opts,
user_creatable_add_opts_foreach,
NULL, &error_fatal);
if (!trace_init_backends()) {
exit(1);
}
trace_init_file(trace_file);
qemu_set_log(LOG_TRACE);
socket_activation = check_socket_activation();
if (socket_activation == 0) {
setup_address_and_port(&bindto, &port);
} else {
/* Using socket activation - check user didn't use -p etc. */
const char *err_msg = socket_activation_validate_opts(device, sockpath,
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
bindto, port,
list);
if (err_msg != NULL) {
error_report("%s", err_msg);
exit(EXIT_FAILURE);
}
/* qemu-nbd can only listen on a single socket. */
if (socket_activation > 1) {
error_report("qemu-nbd does not support socket activation with %s > 1",
"LISTEN_FDS");
exit(EXIT_FAILURE);
}
}
if (tlscredsid) {
if (sockpath) {
error_report("TLS is only supported with IPv4/IPv6");
exit(EXIT_FAILURE);
}
if (device) {
error_report("TLS is not supported with a host device");
exit(EXIT_FAILURE);
}
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
if (tlsauthz && list) {
error_report("TLS authorization is incompatible with export list");
exit(EXIT_FAILURE);
}
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
tlscreds = nbd_get_tls_creds(tlscredsid, list, &local_err);
if (local_err) {
error_report("Failed to get TLS creds %s",
error_get_pretty(local_err));
exit(EXIT_FAILURE);
}
qemu-nbd: add support for authorization of TLS clients Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-27 17:20:33 +01:00
} else {
if (tlsauthz) {
error_report("--tls-authz is not permitted without --tls-creds");
exit(EXIT_FAILURE);
}
}
qemu-nbd: Add --list option We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstream NBD project to list exports, but it's annoying to rely on out-of-tree binaries; furthermore, nbd-client doesn't necessarily know about all of the qemu NBD extensions. Thus, it is time to add a new mode to qemu-nbd that merely sniffs all possible information from the server during handshake phase, then disconnects and dumps the information. This patch actually implements --list/-L, while reusing other options such as --tls-creds for now designating how to connect as the client (rather than their non-list usage of how to operate as the server). I debated about adding this functionality to something akin to 'qemu-img info' - but that tool does not readily lend itself to connecting to an arbitrary NBD server without also tying to a specific export (I may, however, still add ImageInfoSpecificNBD for reporting the bitmaps available when connecting to a single export). And, while it may feel a bit odd that normally qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not really making the qemu-nbd binary that much larger, because 'qemu-nbd -c' has to operate as both server and client simultaneously across two threads when feeding the kernel module for /dev/nbdN access. Sample output: $ qemu-nbd -L exports available: 1 export: '' size: 65536 flags: 0x4ed ( flush fua trim zeroes df cache ) min block: 512 opt block: 4096 max block: 33554432 available meta contexts: 1 base:allocation Note that the output only lists sizes if the server sent NBD_FLAG_HAS_FLAGS, because a newstyle server does not give the size otherwise. It has the side effect that for really old servers that did not send any flags, the size is not output even though it was available. However, I'm not too concerned about that - oldstyle servers are (rightfully) getting less common to encounter (qemu 3.0 was the last version where we even serve it), and most existing servers that still even offer oldstyle negotiation (such as nbdkit) still send flags (since that was added to the NBD protocol in 2007 to permit read-only connections). Not done here, but maybe worth future experiments: capture the meat of NBDExportInfo into a QAPI struct, and use the generated QAPI pretty-printers instead of hand-rolling our output loop. It would also permit us to add a JSON output mode for machine parsing. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-17 20:36:56 +01:00
if (list) {
saddr = nbd_build_socket_address(sockpath, bindto, port);
return qemu_nbd_client_list(saddr, tlscreds, bindto);
}
#if !HAVE_NBD_DEVICE
if (disconnect || device) {
error_report("Kernel /dev/nbdN support not available");
exit(EXIT_FAILURE);
}
#else /* HAVE_NBD_DEVICE */
if (disconnect) {
int nbdfd = open(argv[optind], O_RDWR);
if (nbdfd < 0) {
error_report("Cannot open %s: %s", argv[optind],
strerror(errno));
exit(EXIT_FAILURE);
}
nbd_disconnect(nbdfd);
close(nbdfd);
printf("%s disconnected\n", argv[optind]);
return 0;
}
#endif
if ((device && !verbose) || fork_process) {
int stderr_fd[2];
pid_t pid;
int ret;
if (qemu_pipe(stderr_fd) < 0) {
error_report("Error setting up communication pipe: %s",
strerror(errno));
exit(EXIT_FAILURE);
}
/* Now daemonize, but keep a communication channel open to
* print errors and exit with the proper status code.
*/
pid = fork();
if (pid < 0) {
error_report("Failed to fork: %s", strerror(errno));
exit(EXIT_FAILURE);
} else if (pid == 0) {
close(stderr_fd[0]);
ret = qemu_daemon(1, 0);
/* Temporarily redirect stderr to the parent's pipe... */
old_stderr = dup(STDERR_FILENO);
dup2(stderr_fd[1], STDERR_FILENO);
if (ret < 0) {
error_report("Failed to daemonize: %s", strerror(errno));
exit(EXIT_FAILURE);
}
/* ... close the descriptor we inherited and go on. */
close(stderr_fd[1]);
} else {
bool errors = false;
char *buf;
/* In the parent. Print error messages from the child until
* it closes the pipe.
*/
close(stderr_fd[1]);
buf = g_malloc(1024);
while ((ret = read(stderr_fd[0], buf, 1024)) > 0) {
errors = true;
ret = qemu_write_full(STDERR_FILENO, buf, ret);
if (ret < 0) {
exit(EXIT_FAILURE);
}
}
if (ret < 0) {
error_report("Cannot read from daemon: %s",
strerror(errno));
exit(EXIT_FAILURE);
}
/* Usually the daemon should not print any message.
* Exit with zero status in that case.
*/
exit(errors);
}
}
if (device != NULL && sockpath == NULL) {
sockpath = g_malloc(128);
snprintf(sockpath, 128, SOCKET_PATH, basename(device));
}
server = qio_net_listener_new();
if (socket_activation == 0) {
saddr = nbd_build_socket_address(sockpath, bindto, port);
if (qio_net_listener_open_sync(server, saddr, &local_err) < 0) {
object_unref(OBJECT(server));
error_report_err(local_err);
exit(EXIT_FAILURE);
}
} else {
size_t i;
/* See comment in check_socket_activation above. */
for (i = 0; i < socket_activation; i++) {
QIOChannelSocket *sioc;
sioc = qio_channel_socket_new_fd(FIRST_SOCKET_ACTIVATION_FD + i,
&local_err);
if (sioc == NULL) {
object_unref(OBJECT(server));
error_report("Failed to use socket activation: %s",
error_get_pretty(local_err));
exit(EXIT_FAILURE);
}
qio_net_listener_add(server, sioc);
object_unref(OBJECT(sioc));
}
}
if (qemu_init_main_loop(&local_err)) {
error_report_err(local_err);
exit(EXIT_FAILURE);
}
bdrv_init();
atexit(qemu_nbd_shutdown);
srcpath = argv[optind];
if (imageOpts) {
QemuOpts *opts;
if (fmt) {
error_report("--image-opts and -f are mutually exclusive");
exit(EXIT_FAILURE);
}
opts = qemu_opts_parse_noisily(&file_opts, srcpath, true);
if (!opts) {
qemu_opts_reset(&file_opts);
exit(EXIT_FAILURE);
}
options = qemu_opts_to_qdict(opts, NULL);
qemu_opts_reset(&file_opts);
blk = blk_new_open(NULL, NULL, options, flags, &local_err);
} else {
if (fmt) {
options = qdict_new();
qdict_put_str(options, "driver", fmt);
}
blk = blk_new_open(srcpath, NULL, options, flags, &local_err);
}
if (!blk) {
error_reportf_err(local_err, "Failed to blk_new_open '%s': ",
argv[optind]);
exit(EXIT_FAILURE);
}
bs = blk_bs(blk);
blk_set_enable_write_cache(blk, !writethrough);
if (sn_opts) {
ret = bdrv_snapshot_load_tmp(bs,
qemu_opt_get(sn_opts, SNAPSHOT_OPT_ID),
qemu_opt_get(sn_opts, SNAPSHOT_OPT_NAME),
&local_err);
} else if (sn_id_or_name) {
ret = bdrv_snapshot_load_tmp_by_id_or_name(bs, sn_id_or_name,
&local_err);
}
if (ret < 0) {
error_reportf_err(local_err, "Failed to load snapshot: ");
exit(EXIT_FAILURE);
}
bs->detect_zeroes = detect_zeroes;
fd_size = blk_getlength(blk);
if (fd_size < 0) {
error_report("Failed to determine the image length: %s",
strerror(-fd_size));
exit(EXIT_FAILURE);
}
if (dev_offset >= fd_size) {
error_report("Offset (%" PRIu64 ") has to be smaller than the image "
"size (%" PRId64 ")", dev_offset, fd_size);
exit(EXIT_FAILURE);
}
fd_size -= dev_offset;
if (partition) {
uint64_t limit;
qemu-nbd: Sanity check partition bounds When the user requests a partition, we were using data read from the disk as disk offsets without a bounds check. We got lucky that even when computed offsets are out-of-bounds, blk_pread() will gracefully catch the error later (so I don't think a malicious image can crash or exploit qemu-nbd, and am not treating this as a security flaw), but it's better to flag the problem up front than to risk permanent EIO death of the block device down the road. The new bounds check adds an assertion that will never fail, but rather exists to help the compiler see that adding two positive 41-bit values (given MBR constraints) can't overflow 64-bit off_t. Using off_t to represent a partition length is a bit of a misnomer; a later patch will update to saner types, but it is left separate in case the bounds check needs to be backported in isolation. Also, note that the partition code blindly overwrites any non-zero offset passed in by the user; so for now, make the -o/-P combo an error for less confusion. In the future, we may let -o and -P work together (selecting a subset of a partition); so it is okay that an explicit '-o 0' behaves no differently from omitting -o. This can be tested with nbdkit: $ echo hi > file $ nbdkit -fv --filter=truncate partitioning file truncate=64k Pre-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 & $ qemu-io -f raw nbd://localhost:10810 qemu-io> r -v 0 1 Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error Connection closed read failed: Input/output error qemu-io> q [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 Post-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536 Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-5-eblake@redhat.com>
2019-01-17 20:36:41 +01:00
if (dev_offset) {
error_report("Cannot request partition and offset together");
exit(EXIT_FAILURE);
}
ret = find_partition(blk, partition, &dev_offset, &limit);
if (ret < 0) {
error_report("Could not find partition %d: %s", partition,
strerror(-ret));
exit(EXIT_FAILURE);
}
qemu-nbd: Sanity check partition bounds When the user requests a partition, we were using data read from the disk as disk offsets without a bounds check. We got lucky that even when computed offsets are out-of-bounds, blk_pread() will gracefully catch the error later (so I don't think a malicious image can crash or exploit qemu-nbd, and am not treating this as a security flaw), but it's better to flag the problem up front than to risk permanent EIO death of the block device down the road. The new bounds check adds an assertion that will never fail, but rather exists to help the compiler see that adding two positive 41-bit values (given MBR constraints) can't overflow 64-bit off_t. Using off_t to represent a partition length is a bit of a misnomer; a later patch will update to saner types, but it is left separate in case the bounds check needs to be backported in isolation. Also, note that the partition code blindly overwrites any non-zero offset passed in by the user; so for now, make the -o/-P combo an error for less confusion. In the future, we may let -o and -P work together (selecting a subset of a partition); so it is okay that an explicit '-o 0' behaves no differently from omitting -o. This can be tested with nbdkit: $ echo hi > file $ nbdkit -fv --filter=truncate partitioning file truncate=64k Pre-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 & $ qemu-io -f raw nbd://localhost:10810 qemu-io> r -v 0 1 Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error Connection closed read failed: Input/output error qemu-io> q [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 Post-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536 Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-5-eblake@redhat.com>
2019-01-17 20:36:41 +01:00
/*
* MBR partition limits are (32-bit << 9); this assert lets
* the compiler know that we can't overflow 64 bits.
qemu-nbd: Sanity check partition bounds When the user requests a partition, we were using data read from the disk as disk offsets without a bounds check. We got lucky that even when computed offsets are out-of-bounds, blk_pread() will gracefully catch the error later (so I don't think a malicious image can crash or exploit qemu-nbd, and am not treating this as a security flaw), but it's better to flag the problem up front than to risk permanent EIO death of the block device down the road. The new bounds check adds an assertion that will never fail, but rather exists to help the compiler see that adding two positive 41-bit values (given MBR constraints) can't overflow 64-bit off_t. Using off_t to represent a partition length is a bit of a misnomer; a later patch will update to saner types, but it is left separate in case the bounds check needs to be backported in isolation. Also, note that the partition code blindly overwrites any non-zero offset passed in by the user; so for now, make the -o/-P combo an error for less confusion. In the future, we may let -o and -P work together (selecting a subset of a partition); so it is okay that an explicit '-o 0' behaves no differently from omitting -o. This can be tested with nbdkit: $ echo hi > file $ nbdkit -fv --filter=truncate partitioning file truncate=64k Pre-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 & $ qemu-io -f raw nbd://localhost:10810 qemu-io> r -v 0 1 Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error Connection closed read failed: Input/output error qemu-io> q [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 Post-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536 Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-5-eblake@redhat.com>
2019-01-17 20:36:41 +01:00
*/
assert(dev_offset + limit >= dev_offset);
qemu-nbd: Sanity check partition bounds When the user requests a partition, we were using data read from the disk as disk offsets without a bounds check. We got lucky that even when computed offsets are out-of-bounds, blk_pread() will gracefully catch the error later (so I don't think a malicious image can crash or exploit qemu-nbd, and am not treating this as a security flaw), but it's better to flag the problem up front than to risk permanent EIO death of the block device down the road. The new bounds check adds an assertion that will never fail, but rather exists to help the compiler see that adding two positive 41-bit values (given MBR constraints) can't overflow 64-bit off_t. Using off_t to represent a partition length is a bit of a misnomer; a later patch will update to saner types, but it is left separate in case the bounds check needs to be backported in isolation. Also, note that the partition code blindly overwrites any non-zero offset passed in by the user; so for now, make the -o/-P combo an error for less confusion. In the future, we may let -o and -P work together (selecting a subset of a partition); so it is okay that an explicit '-o 0' behaves no differently from omitting -o. This can be tested with nbdkit: $ echo hi > file $ nbdkit -fv --filter=truncate partitioning file truncate=64k Pre-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 & $ qemu-io -f raw nbd://localhost:10810 qemu-io> r -v 0 1 Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error Connection closed read failed: Input/output error qemu-io> q [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 Post-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536 Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-5-eblake@redhat.com>
2019-01-17 20:36:41 +01:00
if (dev_offset + limit > fd_size) {
error_report("Discovered partition %d at offset %" PRIu64
" size %" PRIu64 ", but size exceeds file length %"
PRId64, partition, dev_offset, limit, fd_size);
qemu-nbd: Sanity check partition bounds When the user requests a partition, we were using data read from the disk as disk offsets without a bounds check. We got lucky that even when computed offsets are out-of-bounds, blk_pread() will gracefully catch the error later (so I don't think a malicious image can crash or exploit qemu-nbd, and am not treating this as a security flaw), but it's better to flag the problem up front than to risk permanent EIO death of the block device down the road. The new bounds check adds an assertion that will never fail, but rather exists to help the compiler see that adding two positive 41-bit values (given MBR constraints) can't overflow 64-bit off_t. Using off_t to represent a partition length is a bit of a misnomer; a later patch will update to saner types, but it is left separate in case the bounds check needs to be backported in isolation. Also, note that the partition code blindly overwrites any non-zero offset passed in by the user; so for now, make the -o/-P combo an error for less confusion. In the future, we may let -o and -P work together (selecting a subset of a partition); so it is okay that an explicit '-o 0' behaves no differently from omitting -o. This can be tested with nbdkit: $ echo hi > file $ nbdkit -fv --filter=truncate partitioning file truncate=64k Pre-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 & $ qemu-io -f raw nbd://localhost:10810 qemu-io> r -v 0 1 Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error Connection closed read failed: Input/output error qemu-io> q [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 Post-patch: $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536 Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Message-Id: <20190117193658.16413-5-eblake@redhat.com>
2019-01-17 20:36:41 +01:00
exit(EXIT_FAILURE);
}
fd_size = limit;
}
nbd: Merge nbd_export_set_name into nbd_export_new The existing NBD code had a weird split where nbd_export_new() created an export but did not add it to the list of exported names until a later nbd_export_set_name() came along and grabbed a second reference on the object; later, the first call to nbd_export_close() drops the second reference while removing the export from the list. This is in part because the QAPI NbdServerRemoveNode enum documents the possibility of adding a mode where we could do a soft disconnect: preventing new clients, but waiting for existing clients to gracefully quit, based on the mode used when calling nbd_export_close(). But in spite of all that, note that we never change the name of an NBD export while it is exposed, which means it is easier to just inline the process of setting the name as part of creating the export. Inline the contents of nbd_export_set_name() and nbd_export_set_description() into the two points in an export lifecycle where they matter, then adjust both callers to pass the name up front. Note that for creation, all callers pass a non-NULL name, (passing NULL at creation was for old style servers, but we removed support for that in commit 7f7dfe2a), so we can add an assert and do things unconditionally; but for cleanup, because of the dual nature of nbd_export_close(), we still have to be careful to avoid use-after-free. Along the way, add a comment reminding ourselves of the potential of adding a middle mode disconnect. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20190111194720.15671-5-eblake@redhat.com>
2019-01-11 20:47:16 +01:00
export = nbd_export_new(bs, dev_offset, fd_size, export_name,
export_description, bitmap, nbdflags,
nbd_export_closed, writethrough, NULL,
&error_fatal);
if (device) {
#if HAVE_NBD_DEVICE
int ret;
ret = pthread_create(&client_thread, NULL, nbd_client_thread, device);
if (ret != 0) {
error_report("Failed to create client thread: %s", strerror(ret));
exit(EXIT_FAILURE);
}
#endif
} else {
/* Shut up GCC warnings. */
memset(&client_thread, 0, sizeof(client_thread));
}
nbd_update_server_watch();
/* now when the initialization is (almost) complete, chdir("/")
* to free any busy filesystems */
if (chdir("/") < 0) {
error_report("Could not chdir to root directory: %s",
strerror(errno));
exit(EXIT_FAILURE);
}
if (fork_process) {
dup2(old_stderr, STDERR_FILENO);
close(old_stderr);
}
state = RUNNING;
do {
main_loop_wait(false);
if (state == TERMINATE) {
state = TERMINATING;
nbd_export_close(export);
nbd_export_put(export);
export = NULL;
}
} while (state != TERMINATED);
block: New BlockBackend A block device consists of a frontend device model and a backend. A block backend has a tree of block drivers doing the actual work. The tree is managed by the block layer. We currently use a single abstraction BlockDriverState both for tree nodes and the backend as a whole. Drawbacks: * Its API includes both stuff that makes sense only at the block backend level (root of the tree) and stuff that's only for use within the block layer. This makes the API bigger and more complex than necessary. Moreover, it's not obvious which interfaces are meant for device models, and which really aren't. * Since device models keep a reference to their backend, the backend object can't just be destroyed. But for media change, we need to replace the tree. Our solution is to make the BlockDriverState generic, with actual driver state in a separate object, pointed to by member opaque. That lets us replace the tree by deinitializing and reinitializing its root. This special need of the root makes the data structure awkward everywhere in the tree. The general plan is to separate the APIs into "block backend", for use by device models, monitor and whatever other code dealing with block backends, and "block driver", for use by the block layer and whatever other code (if any) dealing with trees and tree nodes. Code dealing with block backends, device models in particular, should become completely oblivious of BlockDriverState. This should let us clean up both APIs, and the tree data structures. This commit is a first step. It creates a minimal "block backend" API: type BlockBackend and functions to create, destroy and find them. BlockBackend objects are created and destroyed exactly when root BlockDriverState objects are created and destroyed. "Root" in the sense of "in bdrv_states". They're not yet used for anything; that'll come shortly. A root BlockDriverState is created with bdrv_new_root(), so where to create a BlockBackend is obvious. Where these roots get destroyed isn't always as obvious. It is obvious in qemu-img.c, qemu-io.c and qemu-nbd.c, and in error paths of blockdev_init(), blk_connect(). That leaves destruction of objects successfully created by blockdev_init() and blk_connect(). blockdev_init() is used only by drive_new() and qmp_blockdev_add(). Objects created by the latter are currently indestructible (see commit 48f364d "blockdev: Refuse to drive_del something added with blockdev-add" and commit 2d246f0 "blockdev: Introduce DriveInfo.enable_auto_del"). Objects created by the former get destroyed by drive_del(). Objects created by blk_connect() get destroyed by blk_disconnect(). BlockBackend is reference-counted. Its reference count never exceeds one so far, but that's going to change. In drive_del(), the BB's reference count is surely one now. The BDS's reference count is greater than one when something else is holding a reference, such as a block job. In this case, the BB is destroyed right away, but the BDS lives on until all extra references get dropped. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2014-10-07 13:59:04 +02:00
blk_unref(blk);
if (sockpath) {
unlink(sockpath);
}
qemu_opts_del(sn_opts);
if (device) {
void *ret;
pthread_join(client_thread, &ret);
exit(ret != NULL);
} else {
exit(EXIT_SUCCESS);
}
}