OpenE2K/qemu-e2k
13
4
Fork 0
Code Issues 4 Pull Requests Projects 2 Releases Activity

hw/display/qxl: fix signed to unsigned comparison

Browse Source 
Several small signedness / overflow corrections to qxl_create_guest_primary:
1. use 64 bit unsigned for size to avoid overflow possible from two 32
bit multiplicants.
2. correct sign for requested_height
3. add a more verbose error message when setting guest bug state (which
causes a complete guess blackout until reset, so it helps if it is
verbose).

Signed-off-by: Alon Levy <alevy@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This commit is contained in:
Alon Levy 2014-01-20 18:57:12 +02:00 committed by Gerd Hoffmann
parent f06b85218a
commit 3761abb167
1 changed files with 9 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
hw/display/qxl.c
View File

@ -19,6 +19,7 @@
*/
#include <zlib.h>
#include <stdint.h>
#include "qemu-common.h"
#include "qemu/timer.h"
@ -1361,14 +1362,16 @@ static void qxl_create_guest_primary(PCIQXLDevice *qxl, int loadvm,
{
QXLDevSurfaceCreate surface;
QXLSurfaceCreate *sc = &qxl->guest_primary.surface;
int size;
int requested_height = le32_to_cpu(sc->height);
uint32_t requested_height = le32_to_cpu(sc->height);
int requested_stride = le32_to_cpu(sc->stride);
size = abs(requested_stride) * requested_height;
if (size > qxl->vgamem_size) {
qxl_set_guest_bug(qxl, "%s: requested primary larger then framebuffer"
" size", __func__);
if (requested_stride == INT32_MIN ||
abs(requested_stride) * (uint64_t)requested_height
> qxl->vgamem_size) {
qxl_set_guest_bug(qxl, "%s: requested primary larger than framebuffer"
" stride %d x height %" PRIu32 " > %" PRIu32,
__func__, requested_stride, requested_height,
qxl->vgamem_size);
return;
}