linux/drivers/gpu/drm/vmwgfx
Xi Wang 8a78389651 vmwgfx: fix incorrect VRAM size check in vmw_kms_fb_create()
Commit e133e737 didn't correctly fix the integer overflow issue.

-	unsigned int required_size;
+	u64 required_size;
	...
	required_size = mode_cmd->pitch * mode_cmd->height;
-	if (unlikely(required_size > dev_priv->vram_size)) {
+	if (unlikely(required_size > (u64) dev_priv->vram_size)) {

Note that both pitch and height are u32.  Their product is still u32 and
would overflow before being assigned to required_size.  A correct way is
to convert pitch and height to u64 before the multiplication.

	required_size = (u64)mode_cmd->pitch * (u64)mode_cmd->height;

This patch calls the existing vmw_kms_validate_mode_vram() for
validation.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Reviewed-and-tested-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
2011-12-22 12:31:02 +00:00
..
Kconfig
Makefile
svga3d_reg.h
svga_escape.h
svga_overlay.h
svga_reg.h
svga_types.h
vmwgfx_buffer.c
vmwgfx_dmabuf.c
vmwgfx_drv.c
vmwgfx_drv.h vmwgfx: Add helper function to get surface or dmabuf 2011-12-19 14:06:02 +00:00
vmwgfx_execbuf.c
vmwgfx_fb.c
vmwgfx_fence.c
vmwgfx_fence.h
vmwgfx_fifo.c vmwgfx: Use the revised fifo hw version register when present 2011-12-19 14:06:00 +00:00
vmwgfx_gmr.c
vmwgfx_gmrid_manager.c
vmwgfx_ioctl.c vmwgfx: Remove dmabuf check in present ioctl 2011-12-19 14:06:01 +00:00
vmwgfx_irq.c
vmwgfx_kms.c vmwgfx: fix incorrect VRAM size check in vmw_kms_fb_create() 2011-12-22 12:31:02 +00:00
vmwgfx_kms.h vmwgfx: Resend the cursor after legacy modeset 2011-12-19 14:06:04 +00:00
vmwgfx_ldu.c vmwgfx: Resend the cursor after legacy modeset 2011-12-19 14:06:04 +00:00
vmwgfx_marker.c
vmwgfx_overlay.c
vmwgfx_reg.h
vmwgfx_resource.c vmwgfx: Add helper function to get surface or dmabuf 2011-12-19 14:06:02 +00:00
vmwgfx_scrn.c
vmwgfx_ttm_glue.c