Avoid undefined behavior in extract_integer

-fsanitize=undefined showed that extract_integer could left-shift a
negative value, which is undefined.  This patch fixes the problem by
doing all the work in an unsigned type.  This relies on
implementation-defined behavior, but I tend to think we are on safe
ground there.  (Also, if need be, violations of this could probably be
detected, either by configure or by a static_assert.)

gdb/ChangeLog
2018-10-03  Tom Tromey  <tom@tromey.com>

	* findvar.c (extract_integer): Do work in an unsigned type.
This commit is contained in:
Tom Tromey 2018-07-29 17:16:03 -06:00
parent ad69edbb4b
commit 0101665f86
2 changed files with 5 additions and 1 deletions

View File

@ -1,3 +1,7 @@
2018-10-03 Tom Tromey <tom@tromey.com>
* findvar.c (extract_integer): Do work in an unsigned type.
2018-10-03 Tom Tromey <tom@tromey.com>
* common/enum-flags.h (enum_flags::operator~): Add static assert.

View File

@ -50,7 +50,7 @@ template<typename T, typename>
T
extract_integer (const gdb_byte *addr, int len, enum bfd_endian byte_order)
{
T retval = 0;
typename std::make_unsigned<T>::type retval = 0;
const unsigned char *p;
const unsigned char *startaddr = addr;
const unsigned char *endaddr = startaddr + len;