Auto merge of #33612 - royalstream:royalstream-enc-enum-ptr, r=michaelwoerister
gdb Pretty Print: generic encoded was failing on reference/pointer types If you debug this program using **gdb** ```rust fn main() { let x = 10; let y = Some(&x); // additional code } ``` And you try to print **y**'s value from the debugger, you get the following: ``` (gdb) print y Python Exception <class 'gdb.error'> Cannot convert value to int.: $1 = {RUST$ENCODED$ENUM$0$None = Some = {0x7fff5fbff97c}} ``` What happens is that inside **debugger_pretty_printers_common.py** the method `is_null_variant` doesn't have any special handling for pointer values so it ends up calling `.as_integer()` on `discriminant_val` (which holds a pointer) and fails. Considering it needs to handle pointers and return _true_ when the pointer is _null_, I modified the `.as_integer()` method in **gdb_rust_pretty_printing.py** to take pointers into consideration. After this modification **gdb** prints **y** like this: ``` (gdb) print y $1 = Some = {0x7fff5fbff97c} ``` Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.
This commit is contained in:
commit
b358319353
@ -70,6 +70,8 @@ class GdbValue(rustpp.Value):
|
||||
return child
|
||||
|
||||
def as_integer(self):
|
||||
if self.gdb_val.type.code == gdb.TYPE_CODE_PTR:
|
||||
return int(str(self.gdb_val), 0)
|
||||
return int(self.gdb_val)
|
||||
|
||||
def get_wrapped_value(self):
|
||||
|
Loading…
Reference in New Issue
Block a user