simplify-rtx: Punt on simplify_associative_operation with large operands [PR102356]

Seems simplify_associate_operation is quadratic, which isn't a big deal
for use during combine and other similar RTL passes, because those never
try to combine expressions from more than a few instructions and because
those instructions need to be recognized the machine description also bounds
how many expressions can appear in there.
var-tracking has depth limits only for some cases and unlimited depth
for the vt_expand_loc though:
 /* This is the value used during expansion of locations.  We want it
    to be unbounded, so that variables expanded deep in a recursion
    nest are fully evaluated, so that their values are cached
    correctly.  We avoid recursion cycles through other means, and we
    don't unshare RTL, so excess complexity is not a problem.  */
 #define EXPR_DEPTH (INT_MAX)
 /* We use this to keep too-complex expressions from being emitted as
    location notes, and then to debug information.  Users can trade
    compile time for ridiculously complex expressions, although they're
    seldom useful, and they may often have to be discarded as not
    representable anyway.  */
 #define EXPR_USE_DEPTH (param_max_vartrack_expr_depth)

IMO for very large expressions it isn't worth trying to reassociate though,
in fact e.g. for the new testcase below keeping it as is has bigger chance
of generating smaller debug info which the dwarf2out.c part of the change
tries to achieve - if a binary operation has the same operands, we can
use DW_OP_dup and not bother computing the possibly large operand again.

The patch fixes it by adding a counter to simplify_context and counting
how many times simplify_associative_operation has been called during
a single outermost simplify_* call, and once it reaches some maximum
(currently 64), it stops reassociating.

Another possibility to deal with the power expressions in debug info
would be to introduce some new RTL operation for the pow{,i} (x, n)
case, allow that solely in debug insns and expand those into DWARF
using a loop.  But that seems like quite a lot of work for something rarely
used (especially when powi for larger n is only useful for 0 and 1 inputs
because anything else overflows).

2021-12-01  Jakub Jelinek  <jakub@redhat.com>

	PR rtl-optimization/102356
	* rtl.h (simplify_context): Add assoc_count member and
	max_assoc_count static member.
	* simplify-rtx.c (simplify_associative_operation): Don't reassociate
	more than max_assoc_count times within one outermost simplify_* call.
	* dwarf2out.c (mem_loc_descriptor): Optimize binary operation
	with both operands the same using DW_OP_dup.

	* gcc.dg/pr102356.c: New test.

gcc/dwarf2out.c

@@ -16363,6 +16363,15 @@ mem_loc_descriptor (rtx rtl, machine_mode mode,
     do_binop:
       op0 = mem_loc_descriptor (XEXP (rtl, 0), mode, mem_mode,
 				VAR_INIT_STATUS_INITIALIZED);
+      if (XEXP (rtl, 0) == XEXP (rtl, 1))
+	{
+	  if (op0 == 0)
+	    break;
+	  mem_loc_result = op0;
+	  add_loc_descr (&mem_loc_result, new_loc_descr (DW_OP_dup, 0, 0));
+	  add_loc_descr (&mem_loc_result, new_loc_descr (op, 0, 0));
+	  break;
+	}
       op1 = mem_loc_descriptor (XEXP (rtl, 1), mode, mem_mode,
 				VAR_INIT_STATUS_INITIALIZED);
 

gcc/rtl.h

@@ -3433,6 +3433,14 @@ public:
      inside a MEM than outside.  */
   unsigned int mem_depth = 0;
 
+  /* Tracks number of simplify_associative_operation calls performed during
+     outermost simplify* call.  */
+  unsigned int assoc_count = 0;
+
+  /* Limit for the above number, return NULL from
+     simplify_associative_operation after we reach that assoc_count.  */
+  static const unsigned int max_assoc_count = 64;
+
 private:
   rtx simplify_truncation (machine_mode, rtx, machine_mode);
   rtx simplify_byte_swapping_operation (rtx_code, machine_mode, rtx, rtx);

gcc/simplify-rtx.c

@@ -2263,6 +2263,16 @@ simplify_context::simplify_associative_operation (rtx_code code,
 {
   rtx tem;
 
+  /* Normally expressions simplified by simplify-rtx.c are combined
+     at most from a few machine instructions and therefore the
+     expressions should be fairly small.  During var-tracking
+     we can see arbitrarily large expressions though and reassociating
+     those can be quadratic, so punt after encountering max_assoc_count
+     simplify_associative_operation calls during outermost simplify_*
+     call.  */
+  if (++assoc_count >= max_assoc_count)
+    return NULL_RTX;
+
   /* Linearize the operator to the left.  */
   if (GET_CODE (op1) == code)
     {

gcc/testsuite/gcc.dg/pr102356.c

@@ -0,0 +1,33 @@
+/* PR rtl-optimization/102356 */
+/* { dg-do compile { target int32plus } } */
+/* { dg-options "-O3 -g" } */
+
+signed char a = 0;
+unsigned char b = 9;
+unsigned long long c = 0xF1FBFC17225F7A57ULL;
+int d = 0x3A6667C6;
+
+unsigned char
+foo (unsigned int x)
+{
+  unsigned int *e = &x;
+  if ((c /= ((0 * (*e *= b)) <= 0)))
+    ;
+  for (d = 9; d > 2; d -= 2)
+    {
+      c = -2;
+      do
+	if ((*e *= *e))
+	  {
+	    a = 4;
+	    do
+	      {
+		a -= 3;
+		if ((*e *= *e))
+		  b = 9;
+	      }
+	    while (a > 2);
+	  }
+      while (c++);
+    }
+}