OpenE2K/gcc
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

ipa-sra: Restructure how cloning and call redirection communicate (PR 93385)

Browse Source 
I was asked by Richi to split my fix for PR 93385 for easier review
into IPA-SRA materialization refactoring and the actual DCE addition.
Fortunately it was mostly natural except for a temporary weird
condition in ipa_param_body_adjustments::modify_call_stmt.
Additionally.  In addition to the patch I posted previously, this one
also deallocated the newly added summary in toplev::finalize and fixes
a mistakenly uninitialized field.

This is the first part which basically replaces performed_splits in
clone_info and the code which generates it, keeps it up-to-date and
consumes it with new edge summaries which are much nicer.  It simply
contains 1) a mapping from the original argument indices to the actual
indices in the call statement as it is now, 2) information needed to
identify arguments representing pass-through IPA-SRA splits with which
have been added to the call arguments in place of an original
argument/reference and 3) a delta to the index where va_args may start
- so basically directly all the information that the consumer of
performed_splits had to compute and we also do not need the weird
dummy declarations.

The main disadvantage is that the information has to be created (and
kept up-to-date) for all call graph edges associated with the given
statement from all clones (including inline clones) of the clone where
splitting or removal happened first.  But all of this happens during
clone materialization so the only effect on WPA memory consumption is
the removal of a pointer from clone_info.

The statement modification code also has to know the statement from
the original function in order to be able to locate the edge summaries
which at this point are still keyed to these.  However, the code is
already quite heavily dependant on how things are structured in
tree-inline.c and in order to fix bugs like these it probably has to
be.

The subsequent patch needs this new information to be able to remove
arguments from calls during materialization and communicate this
information to the call redirection.

gcc/ChangeLog:

2021-05-17  Martin Jambor  <mjambor@suse.cz>

	PR ipa/93385
	* symtab-clones.h (clone_info): Removed member param_adjustments.
	* ipa-param-manipulation.h: Adjust initial comment to reflect how we
	deal with pass-through splits now.
	(ipa_param_performed_split): Removed.
	(ipa_param_adjustments::modify_call): Adjusted parameters.
	(class ipa_param_body_adjustments): Adjusted parameters of
	register_replacement, modify_gimple_stmt and modify_call_stmt.
	(ipa_verify_edge_has_no_modifications): Declare.
	(ipa_edge_modifications_finalize): Declare.
	* cgraph.c (cgraph_edge::redirect_call_stmt_to_callee): Remove
	performed_splits processing, pas only edge to padjs->modify_call,
	check that call arguments were not modified if they should not have
	been.
	* cgraphclones.c (cgraph_node::create_clone): Do not copy performed
	splits.
	* ipa-param-manipulation.c (struct pass_through_split_map): New type.
	(ipa_edge_modification_info): Likewise.
	(ipa_edge_modification_sum): Likewise.
	(ipa_edge_modifications): New edge summary.
	(ipa_verify_edge_has_no_modifications): New function.
	(transitive_split_p): Removed.
	(transitive_split_map): Likewise.
	(init_transitive_splits): Likewise.
	(ipa_param_adjustments::modify_call): Adjusted to use the new edge
	summary instead of performed_splits.
	(ipa_param_body_adjustments::register_replacement): Drop dummy
	parameter, set base_index of the created ipa_param_body_replacement.
	(phi_arg_will_live_p): New function.
	(ipa_param_body_adjustments::common_initialization): Do not create
	IPA_SRA dummy decls.
	(simple_tree_swap_info): Removed.
	(remap_split_decl_to_dummy): Likewise.
	(record_argument_state_1): New function.
	(record_argument_state): Likewise.
	(ipa_param_body_adjustments::modify_call_stmt): New parameter
	orig_stmt.  Do not work with dummy decls, save necessary info about
	changes to ipa_edge_modifications.
	(ipa_param_body_adjustments::modify_gimple_stmt): New parameter
	orig_stmt, pass it to modify_call_stmt.
	(ipa_param_body_adjustments::modify_cfun_body): Adjust call to
	modify_gimple_stmt.
	(ipa_edge_modifications_finalize): New function.
	* tree-inline.c (remap_gimple_stmt): Pass original statement to
	modify_gimple_stmt.
	(copy_phis_for_bb): Do not copy dead PHI nodes.
	(expand_call_inline): Do not remap performed_splits.
	(update_clone_info): Likewise.
	* toplev.c: Include ipa-param-manipulation.h.
	(toplev::finalize): Call ipa_edge_modifications_finalize.
This commit is contained in:
Martin Jambor 2021-06-28 18:20:00 +02:00
parent 084635aa80
commit 87467f45e8
No known key found for this signature in database
GPG Key ID: BF63C1BC3FA43540
7 changed files with 485 additions and 545 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
gcc/cgraph.c
View File

@ -1506,8 +1506,6 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e)
} }
clone_info *callee_info = clone_info::get (e->callee); clone_info *callee_info = clone_info::get (e->callee);
clone_info *caller_info = clone_info::get (e->caller);
if (symtab->dump_file) if (symtab->dump_file)
{ {
fprintf (symtab->dump_file, "updating call of %s -> %s: ", fprintf (symtab->dump_file, "updating call of %s -> %s: ",
@ -1515,18 +1513,6 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e)
print_gimple_stmt (symtab->dump_file, e->call_stmt, 0, dump_flags); print_gimple_stmt (symtab->dump_file, e->call_stmt, 0, dump_flags);
if (callee_info && callee_info->param_adjustments) if (callee_info && callee_info->param_adjustments)
callee_info->param_adjustments->dump (symtab->dump_file); callee_info->param_adjustments->dump (symtab->dump_file);
unsigned performed_len
= caller_info ? vec_safe_length (caller_info->performed_splits) : 0;
if (performed_len > 0)
fprintf (symtab->dump_file, "Performed splits records:\n");
for (unsigned i = 0; i < performed_len; i++)
{
ipa_param_performed_split *sm
= &(*caller_info->performed_splits)[i];
print_node_brief (symtab->dump_file, " dummy_decl: ", sm->dummy_decl,
TDF_UID);
fprintf (symtab->dump_file, ", unit_offset: %u\n", sm->unit_offset);
}
} }
if (ipa_param_adjustments *padjs if (ipa_param_adjustments *padjs
@ -1541,10 +1527,7 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e)
remove_stmt_from_eh_lp (e->call_stmt); remove_stmt_from_eh_lp (e->call_stmt);
tree old_fntype = gimple_call_fntype (e->call_stmt); tree old_fntype = gimple_call_fntype (e->call_stmt);
new_stmt = padjs->modify_call (e->call_stmt, new_stmt = padjs->modify_call (e, false);
caller_info
? caller_info->performed_splits : NULL,
e->callee->decl, false);
cgraph_node *origin = e->callee; cgraph_node *origin = e->callee;
while (origin->clone_of) while (origin->clone_of)
origin = origin->clone_of; origin = origin->clone_of;
@ -1564,6 +1547,9 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e)
} }
else else
{ {
if (flag_checking
&& !fndecl_built_in_p (e->callee->decl, BUILT_IN_UNREACHABLE))
ipa_verify_edge_has_no_modifications (e);
new_stmt = e->call_stmt; new_stmt = e->call_stmt;
gimple_call_set_fndecl (new_stmt, e->callee->decl); gimple_call_set_fndecl (new_stmt, e->callee->decl);
update_stmt_fn (DECL_STRUCT_FUNCTION (e->caller->decl), new_stmt); update_stmt_fn (DECL_STRUCT_FUNCTION (e->caller->decl), new_stmt);

3
gcc/cgraphclones.c
View File

@ -414,9 +414,6 @@ cgraph_node::create_clone (tree new_decl, profile_count prof_count,
else if (info && info->param_adjustments) else if (info && info->param_adjustments)
clone_info::get_create (new_node)->param_adjustments clone_info::get_create (new_node)->param_adjustments
= info->param_adjustments; = info->param_adjustments;
if (info && info->performed_splits)
clone_info::get_create (new_node)->performed_splits
= vec_safe_copy (info->performed_splits);
new_node->split_part = split_part; new_node->split_part = split_part;
FOR_EACH_VEC_ELT (redirect_callers, i, e) FOR_EACH_VEC_ELT (redirect_callers, i, e)

803
gcc/ipa-param-manipulation.c
View File

@ -62,6 +62,80 @@ static const char *ipa_param_op_names[IPA_PARAM_PREFIX_COUNT]
"IPA_PARAM_OP_NEW", "IPA_PARAM_OP_NEW",
"IPA_PARAM_OP_SPLIT"}; "IPA_PARAM_OP_SPLIT"};
/* Structure to hold declarations representing pass-through IPA-SRA splits. In
essence, it tells new index for a combination of original index and
offset. */
struct pass_through_split_map
{
/* Original argument index. */
unsigned base_index;
/* Offset of the split part in the original argument. */
unsigned unit_offset;
/* Index of the split part in the call statement - where clone
materialization put it. */
int new_index;
};
/* Information about some call statements that needs to be conveyed from clone
materialization to edge redirection. */
class ipa_edge_modification_info
{
public:
ipa_edge_modification_info ()
{}
/* Mapping of original argument indices to where those arguments sit in the
call statement now or to a negative index if they were removed. */
auto_vec<int> index_map;
/* Information about ISRA replacements put into the call statement at the
clone materialization stages. */
auto_vec<pass_through_split_map> pass_through_map;
/* Necessary adjustment to ipa_param_adjustments::m_always_copy_start when
redirecting the call. */
int always_copy_delta = 0;
};
/* Class for storing and retrieving summaries about cal statement
modifications. */
class ipa_edge_modification_sum
: public call_summary <ipa_edge_modification_info *>
{
public:
ipa_edge_modification_sum (symbol_table *table)
: call_summary<ipa_edge_modification_info *> (table)
{
}
/* Hook that is called by summary when an edge is duplicated. */
virtual void duplicate (cgraph_edge *,
cgraph_edge *,
ipa_edge_modification_info *old_info,
ipa_edge_modification_info *new_info)
{
new_info->index_map.safe_splice (old_info->index_map);
new_info->pass_through_map.safe_splice (old_info->pass_through_map);
new_info->always_copy_delta = old_info->always_copy_delta;
}
};
/* Call summary to store information about edges which have had their arguments
partially modified already. */
static ipa_edge_modification_sum *ipa_edge_modifications;
/* Fail compilation if CS has any summary associated with it in
ipa_edge_modifications. */
DEBUG_FUNCTION void
ipa_verify_edge_has_no_modifications (cgraph_edge *cs)
{
gcc_assert (!ipa_edge_modifications || !ipa_edge_modifications->get (cs));
}
/* Fill an empty vector ARGS with PARM_DECLs representing formal parameters of /* Fill an empty vector ARGS with PARM_DECLs representing formal parameters of
FNDECL. The function should not be called during LTO WPA phase except for FNDECL. The function should not be called during LTO WPA phase except for
thunks (or functions with bodies streamed in). */ thunks (or functions with bodies streamed in). */
@ -459,147 +533,46 @@ isra_get_ref_base_and_offset (tree expr, tree *base_p, unsigned *unit_offset_p)
return true; return true;
} }
/* Return true if EXPR describes a transitive split (i.e. one that happened for /* Modify actual arguments of a function call in statement currently belonging
both the caller and the callee) as recorded in PERFORMED_SPLITS. In that to CS, and make it call CS->callee->decl. Return the new statement that
case, store index of the respective record in PERFORMED_SPLITS into replaced the old one. When invoked, cfun and current_function_decl have to
*SM_IDX_P and the unit offset from all handled components in EXPR into be set to the caller. */
*UNIT_OFFSET_P. */
static bool
transitive_split_p (vec<ipa_param_performed_split, va_gc> *performed_splits,
tree expr, unsigned *sm_idx_p, unsigned *unit_offset_p)
{
tree base;
if (!isra_get_ref_base_and_offset (expr, &base, unit_offset_p))
return false;
if (TREE_CODE (base) == SSA_NAME)
{
base = SSA_NAME_VAR (base);
if (!base)
return false;
}
unsigned len = vec_safe_length (performed_splits);
for (unsigned i = 0 ; i < len; i++)
{
ipa_param_performed_split *sm = &(*performed_splits)[i];
if (sm->dummy_decl == base)
{
*sm_idx_p = i;
return true;
}
}
return false;
}
/* Structure to hold declarations representing transitive IPA-SRA splits. In
essence, if we need to pass UNIT_OFFSET of a parameter which originally has
number BASE_INDEX, we should pass down REPL. */
struct transitive_split_map
{
tree repl;
unsigned base_index;
unsigned unit_offset;
};
/* If call STMT contains any parameters representing transitive splits as
described by PERFORMED_SPLITS, return the number of extra parameters that
were addded during clone materialization and fill in INDEX_MAP with adjusted
indices of corresponding original parameters and TRANS_MAP with description
of all transitive replacement descriptions. Otherwise return zero. */
static unsigned
init_transitive_splits (vec<ipa_param_performed_split, va_gc> *performed_splits,
gcall *stmt, vec <unsigned> *index_map,
auto_vec <transitive_split_map> *trans_map)
{
unsigned phony_arguments = 0;
unsigned stmt_idx = 0, base_index = 0;
unsigned nargs = gimple_call_num_args (stmt);
while (stmt_idx < nargs)
{
unsigned unit_offset_delta;
tree base_arg = gimple_call_arg (stmt, stmt_idx);
if (phony_arguments > 0)
index_map->safe_push (stmt_idx);
unsigned sm_idx;
stmt_idx++;
if (transitive_split_p (performed_splits, base_arg, &sm_idx,
&unit_offset_delta))
{
if (phony_arguments == 0)
/* We have optimistically avoided constructing index_map do far but
now it is clear it will be necessary, so let's create the easy
bit we skipped until now. */
for (unsigned k = 0; k < stmt_idx; k++)
index_map->safe_push (k);
tree dummy = (*performed_splits)[sm_idx].dummy_decl;
for (unsigned j = sm_idx; j < performed_splits->length (); j++)
{
ipa_param_performed_split *caller_split
= &(*performed_splits)[j];
if (caller_split->dummy_decl != dummy)
break;
tree arg = gimple_call_arg (stmt, stmt_idx);
struct transitive_split_map tsm;
tsm.repl = arg;
tsm.base_index = base_index;
if (caller_split->unit_offset >= unit_offset_delta)
{
tsm.unit_offset
= (caller_split->unit_offset - unit_offset_delta);
trans_map->safe_push (tsm);
}
phony_arguments++;
stmt_idx++;
}
}
base_index++;
}
return phony_arguments;
}
/* Modify actual arguments of a function call in statement STMT, assuming it
calls CALLEE_DECL. CALLER_ADJ must be the description of parameter
adjustments of the caller or NULL if there are none. Return the new
statement that replaced the old one. When invoked, cfun and
current_function_decl have to be set to the caller. */
gcall * gcall *
ipa_param_adjustments::modify_call (gcall *stmt, ipa_param_adjustments::modify_call (cgraph_edge *cs,
vec<ipa_param_performed_split, bool update_references)
va_gc> *performed_splits,
tree callee_decl, bool update_references)
{ {
gcall *stmt = cs->call_stmt;
tree callee_decl = cs->callee->decl;
ipa_edge_modification_info *mod_info
= ipa_edge_modifications ? ipa_edge_modifications->get (cs) : NULL;
if (mod_info && symtab->dump_file)
{
fprintf (symtab->dump_file, "Information about pre-exiting "
"modifications.\n Index map:");
unsigned idx_len = mod_info->index_map.length ();
for (unsigned i = 0; i < idx_len; i++)
fprintf (symtab->dump_file, " %i", mod_info->index_map[i]);
fprintf (symtab->dump_file, "\n Pass-through split map: ");
unsigned ptm_len = mod_info->pass_through_map.length ();
for (unsigned i = 0; i < ptm_len; i++)
fprintf (symtab->dump_file,
" (base_index: %u, offset: %u, new_index: %i)",
mod_info->pass_through_map[i].base_index,
mod_info->pass_through_map[i].unit_offset,
mod_info->pass_through_map[i].new_index);
fprintf (symtab->dump_file, "\n Always-copy delta: %i\n",
mod_info->always_copy_delta);
}
unsigned len = vec_safe_length (m_adj_params); unsigned len = vec_safe_length (m_adj_params);
auto_vec<tree, 16> vargs (len); auto_vec<tree, 16> vargs (len);
tree old_decl = gimple_call_fndecl (stmt);
unsigned old_nargs = gimple_call_num_args (stmt); unsigned old_nargs = gimple_call_num_args (stmt);
unsigned orig_nargs = mod_info ? mod_info->index_map.length () : old_nargs;
auto_vec<bool, 16> kept (old_nargs); auto_vec<bool, 16> kept (old_nargs);
kept.quick_grow_cleared (old_nargs); kept.quick_grow_cleared (old_nargs);
auto_vec <unsigned, 16> index_map;
auto_vec <transitive_split_map> trans_map;
bool transitive_remapping = false;
if (performed_splits)
{
unsigned removed = init_transitive_splits (performed_splits,
stmt, &index_map, &trans_map);
if (removed > 0)
{
transitive_remapping = true;
old_nargs -= removed;
}
}
cgraph_node *current_node = cgraph_node::get (current_function_decl); cgraph_node *current_node = cgraph_node::get (current_function_decl);
if (update_references) if (update_references)
current_node->remove_stmt_references (stmt); current_node->remove_stmt_references (stmt);
@ -612,13 +585,16 @@ ipa_param_adjustments::modify_call (gcall *stmt,
ipa_adjusted_param *apm = &(*m_adj_params)[i]; ipa_adjusted_param *apm = &(*m_adj_params)[i];
if (apm->op == IPA_PARAM_OP_COPY) if (apm->op == IPA_PARAM_OP_COPY)
{ {
unsigned index = apm->base_index; int index = apm->base_index;
if (index >= old_nargs) if ((unsigned) index >= orig_nargs)
/* Can happen if the original call has argument mismatch, /* Can happen if the original call has argument mismatch,
ignore. */ ignore. */
continue; continue;
if (transitive_remapping) if (mod_info)
index = index_map[apm->base_index]; {
index = mod_info->index_map[apm->base_index];
gcc_assert (index >= 0);
}
tree arg = gimple_call_arg (stmt, index); tree arg = gimple_call_arg (stmt, index);
@ -636,14 +612,17 @@ ipa_param_adjustments::modify_call (gcall *stmt,
materialization. */ materialization. */
gcc_assert (apm->op == IPA_PARAM_OP_SPLIT); gcc_assert (apm->op == IPA_PARAM_OP_SPLIT);
/* We have to handle transitive changes differently using the maps we /* We have to handle pass-through changes differently using the map
have created before. So look into them first. */ clone materialziation might have left behind. */
tree repl = NULL_TREE; tree repl = NULL_TREE;
for (unsigned j = 0; j < trans_map.length (); j++) unsigned ptm_len = mod_info ? mod_info->pass_through_map.length () : 0;
if (trans_map[j].base_index == apm->base_index for (unsigned j = 0; j < ptm_len; j++)
&& trans_map[j].unit_offset == apm->unit_offset) if (mod_info->pass_through_map[j].base_index == apm->base_index
&& mod_info->pass_through_map[j].unit_offset == apm->unit_offset)
{ {
repl = trans_map[j].repl; int repl_idx = mod_info->pass_through_map[j].new_index;
gcc_assert (repl_idx >= 0);
repl = gimple_call_arg (stmt, repl_idx);
break; break;
} }
if (repl) if (repl)
@ -652,12 +631,15 @@ ipa_param_adjustments::modify_call (gcall *stmt,
continue; continue;
} }
unsigned index = apm->base_index; int index = apm->base_index;
if (index >= old_nargs) if ((unsigned) index >= orig_nargs)
/* Can happen if the original call has argument mismatch, ignore. */ /* Can happen if the original call has argument mismatch, ignore. */
continue; continue;
if (transitive_remapping) if (mod_info)
index = index_map[apm->base_index]; {
index = mod_info->index_map[apm->base_index];
gcc_assert (index >= 0);
}
tree base = gimple_call_arg (stmt, index); tree base = gimple_call_arg (stmt, index);
/* We create a new parameter out of the value of the old one, we can /* We create a new parameter out of the value of the old one, we can
@ -773,8 +755,16 @@ ipa_param_adjustments::modify_call (gcall *stmt,
} }
if (m_always_copy_start >= 0) if (m_always_copy_start >= 0)
for (unsigned i = m_always_copy_start; i < old_nargs; i++) {
vargs.safe_push (gimple_call_arg (stmt, i)); int always_copy_start = m_always_copy_start;
if (mod_info)
{
always_copy_start += mod_info->always_copy_delta;
gcc_assert (always_copy_start >= 0);
}
for (unsigned i = always_copy_start; i < old_nargs; i++)
vargs.safe_push (gimple_call_arg (stmt, i));
}
/* For optimized away parameters, add on the caller side /* For optimized away parameters, add on the caller side
before the call before the call
@ -782,6 +772,7 @@ ipa_param_adjustments::modify_call (gcall *stmt,
stmts and associate D#X with parm in decl_debug_args_lookup stmts and associate D#X with parm in decl_debug_args_lookup
vector to say for debug info that if parameter parm had been passed, vector to say for debug info that if parameter parm had been passed,
it would have value parm_Y(D). */ it would have value parm_Y(D). */
tree old_decl = gimple_call_fndecl (stmt);
if (MAY_HAVE_DEBUG_BIND_STMTS && old_decl && callee_decl) if (MAY_HAVE_DEBUG_BIND_STMTS && old_decl && callee_decl)
{ {
vec<tree, va_gc> **debug_args = NULL; vec<tree, va_gc> **debug_args = NULL;
@ -799,13 +790,17 @@ ipa_param_adjustments::modify_call (gcall *stmt,
{ {
if (!is_gimple_reg (old_parm) || kept[i]) if (!is_gimple_reg (old_parm) || kept[i])
continue; continue;
tree origin = DECL_ORIGIN (old_parm);
tree arg; tree arg;
if (transitive_remapping) if (mod_info)
arg = gimple_call_arg (stmt, index_map[i]); {
if (mod_info->index_map[i] < 0)
continue;
arg = gimple_call_arg (stmt, mod_info->index_map[i]);
}
else else
arg = gimple_call_arg (stmt, i); arg = gimple_call_arg (stmt, i);
tree origin = DECL_ORIGIN (old_parm);
if (!useless_type_conversion_p (TREE_TYPE (origin), TREE_TYPE (arg))) if (!useless_type_conversion_p (TREE_TYPE (origin), TREE_TYPE (arg)))
{ {
if (!fold_convertible_p (TREE_TYPE (origin), arg)) if (!fold_convertible_p (TREE_TYPE (origin), arg))
@ -909,6 +904,9 @@ ipa_param_adjustments::modify_call (gcall *stmt,
gsi_prev (&gsi); gsi_prev (&gsi);
} }
while (gsi_stmt (gsi) != gsi_stmt (prev_gsi)); while (gsi_stmt (gsi) != gsi_stmt (prev_gsi));
if (mod_info)
ipa_edge_modifications->remove (cs);
return new_stmt; return new_stmt;
} }
@ -931,13 +929,11 @@ ipa_param_adjustments::debug ()
dump (stderr); dump (stderr);
} }
/* Register that REPLACEMENT should replace parameter described in APM and /* Register that REPLACEMENT should replace parameter described in APM. */
optionally as DUMMY to mark transitive splits across calls. */
void void
ipa_param_body_adjustments::register_replacement (ipa_adjusted_param *apm, ipa_param_body_adjustments::register_replacement (ipa_adjusted_param *apm,
tree replacement, tree replacement)
tree dummy)
{ {
gcc_checking_assert (apm->op == IPA_PARAM_OP_SPLIT gcc_checking_assert (apm->op == IPA_PARAM_OP_SPLIT
|| apm->op == IPA_PARAM_OP_NEW); || apm->op == IPA_PARAM_OP_NEW);
@ -945,7 +941,7 @@ ipa_param_body_adjustments::register_replacement (ipa_adjusted_param *apm,
ipa_param_body_replacement psr; ipa_param_body_replacement psr;
psr.base = m_oparms[apm->prev_clone_index]; psr.base = m_oparms[apm->prev_clone_index];
psr.repl = replacement; psr.repl = replacement;
psr.dummy = dummy; psr.dummy = NULL_TREE;
psr.unit_offset = apm->unit_offset; psr.unit_offset = apm->unit_offset;
m_replacements.safe_push (psr); m_replacements.safe_push (psr);
} }
@ -1007,9 +1003,6 @@ ipa_param_body_adjustments::common_initialization (tree old_fndecl,
auto_vec<bool, 16> kept; auto_vec<bool, 16> kept;
kept.reserve_exact (m_oparms.length ()); kept.reserve_exact (m_oparms.length ());
kept.quick_grow_cleared (m_oparms.length ()); kept.quick_grow_cleared (m_oparms.length ());
auto_vec<tree, 16> isra_dummy_decls;
isra_dummy_decls.reserve_exact (m_oparms.length ());
isra_dummy_decls.quick_grow_cleared (m_oparms.length ());
unsigned adj_len = vec_safe_length (m_adj_params); unsigned adj_len = vec_safe_length (m_adj_params);
m_method2func = ((TREE_CODE (TREE_TYPE (m_fndecl)) == METHOD_TYPE) m_method2func = ((TREE_CODE (TREE_TYPE (m_fndecl)) == METHOD_TYPE)
@ -1055,35 +1048,7 @@ ipa_param_body_adjustments::common_initialization (tree old_fndecl,
if (apm->op == IPA_PARAM_OP_SPLIT) if (apm->op == IPA_PARAM_OP_SPLIT)
{ {
m_split_modifications_p = true; m_split_modifications_p = true;
register_replacement (apm, new_parm);
if (m_id)
{
tree dummy_decl;
if (!isra_dummy_decls[prev_index])
{
dummy_decl = copy_decl_to_var (m_oparms[prev_index],
m_id);
/* Any attempt to remap this dummy in this particular
instance of clone materialization should yield
itself. */
insert_decl_map (m_id, dummy_decl, dummy_decl);
DECL_CHAIN (dummy_decl) = *vars;
*vars = dummy_decl;
isra_dummy_decls[prev_index] = dummy_decl;
}
else
dummy_decl = isra_dummy_decls[prev_index];
register_replacement (apm, new_parm, dummy_decl);
ipa_param_performed_split ps;
ps.dummy_decl = dummy_decl;
ps.unit_offset = apm->unit_offset;
vec_safe_push (clone_info::get_create
(m_id->dst_node)->performed_splits, ps);
}
else
register_replacement (apm, new_parm);
} }
} }
else else
@ -1110,8 +1075,6 @@ ipa_param_body_adjustments::common_initialization (tree old_fndecl,
{ {
if (!m_id->decl_map->get (m_oparms[i])) if (!m_id->decl_map->get (m_oparms[i]))
{ {
/* TODO: Perhaps at least aggregate-type params could re-use
their isra_dummy_decl here? */
tree var = copy_decl_to_var (m_oparms[i], m_id); tree var = copy_decl_to_var (m_oparms[i], m_id);
insert_decl_map (m_id, m_oparms[i], var); insert_decl_map (m_id, m_oparms[i], var);
/* Declare this new variable. */ /* Declare this new variable. */
@ -1502,220 +1465,329 @@ ipa_param_body_adjustments::modify_assignment (gimple *stmt,
return any; return any;
} }
/* Data passed to remap_split_decl_to_dummy through walk_tree. */ /* Record information about what modifications to call arguments have already
been done by clone materialization into a summary describing CS. The
information is stored in NEW_INDEX_MAP, NEW_PT_MAP and NEW_ALWAYS_COPY_DELTA
and correspond to equivalent fields in ipa_edge_modification_info. Return
the edge summary. */
static ipa_edge_modification_info *
record_argument_state_1 (cgraph_edge *cs, const vec<int> &new_index_map,
const vec<pass_through_split_map> &new_pt_map,
int new_always_copy_delta)
struct simple_tree_swap_info
{ {
/* Change FROM to TO. */ ipa_edge_modification_info *sum = ipa_edge_modifications->get_create (cs);
tree from, to;
/* And set DONE to true when doing so. */
bool done;
};
/* Simple remapper to remap a split parameter to the same expression based on a unsigned len = sum->pass_through_map.length ();
special dummy decl so that edge redirections can detect transitive splitting for (unsigned i = 0; i < len; i++)
and finish them. */
static tree
remap_split_decl_to_dummy (tree *tp, int *walk_subtrees, void *data)
{
tree t = *tp;
if (DECL_P (t) || TREE_CODE (t) == SSA_NAME)
{ {
struct simple_tree_swap_info *swapinfo unsigned oldnew = sum->pass_through_map[i].new_index;
= (struct simple_tree_swap_info *) data; sum->pass_through_map[i].new_index = new_index_map[oldnew];
if (t == swapinfo->from }
|| (TREE_CODE (t) == SSA_NAME
&& SSA_NAME_VAR (t) == swapinfo->from)) len = sum->index_map.length ();
{ if (len > 0)
*tp = swapinfo->to; {
swapinfo->done = true; unsigned nptlen = new_pt_map.length ();
} for (unsigned j = 0; j < nptlen; j++)
*walk_subtrees = 0; {
int inverse = -1;
for (unsigned i = 0; i < len ; i++)
if ((unsigned) sum->index_map[i] == new_pt_map[j].base_index)
{
inverse = i;
break;
}
gcc_assert (inverse >= 0);
pass_through_split_map ptm_item;
ptm_item.base_index = inverse;
ptm_item.unit_offset = new_pt_map[j].unit_offset;
ptm_item.new_index = new_pt_map[j].new_index;
sum->pass_through_map.safe_push (ptm_item);
}
for (unsigned i = 0; i < len; i++)
{
int idx = sum->index_map[i];
if (idx < 0)
continue;
sum->index_map[i] = new_index_map[idx];
}
} }
else if (TYPE_P (t))
*walk_subtrees = 0;
else else
*walk_subtrees = 1; {
return NULL_TREE; sum->pass_through_map.safe_splice (new_pt_map);
sum->index_map.safe_splice (new_index_map);
}
sum->always_copy_delta += new_always_copy_delta;
return sum;
} }
/* Record information about what modifications to call arguments have already
been done by clone materialization into a summary of an edge describing the
call in this clone and all its clones. NEW_INDEX_MAP, NEW_PT_MAP and
NEW_ALWAYS_COPY_DELTA have the same meaning as record_argument_state_1.
In order to associate the info with the right edge summaries, we need
address of the ORIG_STMT in the function from which we are cloning (because
the edges have not yet been re-assigned to the new statement that has just
been created) and ID, the structure governing function body copying. */
static void
record_argument_state (copy_body_data *id, gimple *orig_stmt,
const vec<int> &new_index_map,
const vec<pass_through_split_map> &new_pt_map,
int new_always_copy_delta)
{
if (!ipa_edge_modifications)
ipa_edge_modifications = new ipa_edge_modification_sum (symtab);
struct cgraph_node *this_node = id->dst_node;
ipa_edge_modification_info *first_sum = NULL;
cgraph_edge *cs = this_node->get_edge (orig_stmt);
if (cs)
first_sum = record_argument_state_1 (cs, new_index_map, new_pt_map,
new_always_copy_delta);
else
gcc_assert (this_node->clones);
if (!this_node->clones)
return;
for (cgraph_node *subclone = this_node->clones; subclone != this_node;)
{
cs = subclone->get_edge (orig_stmt);
if (cs)
{
if (!first_sum)
first_sum = record_argument_state_1 (cs, new_index_map, new_pt_map,
new_always_copy_delta);
else
{
ipa_edge_modification_info *s2
= ipa_edge_modifications->get_create (cs);
s2->index_map.truncate (0);
s2->index_map.safe_splice (first_sum->index_map);
s2->pass_through_map.truncate (0);
s2->pass_through_map.safe_splice (first_sum->pass_through_map);
s2->always_copy_delta = first_sum->always_copy_delta;
}
}
else
gcc_assert (subclone->clones);
if (subclone->clones)
subclone = subclone->clones;
else if (subclone->next_sibling_clone)
subclone = subclone->next_sibling_clone;
else
{
while (subclone != this_node && !subclone->next_sibling_clone)
subclone = subclone->clone_of;
if (subclone != this_node)
subclone = subclone->next_sibling_clone;
}
}
}
/* If the call statement pointed at by STMT_P contains any expressions that /* If the call statement pointed at by STMT_P contains any expressions that
need to replaced with a different one as noted by ADJUSTMENTS, do so. f the need to replaced with a different one as noted by ADJUSTMENTS, do so. f the
statement needs to be rebuilt, do so. Return true if any modifications have statement needs to be rebuilt, do so. Return true if any modifications have
been performed. been performed. ORIG_STMT, if not NULL, is the original statement in the
function that is being cloned from, which at this point can be used to look
up call_graph edges.
If the method is invoked as a part of IPA clone materialization and if any If the method is invoked as a part of IPA clone materialization and if any
parameter split is transitive, i.e. it applies to the functin that is being parameter split is pass-through, i.e. it applies to the functin that is
modified and also to the callee of the statement, replace the parameter being modified and also to the callee of the statement, replace the
passed to old callee with an equivalent expression based on a dummy decl parameter passed to old callee with all of the replacement a callee might
followed by PARM_DECLs representing the actual replacements. The actual possibly want and record the performed argument modifications in
replacements will be then converted into SSA_NAMEs and then ipa_edge_modifications. Likewise if any argument has already been left out
ipa_param_adjustments::modify_call will find the appropriate ones and leave because it is not necessary. */
only those in the call. */
bool bool
ipa_param_body_adjustments::modify_call_stmt (gcall **stmt_p) ipa_param_body_adjustments::modify_call_stmt (gcall **stmt_p,
gimple *orig_stmt)
{ {
gcall *stmt = *stmt_p;
auto_vec <unsigned, 4> pass_through_args; auto_vec <unsigned, 4> pass_through_args;
auto_vec <unsigned, 4> pass_through_pbr_indices; auto_vec <unsigned, 4> pass_through_pbr_indices;
auto_vec <HOST_WIDE_INT, 4> pass_through_offsets;
gcall *stmt = *stmt_p;
unsigned nargs = gimple_call_num_args (stmt);
bool recreate = false;
if (m_split_modifications_p && m_id) for (unsigned i = 0; i < gimple_call_num_args (stmt); i++)
{ {
for (unsigned i = 0; i < gimple_call_num_args (stmt); i++) tree t = gimple_call_arg (stmt, i);
gcc_assert (TREE_CODE (t) != BIT_FIELD_REF
&& TREE_CODE (t) != IMAGPART_EXPR
&& TREE_CODE (t) != REALPART_EXPR);
/* The follow-up patch will check whether t needs to be removed, that's
why this condition is in the loop. */
if (!m_split_modifications_p)
continue;
tree base;
unsigned agg_arg_offset;
if (!isra_get_ref_base_and_offset (t, &base, &agg_arg_offset))
continue;
bool by_ref = false;
if (TREE_CODE (base) == SSA_NAME)
{ {
tree t = gimple_call_arg (stmt, i); if (!SSA_NAME_IS_DEFAULT_DEF (base))
gcc_assert (TREE_CODE (t) != BIT_FIELD_REF
&& TREE_CODE (t) != IMAGPART_EXPR
&& TREE_CODE (t) != REALPART_EXPR);
tree base;
unsigned unit_offset;
if (!isra_get_ref_base_and_offset (t, &base, &unit_offset))
continue; continue;
base = SSA_NAME_VAR (base);
gcc_checking_assert (base);
by_ref = true;
}
if (TREE_CODE (base) != PARM_DECL)
continue;
bool by_ref = false; bool base_among_replacements = false;
if (TREE_CODE (base) == SSA_NAME) unsigned j, repl_list_len = m_replacements.length ();
for (j = 0; j < repl_list_len; j++)
{
ipa_param_body_replacement *pbr = &m_replacements[j];
if (pbr->base == base)
{ {
if (!SSA_NAME_IS_DEFAULT_DEF (base)) base_among_replacements = true;
continue; break;
base = SSA_NAME_VAR (base);
gcc_checking_assert (base);
by_ref = true;
} }
if (TREE_CODE (base) != PARM_DECL) }
continue; if (!base_among_replacements)
continue;
bool base_among_replacements = false; /* We still have to distinguish between an end-use that we have to
unsigned j, repl_list_len = m_replacements.length (); transform now and a pass-through, which happens in the following
for (j = 0; j < repl_list_len; j++) two cases. */
/* TODO: After we adjust ptr_parm_has_nonarg_uses to also consider
&MEM_REF[ssa_name + offset], we will also have to detect that case
here. */
if (TREE_CODE (t) == SSA_NAME
&& SSA_NAME_IS_DEFAULT_DEF (t)
&& SSA_NAME_VAR (t)
&& TREE_CODE (SSA_NAME_VAR (t)) == PARM_DECL)
{
/* This must be a by_reference pass-through. */
recreate = true;
gcc_assert (POINTER_TYPE_P (TREE_TYPE (t)));
pass_through_args.safe_push (i);
pass_through_pbr_indices.safe_push (j);
pass_through_offsets.safe_push (agg_arg_offset);
}
else if (!by_ref && AGGREGATE_TYPE_P (TREE_TYPE (t)))
{
/* Currently IPA-SRA guarantees the aggregate access type
exactly matches in this case. So if it does not match, it is
a pass-through argument that will be sorted out at edge
redirection time. */
ipa_param_body_replacement *pbr
= lookup_replacement_1 (base, agg_arg_offset);
if (!pbr
|| (TYPE_MAIN_VARIANT (TREE_TYPE (t))
!= TYPE_MAIN_VARIANT (TREE_TYPE (pbr->repl))))
{ {
ipa_param_body_replacement *pbr = &m_replacements[j]; recreate = true;
if (pbr->base == base)
{
base_among_replacements = true;
break;
}
}
if (!base_among_replacements)
continue;
/* We still have to distinguish between an end-use that we have to
transform now and a pass-through, which happens in the following
two cases. */
/* TODO: After we adjust ptr_parm_has_nonarg_uses to also consider
&MEM_REF[ssa_name + offset], we will also have to detect that case
here. */
if (TREE_CODE (t) == SSA_NAME
&& SSA_NAME_IS_DEFAULT_DEF (t)
&& SSA_NAME_VAR (t)
&& TREE_CODE (SSA_NAME_VAR (t)) == PARM_DECL)
{
/* This must be a by_reference pass-through. */
gcc_assert (POINTER_TYPE_P (TREE_TYPE (t)));
pass_through_args.safe_push (i); pass_through_args.safe_push (i);
pass_through_pbr_indices.safe_push (j); pass_through_pbr_indices.safe_push (j);
} pass_through_offsets.safe_push (agg_arg_offset);
else if (!by_ref && AGGREGATE_TYPE_P (TREE_TYPE (t)))
{
/* Currently IPA-SRA guarantees the aggregate access type
exactly matches in this case. So if it does not match, it is
a pass-through argument that will be sorted out at edge
redirection time. */
ipa_param_body_replacement *pbr
= lookup_replacement_1 (base, unit_offset);
if (!pbr
|| (TYPE_MAIN_VARIANT (TREE_TYPE (t))
!= TYPE_MAIN_VARIANT (TREE_TYPE (pbr->repl))))
{
pass_through_args.safe_push (i);
pass_through_pbr_indices.safe_push (j);
}
} }
} }
} }
unsigned nargs = gimple_call_num_args (stmt); if (!recreate)
if (!pass_through_args.is_empty ())
{ {
auto_vec<tree, 16> vargs; /* No need to rebuild the statement, let's just modify arguments
unsigned pt_idx = 0; and the LHS if/as appropriate. */
bool modified = false;
for (unsigned i = 0; i < nargs; i++) for (unsigned i = 0; i < nargs; i++)
{ {
if (pt_idx < pass_through_args.length () tree *t = gimple_call_arg_ptr (stmt, i);
&& i == pass_through_args[pt_idx]) modified |= modify_expression (t, true);
{
unsigned j = pass_through_pbr_indices[pt_idx];
pt_idx++;
tree base = m_replacements[j].base;
/* Map base will get mapped to the special transitive-isra marker
dummy decl. */
struct simple_tree_swap_info swapinfo;
swapinfo.from = base;
swapinfo.to = m_replacements[j].dummy;
swapinfo.done = false;
tree arg = gimple_call_arg (stmt, i);
walk_tree (&arg, remap_split_decl_to_dummy, &swapinfo, NULL);
gcc_assert (swapinfo.done);
vargs.safe_push (arg);
/* Now let's push all replacements pertaining to this parameter
so that all gimple register ones get correct SSA_NAMES. Edge
redirection will weed out the dummy argument as well as all
unused replacements later. */
unsigned int repl_list_len = m_replacements.length ();
for (; j < repl_list_len; j++)
{
if (m_replacements[j].base != base)
break;
vargs.safe_push (m_replacements[j].repl);
}
}
else
{
tree t = gimple_call_arg (stmt, i);
modify_expression (&t, true);
vargs.safe_push (t);
}
} }
gcall *new_stmt = gimple_build_call_vec (gimple_call_fn (stmt), vargs); if (gimple_call_lhs (stmt))
if (gimple_has_location (stmt))
gimple_set_location (new_stmt, gimple_location (stmt));
gimple_call_set_chain (new_stmt, gimple_call_chain (stmt));
gimple_call_copy_flags (new_stmt, stmt);
if (tree lhs = gimple_call_lhs (stmt))
{ {
modify_expression (&lhs, false); tree *t = gimple_call_lhs_ptr (stmt);
/* Avoid adjusting SSA_NAME_DEF_STMT of a SSA lhs, SSA names modified |= modify_expression (t, false);
have not yet been remapped. */
*gimple_call_lhs_ptr (new_stmt) = lhs;
} }
*stmt_p = new_stmt; return modified;
return true;
} }
/* Otherwise, no need to rebuild the statement, let's just modify arguments auto_vec<int, 16> index_map;
and the LHS if/as appropriate. */ auto_vec<pass_through_split_map, 4> pass_through_map;
bool modified = false; auto_vec<tree, 16> vargs;
int always_copy_delta = 0;
unsigned pt_idx = 0;
int new_arg_idx = 0;
for (unsigned i = 0; i < nargs; i++) for (unsigned i = 0; i < nargs; i++)
{ {
tree *t = gimple_call_arg_ptr (stmt, i); if (pt_idx < pass_through_args.length ()
modified |= modify_expression (t, true); && i == pass_through_args[pt_idx])
{
unsigned j = pass_through_pbr_indices[pt_idx];
unsigned agg_arg_offset = pass_through_offsets[pt_idx];
pt_idx++;
always_copy_delta--;
tree base = m_replacements[j].base;
/* In order to be put into SSA form, we have to push all replacements
pertaining to this parameter as parameters to the call statement.
Edge redirection will need to use edge summary to weed out the
unnecessary ones. */
unsigned repl_list_len = m_replacements.length ();
for (; j < repl_list_len; j++)
{
if (m_replacements[j].base != base)
break;
if (m_replacements[j].unit_offset < agg_arg_offset)
continue;
pass_through_split_map pt_map;
pt_map.base_index = i;
pt_map.unit_offset
= m_replacements[j].unit_offset - agg_arg_offset;
pt_map.new_index = new_arg_idx;
pass_through_map.safe_push (pt_map);
vargs.safe_push (m_replacements[j].repl);
new_arg_idx++;
always_copy_delta++;
}
index_map.safe_push (-1);
}
else
{
tree t = gimple_call_arg (stmt, i);
modify_expression (&t, true);
vargs.safe_push (t);
index_map.safe_push (new_arg_idx);
new_arg_idx++;
}
} }
if (gimple_call_lhs (stmt)) gcall *new_stmt = gimple_build_call_vec (gimple_call_fn (stmt), vargs);
if (gimple_has_location (stmt))
gimple_set_location (new_stmt, gimple_location (stmt));
gimple_call_set_chain (new_stmt, gimple_call_chain (stmt));
gimple_call_copy_flags (new_stmt, stmt);
if (tree lhs = gimple_call_lhs (stmt))
{ {
tree *t = gimple_call_lhs_ptr (stmt); modify_expression (&lhs, false);
modified |= modify_expression (t, false); /* Avoid adjusting SSA_NAME_DEF_STMT of a SSA lhs, SSA names
have not yet been remapped. */
*gimple_call_lhs_ptr (new_stmt) = lhs;
} }
*stmt_p = new_stmt;
return modified; if (orig_stmt)
record_argument_state (m_id, orig_stmt, index_map, pass_through_map,
always_copy_delta);
return true;
} }
/* If the statement STMT contains any expressions that need to replaced with a /* If the statement STMT contains any expressions that need to replaced with a
@ -1726,7 +1798,8 @@ ipa_param_body_adjustments::modify_call_stmt (gcall **stmt_p)
bool bool
ipa_param_body_adjustments::modify_gimple_stmt (gimple **stmt, ipa_param_body_adjustments::modify_gimple_stmt (gimple **stmt,
gimple_seq *extra_stmts) gimple_seq *extra_stmts,
gimple *orig_stmt)
{ {
bool modified = false; bool modified = false;
tree *t; tree *t;
@ -1746,7 +1819,7 @@ ipa_param_body_adjustments::modify_gimple_stmt (gimple **stmt,
break; break;
case GIMPLE_CALL: case GIMPLE_CALL:
modified |= modify_call_stmt ((gcall **) stmt); modified |= modify_call_stmt ((gcall **) stmt, orig_stmt);
break; break;
case GIMPLE_ASM: case GIMPLE_ASM:
@ -1803,7 +1876,7 @@ ipa_param_body_adjustments::modify_cfun_body ()
gimple *stmt = gsi_stmt (gsi); gimple *stmt = gsi_stmt (gsi);
gimple *stmt_copy = stmt; gimple *stmt_copy = stmt;
gimple_seq extra_stmts = NULL; gimple_seq extra_stmts = NULL;
bool modified = modify_gimple_stmt (&stmt, &extra_stmts); bool modified = modify_gimple_stmt (&stmt, &extra_stmts, NULL);
if (stmt != stmt_copy) if (stmt != stmt_copy)
{ {
gcc_checking_assert (modified); gcc_checking_assert (modified);
@ -1952,3 +2025,17 @@ ipa_param_body_adjustments::perform_cfun_body_modifications ()
return cfg_changed; return cfg_changed;
} }
/* Deallocate summaries which otherwise stay alive until the end of
compilation. */
void
ipa_edge_modifications_finalize ()
{
if (!ipa_edge_modifications)
return;
delete ipa_edge_modifications;
ipa_edge_modifications = NULL;
}

82
gcc/ipa-param-manipulation.h
View File

@ -54,7 +54,7 @@ or only a vector of ipa_adjusted_params.
When these classes are used in the context of call graph clone materialization When these classes are used in the context of call graph clone materialization
and subsequent call statement redirection - which is the point at which we and subsequent call statement redirection - which is the point at which we
modify arguments in call statements - they need to cooperate with each other in modify arguments in call statements - they need to cooperate with each other in
order to handle what we refer to as transitive (IPA-SRA) splits. These are order to handle what we refer to as pass-through (IPA-SRA) splits. These are
situations when a formal parameter of one function is split into several situations when a formal parameter of one function is split into several
smaller ones and some of them are then passed on in a call to another function smaller ones and some of them are then passed on in a call to another function
because the formal parameter of this callee has also been split. because the formal parameter of this callee has also been split.
@ -83,7 +83,7 @@ baz ()
Both bar and foo would have their parameter split. Foo would receive one Both bar and foo would have their parameter split. Foo would receive one
replacement representing s.b. Function bar would see its parameter split into replacement representing s.b. Function bar would see its parameter split into
one replacement representing z.s.a and another representing z.s.b which would one replacement representing z.s.a and another representing z.s.b which would
be passed on to foo. It would be a so called transitive split IPA-SRA be passed on to foo. It would be a so called pass-through split IPA-SRA
replacement, one which is passed in a call as an actual argument to another replacement, one which is passed in a call as an actual argument to another
IPA-SRA replacement in another function. IPA-SRA replacement in another function.
@ -95,30 +95,25 @@ all of the above.
Call redirection has to be able to find the right decl or SSA_NAME that Call redirection has to be able to find the right decl or SSA_NAME that
corresponds to the transitive split in the caller. The SSA names are assigned corresponds to the transitive split in the caller. The SSA names are assigned
right after clone materialization/ modification and cannot be "added" right after clone materialization/ modification and cannot be "added" to call
afterwards. Moreover, if the caller has been inlined the SSA_NAMEs in question arguments at any later point. Moreover, if the caller has been inlined the
no longer belong to PARM_DECLs but to VAR_DECLs, indistinguishable from any SSA_NAMEs in question no longer belong to PARM_DECLs but to VAR_DECLs,
others. indistinguishable from any others.
Therefore, when clone materialization finds a call statement which it knows is Therefore, when clone materialization finds a call statement which it knows is
a part of a transitive split, it will modify it into: a part of a transitive split, it will simply add as arguments all new "split"
replacements (that have grater or equal offset than the original call
argument):
foo (DUMMY_Z_VAR.s, repl_for_a, repl_for_b, <rest of original arguments>); foo (repl_for_a, repl_for_b, <rest of original arguments>);
It will also store {DUMMY_S_VAR, 32} and {DUMMY_S_VAR, 64} representing offsets It will also store into ipa_edge_modification_info (which is internal to
of z.s.a and z.s.b (assuming a 32-bit int) into foo's cgraph node ipa-param-modification.c) information about which replacement is which and
clone->performed_splits vector (which is storing structures of type where original arguments are. Call redirection will then invoke
ipa_param_performed_split also defined in this header file). ipa_param_adjustments::modify_call which will access this information and
eliminate all replacements which the callee does not expect (repl_for_a in our
Call redirection will identify that expression DUMMY_Z_VAR.s is based on a example above). In between these two steps, however, a call statement might
variable stored in performed_splits vector and learn that the following have extraneous arguments. */
arguments, already in SSA form, represent offsets 32 and 64 in a split original
parameter. It subtracts offset of DUMMY_Z_VAR.s from 32 and 64 and arrives at
offsets 0 and 32 within callee's original parameter. At this point it also
knows from the call graph that only the bit with offset 32 is needed and so
changes the call statement into final:
bar (repl_for_b, <rest of original arguments>); */
#ifndef IPA_PARAM_MANIPULATION_H #ifndef IPA_PARAM_MANIPULATION_H
#define IPA_PARAM_MANIPULATION_H #define IPA_PARAM_MANIPULATION_H
@ -207,21 +202,6 @@ struct GTY(()) ipa_adjusted_param
void ipa_dump_adjusted_parameters (FILE *f, void ipa_dump_adjusted_parameters (FILE *f,
vec<ipa_adjusted_param, va_gc> *adj_params); vec<ipa_adjusted_param, va_gc> *adj_params);
/* Structure to remember the split performed on a node so that edge redirection
(i.e. splitting arguments of call statements) know how split formal
parameters of the caller are represented. */
struct GTY(()) ipa_param_performed_split
{
/* The dummy VAR_DECL that was created instead of the split parameter that
sits in the call in the meantime between clone materialization and call
redirection. All entries in a vector of performed splits that correspond
to the same dumy decl must be grouped together. */
tree dummy_decl;
/* Offset into the original parameter. */
unsigned unit_offset;
};
/* Class used to record planned modifications to parameters of a function and /* Class used to record planned modifications to parameters of a function and
also to perform necessary modifications at the caller side at the gimple also to perform necessary modifications at the caller side at the gimple
level. Used to describe all cgraph node clones that have their parameters level. Used to describe all cgraph node clones that have their parameters
@ -244,9 +224,7 @@ public:
/* Modify a call statement arguments (and possibly remove the return value) /* Modify a call statement arguments (and possibly remove the return value)
as described in the data fields of this class. */ as described in the data fields of this class. */
gcall *modify_call (gcall *stmt, gcall *modify_call (cgraph_edge *cs, bool update_references);
vec<ipa_param_performed_split, va_gc> *performed_splits,
tree callee_decl, bool update_references);
/* Return if the first parameter is left intact. */ /* Return if the first parameter is left intact. */
bool first_param_intact_p (); bool first_param_intact_p ();
/* Build a function type corresponding to the modified call. */ /* Build a function type corresponding to the modified call. */
@ -293,15 +271,9 @@ struct ipa_param_body_replacement
tree base; tree base;
/* The new decl it should be replaced with. */ /* The new decl it should be replaced with. */
tree repl; tree repl;
/* When modifying clones during IPA clone materialization, this is a dummy /* Users of ipa_param_body_adjustments that modify standalone functions
decl used to mark calls in which we need to apply transitive splitting, outside of IPA clone materialization can use the following field for their
these dummy delcls are inserted as arguments to such calls and then internal purposes. */
followed by all the replacements with offset info stored in
ipa_param_performed_split.
Users of ipa_param_body_adjustments that modify standalone functions
outside of IPA clone materialization can use this field for their internal
purposes. */
tree dummy; tree dummy;
/* The offset within BASE that REPL represents. */ /* The offset within BASE that REPL represents. */
unsigned unit_offset; unsigned unit_offset;
@ -342,8 +314,7 @@ public:
/* Change the PARM_DECLs. */ /* Change the PARM_DECLs. */
void modify_formal_parameters (); void modify_formal_parameters ();
/* Register a replacement decl for the transformation done in APM. */ /* Register a replacement decl for the transformation done in APM. */
void register_replacement (ipa_adjusted_param *apm, tree replacement, void register_replacement (ipa_adjusted_param *apm, tree replacement);
tree dummy = NULL_TREE);
/* Lookup a replacement for a given offset within a given parameter. */ /* Lookup a replacement for a given offset within a given parameter. */
tree lookup_replacement (tree base, unsigned unit_offset); tree lookup_replacement (tree base, unsigned unit_offset);
/* Lookup a replacement for an expression, if there is one. */ /* Lookup a replacement for an expression, if there is one. */
@ -353,7 +324,8 @@ public:
parameter. */ parameter. */
tree get_replacement_ssa_base (tree old_decl); tree get_replacement_ssa_base (tree old_decl);
/* Modify a statement. */ /* Modify a statement. */
bool modify_gimple_stmt (gimple **stmt, gimple_seq *extra_stmts); bool modify_gimple_stmt (gimple **stmt, gimple_seq *extra_stmts,
gimple *orig_stmt);
/* Return the new chain of parameters. */ /* Return the new chain of parameters. */
tree get_new_param_chain (); tree get_new_param_chain ();
@ -380,9 +352,10 @@ private:
tree replace_removed_params_ssa_names (tree old_name, gimple *stmt); tree replace_removed_params_ssa_names (tree old_name, gimple *stmt);
bool modify_expression (tree *expr_p, bool convert); bool modify_expression (tree *expr_p, bool convert);
bool modify_assignment (gimple *stmt, gimple_seq *extra_stmts); bool modify_assignment (gimple *stmt, gimple_seq *extra_stmts);
bool modify_call_stmt (gcall **stmt_p); bool modify_call_stmt (gcall **stmt_p, gimple *orig_stmt);
bool modify_cfun_body (); bool modify_cfun_body ();
void reset_debug_stmts (); void reset_debug_stmts ();
void mark_dead_statements (tree dead_param);
/* Declaration of the function that is being transformed. */ /* Declaration of the function that is being transformed. */
@ -431,5 +404,8 @@ private:
void push_function_arg_decls (vec<tree> *args, tree fndecl); void push_function_arg_decls (vec<tree> *args, tree fndecl);
void push_function_arg_types (vec<tree> *types, tree fntype); void push_function_arg_types (vec<tree> *types, tree fntype);
void ipa_verify_edge_has_no_modifications (cgraph_edge *cs);
void ipa_edge_modifications_finalize ();
#endif /* IPA_PARAM_MANIPULATION_H */ #endif /* IPA_PARAM_MANIPULATION_H */

15
gcc/symtab-clones.h
View File

@ -26,8 +26,7 @@ struct GTY(()) clone_info
/* Constructor. */ /* Constructor. */
clone_info () clone_info ()
: tree_map (NULL), : tree_map (NULL),
param_adjustments (NULL), param_adjustments (NULL)
performed_splits (NULL)
{ {
} }
/* Constants discovered by IPA-CP, i.e. which parameter should be replaced /* Constants discovered by IPA-CP, i.e. which parameter should be replaced
@ -35,18 +34,6 @@ struct GTY(()) clone_info
vec<ipa_replace_map *, va_gc> *tree_map; vec<ipa_replace_map *, va_gc> *tree_map;
/* Parameter modification that IPA-SRA decided to perform. */ /* Parameter modification that IPA-SRA decided to perform. */
ipa_param_adjustments *param_adjustments; ipa_param_adjustments *param_adjustments;
/* Lists of dummy-decl and offset pairs representing split formal parameters
in the caller. Offsets of all new replacements are enumerated, those
coming from the same original parameter have the same dummy decl stored
along with them.
Dummy decls sit in call statement arguments followed by new parameter
decls (or their SSA names) in between (caller) clone materialization and
call redirection. Redirection then recognizes the dummy variable and
together with the stored offsets can reconstruct what exactly the new
parameter decls represent and can leave in place only those that the
callee expects. */
vec<ipa_param_performed_split, va_gc> *performed_splits;
/* Return clone_info, if available. */ /* Return clone_info, if available. */
static clone_info *get (cgraph_node *node); static clone_info *get (cgraph_node *node);

2
gcc/toplev.c
View File

@ -86,6 +86,7 @@ along with GCC; see the file COPYING3. If not see
#include "optinfo-emit-json.h" #include "optinfo-emit-json.h"
#include "ipa-modref-tree.h" #include "ipa-modref-tree.h"
#include "ipa-modref.h" #include "ipa-modref.h"
#include "ipa-param-manipulation.h"
#include "dbgcnt.h" #include "dbgcnt.h"
#if defined(DBX_DEBUGGING_INFO) || defined(XCOFF_DEBUGGING_INFO) #if defined(DBX_DEBUGGING_INFO) || defined(XCOFF_DEBUGGING_INFO)
@ -2393,6 +2394,7 @@ toplev::finalize (void)
ipa_reference_c_finalize (); ipa_reference_c_finalize ();
ipa_fnsummary_c_finalize (); ipa_fnsummary_c_finalize ();
ipa_modref_c_finalize (); ipa_modref_c_finalize ();
ipa_edge_modifications_finalize ();
cgraph_c_finalize (); cgraph_c_finalize ();
cgraphunit_c_finalize (); cgraphunit_c_finalize ();

103
gcc/tree-inline.c
View File

@ -1922,7 +1922,7 @@ remap_gimple_stmt (gimple *stmt, copy_body_data *id)
if (id->param_body_adjs) if (id->param_body_adjs)
{ {
gimple_seq extra_stmts = NULL; gimple_seq extra_stmts = NULL;
id->param_body_adjs->modify_gimple_stmt (&copy, &extra_stmts); id->param_body_adjs->modify_gimple_stmt (&copy, &extra_stmts, stmt);
if (!gimple_seq_empty_p (extra_stmts)) if (!gimple_seq_empty_p (extra_stmts))
{ {
memset (&wi, 0, sizeof (wi)); memset (&wi, 0, sizeof (wi));
@ -4730,7 +4730,6 @@ expand_call_inline (basic_block bb, gimple *stmt, copy_body_data *id,
use_operand_p use; use_operand_p use;
gimple *simtenter_stmt = NULL; gimple *simtenter_stmt = NULL;
vec<tree> *simtvars_save; vec<tree> *simtvars_save;
clone_info *info;
/* The gimplifier uses input_location in too many places, such as /* The gimplifier uses input_location in too many places, such as
internal_get_tmp_var (). */ internal_get_tmp_var (). */
@ -5055,40 +5054,6 @@ expand_call_inline (basic_block bb, gimple *stmt, copy_body_data *id,
/* Add local vars in this inlined callee to caller. */ /* Add local vars in this inlined callee to caller. */
add_local_variables (id->src_cfun, cfun, id); add_local_variables (id->src_cfun, cfun, id);
info = clone_info::get (id->src_node);
if (info && info->performed_splits)
{
clone_info *dst_info = clone_info::get_create (id->dst_node);
/* Any calls from the inlined function will be turned into calls from the
function we inline into. We must preserve notes about how to split
parameters such calls should be redirected/updated. */
unsigned len = vec_safe_length (info->performed_splits);
for (unsigned i = 0; i < len; i++)
{
ipa_param_performed_split ps
= (*info->performed_splits)[i];
ps.dummy_decl = remap_decl (ps.dummy_decl, id);
vec_safe_push (dst_info->performed_splits, ps);
}
if (flag_checking)
{
len = vec_safe_length (dst_info->performed_splits);
for (unsigned i = 0; i < len; i++)
{
ipa_param_performed_split *ps1
= &(*dst_info->performed_splits)[i];
for (unsigned j = i + 1; j < len; j++)
{
ipa_param_performed_split *ps2
= &(*dst_info->performed_splits)[j];
gcc_assert (ps1->dummy_decl != ps2->dummy_decl
|| ps1->unit_offset != ps2->unit_offset);
}
}
}
}
if (dump_enabled_p ()) if (dump_enabled_p ())
{ {
char buf[128]; char buf[128];
@ -6112,23 +6077,10 @@ tree_versionable_function_p (tree fndecl)
static void static void
update_clone_info (copy_body_data * id) update_clone_info (copy_body_data * id)
{ {
clone_info *dst_info = clone_info::get (id->dst_node); struct cgraph_node *this_node = id->dst_node;
vec<ipa_param_performed_split, va_gc> *cur_performed_splits if (!this_node->clones)
= dst_info ? dst_info->performed_splits : NULL;
if (cur_performed_splits)
{
unsigned len = cur_performed_splits->length ();
for (unsigned i = 0; i < len; i++)
{
ipa_param_performed_split *ps = &(*cur_performed_splits)[i];
ps->dummy_decl = remap_decl (ps->dummy_decl, id);
}
}
struct cgraph_node *node;
if (!id->dst_node->clones)
return; return;
for (node = id->dst_node->clones; node != id->dst_node;) for (cgraph_node *node = this_node->clones; node != this_node;)
{ {
/* First update replace maps to match the new body. */ /* First update replace maps to match the new body. */
clone_info *info = clone_info::get (node); clone_info *info = clone_info::get (node);
@ -6142,53 +6094,6 @@ update_clone_info (copy_body_data * id)
walk_tree (&replace_info->new_tree, copy_tree_body_r, id, NULL); walk_tree (&replace_info->new_tree, copy_tree_body_r, id, NULL);
} }
} }
if (info && info->performed_splits)
{
unsigned len = vec_safe_length (info->performed_splits);
for (unsigned i = 0; i < len; i++)
{
ipa_param_performed_split *ps
= &(*info->performed_splits)[i];
ps->dummy_decl = remap_decl (ps->dummy_decl, id);
}
}
if (unsigned len = vec_safe_length (cur_performed_splits))
{
/* We do not want to add current performed splits when we are saving
a copy of function body for later during inlining, that would just
duplicate all entries. So let's have a look whether anything
referring to the first dummy_decl is present. */
if (!info)
info = clone_info::get_create (node);
unsigned dst_len = vec_safe_length (info->performed_splits);
ipa_param_performed_split *first = &(*cur_performed_splits)[0];
for (unsigned i = 0; i < dst_len; i++)
if ((*info->performed_splits)[i].dummy_decl
== first->dummy_decl)
{
len = 0;
break;
}
for (unsigned i = 0; i < len; i++)
vec_safe_push (info->performed_splits,
(*cur_performed_splits)[i]);
if (flag_checking)
{
for (unsigned i = 0; i < dst_len; i++)
{
ipa_param_performed_split *ps1
= &(*info->performed_splits)[i];
for (unsigned j = i + 1; j < dst_len; j++)
{
ipa_param_performed_split *ps2
= &(*info->performed_splits)[j];
gcc_assert (ps1->dummy_decl != ps2->dummy_decl
|| ps1->unit_offset != ps2->unit_offset);
}
}
}
}
if (node->clones) if (node->clones)
node = node->clones; node = node->clones;