tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
/* GIMPLE store merging and byte swapping passes.
|
2021-01-04 10:26:59 +01:00
|
|
|
Copyright (C) 2009-2021 Free Software Foundation, Inc.
|
2016-10-28 16:18:50 +02:00
|
|
|
Contributed by ARM Ltd.
|
|
|
|
|
|
|
|
This file is part of GCC.
|
|
|
|
|
|
|
|
GCC is free software; you can redistribute it and/or modify it
|
|
|
|
under the terms of the GNU General Public License as published by
|
|
|
|
the Free Software Foundation; either version 3, or (at your option)
|
|
|
|
any later version.
|
|
|
|
|
|
|
|
GCC is distributed in the hope that it will be useful, but
|
|
|
|
WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
General Public License for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
|
|
along with GCC; see the file COPYING3. If not see
|
|
|
|
<http://www.gnu.org/licenses/>. */
|
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
/* The purpose of the store merging pass is to combine multiple memory stores
|
|
|
|
of constant values, values loaded from memory, bitwise operations on those,
|
|
|
|
or bit-field values, to consecutive locations, into fewer wider stores.
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
For example, if we have a sequence peforming four byte stores to
|
|
|
|
consecutive memory locations:
|
|
|
|
[p ] := imm1;
|
|
|
|
[p + 1B] := imm2;
|
|
|
|
[p + 2B] := imm3;
|
|
|
|
[p + 3B] := imm4;
|
|
|
|
we can transform this into a single 4-byte store if the target supports it:
|
2018-06-01 23:56:17 +02:00
|
|
|
[p] := imm1:imm2:imm3:imm4 concatenated according to endianness.
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
Or:
|
|
|
|
[p ] := [q ];
|
|
|
|
[p + 1B] := [q + 1B];
|
|
|
|
[p + 2B] := [q + 2B];
|
|
|
|
[p + 3B] := [q + 3B];
|
|
|
|
if there is no overlap can be transformed into a single 4-byte
|
|
|
|
load followed by single 4-byte store.
|
|
|
|
|
|
|
|
Or:
|
|
|
|
[p ] := [q ] ^ imm1;
|
|
|
|
[p + 1B] := [q + 1B] ^ imm2;
|
|
|
|
[p + 2B] := [q + 2B] ^ imm3;
|
|
|
|
[p + 3B] := [q + 3B] ^ imm4;
|
|
|
|
if there is no overlap can be transformed into a single 4-byte
|
|
|
|
load, xored with imm1:imm2:imm3:imm4 and stored using a single 4-byte store.
|
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
Or:
|
|
|
|
[p:1 ] := imm;
|
|
|
|
[p:31] := val & 0x7FFFFFFF;
|
|
|
|
we can transform this into a single 4-byte store if the target supports it:
|
|
|
|
[p] := imm:(val & 0x7FFFFFFF) concatenated according to endianness.
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
The algorithm is applied to each basic block in three phases:
|
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
1) Scan through the basic block and record assignments to destinations
|
|
|
|
that can be expressed as a store to memory of a certain size at a certain
|
|
|
|
bit offset from base expressions we can handle. For bit-fields we also
|
|
|
|
record the surrounding bit region, i.e. bits that could be stored in
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
a read-modify-write operation when storing the bit-field. Record store
|
|
|
|
chains to different bases in a hash_map (m_stores) and make sure to
|
Fix up duplicated duplicated words mostly in comments
In the r10-7197-gbae7b38cf8a21e068ad5c0bab089dedb78af3346 commit I've
noticed duplicated word in a message, which lead me to grep for those and
we have a tons of them.
I've used
grep -v 'long long\|optab optab\|template template\|double double' *.[chS] */*.[chS] *.def config/*/* 2>/dev/null | grep ' \([a-zA-Z]\+\) \1 '
Note, the command will not detect the doubled words at the start or end of
line or when one of the words is at the end of line and the next one at the
start of another one.
Some of it is fairly obvious, e.g. all the "the the" cases which is
something I've posted and committed patch for already e.g. in 2016,
other cases are often valid, e.g. "that that" seems to look mostly ok to me.
Some cases are quite hard to figure out, I've left out some of them from the
patch (e.g. "and and" in some cases isn't talking about bitwise/logical and
and so looks incorrect, but in other cases it is talking about those
operations).
In most cases the right solution seems to be to remove one of the duplicated
words, but not always.
I think most important are the ones with user visible messages (in the patch
3 of the first 4 hunks), the rest is just comments (and internal
documentation; for that see the doc/tm.texi changes).
2020-03-17 Jakub Jelinek <jakub@redhat.com>
* lra-spills.c (remove_pseudos): Fix up duplicated word issue in
a dump message.
* tree-sra.c (create_access_replacement): Fix up duplicated word issue
in a comment.
* read-rtl-function.c (find_param_by_name,
function_reader::parse_enum_value, function_reader::get_insn_by_uid):
Likewise.
* spellcheck.c (get_edit_distance_cutoff): Likewise.
* tree-data-ref.c (create_ifn_alias_checks): Likewise.
* tree.def (SWITCH_EXPR): Likewise.
* selftest.c (assert_str_contains): Likewise.
* ipa-param-manipulation.h (class ipa_param_body_adjustments):
Likewise.
* tree-ssa-math-opts.c (convert_expand_mult_copysign): Likewise.
* tree-ssa-loop-split.c (find_vdef_in_loop): Likewise.
* langhooks.h (struct lang_hooks_for_decls): Likewise.
* ipa-prop.h (struct ipa_param_descriptor): Likewise.
* tree-ssa-strlen.c (handle_builtin_string_cmp, handle_store):
Likewise.
* tree-ssa-dom.c (simplify_stmt_for_jump_threading): Likewise.
* tree-ssa-reassoc.c (reassociate_bb): Likewise.
* tree.c (component_ref_size): Likewise.
* hsa-common.c (hsa_init_compilation_unit_data): Likewise.
* gimple-ssa-sprintf.c (get_string_length, format_string,
format_directive): Likewise.
* omp-grid.c (grid_process_kernel_body_copy): Likewise.
* input.c (string_concat_db::get_string_concatenation,
test_lexer_string_locations_ucn4): Likewise.
* cfgexpand.c (pass_expand::execute): Likewise.
* gimple-ssa-warn-restrict.c (builtin_memref::offset_out_of_bounds,
maybe_diag_overlap): Likewise.
* rtl.c (RTX_CODE_HWINT_P_1): Likewise.
* shrink-wrap.c (spread_components): Likewise.
* tree-ssa-dse.c (initialize_ao_ref_for_dse, valid_ao_ref_for_dse):
Likewise.
* tree-call-cdce.c (shrink_wrap_one_built_in_call_with_conds):
Likewise.
* dwarf2out.c (dwarf2out_early_finish): Likewise.
* gimple-ssa-store-merging.c: Likewise.
* ira-costs.c (record_operand_costs): Likewise.
* tree-vect-loop.c (vectorizable_reduction): Likewise.
* target.def (dispatch): Likewise.
(validate_dims, gen_ccmp_first): Fix up duplicated word issue
in documentation text.
* doc/tm.texi: Regenerated.
* config/i386/x86-tune.def (X86_TUNE_PARTIAL_FLAG_REG_STALL): Fix up
duplicated word issue in a comment.
* config/i386/i386.c (ix86_test_loading_unspec): Likewise.
* config/i386/i386-features.c (remove_partial_avx_dependency):
Likewise.
* config/msp430/msp430.c (msp430_select_section): Likewise.
* config/gcn/gcn-run.c (load_image): Likewise.
* config/aarch64/aarch64-sve.md (sve_ld1r<mode>): Likewise.
* config/aarch64/aarch64.c (aarch64_gen_adjusted_ldpstp): Likewise.
* config/aarch64/falkor-tag-collision-avoidance.c
(single_dest_per_chain): Likewise.
* config/nvptx/nvptx.c (nvptx_record_fndecl): Likewise.
* config/fr30/fr30.c (fr30_arg_partial_bytes): Likewise.
* config/rs6000/rs6000-string.c (expand_cmp_vec_sequence): Likewise.
* config/rs6000/rs6000-p8swap.c (replace_swapped_load_constant):
Likewise.
* config/rs6000/rs6000-c.c (rs6000_target_modify_macros): Likewise.
* config/rs6000/rs6000.c (rs6000_option_override_internal): Likewise.
* config/rs6000/rs6000-logue.c
(rs6000_emit_probe_stack_range_stack_clash): Likewise.
* config/nds32/nds32-md-auxiliary.c (nds32_split_ashiftdi3): Likewise.
Fix various other issues in the comment.
c-family/
* c-common.c (resolve_overloaded_builtin): Fix up duplicated word
issue in a diagnostic message.
cp/
* pt.c (tsubst): Fix up duplicated word issue in a diagnostic message.
(lookup_template_class_1, tsubst_expr): Fix up duplicated word issue
in a comment.
* parser.c (cp_parser_statement, cp_parser_linkage_specification,
cp_parser_placeholder_type_specifier,
cp_parser_constraint_requires_parens): Likewise.
* name-lookup.c (suggest_alternative_in_explicit_scope): Likewise.
fortran/
* array.c (gfc_check_iter_variable): Fix up duplicated word issue
in a comment.
* arith.c (gfc_arith_concat): Likewise.
* resolve.c (gfc_resolve_ref): Likewise.
* frontend-passes.c (matmul_lhs_realloc): Likewise.
* module.c (gfc_match_submodule, load_needed): Likewise.
* trans-expr.c (gfc_init_se): Likewise.
2020-03-17 13:52:19 +01:00
|
|
|
terminate such chains when appropriate (for example when the stored
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
values get used subsequently).
|
2016-10-28 16:18:50 +02:00
|
|
|
These stores can be a result of structure element initializers, array stores
|
|
|
|
etc. A store_immediate_info object is recorded for every such store.
|
|
|
|
Record as many such assignments to a single base as possible until a
|
|
|
|
statement that interferes with the store sequence is encountered.
|
2018-06-01 23:56:17 +02:00
|
|
|
Each store has up to 2 operands, which can be a either constant, a memory
|
|
|
|
load or an SSA name, from which the value to be stored can be computed.
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
At most one of the operands can be a constant. The operands are recorded
|
|
|
|
in store_operand_info struct.
|
2016-10-28 16:18:50 +02:00
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
2) Analyze the chains of stores recorded in phase 1) (i.e. the vector of
|
2016-10-28 16:18:50 +02:00
|
|
|
store_immediate_info objects) and coalesce contiguous stores into
|
2018-06-01 23:56:17 +02:00
|
|
|
merged_store_group objects. For bit-field stores, we don't need to
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
require the stores to be contiguous, just their surrounding bit regions
|
|
|
|
have to be contiguous. If the expression being stored is different
|
|
|
|
between adjacent stores, such as one store storing a constant and
|
|
|
|
following storing a value loaded from memory, or if the loaded memory
|
|
|
|
objects are not adjacent, a new merged_store_group is created as well.
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
For example, given the stores:
|
|
|
|
[p ] := 0;
|
|
|
|
[p + 1B] := 1;
|
|
|
|
[p + 3B] := 0;
|
|
|
|
[p + 4B] := 1;
|
|
|
|
[p + 5B] := 0;
|
|
|
|
[p + 6B] := 0;
|
|
|
|
This phase would produce two merged_store_group objects, one recording the
|
|
|
|
two bytes stored in the memory region [p : p + 1] and another
|
|
|
|
recording the four bytes stored in the memory region [p + 3 : p + 6].
|
|
|
|
|
|
|
|
3) The merged_store_group objects produced in phase 2) are processed
|
|
|
|
to generate the sequence of wider stores that set the contiguous memory
|
|
|
|
regions to the sequence of bytes that correspond to it. This may emit
|
|
|
|
multiple stores per store group to handle contiguous stores that are not
|
|
|
|
of a size that is a power of 2. For example it can try to emit a 40-bit
|
|
|
|
store as a 32-bit store followed by an 8-bit store.
|
2018-06-01 23:56:17 +02:00
|
|
|
We try to emit as wide stores as we can while respecting STRICT_ALIGNMENT
|
|
|
|
or TARGET_SLOW_UNALIGNED_ACCESS settings.
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
Note on endianness and example:
|
|
|
|
Consider 2 contiguous 16-bit stores followed by 2 contiguous 8-bit stores:
|
|
|
|
[p ] := 0x1234;
|
|
|
|
[p + 2B] := 0x5678;
|
|
|
|
[p + 4B] := 0xab;
|
|
|
|
[p + 5B] := 0xcd;
|
|
|
|
|
|
|
|
The memory layout for little-endian (LE) and big-endian (BE) must be:
|
|
|
|
p |LE|BE|
|
|
|
|
---------
|
|
|
|
0 |34|12|
|
|
|
|
1 |12|34|
|
|
|
|
2 |78|56|
|
|
|
|
3 |56|78|
|
|
|
|
4 |ab|ab|
|
|
|
|
5 |cd|cd|
|
|
|
|
|
|
|
|
To merge these into a single 48-bit merged value 'val' in phase 2)
|
|
|
|
on little-endian we insert stores to higher (consecutive) bitpositions
|
|
|
|
into the most significant bits of the merged value.
|
|
|
|
The final merged value would be: 0xcdab56781234
|
|
|
|
|
|
|
|
For big-endian we insert stores to higher bitpositions into the least
|
|
|
|
significant bits of the merged value.
|
|
|
|
The final merged value would be: 0x12345678abcd
|
|
|
|
|
|
|
|
Then, in phase 3), we want to emit this 48-bit value as a 32-bit store
|
|
|
|
followed by a 16-bit store. Again, we must consider endianness when
|
|
|
|
breaking down the 48-bit value 'val' computed above.
|
|
|
|
For little endian we emit:
|
|
|
|
[p] (32-bit) := 0x56781234; // val & 0x0000ffffffff;
|
|
|
|
[p + 4B] (16-bit) := 0xcdab; // (val & 0xffff00000000) >> 32;
|
|
|
|
|
|
|
|
Whereas for big-endian we emit:
|
|
|
|
[p] (32-bit) := 0x12345678; // (val & 0xffffffff0000) >> 16;
|
|
|
|
[p + 4B] (16-bit) := 0xabcd; // val & 0x00000000ffff; */
|
|
|
|
|
|
|
|
#include "config.h"
|
|
|
|
#include "system.h"
|
|
|
|
#include "coretypes.h"
|
|
|
|
#include "backend.h"
|
|
|
|
#include "tree.h"
|
|
|
|
#include "gimple.h"
|
|
|
|
#include "builtins.h"
|
|
|
|
#include "fold-const.h"
|
|
|
|
#include "tree-pass.h"
|
|
|
|
#include "ssa.h"
|
|
|
|
#include "gimple-pretty-print.h"
|
|
|
|
#include "alias.h"
|
|
|
|
#include "fold-const.h"
|
|
|
|
#include "print-tree.h"
|
|
|
|
#include "tree-hash-traits.h"
|
|
|
|
#include "gimple-iterator.h"
|
|
|
|
#include "gimplify.h"
|
2018-06-01 23:56:17 +02:00
|
|
|
#include "gimple-fold.h"
|
2016-10-28 16:18:50 +02:00
|
|
|
#include "stor-layout.h"
|
|
|
|
#include "timevar.h"
|
2019-10-02 15:35:40 +02:00
|
|
|
#include "cfganal.h"
|
|
|
|
#include "cfgcleanup.h"
|
2016-10-28 16:18:50 +02:00
|
|
|
#include "tree-cfg.h"
|
2019-10-02 15:35:40 +02:00
|
|
|
#include "except.h"
|
2016-10-28 16:18:50 +02:00
|
|
|
#include "tree-eh.h"
|
|
|
|
#include "target.h"
|
2016-11-02 16:00:48 +01:00
|
|
|
#include "gimplify-me.h"
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
#include "rtl.h"
|
|
|
|
#include "expr.h" /* For get_bit_range. */
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
#include "optabs-tree.h"
|
2019-09-18 11:03:14 +02:00
|
|
|
#include "dbgcnt.h"
|
2016-11-08 17:04:20 +01:00
|
|
|
#include "selftest.h"
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
/* The maximum size (in bits) of the stores this pass should generate. */
|
|
|
|
#define MAX_STORE_BITSIZE (BITS_PER_WORD)
|
|
|
|
#define MAX_STORE_BYTES (MAX_STORE_BITSIZE / BITS_PER_UNIT)
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Limit to bound the number of aliasing checks for loads with the same
|
|
|
|
vuse as the corresponding store. */
|
|
|
|
#define MAX_STORE_ALIAS_CHECKS 64
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
namespace {
|
|
|
|
|
2017-12-01 09:18:52 +01:00
|
|
|
struct bswap_stat
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
|
|
|
/* Number of hand-written 16-bit nop / bswaps found. */
|
|
|
|
int found_16bit;
|
|
|
|
|
|
|
|
/* Number of hand-written 32-bit nop / bswaps found. */
|
|
|
|
int found_32bit;
|
|
|
|
|
|
|
|
/* Number of hand-written 64-bit nop / bswaps found. */
|
|
|
|
int found_64bit;
|
|
|
|
} nop_stats, bswap_stats;
|
|
|
|
|
|
|
|
/* A symbolic number structure is used to detect byte permutation and selection
|
|
|
|
patterns of a source. To achieve that, its field N contains an artificial
|
|
|
|
number consisting of BITS_PER_MARKER sized markers tracking where does each
|
|
|
|
byte come from in the source:
|
|
|
|
|
|
|
|
0 - target byte has the value 0
|
|
|
|
FF - target byte has an unknown value (eg. due to sign extension)
|
|
|
|
1..size - marker value is the byte index in the source (0 for lsb).
|
|
|
|
|
|
|
|
To detect permutations on memory sources (arrays and structures), a symbolic
|
|
|
|
number is also associated:
|
|
|
|
- a base address BASE_ADDR and an OFFSET giving the address of the source;
|
|
|
|
- a range which gives the difference between the highest and lowest accessed
|
|
|
|
memory location to make such a symbolic number;
|
|
|
|
- the address SRC of the source element of lowest address as a convenience
|
|
|
|
to easily get BASE_ADDR + offset + lowest bytepos;
|
|
|
|
- number of expressions N_OPS bitwise ored together to represent
|
|
|
|
approximate cost of the computation.
|
|
|
|
|
|
|
|
Note 1: the range is different from size as size reflects the size of the
|
|
|
|
type of the current expression. For instance, for an array char a[],
|
|
|
|
(short) a[0] | (short) a[3] would have a size of 2 but a range of 4 while
|
|
|
|
(short) a[0] | ((short) a[0] << 1) would still have a size of 2 but this
|
|
|
|
time a range of 1.
|
|
|
|
|
|
|
|
Note 2: for non-memory sources, range holds the same value as size.
|
|
|
|
|
|
|
|
Note 3: SRC points to the SSA_NAME in case of non-memory source. */
|
|
|
|
|
|
|
|
struct symbolic_number {
|
|
|
|
uint64_t n;
|
|
|
|
tree type;
|
|
|
|
tree base_addr;
|
|
|
|
tree offset;
|
2017-12-20 13:55:57 +01:00
|
|
|
poly_int64_pod bytepos;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
tree src;
|
|
|
|
tree alias_set;
|
|
|
|
tree vuse;
|
|
|
|
unsigned HOST_WIDE_INT range;
|
|
|
|
int n_ops;
|
|
|
|
};
|
|
|
|
|
|
|
|
#define BITS_PER_MARKER 8
|
|
|
|
#define MARKER_MASK ((1 << BITS_PER_MARKER) - 1)
|
|
|
|
#define MARKER_BYTE_UNKNOWN MARKER_MASK
|
|
|
|
#define HEAD_MARKER(n, size) \
|
|
|
|
((n) & ((uint64_t) MARKER_MASK << (((size) - 1) * BITS_PER_MARKER)))
|
|
|
|
|
|
|
|
/* The number which the find_bswap_or_nop_1 result should match in
|
|
|
|
order to have a nop. The number is masked according to the size of
|
|
|
|
the symbolic number before using it. */
|
|
|
|
#define CMPNOP (sizeof (int64_t) < 8 ? 0 : \
|
|
|
|
(uint64_t)0x08070605 << 32 | 0x04030201)
|
|
|
|
|
|
|
|
/* The number which the find_bswap_or_nop_1 result should match in
|
|
|
|
order to have a byte swap. The number is masked according to the
|
|
|
|
size of the symbolic number before using it. */
|
|
|
|
#define CMPXCHG (sizeof (int64_t) < 8 ? 0 : \
|
|
|
|
(uint64_t)0x01020304 << 32 | 0x05060708)
|
|
|
|
|
|
|
|
/* Perform a SHIFT or ROTATE operation by COUNT bits on symbolic
|
|
|
|
number N. Return false if the requested operation is not permitted
|
|
|
|
on a symbolic number. */
|
|
|
|
|
|
|
|
inline bool
|
|
|
|
do_shift_rotate (enum tree_code code,
|
|
|
|
struct symbolic_number *n,
|
|
|
|
int count)
|
|
|
|
{
|
|
|
|
int i, size = TYPE_PRECISION (n->type) / BITS_PER_UNIT;
|
|
|
|
unsigned head_marker;
|
|
|
|
|
2018-11-01 13:31:45 +01:00
|
|
|
if (count < 0
|
|
|
|
|| count >= TYPE_PRECISION (n->type)
|
|
|
|
|| count % BITS_PER_UNIT != 0)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
return false;
|
|
|
|
count = (count / BITS_PER_UNIT) * BITS_PER_MARKER;
|
|
|
|
|
|
|
|
/* Zero out the extra bits of N in order to avoid them being shifted
|
|
|
|
into the significant bits. */
|
|
|
|
if (size < 64 / BITS_PER_MARKER)
|
|
|
|
n->n &= ((uint64_t) 1 << (size * BITS_PER_MARKER)) - 1;
|
|
|
|
|
|
|
|
switch (code)
|
|
|
|
{
|
|
|
|
case LSHIFT_EXPR:
|
|
|
|
n->n <<= count;
|
|
|
|
break;
|
|
|
|
case RSHIFT_EXPR:
|
|
|
|
head_marker = HEAD_MARKER (n->n, size);
|
|
|
|
n->n >>= count;
|
|
|
|
/* Arithmetic shift of signed type: result is dependent on the value. */
|
|
|
|
if (!TYPE_UNSIGNED (n->type) && head_marker)
|
|
|
|
for (i = 0; i < count / BITS_PER_MARKER; i++)
|
|
|
|
n->n |= (uint64_t) MARKER_BYTE_UNKNOWN
|
|
|
|
<< ((size - 1 - i) * BITS_PER_MARKER);
|
|
|
|
break;
|
|
|
|
case LROTATE_EXPR:
|
|
|
|
n->n = (n->n << count) | (n->n >> ((size * BITS_PER_MARKER) - count));
|
|
|
|
break;
|
|
|
|
case RROTATE_EXPR:
|
|
|
|
n->n = (n->n >> count) | (n->n << ((size * BITS_PER_MARKER) - count));
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
/* Zero unused bits for size. */
|
|
|
|
if (size < 64 / BITS_PER_MARKER)
|
|
|
|
n->n &= ((uint64_t) 1 << (size * BITS_PER_MARKER)) - 1;
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Perform sanity checking for the symbolic number N and the gimple
|
|
|
|
statement STMT. */
|
|
|
|
|
|
|
|
inline bool
|
|
|
|
verify_symbolic_number_p (struct symbolic_number *n, gimple *stmt)
|
|
|
|
{
|
|
|
|
tree lhs_type;
|
|
|
|
|
|
|
|
lhs_type = gimple_expr_type (stmt);
|
|
|
|
|
2020-03-31 11:04:32 +02:00
|
|
|
if (TREE_CODE (lhs_type) != INTEGER_TYPE
|
|
|
|
&& TREE_CODE (lhs_type) != ENUMERAL_TYPE)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
return false;
|
|
|
|
|
|
|
|
if (TYPE_PRECISION (lhs_type) != TYPE_PRECISION (n->type))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Initialize the symbolic number N for the bswap pass from the base element
|
|
|
|
SRC manipulated by the bitwise OR expression. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
init_symbolic_number (struct symbolic_number *n, tree src)
|
|
|
|
{
|
|
|
|
int size;
|
|
|
|
|
2021-04-01 10:51:03 +02:00
|
|
|
if (!INTEGRAL_TYPE_P (TREE_TYPE (src)) && !POINTER_TYPE_P (TREE_TYPE (src)))
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
return false;
|
|
|
|
|
|
|
|
n->base_addr = n->offset = n->alias_set = n->vuse = NULL_TREE;
|
|
|
|
n->src = src;
|
|
|
|
|
|
|
|
/* Set up the symbolic number N by setting each byte to a value between 1 and
|
|
|
|
the byte size of rhs1. The highest order byte is set to n->size and the
|
|
|
|
lowest order byte to 1. */
|
|
|
|
n->type = TREE_TYPE (src);
|
|
|
|
size = TYPE_PRECISION (n->type);
|
|
|
|
if (size % BITS_PER_UNIT != 0)
|
|
|
|
return false;
|
|
|
|
size /= BITS_PER_UNIT;
|
|
|
|
if (size > 64 / BITS_PER_MARKER)
|
|
|
|
return false;
|
|
|
|
n->range = size;
|
|
|
|
n->n = CMPNOP;
|
|
|
|
n->n_ops = 1;
|
|
|
|
|
|
|
|
if (size < 64 / BITS_PER_MARKER)
|
|
|
|
n->n &= ((uint64_t) 1 << (size * BITS_PER_MARKER)) - 1;
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Check if STMT might be a byte swap or a nop from a memory source and returns
|
|
|
|
the answer. If so, REF is that memory source and the base of the memory area
|
|
|
|
accessed and the offset of the access from that base are recorded in N. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
find_bswap_or_nop_load (gimple *stmt, tree ref, struct symbolic_number *n)
|
|
|
|
{
|
|
|
|
/* Leaf node is an array or component ref. Memorize its base and
|
|
|
|
offset from base to compare to other such leaf node. */
|
2017-12-21 07:57:41 +01:00
|
|
|
poly_int64 bitsize, bitpos, bytepos;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
machine_mode mode;
|
|
|
|
int unsignedp, reversep, volatilep;
|
|
|
|
tree offset, base_addr;
|
|
|
|
|
|
|
|
/* Not prepared to handle PDP endian. */
|
|
|
|
if (BYTES_BIG_ENDIAN != WORDS_BIG_ENDIAN)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
if (!gimple_assign_load_p (stmt) || gimple_has_volatile_ops (stmt))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
base_addr = get_inner_reference (ref, &bitsize, &bitpos, &offset, &mode,
|
|
|
|
&unsignedp, &reversep, &volatilep);
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (TREE_CODE (base_addr) == TARGET_MEM_REF)
|
|
|
|
/* Do not rewrite TARGET_MEM_REF. */
|
|
|
|
return false;
|
|
|
|
else if (TREE_CODE (base_addr) == MEM_REF)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
2017-12-21 08:02:06 +01:00
|
|
|
poly_offset_int bit_offset = 0;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
tree off = TREE_OPERAND (base_addr, 1);
|
|
|
|
|
|
|
|
if (!integer_zerop (off))
|
|
|
|
{
|
2017-12-21 08:02:06 +01:00
|
|
|
poly_offset_int boff = mem_ref_offset (base_addr);
|
|
|
|
boff <<= LOG2_BITS_PER_UNIT;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
bit_offset += boff;
|
|
|
|
}
|
|
|
|
|
|
|
|
base_addr = TREE_OPERAND (base_addr, 0);
|
|
|
|
|
|
|
|
/* Avoid returning a negative bitpos as this may wreak havoc later. */
|
2017-12-21 08:02:06 +01:00
|
|
|
if (maybe_lt (bit_offset, 0))
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
2017-12-21 08:02:06 +01:00
|
|
|
tree byte_offset = wide_int_to_tree
|
|
|
|
(sizetype, bits_to_bytes_round_down (bit_offset));
|
|
|
|
bit_offset = num_trailing_bits (bit_offset);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
if (offset)
|
2017-12-21 08:02:06 +01:00
|
|
|
offset = size_binop (PLUS_EXPR, offset, byte_offset);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
else
|
2017-12-21 08:02:06 +01:00
|
|
|
offset = byte_offset;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
|
2017-12-21 08:02:06 +01:00
|
|
|
bitpos += bit_offset.force_shwi ();
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
else
|
|
|
|
base_addr = build_fold_addr_expr (base_addr);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
2017-12-21 07:57:41 +01:00
|
|
|
if (!multiple_p (bitpos, BITS_PER_UNIT, &bytepos))
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
return false;
|
2017-12-21 07:57:41 +01:00
|
|
|
if (!multiple_p (bitsize, BITS_PER_UNIT))
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
return false;
|
|
|
|
if (reversep)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
if (!init_symbolic_number (n, ref))
|
|
|
|
return false;
|
|
|
|
n->base_addr = base_addr;
|
|
|
|
n->offset = offset;
|
2017-12-21 07:57:41 +01:00
|
|
|
n->bytepos = bytepos;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
n->alias_set = reference_alias_ptr_type (ref);
|
|
|
|
n->vuse = gimple_vuse (stmt);
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Compute the symbolic number N representing the result of a bitwise OR on 2
|
|
|
|
symbolic number N1 and N2 whose source statements are respectively
|
|
|
|
SOURCE_STMT1 and SOURCE_STMT2. */
|
|
|
|
|
|
|
|
gimple *
|
|
|
|
perform_symbolic_merge (gimple *source_stmt1, struct symbolic_number *n1,
|
|
|
|
gimple *source_stmt2, struct symbolic_number *n2,
|
|
|
|
struct symbolic_number *n)
|
|
|
|
{
|
|
|
|
int i, size;
|
|
|
|
uint64_t mask;
|
|
|
|
gimple *source_stmt;
|
|
|
|
struct symbolic_number *n_start;
|
|
|
|
|
|
|
|
tree rhs1 = gimple_assign_rhs1 (source_stmt1);
|
|
|
|
if (TREE_CODE (rhs1) == BIT_FIELD_REF
|
|
|
|
&& TREE_CODE (TREE_OPERAND (rhs1, 0)) == SSA_NAME)
|
|
|
|
rhs1 = TREE_OPERAND (rhs1, 0);
|
|
|
|
tree rhs2 = gimple_assign_rhs1 (source_stmt2);
|
|
|
|
if (TREE_CODE (rhs2) == BIT_FIELD_REF
|
|
|
|
&& TREE_CODE (TREE_OPERAND (rhs2, 0)) == SSA_NAME)
|
|
|
|
rhs2 = TREE_OPERAND (rhs2, 0);
|
|
|
|
|
|
|
|
/* Sources are different, cancel bswap if they are not memory location with
|
|
|
|
the same base (array, structure, ...). */
|
|
|
|
if (rhs1 != rhs2)
|
|
|
|
{
|
|
|
|
uint64_t inc;
|
2017-12-20 13:55:57 +01:00
|
|
|
HOST_WIDE_INT start1, start2, start_sub, end_sub, end1, end2, end;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
struct symbolic_number *toinc_n_ptr, *n_end;
|
|
|
|
basic_block bb1, bb2;
|
|
|
|
|
|
|
|
if (!n1->base_addr || !n2->base_addr
|
|
|
|
|| !operand_equal_p (n1->base_addr, n2->base_addr, 0))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
if (!n1->offset != !n2->offset
|
|
|
|
|| (n1->offset && !operand_equal_p (n1->offset, n2->offset, 0)))
|
|
|
|
return NULL;
|
|
|
|
|
2017-12-20 13:55:57 +01:00
|
|
|
start1 = 0;
|
|
|
|
if (!(n2->bytepos - n1->bytepos).is_constant (&start2))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
if (start1 < start2)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
|
|
|
n_start = n1;
|
2017-12-20 13:55:57 +01:00
|
|
|
start_sub = start2 - start1;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
n_start = n2;
|
2017-12-20 13:55:57 +01:00
|
|
|
start_sub = start1 - start2;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
bb1 = gimple_bb (source_stmt1);
|
|
|
|
bb2 = gimple_bb (source_stmt2);
|
|
|
|
if (dominated_by_p (CDI_DOMINATORS, bb1, bb2))
|
|
|
|
source_stmt = source_stmt1;
|
|
|
|
else
|
|
|
|
source_stmt = source_stmt2;
|
|
|
|
|
|
|
|
/* Find the highest address at which a load is performed and
|
|
|
|
compute related info. */
|
2017-12-20 13:55:57 +01:00
|
|
|
end1 = start1 + (n1->range - 1);
|
|
|
|
end2 = start2 + (n2->range - 1);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
if (end1 < end2)
|
|
|
|
{
|
|
|
|
end = end2;
|
|
|
|
end_sub = end2 - end1;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
end = end1;
|
|
|
|
end_sub = end1 - end2;
|
|
|
|
}
|
|
|
|
n_end = (end2 > end1) ? n2 : n1;
|
|
|
|
|
|
|
|
/* Find symbolic number whose lsb is the most significant. */
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
toinc_n_ptr = (n_end == n1) ? n2 : n1;
|
|
|
|
else
|
|
|
|
toinc_n_ptr = (n_start == n1) ? n2 : n1;
|
|
|
|
|
2017-12-20 13:55:57 +01:00
|
|
|
n->range = end - MIN (start1, start2) + 1;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
|
|
|
/* Check that the range of memory covered can be represented by
|
|
|
|
a symbolic number. */
|
|
|
|
if (n->range > 64 / BITS_PER_MARKER)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
/* Reinterpret byte marks in symbolic number holding the value of
|
|
|
|
bigger weight according to target endianness. */
|
|
|
|
inc = BYTES_BIG_ENDIAN ? end_sub : start_sub;
|
|
|
|
size = TYPE_PRECISION (n1->type) / BITS_PER_UNIT;
|
|
|
|
for (i = 0; i < size; i++, inc <<= BITS_PER_MARKER)
|
|
|
|
{
|
|
|
|
unsigned marker
|
|
|
|
= (toinc_n_ptr->n >> (i * BITS_PER_MARKER)) & MARKER_MASK;
|
|
|
|
if (marker && marker != MARKER_BYTE_UNKNOWN)
|
|
|
|
toinc_n_ptr->n += inc;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
n->range = n1->range;
|
|
|
|
n_start = n1;
|
|
|
|
source_stmt = source_stmt1;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!n1->alias_set
|
|
|
|
|| alias_ptr_types_compatible_p (n1->alias_set, n2->alias_set))
|
|
|
|
n->alias_set = n1->alias_set;
|
|
|
|
else
|
|
|
|
n->alias_set = ptr_type_node;
|
|
|
|
n->vuse = n_start->vuse;
|
|
|
|
n->base_addr = n_start->base_addr;
|
|
|
|
n->offset = n_start->offset;
|
|
|
|
n->src = n_start->src;
|
|
|
|
n->bytepos = n_start->bytepos;
|
|
|
|
n->type = n_start->type;
|
|
|
|
size = TYPE_PRECISION (n->type) / BITS_PER_UNIT;
|
|
|
|
|
|
|
|
for (i = 0, mask = MARKER_MASK; i < size; i++, mask <<= BITS_PER_MARKER)
|
|
|
|
{
|
|
|
|
uint64_t masked1, masked2;
|
|
|
|
|
|
|
|
masked1 = n1->n & mask;
|
|
|
|
masked2 = n2->n & mask;
|
|
|
|
if (masked1 && masked2 && masked1 != masked2)
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
n->n = n1->n | n2->n;
|
|
|
|
n->n_ops = n1->n_ops + n2->n_ops;
|
|
|
|
|
|
|
|
return source_stmt;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* find_bswap_or_nop_1 invokes itself recursively with N and tries to perform
|
|
|
|
the operation given by the rhs of STMT on the result. If the operation
|
|
|
|
could successfully be executed the function returns a gimple stmt whose
|
|
|
|
rhs's first tree is the expression of the source operand and NULL
|
|
|
|
otherwise. */
|
|
|
|
|
|
|
|
gimple *
|
|
|
|
find_bswap_or_nop_1 (gimple *stmt, struct symbolic_number *n, int limit)
|
|
|
|
{
|
|
|
|
enum tree_code code;
|
|
|
|
tree rhs1, rhs2 = NULL;
|
|
|
|
gimple *rhs1_stmt, *rhs2_stmt, *source_stmt1;
|
|
|
|
enum gimple_rhs_class rhs_class;
|
|
|
|
|
|
|
|
if (!limit || !is_gimple_assign (stmt))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
rhs1 = gimple_assign_rhs1 (stmt);
|
|
|
|
|
|
|
|
if (find_bswap_or_nop_load (stmt, rhs1, n))
|
|
|
|
return stmt;
|
|
|
|
|
|
|
|
/* Handle BIT_FIELD_REF. */
|
|
|
|
if (TREE_CODE (rhs1) == BIT_FIELD_REF
|
|
|
|
&& TREE_CODE (TREE_OPERAND (rhs1, 0)) == SSA_NAME)
|
|
|
|
{
|
2020-06-24 16:48:45 +02:00
|
|
|
if (!tree_fits_uhwi_p (TREE_OPERAND (rhs1, 1))
|
|
|
|
|| !tree_fits_uhwi_p (TREE_OPERAND (rhs1, 2)))
|
|
|
|
return NULL;
|
|
|
|
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
unsigned HOST_WIDE_INT bitsize = tree_to_uhwi (TREE_OPERAND (rhs1, 1));
|
|
|
|
unsigned HOST_WIDE_INT bitpos = tree_to_uhwi (TREE_OPERAND (rhs1, 2));
|
|
|
|
if (bitpos % BITS_PER_UNIT == 0
|
|
|
|
&& bitsize % BITS_PER_UNIT == 0
|
|
|
|
&& init_symbolic_number (n, TREE_OPERAND (rhs1, 0)))
|
|
|
|
{
|
|
|
|
/* Handle big-endian bit numbering in BIT_FIELD_REF. */
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
bitpos = TYPE_PRECISION (n->type) - bitpos - bitsize;
|
|
|
|
|
|
|
|
/* Shift. */
|
|
|
|
if (!do_shift_rotate (RSHIFT_EXPR, n, bitpos))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
/* Mask. */
|
|
|
|
uint64_t mask = 0;
|
|
|
|
uint64_t tmp = (1 << BITS_PER_UNIT) - 1;
|
|
|
|
for (unsigned i = 0; i < bitsize / BITS_PER_UNIT;
|
|
|
|
i++, tmp <<= BITS_PER_UNIT)
|
|
|
|
mask |= (uint64_t) MARKER_MASK << (i * BITS_PER_MARKER);
|
|
|
|
n->n &= mask;
|
|
|
|
|
|
|
|
/* Convert. */
|
|
|
|
n->type = TREE_TYPE (rhs1);
|
|
|
|
if (!n->base_addr)
|
|
|
|
n->range = TYPE_PRECISION (n->type) / BITS_PER_UNIT;
|
|
|
|
|
|
|
|
return verify_symbolic_number_p (n, stmt) ? stmt : NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (TREE_CODE (rhs1) != SSA_NAME)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
code = gimple_assign_rhs_code (stmt);
|
|
|
|
rhs_class = gimple_assign_rhs_class (stmt);
|
|
|
|
rhs1_stmt = SSA_NAME_DEF_STMT (rhs1);
|
|
|
|
|
|
|
|
if (rhs_class == GIMPLE_BINARY_RHS)
|
|
|
|
rhs2 = gimple_assign_rhs2 (stmt);
|
|
|
|
|
|
|
|
/* Handle unary rhs and binary rhs with integer constants as second
|
|
|
|
operand. */
|
|
|
|
|
|
|
|
if (rhs_class == GIMPLE_UNARY_RHS
|
|
|
|
|| (rhs_class == GIMPLE_BINARY_RHS
|
|
|
|
&& TREE_CODE (rhs2) == INTEGER_CST))
|
|
|
|
{
|
|
|
|
if (code != BIT_AND_EXPR
|
|
|
|
&& code != LSHIFT_EXPR
|
|
|
|
&& code != RSHIFT_EXPR
|
|
|
|
&& code != LROTATE_EXPR
|
|
|
|
&& code != RROTATE_EXPR
|
|
|
|
&& !CONVERT_EXPR_CODE_P (code))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
source_stmt1 = find_bswap_or_nop_1 (rhs1_stmt, n, limit - 1);
|
|
|
|
|
|
|
|
/* If find_bswap_or_nop_1 returned NULL, STMT is a leaf node and
|
|
|
|
we have to initialize the symbolic number. */
|
|
|
|
if (!source_stmt1)
|
|
|
|
{
|
|
|
|
if (gimple_assign_load_p (stmt)
|
|
|
|
|| !init_symbolic_number (n, rhs1))
|
|
|
|
return NULL;
|
|
|
|
source_stmt1 = stmt;
|
|
|
|
}
|
|
|
|
|
|
|
|
switch (code)
|
|
|
|
{
|
|
|
|
case BIT_AND_EXPR:
|
|
|
|
{
|
|
|
|
int i, size = TYPE_PRECISION (n->type) / BITS_PER_UNIT;
|
|
|
|
uint64_t val = int_cst_value (rhs2), mask = 0;
|
|
|
|
uint64_t tmp = (1 << BITS_PER_UNIT) - 1;
|
|
|
|
|
|
|
|
/* Only constants masking full bytes are allowed. */
|
|
|
|
for (i = 0; i < size; i++, tmp <<= BITS_PER_UNIT)
|
|
|
|
if ((val & tmp) != 0 && (val & tmp) != tmp)
|
|
|
|
return NULL;
|
|
|
|
else if (val & tmp)
|
|
|
|
mask |= (uint64_t) MARKER_MASK << (i * BITS_PER_MARKER);
|
|
|
|
|
|
|
|
n->n &= mask;
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
case LSHIFT_EXPR:
|
|
|
|
case RSHIFT_EXPR:
|
|
|
|
case LROTATE_EXPR:
|
|
|
|
case RROTATE_EXPR:
|
|
|
|
if (!do_shift_rotate (code, n, (int) TREE_INT_CST_LOW (rhs2)))
|
|
|
|
return NULL;
|
|
|
|
break;
|
|
|
|
CASE_CONVERT:
|
|
|
|
{
|
|
|
|
int i, type_size, old_type_size;
|
|
|
|
tree type;
|
|
|
|
|
|
|
|
type = gimple_expr_type (stmt);
|
|
|
|
type_size = TYPE_PRECISION (type);
|
|
|
|
if (type_size % BITS_PER_UNIT != 0)
|
|
|
|
return NULL;
|
|
|
|
type_size /= BITS_PER_UNIT;
|
|
|
|
if (type_size > 64 / BITS_PER_MARKER)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
/* Sign extension: result is dependent on the value. */
|
|
|
|
old_type_size = TYPE_PRECISION (n->type) / BITS_PER_UNIT;
|
|
|
|
if (!TYPE_UNSIGNED (n->type) && type_size > old_type_size
|
|
|
|
&& HEAD_MARKER (n->n, old_type_size))
|
|
|
|
for (i = 0; i < type_size - old_type_size; i++)
|
|
|
|
n->n |= (uint64_t) MARKER_BYTE_UNKNOWN
|
|
|
|
<< ((type_size - 1 - i) * BITS_PER_MARKER);
|
|
|
|
|
|
|
|
if (type_size < 64 / BITS_PER_MARKER)
|
|
|
|
{
|
|
|
|
/* If STMT casts to a smaller type mask out the bits not
|
|
|
|
belonging to the target type. */
|
|
|
|
n->n &= ((uint64_t) 1 << (type_size * BITS_PER_MARKER)) - 1;
|
|
|
|
}
|
|
|
|
n->type = type;
|
|
|
|
if (!n->base_addr)
|
|
|
|
n->range = type_size;
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
return NULL;
|
|
|
|
};
|
|
|
|
return verify_symbolic_number_p (n, stmt) ? source_stmt1 : NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Handle binary rhs. */
|
|
|
|
|
|
|
|
if (rhs_class == GIMPLE_BINARY_RHS)
|
|
|
|
{
|
|
|
|
struct symbolic_number n1, n2;
|
|
|
|
gimple *source_stmt, *source_stmt2;
|
|
|
|
|
|
|
|
if (code != BIT_IOR_EXPR)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
if (TREE_CODE (rhs2) != SSA_NAME)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
rhs2_stmt = SSA_NAME_DEF_STMT (rhs2);
|
|
|
|
|
|
|
|
switch (code)
|
|
|
|
{
|
|
|
|
case BIT_IOR_EXPR:
|
|
|
|
source_stmt1 = find_bswap_or_nop_1 (rhs1_stmt, &n1, limit - 1);
|
|
|
|
|
|
|
|
if (!source_stmt1)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
source_stmt2 = find_bswap_or_nop_1 (rhs2_stmt, &n2, limit - 1);
|
|
|
|
|
|
|
|
if (!source_stmt2)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
if (TYPE_PRECISION (n1.type) != TYPE_PRECISION (n2.type))
|
|
|
|
return NULL;
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (n1.vuse != n2.vuse)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
return NULL;
|
|
|
|
|
|
|
|
source_stmt
|
|
|
|
= perform_symbolic_merge (source_stmt1, &n1, source_stmt2, &n2, n);
|
|
|
|
|
|
|
|
if (!source_stmt)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
if (!verify_symbolic_number_p (n, stmt))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
return source_stmt;
|
|
|
|
}
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
/* Helper for find_bswap_or_nop and try_coalesce_bswap to compute
|
|
|
|
*CMPXCHG, *CMPNOP and adjust *N. */
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
void
|
|
|
|
find_bswap_or_nop_finalize (struct symbolic_number *n, uint64_t *cmpxchg,
|
|
|
|
uint64_t *cmpnop)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
|
|
|
unsigned rsize;
|
|
|
|
uint64_t tmpn, mask;
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
/* The number which the find_bswap_or_nop_1 result should match in order
|
|
|
|
to have a full byte swap. The number is shifted to the right
|
|
|
|
according to the size of the symbolic number before using it. */
|
|
|
|
*cmpxchg = CMPXCHG;
|
|
|
|
*cmpnop = CMPNOP;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
|
|
|
/* Find real size of result (highest non-zero byte). */
|
|
|
|
if (n->base_addr)
|
|
|
|
for (tmpn = n->n, rsize = 0; tmpn; tmpn >>= BITS_PER_MARKER, rsize++);
|
|
|
|
else
|
|
|
|
rsize = n->range;
|
|
|
|
|
|
|
|
/* Zero out the bits corresponding to untouched bytes in original gimple
|
|
|
|
expression. */
|
|
|
|
if (n->range < (int) sizeof (int64_t))
|
|
|
|
{
|
|
|
|
mask = ((uint64_t) 1 << (n->range * BITS_PER_MARKER)) - 1;
|
2017-11-20 11:10:23 +01:00
|
|
|
*cmpxchg >>= (64 / BITS_PER_MARKER - n->range) * BITS_PER_MARKER;
|
|
|
|
*cmpnop &= mask;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Zero out the bits corresponding to unused bytes in the result of the
|
|
|
|
gimple expression. */
|
|
|
|
if (rsize < n->range)
|
|
|
|
{
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
{
|
|
|
|
mask = ((uint64_t) 1 << (rsize * BITS_PER_MARKER)) - 1;
|
2017-11-20 11:10:23 +01:00
|
|
|
*cmpxchg &= mask;
|
|
|
|
*cmpnop >>= (n->range - rsize) * BITS_PER_MARKER;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
mask = ((uint64_t) 1 << (rsize * BITS_PER_MARKER)) - 1;
|
2017-11-20 11:10:23 +01:00
|
|
|
*cmpxchg >>= (n->range - rsize) * BITS_PER_MARKER;
|
|
|
|
*cmpnop &= mask;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
n->range = rsize;
|
|
|
|
}
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
n->range *= BITS_PER_UNIT;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Check if STMT completes a bswap implementation or a read in a given
|
|
|
|
endianness consisting of ORs, SHIFTs and ANDs and sets *BSWAP
|
|
|
|
accordingly. It also sets N to represent the kind of operations
|
|
|
|
performed: size of the resulting expression and whether it works on
|
|
|
|
a memory source, and if so alias-set and vuse. At last, the
|
|
|
|
function returns a stmt whose rhs's first tree is the source
|
|
|
|
expression. */
|
|
|
|
|
|
|
|
gimple *
|
|
|
|
find_bswap_or_nop (gimple *stmt, struct symbolic_number *n, bool *bswap)
|
|
|
|
{
|
2020-10-26 12:42:18 +01:00
|
|
|
tree type_size = TYPE_SIZE_UNIT (gimple_expr_type (stmt));
|
|
|
|
if (!tree_fits_uhwi_p (type_size))
|
|
|
|
return NULL;
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
/* The last parameter determines the depth search limit. It usually
|
|
|
|
correlates directly to the number n of bytes to be touched. We
|
2020-01-09 11:29:54 +01:00
|
|
|
increase that number by 2 * (log2(n) + 1) here in order to also
|
2017-11-20 11:10:23 +01:00
|
|
|
cover signed -> unsigned conversions of the src operand as can be seen
|
|
|
|
in libgcc, and for initial shift/and operation of the src operand. */
|
2020-10-26 12:42:18 +01:00
|
|
|
int limit = tree_to_uhwi (type_size);
|
2020-01-09 11:29:54 +01:00
|
|
|
limit += 2 * (1 + (int) ceil_log2 ((unsigned HOST_WIDE_INT) limit));
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple *ins_stmt = find_bswap_or_nop_1 (stmt, n, limit);
|
|
|
|
|
|
|
|
if (!ins_stmt)
|
2020-12-16 13:08:07 +01:00
|
|
|
{
|
|
|
|
if (gimple_assign_rhs_code (stmt) != CONSTRUCTOR
|
|
|
|
|| BYTES_BIG_ENDIAN != WORDS_BIG_ENDIAN)
|
|
|
|
return NULL;
|
|
|
|
unsigned HOST_WIDE_INT sz = tree_to_uhwi (type_size) * BITS_PER_UNIT;
|
|
|
|
if (sz != 16 && sz != 32 && sz != 64)
|
|
|
|
return NULL;
|
|
|
|
tree rhs = gimple_assign_rhs1 (stmt);
|
2020-12-19 12:46:40 +01:00
|
|
|
if (CONSTRUCTOR_NELTS (rhs) == 0)
|
|
|
|
return NULL;
|
2020-12-16 13:08:07 +01:00
|
|
|
tree eltype = TREE_TYPE (TREE_TYPE (rhs));
|
|
|
|
unsigned HOST_WIDE_INT eltsz
|
|
|
|
= int_size_in_bytes (eltype) * BITS_PER_UNIT;
|
|
|
|
if (TYPE_PRECISION (eltype) != eltsz)
|
|
|
|
return NULL;
|
|
|
|
constructor_elt *elt;
|
|
|
|
unsigned int i;
|
|
|
|
tree type = build_nonstandard_integer_type (sz, 1);
|
|
|
|
FOR_EACH_VEC_SAFE_ELT (CONSTRUCTOR_ELTS (rhs), i, elt)
|
|
|
|
{
|
|
|
|
if (TREE_CODE (elt->value) != SSA_NAME
|
|
|
|
|| !INTEGRAL_TYPE_P (TREE_TYPE (elt->value)))
|
|
|
|
return NULL;
|
|
|
|
struct symbolic_number n1;
|
|
|
|
gimple *source_stmt
|
|
|
|
= find_bswap_or_nop_1 (SSA_NAME_DEF_STMT (elt->value), &n1,
|
|
|
|
limit - 1);
|
|
|
|
|
|
|
|
if (!source_stmt)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
n1.type = type;
|
|
|
|
if (!n1.base_addr)
|
|
|
|
n1.range = sz / BITS_PER_UNIT;
|
|
|
|
|
|
|
|
if (i == 0)
|
|
|
|
{
|
|
|
|
ins_stmt = source_stmt;
|
|
|
|
*n = n1;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
if (n->vuse != n1.vuse)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
struct symbolic_number n0 = *n;
|
|
|
|
|
|
|
|
if (!BYTES_BIG_ENDIAN)
|
|
|
|
{
|
|
|
|
if (!do_shift_rotate (LSHIFT_EXPR, &n1, i * eltsz))
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
else if (!do_shift_rotate (LSHIFT_EXPR, &n0, eltsz))
|
|
|
|
return NULL;
|
|
|
|
ins_stmt
|
|
|
|
= perform_symbolic_merge (ins_stmt, &n0, source_stmt, &n1, n);
|
|
|
|
|
|
|
|
if (!ins_stmt)
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
|
|
|
|
uint64_t cmpxchg, cmpnop;
|
|
|
|
find_bswap_or_nop_finalize (n, &cmpxchg, &cmpnop);
|
|
|
|
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
/* A complete byte swap should make the symbolic number to start with
|
|
|
|
the largest digit in the highest order byte. Unchanged symbolic
|
|
|
|
number indicates a read with same endianness as target architecture. */
|
|
|
|
if (n->n == cmpnop)
|
|
|
|
*bswap = false;
|
|
|
|
else if (n->n == cmpxchg)
|
|
|
|
*bswap = true;
|
|
|
|
else
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
/* Useless bit manipulation performed by code. */
|
|
|
|
if (!n->base_addr && n->n == cmpnop && n->n_ops == 1)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
return ins_stmt;
|
|
|
|
}
|
|
|
|
|
|
|
|
const pass_data pass_data_optimize_bswap =
|
|
|
|
{
|
|
|
|
GIMPLE_PASS, /* type */
|
|
|
|
"bswap", /* name */
|
|
|
|
OPTGROUP_NONE, /* optinfo_flags */
|
|
|
|
TV_NONE, /* tv_id */
|
|
|
|
PROP_ssa, /* properties_required */
|
|
|
|
0, /* properties_provided */
|
|
|
|
0, /* properties_destroyed */
|
|
|
|
0, /* todo_flags_start */
|
|
|
|
0, /* todo_flags_finish */
|
|
|
|
};
|
|
|
|
|
|
|
|
class pass_optimize_bswap : public gimple_opt_pass
|
|
|
|
{
|
|
|
|
public:
|
|
|
|
pass_optimize_bswap (gcc::context *ctxt)
|
|
|
|
: gimple_opt_pass (pass_data_optimize_bswap, ctxt)
|
|
|
|
{}
|
|
|
|
|
|
|
|
/* opt_pass methods: */
|
|
|
|
virtual bool gate (function *)
|
|
|
|
{
|
|
|
|
return flag_expensive_optimizations && optimize && BITS_PER_UNIT == 8;
|
|
|
|
}
|
|
|
|
|
|
|
|
virtual unsigned int execute (function *);
|
|
|
|
|
|
|
|
}; // class pass_optimize_bswap
|
|
|
|
|
2021-01-07 09:57:40 +01:00
|
|
|
/* Helper function for bswap_replace. Build VIEW_CONVERT_EXPR from
|
|
|
|
VAL to TYPE. If VAL has different type size, emit a NOP_EXPR cast
|
|
|
|
first. */
|
|
|
|
|
|
|
|
static tree
|
|
|
|
bswap_view_convert (gimple_stmt_iterator *gsi, tree type, tree val)
|
|
|
|
{
|
2021-04-03 10:08:08 +02:00
|
|
|
gcc_assert (INTEGRAL_TYPE_P (TREE_TYPE (val))
|
|
|
|
|| POINTER_TYPE_P (TREE_TYPE (val)));
|
2021-01-07 09:57:40 +01:00
|
|
|
if (TYPE_SIZE (type) != TYPE_SIZE (TREE_TYPE (val)))
|
|
|
|
{
|
|
|
|
HOST_WIDE_INT prec = TREE_INT_CST_LOW (TYPE_SIZE (type));
|
2021-04-03 10:08:08 +02:00
|
|
|
if (POINTER_TYPE_P (TREE_TYPE (val)))
|
|
|
|
{
|
|
|
|
gimple *g
|
|
|
|
= gimple_build_assign (make_ssa_name (pointer_sized_int_node),
|
|
|
|
NOP_EXPR, val);
|
|
|
|
gsi_insert_before (gsi, g, GSI_SAME_STMT);
|
|
|
|
val = gimple_assign_lhs (g);
|
|
|
|
}
|
2021-01-07 09:57:40 +01:00
|
|
|
tree itype = build_nonstandard_integer_type (prec, 1);
|
|
|
|
gimple *g = gimple_build_assign (make_ssa_name (itype), NOP_EXPR, val);
|
|
|
|
gsi_insert_before (gsi, g, GSI_SAME_STMT);
|
|
|
|
val = gimple_assign_lhs (g);
|
|
|
|
}
|
|
|
|
return build1 (VIEW_CONVERT_EXPR, type, val);
|
|
|
|
}
|
|
|
|
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
/* Perform the bswap optimization: replace the expression computed in the rhs
|
2017-11-20 11:10:23 +01:00
|
|
|
of gsi_stmt (GSI) (or if NULL add instead of replace) by an equivalent
|
|
|
|
bswap, load or load + bswap expression.
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
Which of these alternatives replace the rhs is given by N->base_addr (non
|
|
|
|
null if a load is needed) and BSWAP. The type, VUSE and set-alias of the
|
|
|
|
load to perform are also given in N while the builtin bswap invoke is given
|
2017-11-20 11:10:23 +01:00
|
|
|
in FNDEL. Finally, if a load is involved, INS_STMT refers to one of the
|
|
|
|
load statements involved to construct the rhs in gsi_stmt (GSI) and
|
|
|
|
N->range gives the size of the rhs expression for maintaining some
|
|
|
|
statistics.
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
Note that if the replacement involve a load and if gsi_stmt (GSI) is
|
|
|
|
non-NULL, that stmt is moved just after INS_STMT to do the load with the
|
|
|
|
same VUSE which can lead to gsi_stmt (GSI) changing of basic block. */
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
tree
|
|
|
|
bswap_replace (gimple_stmt_iterator gsi, gimple *ins_stmt, tree fndecl,
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
tree bswap_type, tree load_type, struct symbolic_number *n,
|
|
|
|
bool bswap)
|
|
|
|
{
|
2017-11-20 11:10:23 +01:00
|
|
|
tree src, tmp, tgt = NULL_TREE;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
gimple *bswap_stmt;
|
2020-12-16 13:08:07 +01:00
|
|
|
tree_code conv_code = NOP_EXPR;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple *cur_stmt = gsi_stmt (gsi);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
src = n->src;
|
2017-11-20 11:10:23 +01:00
|
|
|
if (cur_stmt)
|
2020-12-16 13:08:07 +01:00
|
|
|
{
|
|
|
|
tgt = gimple_assign_lhs (cur_stmt);
|
|
|
|
if (gimple_assign_rhs_code (cur_stmt) == CONSTRUCTOR
|
|
|
|
&& tgt
|
|
|
|
&& VECTOR_TYPE_P (TREE_TYPE (tgt)))
|
|
|
|
conv_code = VIEW_CONVERT_EXPR;
|
|
|
|
}
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
|
|
|
/* Need to load the value from memory first. */
|
|
|
|
if (n->base_addr)
|
|
|
|
{
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple_stmt_iterator gsi_ins = gsi;
|
|
|
|
if (ins_stmt)
|
|
|
|
gsi_ins = gsi_for_stmt (ins_stmt);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
tree addr_expr, addr_tmp, val_expr, val_tmp;
|
|
|
|
tree load_offset_ptr, aligned_load_type;
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple *load_stmt;
|
|
|
|
unsigned align = get_object_alignment (src);
|
2017-12-20 13:55:57 +01:00
|
|
|
poly_int64 load_offset = 0;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (cur_stmt)
|
|
|
|
{
|
|
|
|
basic_block ins_bb = gimple_bb (ins_stmt);
|
|
|
|
basic_block cur_bb = gimple_bb (cur_stmt);
|
|
|
|
if (!dominated_by_p (CDI_DOMINATORS, cur_bb, ins_bb))
|
|
|
|
return NULL_TREE;
|
|
|
|
|
|
|
|
/* Move cur_stmt just before one of the load of the original
|
|
|
|
to ensure it has the same VUSE. See PR61517 for what could
|
|
|
|
go wrong. */
|
|
|
|
if (gimple_bb (cur_stmt) != gimple_bb (ins_stmt))
|
|
|
|
reset_flow_sensitive_info (gimple_assign_lhs (cur_stmt));
|
|
|
|
gsi_move_before (&gsi, &gsi_ins);
|
|
|
|
gsi = gsi_for_stmt (cur_stmt);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
gsi = gsi_ins;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
|
|
|
|
/* Compute address to load from and cast according to the size
|
|
|
|
of the load. */
|
2017-11-20 11:10:23 +01:00
|
|
|
addr_expr = build_fold_addr_expr (src);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
if (is_gimple_mem_ref_addr (addr_expr))
|
2017-11-20 11:10:23 +01:00
|
|
|
addr_tmp = unshare_expr (addr_expr);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
else
|
|
|
|
{
|
2017-11-20 11:10:23 +01:00
|
|
|
addr_tmp = unshare_expr (n->base_addr);
|
|
|
|
if (!is_gimple_mem_ref_addr (addr_tmp))
|
|
|
|
addr_tmp = force_gimple_operand_gsi_1 (&gsi, addr_tmp,
|
|
|
|
is_gimple_mem_ref_addr,
|
|
|
|
NULL_TREE, true,
|
|
|
|
GSI_SAME_STMT);
|
|
|
|
load_offset = n->bytepos;
|
|
|
|
if (n->offset)
|
|
|
|
{
|
|
|
|
tree off
|
|
|
|
= force_gimple_operand_gsi (&gsi, unshare_expr (n->offset),
|
|
|
|
true, NULL_TREE, true,
|
|
|
|
GSI_SAME_STMT);
|
|
|
|
gimple *stmt
|
|
|
|
= gimple_build_assign (make_ssa_name (TREE_TYPE (addr_tmp)),
|
|
|
|
POINTER_PLUS_EXPR, addr_tmp, off);
|
|
|
|
gsi_insert_before (&gsi, stmt, GSI_SAME_STMT);
|
|
|
|
addr_tmp = gimple_assign_lhs (stmt);
|
|
|
|
}
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Perform the load. */
|
|
|
|
aligned_load_type = load_type;
|
|
|
|
if (align < TYPE_ALIGN (load_type))
|
|
|
|
aligned_load_type = build_aligned_type (load_type, align);
|
|
|
|
load_offset_ptr = build_int_cst (n->alias_set, load_offset);
|
|
|
|
val_expr = fold_build2 (MEM_REF, aligned_load_type, addr_tmp,
|
|
|
|
load_offset_ptr);
|
|
|
|
|
|
|
|
if (!bswap)
|
|
|
|
{
|
|
|
|
if (n->range == 16)
|
|
|
|
nop_stats.found_16bit++;
|
|
|
|
else if (n->range == 32)
|
|
|
|
nop_stats.found_32bit++;
|
|
|
|
else
|
|
|
|
{
|
|
|
|
gcc_assert (n->range == 64);
|
|
|
|
nop_stats.found_64bit++;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Convert the result of load if necessary. */
|
2017-11-20 11:10:23 +01:00
|
|
|
if (tgt && !useless_type_conversion_p (TREE_TYPE (tgt), load_type))
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
|
|
|
val_tmp = make_temp_ssa_name (aligned_load_type, NULL,
|
|
|
|
"load_dst");
|
|
|
|
load_stmt = gimple_build_assign (val_tmp, val_expr);
|
|
|
|
gimple_set_vuse (load_stmt, n->vuse);
|
|
|
|
gsi_insert_before (&gsi, load_stmt, GSI_SAME_STMT);
|
2020-12-16 13:08:07 +01:00
|
|
|
if (conv_code == VIEW_CONVERT_EXPR)
|
2021-01-07 09:57:40 +01:00
|
|
|
val_tmp = bswap_view_convert (&gsi, TREE_TYPE (tgt), val_tmp);
|
2020-12-16 13:08:07 +01:00
|
|
|
gimple_assign_set_rhs_with_ops (&gsi, conv_code, val_tmp);
|
2017-11-20 11:10:23 +01:00
|
|
|
update_stmt (cur_stmt);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
else if (cur_stmt)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
|
|
|
gimple_assign_set_rhs_with_ops (&gsi, MEM_REF, val_expr);
|
|
|
|
gimple_set_vuse (cur_stmt, n->vuse);
|
2017-11-20 11:10:23 +01:00
|
|
|
update_stmt (cur_stmt);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
tgt = make_ssa_name (load_type);
|
|
|
|
cur_stmt = gimple_build_assign (tgt, MEM_REF, val_expr);
|
|
|
|
gimple_set_vuse (cur_stmt, n->vuse);
|
|
|
|
gsi_insert_before (&gsi, cur_stmt, GSI_SAME_STMT);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if (dump_file)
|
|
|
|
{
|
|
|
|
fprintf (dump_file,
|
|
|
|
"%d bit load in target endianness found at: ",
|
|
|
|
(int) n->range);
|
|
|
|
print_gimple_stmt (dump_file, cur_stmt, 0);
|
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
return tgt;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
val_tmp = make_temp_ssa_name (aligned_load_type, NULL, "load_dst");
|
|
|
|
load_stmt = gimple_build_assign (val_tmp, val_expr);
|
|
|
|
gimple_set_vuse (load_stmt, n->vuse);
|
|
|
|
gsi_insert_before (&gsi, load_stmt, GSI_SAME_STMT);
|
|
|
|
}
|
|
|
|
src = val_tmp;
|
|
|
|
}
|
|
|
|
else if (!bswap)
|
|
|
|
{
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple *g = NULL;
|
|
|
|
if (tgt && !useless_type_conversion_p (TREE_TYPE (tgt), TREE_TYPE (src)))
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
{
|
|
|
|
if (!is_gimple_val (src))
|
2017-11-20 11:10:23 +01:00
|
|
|
return NULL_TREE;
|
2020-12-16 13:08:07 +01:00
|
|
|
if (conv_code == VIEW_CONVERT_EXPR)
|
2021-01-07 09:57:40 +01:00
|
|
|
src = bswap_view_convert (&gsi, TREE_TYPE (tgt), src);
|
2020-12-16 13:08:07 +01:00
|
|
|
g = gimple_build_assign (tgt, conv_code, src);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
else if (cur_stmt)
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
g = gimple_build_assign (tgt, src);
|
2017-11-20 11:10:23 +01:00
|
|
|
else
|
|
|
|
tgt = src;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
if (n->range == 16)
|
|
|
|
nop_stats.found_16bit++;
|
|
|
|
else if (n->range == 32)
|
|
|
|
nop_stats.found_32bit++;
|
|
|
|
else
|
|
|
|
{
|
|
|
|
gcc_assert (n->range == 64);
|
|
|
|
nop_stats.found_64bit++;
|
|
|
|
}
|
|
|
|
if (dump_file)
|
|
|
|
{
|
|
|
|
fprintf (dump_file,
|
|
|
|
"%d bit reshuffle in target endianness found at: ",
|
|
|
|
(int) n->range);
|
2017-11-20 11:10:23 +01:00
|
|
|
if (cur_stmt)
|
|
|
|
print_gimple_stmt (dump_file, cur_stmt, 0);
|
|
|
|
else
|
|
|
|
{
|
Convert dump and optgroup flags to enums
gcc/brig/ChangeLog:
* brigfrontend/brig-to-generic.cc
(brig_to_generic::write_globals): Use TDF_NONE rather than 0.
(dump_function): Likewise.
gcc/c-family/ChangeLog:
* c-pretty-print.c (c_pretty_printer::statement): Use TDF_NONE
rather than 0.
gcc/ChangeLog:
* cfg.c (debug): Use TDF_NONE rather than 0.
* cfghooks.c (debug): Likewise.
* dumpfile.c (DUMP_FILE_INFO): Likewise; also for OPTGROUP.
(struct dump_option_value_info): Convert to...
(struct kv_pair): ...this template type.
(dump_options): Convert to kv_pair<dump_flags_t>; use TDF_NONE
rather than 0.
(optinfo_verbosity_options): Likewise.
(optgroup_options): Convert to kv_pair<optgroup_flags_t>; use
OPTGROUP_NONE.
(gcc::dump_manager::dump_register): Use optgroup_flags_t rather
than int for "optgroup_flags" param.
(dump_generic_expr_loc): Use dump_flags_t rather than int for
"dump_kind" param.
(dump_dec): Likewise.
(dump_finish): Use TDF_NONE rather than 0.
(gcc::dump_manager::opt_info_enable_passes): Use optgroup_flags_t
rather than int for "optgroup_flags" param. Use TDF_NONE rather
than 0. Update for change to option_ptr.
(opt_info_switch_p_1): Convert "optgroup_flags" param from int *
to optgroup_flags_t *. Use TDF_NONE and OPTGROUP_NONE rather than
0. Update for changes to optinfo_verbosity_options and
optgroup_options.
(opt_info_switch_p): Convert optgroup_flags from int to
optgroup_flags_t.
(dump_basic_block): Use dump_flags_t rather than int
for "dump_kind" param.
* dumpfile.h (TDF_ADDRESS, TDF_SLIM, TDF_RAW, TDF_DETAILS,
TDF_STATS, TDF_BLOCKS, TDF_VOPS, TDF_LINENO, TDF_UID)
TDF_STMTADDR, TDF_GRAPH, TDF_MEMSYMS, TDF_RHS_ONLY, TDF_ASMNAME,
TDF_EH, TDF_NOUID, TDF_ALIAS, TDF_ENUMERATE_LOCALS, TDF_CSELIB,
TDF_SCEV, TDF_GIMPLE, TDF_FOLDING, MSG_OPTIMIZED_LOCATIONS,
MSG_MISSED_OPTIMIZATION, MSG_NOTE, MSG_ALL, TDF_COMPARE_DEBUG,
TDF_NONE): Convert from macros to...
(enum dump_flag): ...this new enum.
(dump_flags_t): Update to use enum.
(operator|, operator&, operator~, operator|=, operator&=):
Implement for dump_flags_t.
(OPTGROUP_NONE, OPTGROUP_IPA, OPTGROUP_LOOP, OPTGROUP_INLINE,
OPTGROUP_OMP, OPTGROUP_VEC, OPTGROUP_OTHER, OPTGROUP_ALL):
Convert from macros to...
(enum optgroup_flag): ...this new enum.
(optgroup_flags_t): New typedef.
(operator|, operator|=): Implement for optgroup_flags_t.
(struct dump_file_info): Convert field "alt_flags" to
dump_flags_t. Convert field "optgroup_flags" to
optgroup_flags_t.
(dump_basic_block): Use dump_flags_t rather than int for param.
(dump_generic_expr_loc): Likewise.
(dump_dec): Likewise.
(dump_register): Convert param "optgroup_flags" to
optgroup_flags_t.
(opt_info_enable_passes): Likewise.
* early-remat.c (early_remat::dump_edge_list): Use TDF_NONE rather
than 0.
* gimple-pretty-print.c (debug): Likewise.
* gimple-ssa-store-merging.c (bswap_replace): Likewise.
(merged_store_group::apply_stores): Likewise.
* gimple-ssa-strength-reduction.c (insert_initializers): Likewise.
* gimple.c (verify_gimple_pp): Likewise.
* graphite-poly.c (print_pbb_body): Likewise.
* passes.c (pass_manager::register_one_dump_file): Convert
local "optgroup_flags" to optgroup_flags_t.
* print-tree.c (print_node): Use TDF_NONE rather than 0.
(debug): Likewise.
(debug_body): Likewise.
* tree-pass.h (struct pass_data): Convert field "optgroup_flags"
to optgroup_flags_t.
* tree-pretty-print.c (print_struct_decl): Use TDF_NONE rather
than 0.
* tree-ssa-math-opts.c (convert_mult_to_fma_1): Likewise.
(convert_mult_to_fma): Likewise.
* tree-ssa-reassoc.c (undistribute_ops_list): Likewise.
* tree-ssa-sccvn.c (vn_eliminate): Likewise.
* tree-vect-data-refs.c (dump_lower_bound): Convert param
"dump_kind" to dump_flags_t.
From-SVN: r261325
2018-06-08 14:50:19 +02:00
|
|
|
print_generic_expr (dump_file, tgt, TDF_NONE);
|
2017-11-20 11:10:23 +01:00
|
|
|
fprintf (dump_file, "\n");
|
|
|
|
}
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
if (cur_stmt)
|
|
|
|
gsi_replace (&gsi, g, true);
|
|
|
|
return tgt;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
else if (TREE_CODE (src) == BIT_FIELD_REF)
|
|
|
|
src = TREE_OPERAND (src, 0);
|
|
|
|
|
|
|
|
if (n->range == 16)
|
|
|
|
bswap_stats.found_16bit++;
|
|
|
|
else if (n->range == 32)
|
|
|
|
bswap_stats.found_32bit++;
|
|
|
|
else
|
|
|
|
{
|
|
|
|
gcc_assert (n->range == 64);
|
|
|
|
bswap_stats.found_64bit++;
|
|
|
|
}
|
|
|
|
|
|
|
|
tmp = src;
|
|
|
|
|
|
|
|
/* Convert the src expression if necessary. */
|
|
|
|
if (!useless_type_conversion_p (TREE_TYPE (tmp), bswap_type))
|
|
|
|
{
|
|
|
|
gimple *convert_stmt;
|
|
|
|
|
|
|
|
tmp = make_temp_ssa_name (bswap_type, NULL, "bswapsrc");
|
|
|
|
convert_stmt = gimple_build_assign (tmp, NOP_EXPR, src);
|
|
|
|
gsi_insert_before (&gsi, convert_stmt, GSI_SAME_STMT);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Canonical form for 16 bit bswap is a rotate expression. Only 16bit values
|
|
|
|
are considered as rotation of 2N bit values by N bits is generally not
|
|
|
|
equivalent to a bswap. Consider for instance 0x01020304 r>> 16 which
|
|
|
|
gives 0x03040102 while a bswap for that value is 0x04030201. */
|
|
|
|
if (bswap && n->range == 16)
|
|
|
|
{
|
|
|
|
tree count = build_int_cst (NULL, BITS_PER_UNIT);
|
|
|
|
src = fold_build2 (LROTATE_EXPR, bswap_type, tmp, count);
|
|
|
|
bswap_stmt = gimple_build_assign (NULL, src);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
bswap_stmt = gimple_build_call (fndecl, 1, tmp);
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (tgt == NULL_TREE)
|
|
|
|
tgt = make_ssa_name (bswap_type);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
tmp = tgt;
|
|
|
|
|
|
|
|
/* Convert the result if necessary. */
|
|
|
|
if (!useless_type_conversion_p (TREE_TYPE (tgt), bswap_type))
|
|
|
|
{
|
|
|
|
gimple *convert_stmt;
|
|
|
|
|
|
|
|
tmp = make_temp_ssa_name (bswap_type, NULL, "bswapdst");
|
2020-12-16 13:08:07 +01:00
|
|
|
tree atmp = tmp;
|
|
|
|
if (conv_code == VIEW_CONVERT_EXPR)
|
2021-01-07 09:57:40 +01:00
|
|
|
atmp = bswap_view_convert (&gsi, TREE_TYPE (tgt), tmp);
|
2020-12-16 13:08:07 +01:00
|
|
|
convert_stmt = gimple_build_assign (tgt, conv_code, atmp);
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
gsi_insert_after (&gsi, convert_stmt, GSI_SAME_STMT);
|
|
|
|
}
|
|
|
|
|
|
|
|
gimple_set_lhs (bswap_stmt, tmp);
|
|
|
|
|
|
|
|
if (dump_file)
|
|
|
|
{
|
|
|
|
fprintf (dump_file, "%d bit bswap implementation found at: ",
|
|
|
|
(int) n->range);
|
2017-11-20 11:10:23 +01:00
|
|
|
if (cur_stmt)
|
|
|
|
print_gimple_stmt (dump_file, cur_stmt, 0);
|
|
|
|
else
|
|
|
|
{
|
Convert dump and optgroup flags to enums
gcc/brig/ChangeLog:
* brigfrontend/brig-to-generic.cc
(brig_to_generic::write_globals): Use TDF_NONE rather than 0.
(dump_function): Likewise.
gcc/c-family/ChangeLog:
* c-pretty-print.c (c_pretty_printer::statement): Use TDF_NONE
rather than 0.
gcc/ChangeLog:
* cfg.c (debug): Use TDF_NONE rather than 0.
* cfghooks.c (debug): Likewise.
* dumpfile.c (DUMP_FILE_INFO): Likewise; also for OPTGROUP.
(struct dump_option_value_info): Convert to...
(struct kv_pair): ...this template type.
(dump_options): Convert to kv_pair<dump_flags_t>; use TDF_NONE
rather than 0.
(optinfo_verbosity_options): Likewise.
(optgroup_options): Convert to kv_pair<optgroup_flags_t>; use
OPTGROUP_NONE.
(gcc::dump_manager::dump_register): Use optgroup_flags_t rather
than int for "optgroup_flags" param.
(dump_generic_expr_loc): Use dump_flags_t rather than int for
"dump_kind" param.
(dump_dec): Likewise.
(dump_finish): Use TDF_NONE rather than 0.
(gcc::dump_manager::opt_info_enable_passes): Use optgroup_flags_t
rather than int for "optgroup_flags" param. Use TDF_NONE rather
than 0. Update for change to option_ptr.
(opt_info_switch_p_1): Convert "optgroup_flags" param from int *
to optgroup_flags_t *. Use TDF_NONE and OPTGROUP_NONE rather than
0. Update for changes to optinfo_verbosity_options and
optgroup_options.
(opt_info_switch_p): Convert optgroup_flags from int to
optgroup_flags_t.
(dump_basic_block): Use dump_flags_t rather than int
for "dump_kind" param.
* dumpfile.h (TDF_ADDRESS, TDF_SLIM, TDF_RAW, TDF_DETAILS,
TDF_STATS, TDF_BLOCKS, TDF_VOPS, TDF_LINENO, TDF_UID)
TDF_STMTADDR, TDF_GRAPH, TDF_MEMSYMS, TDF_RHS_ONLY, TDF_ASMNAME,
TDF_EH, TDF_NOUID, TDF_ALIAS, TDF_ENUMERATE_LOCALS, TDF_CSELIB,
TDF_SCEV, TDF_GIMPLE, TDF_FOLDING, MSG_OPTIMIZED_LOCATIONS,
MSG_MISSED_OPTIMIZATION, MSG_NOTE, MSG_ALL, TDF_COMPARE_DEBUG,
TDF_NONE): Convert from macros to...
(enum dump_flag): ...this new enum.
(dump_flags_t): Update to use enum.
(operator|, operator&, operator~, operator|=, operator&=):
Implement for dump_flags_t.
(OPTGROUP_NONE, OPTGROUP_IPA, OPTGROUP_LOOP, OPTGROUP_INLINE,
OPTGROUP_OMP, OPTGROUP_VEC, OPTGROUP_OTHER, OPTGROUP_ALL):
Convert from macros to...
(enum optgroup_flag): ...this new enum.
(optgroup_flags_t): New typedef.
(operator|, operator|=): Implement for optgroup_flags_t.
(struct dump_file_info): Convert field "alt_flags" to
dump_flags_t. Convert field "optgroup_flags" to
optgroup_flags_t.
(dump_basic_block): Use dump_flags_t rather than int for param.
(dump_generic_expr_loc): Likewise.
(dump_dec): Likewise.
(dump_register): Convert param "optgroup_flags" to
optgroup_flags_t.
(opt_info_enable_passes): Likewise.
* early-remat.c (early_remat::dump_edge_list): Use TDF_NONE rather
than 0.
* gimple-pretty-print.c (debug): Likewise.
* gimple-ssa-store-merging.c (bswap_replace): Likewise.
(merged_store_group::apply_stores): Likewise.
* gimple-ssa-strength-reduction.c (insert_initializers): Likewise.
* gimple.c (verify_gimple_pp): Likewise.
* graphite-poly.c (print_pbb_body): Likewise.
* passes.c (pass_manager::register_one_dump_file): Convert
local "optgroup_flags" to optgroup_flags_t.
* print-tree.c (print_node): Use TDF_NONE rather than 0.
(debug): Likewise.
(debug_body): Likewise.
* tree-pass.h (struct pass_data): Convert field "optgroup_flags"
to optgroup_flags_t.
* tree-pretty-print.c (print_struct_decl): Use TDF_NONE rather
than 0.
* tree-ssa-math-opts.c (convert_mult_to_fma_1): Likewise.
(convert_mult_to_fma): Likewise.
* tree-ssa-reassoc.c (undistribute_ops_list): Likewise.
* tree-ssa-sccvn.c (vn_eliminate): Likewise.
* tree-vect-data-refs.c (dump_lower_bound): Convert param
"dump_kind" to dump_flags_t.
From-SVN: r261325
2018-06-08 14:50:19 +02:00
|
|
|
print_generic_expr (dump_file, tgt, TDF_NONE);
|
2017-11-20 11:10:23 +01:00
|
|
|
fprintf (dump_file, "\n");
|
|
|
|
}
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (cur_stmt)
|
|
|
|
{
|
|
|
|
gsi_insert_after (&gsi, bswap_stmt, GSI_SAME_STMT);
|
|
|
|
gsi_remove (&gsi, true);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
gsi_insert_before (&gsi, bswap_stmt, GSI_SAME_STMT);
|
|
|
|
return tgt;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
}
|
|
|
|
|
2021-01-05 16:16:06 +01:00
|
|
|
/* Try to optimize an assignment CUR_STMT with CONSTRUCTOR on the rhs
|
|
|
|
using bswap optimizations. CDI_DOMINATORS need to be
|
|
|
|
computed on entry. Return true if it has been optimized and
|
|
|
|
TODO_update_ssa is needed. */
|
|
|
|
|
|
|
|
static bool
|
|
|
|
maybe_optimize_vector_constructor (gimple *cur_stmt)
|
|
|
|
{
|
|
|
|
tree fndecl = NULL_TREE, bswap_type = NULL_TREE, load_type;
|
|
|
|
struct symbolic_number n;
|
|
|
|
bool bswap;
|
|
|
|
|
|
|
|
gcc_assert (is_gimple_assign (cur_stmt)
|
|
|
|
&& gimple_assign_rhs_code (cur_stmt) == CONSTRUCTOR);
|
|
|
|
|
|
|
|
tree rhs = gimple_assign_rhs1 (cur_stmt);
|
|
|
|
if (!VECTOR_TYPE_P (TREE_TYPE (rhs))
|
|
|
|
|| !INTEGRAL_TYPE_P (TREE_TYPE (TREE_TYPE (rhs)))
|
|
|
|
|| gimple_assign_lhs (cur_stmt) == NULL_TREE)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
HOST_WIDE_INT sz = int_size_in_bytes (TREE_TYPE (rhs)) * BITS_PER_UNIT;
|
|
|
|
switch (sz)
|
|
|
|
{
|
|
|
|
case 16:
|
|
|
|
load_type = bswap_type = uint16_type_node;
|
|
|
|
break;
|
|
|
|
case 32:
|
|
|
|
if (builtin_decl_explicit_p (BUILT_IN_BSWAP32)
|
|
|
|
&& optab_handler (bswap_optab, SImode) != CODE_FOR_nothing)
|
|
|
|
{
|
|
|
|
load_type = uint32_type_node;
|
|
|
|
fndecl = builtin_decl_explicit (BUILT_IN_BSWAP32);
|
|
|
|
bswap_type = TREE_VALUE (TYPE_ARG_TYPES (TREE_TYPE (fndecl)));
|
|
|
|
}
|
|
|
|
else
|
|
|
|
return false;
|
|
|
|
break;
|
|
|
|
case 64:
|
|
|
|
if (builtin_decl_explicit_p (BUILT_IN_BSWAP64)
|
|
|
|
&& (optab_handler (bswap_optab, DImode) != CODE_FOR_nothing
|
|
|
|
|| (word_mode == SImode
|
|
|
|
&& builtin_decl_explicit_p (BUILT_IN_BSWAP32)
|
|
|
|
&& optab_handler (bswap_optab, SImode) != CODE_FOR_nothing)))
|
|
|
|
{
|
|
|
|
load_type = uint64_type_node;
|
|
|
|
fndecl = builtin_decl_explicit (BUILT_IN_BSWAP64);
|
|
|
|
bswap_type = TREE_VALUE (TYPE_ARG_TYPES (TREE_TYPE (fndecl)));
|
|
|
|
}
|
|
|
|
else
|
|
|
|
return false;
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
gimple *ins_stmt = find_bswap_or_nop (cur_stmt, &n, &bswap);
|
|
|
|
if (!ins_stmt || n.range != (unsigned HOST_WIDE_INT) sz)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
if (bswap && !fndecl && n.range != 16)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
memset (&nop_stats, 0, sizeof (nop_stats));
|
|
|
|
memset (&bswap_stats, 0, sizeof (bswap_stats));
|
|
|
|
return bswap_replace (gsi_for_stmt (cur_stmt), ins_stmt, fndecl,
|
|
|
|
bswap_type, load_type, &n, bswap) != NULL_TREE;
|
|
|
|
}
|
|
|
|
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
/* Find manual byte swap implementations as well as load in a given
|
|
|
|
endianness. Byte swaps are turned into a bswap builtin invokation
|
|
|
|
while endian loads are converted to bswap builtin invokation or
|
|
|
|
simple load according to the target endianness. */
|
|
|
|
|
|
|
|
unsigned int
|
|
|
|
pass_optimize_bswap::execute (function *fun)
|
|
|
|
{
|
|
|
|
basic_block bb;
|
|
|
|
bool bswap32_p, bswap64_p;
|
|
|
|
bool changed = false;
|
|
|
|
tree bswap32_type = NULL_TREE, bswap64_type = NULL_TREE;
|
|
|
|
|
|
|
|
bswap32_p = (builtin_decl_explicit_p (BUILT_IN_BSWAP32)
|
|
|
|
&& optab_handler (bswap_optab, SImode) != CODE_FOR_nothing);
|
|
|
|
bswap64_p = (builtin_decl_explicit_p (BUILT_IN_BSWAP64)
|
|
|
|
&& (optab_handler (bswap_optab, DImode) != CODE_FOR_nothing
|
|
|
|
|| (bswap32_p && word_mode == SImode)));
|
|
|
|
|
|
|
|
/* Determine the argument type of the builtins. The code later on
|
|
|
|
assumes that the return and argument type are the same. */
|
|
|
|
if (bswap32_p)
|
|
|
|
{
|
|
|
|
tree fndecl = builtin_decl_explicit (BUILT_IN_BSWAP32);
|
|
|
|
bswap32_type = TREE_VALUE (TYPE_ARG_TYPES (TREE_TYPE (fndecl)));
|
|
|
|
}
|
|
|
|
|
|
|
|
if (bswap64_p)
|
|
|
|
{
|
|
|
|
tree fndecl = builtin_decl_explicit (BUILT_IN_BSWAP64);
|
|
|
|
bswap64_type = TREE_VALUE (TYPE_ARG_TYPES (TREE_TYPE (fndecl)));
|
|
|
|
}
|
|
|
|
|
|
|
|
memset (&nop_stats, 0, sizeof (nop_stats));
|
|
|
|
memset (&bswap_stats, 0, sizeof (bswap_stats));
|
|
|
|
calculate_dominance_info (CDI_DOMINATORS);
|
|
|
|
|
|
|
|
FOR_EACH_BB_FN (bb, fun)
|
|
|
|
{
|
|
|
|
gimple_stmt_iterator gsi;
|
|
|
|
|
|
|
|
/* We do a reverse scan for bswap patterns to make sure we get the
|
|
|
|
widest match. As bswap pattern matching doesn't handle previously
|
|
|
|
inserted smaller bswap replacements as sub-patterns, the wider
|
|
|
|
variant wouldn't be detected. */
|
|
|
|
for (gsi = gsi_last_bb (bb); !gsi_end_p (gsi);)
|
|
|
|
{
|
|
|
|
gimple *ins_stmt, *cur_stmt = gsi_stmt (gsi);
|
|
|
|
tree fndecl = NULL_TREE, bswap_type = NULL_TREE, load_type;
|
|
|
|
enum tree_code code;
|
|
|
|
struct symbolic_number n;
|
|
|
|
bool bswap;
|
|
|
|
|
|
|
|
/* This gsi_prev (&gsi) is not part of the for loop because cur_stmt
|
|
|
|
might be moved to a different basic block by bswap_replace and gsi
|
|
|
|
must not points to it if that's the case. Moving the gsi_prev
|
|
|
|
there make sure that gsi points to the statement previous to
|
|
|
|
cur_stmt while still making sure that all statements are
|
|
|
|
considered in this basic block. */
|
|
|
|
gsi_prev (&gsi);
|
|
|
|
|
|
|
|
if (!is_gimple_assign (cur_stmt))
|
|
|
|
continue;
|
|
|
|
|
|
|
|
code = gimple_assign_rhs_code (cur_stmt);
|
|
|
|
switch (code)
|
|
|
|
{
|
|
|
|
case LROTATE_EXPR:
|
|
|
|
case RROTATE_EXPR:
|
|
|
|
if (!tree_fits_uhwi_p (gimple_assign_rhs2 (cur_stmt))
|
|
|
|
|| tree_to_uhwi (gimple_assign_rhs2 (cur_stmt))
|
|
|
|
% BITS_PER_UNIT)
|
|
|
|
continue;
|
|
|
|
/* Fall through. */
|
|
|
|
case BIT_IOR_EXPR:
|
|
|
|
break;
|
2020-12-16 13:08:07 +01:00
|
|
|
case CONSTRUCTOR:
|
|
|
|
{
|
|
|
|
tree rhs = gimple_assign_rhs1 (cur_stmt);
|
|
|
|
if (VECTOR_TYPE_P (TREE_TYPE (rhs))
|
|
|
|
&& INTEGRAL_TYPE_P (TREE_TYPE (TREE_TYPE (rhs))))
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
continue;
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
default:
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
ins_stmt = find_bswap_or_nop (cur_stmt, &n, &bswap);
|
|
|
|
|
|
|
|
if (!ins_stmt)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
switch (n.range)
|
|
|
|
{
|
|
|
|
case 16:
|
|
|
|
/* Already in canonical form, nothing to do. */
|
|
|
|
if (code == LROTATE_EXPR || code == RROTATE_EXPR)
|
|
|
|
continue;
|
|
|
|
load_type = bswap_type = uint16_type_node;
|
|
|
|
break;
|
|
|
|
case 32:
|
|
|
|
load_type = uint32_type_node;
|
|
|
|
if (bswap32_p)
|
|
|
|
{
|
|
|
|
fndecl = builtin_decl_explicit (BUILT_IN_BSWAP32);
|
|
|
|
bswap_type = bswap32_type;
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
case 64:
|
|
|
|
load_type = uint64_type_node;
|
|
|
|
if (bswap64_p)
|
|
|
|
{
|
|
|
|
fndecl = builtin_decl_explicit (BUILT_IN_BSWAP64);
|
|
|
|
bswap_type = bswap64_type;
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (bswap && !fndecl && n.range != 16)
|
|
|
|
continue;
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (bswap_replace (gsi_for_stmt (cur_stmt), ins_stmt, fndecl,
|
|
|
|
bswap_type, load_type, &n, bswap))
|
tree-ssa-math-opts.c (nop_stats, [...]): Moved to ...
* tree-ssa-math-opts.c (nop_stats, bswap_stats, struct symbolic_number,
BITS_PER_MARKER, MARKER_MASK, MARKER_BYTE_UNKNOWN, HEAD_MARKER, CMPNOP,
CMPXCHG, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, pass_data_optimize_bswap,
class pass_optimize_bswap, bswap_replace,
pass_optimize_bswap::execute): Moved to ...
* gimple-ssa-store-merging.c: ... this file.
Include optabs-tree.h.
(nop_stats, bswap_stats, do_shift_rotate, verify_symbolic_number_p,
init_symbolic_number, find_bswap_or_nop_load, perform_symbolic_merge,
find_bswap_or_nop_1, find_bswap_or_nop, bswap_replace): Put into
anonymous namespace, remove static keywords.
(pass_optimize_bswap::gate): Test BITS_PER_UNIT == 8 here...
(pass_optimize_bswap::execute): ... rather than here. Formatting fix.
From-SVN: r254947
2017-11-20 11:08:48 +01:00
|
|
|
changed = true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
statistics_counter_event (fun, "16-bit nop implementations found",
|
|
|
|
nop_stats.found_16bit);
|
|
|
|
statistics_counter_event (fun, "32-bit nop implementations found",
|
|
|
|
nop_stats.found_32bit);
|
|
|
|
statistics_counter_event (fun, "64-bit nop implementations found",
|
|
|
|
nop_stats.found_64bit);
|
|
|
|
statistics_counter_event (fun, "16-bit bswap implementations found",
|
|
|
|
bswap_stats.found_16bit);
|
|
|
|
statistics_counter_event (fun, "32-bit bswap implementations found",
|
|
|
|
bswap_stats.found_32bit);
|
|
|
|
statistics_counter_event (fun, "64-bit bswap implementations found",
|
|
|
|
bswap_stats.found_64bit);
|
|
|
|
|
|
|
|
return (changed ? TODO_update_ssa : 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
} // anon namespace
|
|
|
|
|
|
|
|
gimple_opt_pass *
|
|
|
|
make_pass_optimize_bswap (gcc::context *ctxt)
|
|
|
|
{
|
|
|
|
return new pass_optimize_bswap (ctxt);
|
|
|
|
}
|
|
|
|
|
|
|
|
namespace {
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Struct recording one operand for the store, which is either a constant,
|
2018-06-01 23:56:17 +02:00
|
|
|
then VAL represents the constant and all the other fields are zero, or
|
|
|
|
a memory load, then VAL represents the reference, BASE_ADDR is non-NULL
|
|
|
|
and the other fields also reflect the memory load, or an SSA name, then
|
|
|
|
VAL represents the SSA name and all the other fields are zero, */
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
2019-07-09 18:36:00 +02:00
|
|
|
class store_operand_info
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
2019-07-09 18:36:00 +02:00
|
|
|
public:
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
tree val;
|
|
|
|
tree base_addr;
|
2017-12-20 13:56:50 +01:00
|
|
|
poly_uint64 bitsize;
|
|
|
|
poly_uint64 bitpos;
|
|
|
|
poly_uint64 bitregion_start;
|
|
|
|
poly_uint64 bitregion_end;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
gimple *stmt;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
bool bit_not_p;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
store_operand_info ();
|
|
|
|
};
|
|
|
|
|
|
|
|
store_operand_info::store_operand_info ()
|
|
|
|
: val (NULL_TREE), base_addr (NULL_TREE), bitsize (0), bitpos (0),
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
bitregion_start (0), bitregion_end (0), stmt (NULL), bit_not_p (false)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Struct recording the information about a single store of an immediate
|
|
|
|
to memory. These are created in the first phase and coalesced into
|
|
|
|
merged_store_group objects in the second phase. */
|
|
|
|
|
2019-07-09 18:36:00 +02:00
|
|
|
class store_immediate_info
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2019-07-09 18:36:00 +02:00
|
|
|
public:
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT bitsize;
|
|
|
|
unsigned HOST_WIDE_INT bitpos;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT bitregion_start;
|
|
|
|
/* This is one past the last bit of the bit region. */
|
|
|
|
unsigned HOST_WIDE_INT bitregion_end;
|
2016-10-28 16:18:50 +02:00
|
|
|
gimple *stmt;
|
|
|
|
unsigned int order;
|
2020-07-03 18:10:25 +02:00
|
|
|
/* INTEGER_CST for constant store, STRING_CST for string store,
|
|
|
|
MEM_REF for memory copy, BIT_*_EXPR for logical bitwise operation,
|
|
|
|
BIT_INSERT_EXPR for bit insertion.
|
2017-11-20 11:10:23 +01:00
|
|
|
LROTATE_EXPR if it can be only bswap optimized and
|
|
|
|
ops are not really meaningful.
|
|
|
|
NOP_EXPR if bswap optimization detected identity, ops
|
|
|
|
are not meaningful. */
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
enum tree_code rhs_code;
|
2017-11-20 11:10:23 +01:00
|
|
|
/* Two fields for bswap optimization purposes. */
|
|
|
|
struct symbolic_number n;
|
|
|
|
gimple *ins_stmt;
|
2017-11-10 15:43:13 +01:00
|
|
|
/* True if BIT_{AND,IOR,XOR}_EXPR result is inverted before storing. */
|
2017-11-09 20:24:59 +01:00
|
|
|
bool bit_not_p;
|
2017-11-10 15:43:13 +01:00
|
|
|
/* True if ops have been swapped and thus ops[1] represents
|
|
|
|
rhs1 of BIT_{AND,IOR,XOR}_EXPR and ops[0] represents rhs2. */
|
|
|
|
bool ops_swapped_p;
|
2019-10-02 15:35:40 +02:00
|
|
|
/* The index number of the landing pad, or 0 if there is none. */
|
|
|
|
int lp_nr;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Operands. For BIT_*_EXPR rhs_code both operands are used, otherwise
|
|
|
|
just the first one. */
|
|
|
|
store_operand_info ops[2];
|
2016-11-02 14:05:07 +01:00
|
|
|
store_immediate_info (unsigned HOST_WIDE_INT, unsigned HOST_WIDE_INT,
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT, unsigned HOST_WIDE_INT,
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple *, unsigned int, enum tree_code,
|
2019-10-02 15:35:40 +02:00
|
|
|
struct symbolic_number &, gimple *, bool, int,
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
const store_operand_info &,
|
|
|
|
const store_operand_info &);
|
2016-10-28 16:18:50 +02:00
|
|
|
};
|
|
|
|
|
|
|
|
store_immediate_info::store_immediate_info (unsigned HOST_WIDE_INT bs,
|
2016-11-02 14:05:07 +01:00
|
|
|
unsigned HOST_WIDE_INT bp,
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT brs,
|
|
|
|
unsigned HOST_WIDE_INT bre,
|
2016-11-02 14:05:07 +01:00
|
|
|
gimple *st,
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
unsigned int ord,
|
|
|
|
enum tree_code rhscode,
|
2017-11-20 11:10:23 +01:00
|
|
|
struct symbolic_number &nr,
|
|
|
|
gimple *ins_stmtp,
|
2017-11-09 20:24:59 +01:00
|
|
|
bool bitnotp,
|
2019-10-02 15:35:40 +02:00
|
|
|
int nr2,
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
const store_operand_info &op0r,
|
|
|
|
const store_operand_info &op1r)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
: bitsize (bs), bitpos (bp), bitregion_start (brs), bitregion_end (bre),
|
2017-11-20 11:10:23 +01:00
|
|
|
stmt (st), order (ord), rhs_code (rhscode), n (nr),
|
2019-10-02 15:35:40 +02:00
|
|
|
ins_stmt (ins_stmtp), bit_not_p (bitnotp), ops_swapped_p (false),
|
2021-04-22 09:39:39 +02:00
|
|
|
lp_nr (nr2), ops { op0r, op1r }
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
/* Struct representing a group of stores to contiguous memory locations.
|
|
|
|
These are produced by the second phase (coalescing) and consumed in the
|
|
|
|
third phase that outputs the widened stores. */
|
|
|
|
|
2019-07-09 18:36:00 +02:00
|
|
|
class merged_store_group
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2019-07-09 18:36:00 +02:00
|
|
|
public:
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT start;
|
|
|
|
unsigned HOST_WIDE_INT width;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT bitregion_start;
|
|
|
|
unsigned HOST_WIDE_INT bitregion_end;
|
|
|
|
/* The size of the allocated memory for val and mask. */
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT buf_size;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT align_base;
|
2017-12-20 13:56:50 +01:00
|
|
|
poly_uint64 load_align_base[2];
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
unsigned int align;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
unsigned int load_align[2];
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned int first_order;
|
|
|
|
unsigned int last_order;
|
2018-06-05 00:12:51 +02:00
|
|
|
bool bit_insertion;
|
2020-07-03 18:10:25 +02:00
|
|
|
bool string_concatenation;
|
2018-11-05 11:28:19 +01:00
|
|
|
bool only_constants;
|
2020-11-20 12:21:12 +01:00
|
|
|
bool consecutive;
|
2018-11-05 11:28:19 +01:00
|
|
|
unsigned int first_nonmergeable_order;
|
2019-10-02 15:35:40 +02:00
|
|
|
int lp_nr;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
auto_vec<store_immediate_info *> stores;
|
2016-10-28 16:18:50 +02:00
|
|
|
/* We record the first and last original statements in the sequence because
|
|
|
|
we'll need their vuse/vdef and replacement position. It's easier to keep
|
|
|
|
track of them separately as 'stores' is reordered by apply_stores. */
|
|
|
|
gimple *last_stmt;
|
|
|
|
gimple *first_stmt;
|
|
|
|
unsigned char *val;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned char *mask;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
merged_store_group (store_immediate_info *);
|
|
|
|
~merged_store_group ();
|
2018-06-05 00:12:51 +02:00
|
|
|
bool can_be_merged_into (store_immediate_info *);
|
2016-10-28 16:18:50 +02:00
|
|
|
void merge_into (store_immediate_info *);
|
|
|
|
void merge_overlapping (store_immediate_info *);
|
|
|
|
bool apply_stores ();
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
private:
|
|
|
|
void do_merge (store_immediate_info *);
|
2016-10-28 16:18:50 +02:00
|
|
|
};
|
|
|
|
|
|
|
|
/* Debug helper. Dump LEN elements of byte array PTR to FD in hex. */
|
|
|
|
|
|
|
|
static void
|
|
|
|
dump_char_array (FILE *fd, unsigned char *ptr, unsigned int len)
|
|
|
|
{
|
|
|
|
if (!fd)
|
|
|
|
return;
|
|
|
|
|
|
|
|
for (unsigned int i = 0; i < len; i++)
|
2018-06-01 23:56:17 +02:00
|
|
|
fprintf (fd, "%02x ", ptr[i]);
|
2016-10-28 16:18:50 +02:00
|
|
|
fprintf (fd, "\n");
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Clear out LEN bits starting from bit START in the byte array
|
|
|
|
PTR. This clears the bits to the *right* from START.
|
|
|
|
START must be within [0, BITS_PER_UNIT) and counts starting from
|
|
|
|
the least significant bit. */
|
|
|
|
|
|
|
|
static void
|
|
|
|
clear_bit_region_be (unsigned char *ptr, unsigned int start,
|
|
|
|
unsigned int len)
|
|
|
|
{
|
|
|
|
if (len == 0)
|
|
|
|
return;
|
|
|
|
/* Clear len bits to the right of start. */
|
|
|
|
else if (len <= start + 1)
|
|
|
|
{
|
|
|
|
unsigned char mask = (~(~0U << len));
|
|
|
|
mask = mask << (start + 1U - len);
|
|
|
|
ptr[0] &= ~mask;
|
|
|
|
}
|
|
|
|
else if (start != BITS_PER_UNIT - 1)
|
|
|
|
{
|
|
|
|
clear_bit_region_be (ptr, start, (start % BITS_PER_UNIT) + 1);
|
|
|
|
clear_bit_region_be (ptr + 1, BITS_PER_UNIT - 1,
|
|
|
|
len - (start % BITS_PER_UNIT) - 1);
|
|
|
|
}
|
|
|
|
else if (start == BITS_PER_UNIT - 1
|
|
|
|
&& len > BITS_PER_UNIT)
|
|
|
|
{
|
|
|
|
unsigned int nbytes = len / BITS_PER_UNIT;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
memset (ptr, 0, nbytes);
|
2016-10-28 16:18:50 +02:00
|
|
|
if (len % BITS_PER_UNIT != 0)
|
|
|
|
clear_bit_region_be (ptr + nbytes, BITS_PER_UNIT - 1,
|
|
|
|
len % BITS_PER_UNIT);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
gcc_unreachable ();
|
|
|
|
}
|
|
|
|
|
|
|
|
/* In the byte array PTR clear the bit region starting at bit
|
|
|
|
START and is LEN bits wide.
|
|
|
|
For regions spanning multiple bytes do this recursively until we reach
|
|
|
|
zero LEN or a region contained within a single byte. */
|
|
|
|
|
|
|
|
static void
|
|
|
|
clear_bit_region (unsigned char *ptr, unsigned int start,
|
|
|
|
unsigned int len)
|
|
|
|
{
|
|
|
|
/* Degenerate base case. */
|
|
|
|
if (len == 0)
|
|
|
|
return;
|
|
|
|
else if (start >= BITS_PER_UNIT)
|
|
|
|
clear_bit_region (ptr + 1, start - BITS_PER_UNIT, len);
|
|
|
|
/* Second base case. */
|
|
|
|
else if ((start + len) <= BITS_PER_UNIT)
|
|
|
|
{
|
2016-11-22 11:15:43 +01:00
|
|
|
unsigned char mask = (~0U) << (unsigned char) (BITS_PER_UNIT - len);
|
2016-10-28 16:18:50 +02:00
|
|
|
mask >>= BITS_PER_UNIT - (start + len);
|
|
|
|
|
|
|
|
ptr[0] &= ~mask;
|
|
|
|
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
/* Clear most significant bits in a byte and proceed with the next byte. */
|
|
|
|
else if (start != 0)
|
|
|
|
{
|
|
|
|
clear_bit_region (ptr, start, BITS_PER_UNIT - start);
|
2016-11-08 13:31:31 +01:00
|
|
|
clear_bit_region (ptr + 1, 0, len - (BITS_PER_UNIT - start));
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
/* Whole bytes need to be cleared. */
|
|
|
|
else if (start == 0 && len > BITS_PER_UNIT)
|
|
|
|
{
|
|
|
|
unsigned int nbytes = len / BITS_PER_UNIT;
|
2017-07-14 11:11:20 +02:00
|
|
|
/* We could recurse on each byte but we clear whole bytes, so a simple
|
|
|
|
memset will do. */
|
2016-11-22 11:15:43 +01:00
|
|
|
memset (ptr, '\0', nbytes);
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Clear the remaining sub-byte region if there is one. */
|
|
|
|
if (len % BITS_PER_UNIT != 0)
|
|
|
|
clear_bit_region (ptr + nbytes, 0, len % BITS_PER_UNIT);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
gcc_unreachable ();
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Write BITLEN bits of EXPR to the byte array PTR at
|
|
|
|
bit position BITPOS. PTR should contain TOTAL_BYTES elements.
|
|
|
|
Return true if the operation succeeded. */
|
|
|
|
|
|
|
|
static bool
|
|
|
|
encode_tree_to_bitpos (tree expr, unsigned char *ptr, int bitlen, int bitpos,
|
2016-11-22 11:15:43 +01:00
|
|
|
unsigned int total_bytes)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
unsigned int first_byte = bitpos / BITS_PER_UNIT;
|
2017-02-28 23:39:11 +01:00
|
|
|
bool sub_byte_op_p = ((bitlen % BITS_PER_UNIT)
|
|
|
|
|| (bitpos % BITS_PER_UNIT)
|
Make more use of int_mode_for_size
This patch converts more places that could use int_mode_for_size instead
of mode_for_size. This is in preparation for an upcoming patch that
makes mode_for_size itself return an opt_mode.
require () seems like the right choice in expand_builtin_powi
because we have got past the point of backing out. We go on to do:
op1 = expand_expr (arg1, NULL_RTX, mode2, EXPAND_NORMAL);
if (GET_MODE (op1) != mode2)
op1 = convert_to_mode (mode2, op1, 0);
which would be invalid for (and have failed for) BLKmode.
In get_builtin_sync_mode and expand_ifn_atomic_compare_exchange,
the possible bitsizes are {8, 16, 32, 64, 128}, all of which give
target-independent integer modes (up to TImode). The comment above
the call in get_builtin_sync_mode makes clear that an integer mode
must be found.
We can use require () in expand_builtin_atomic_clear and
expand_builtin_atomic_test_and_set because there's always an integer
mode for the boolean type. The same goes for the POINTER_SIZE request
in layout_type. Similarly we can use require () in combine_instructions
and gen_lowpart_common because there's always an integer mode for
HOST_BITS_PER_WIDE_INT (DImode when BITS_PER_UNIT == 8), and
HOST_BITS_PER_DOUBLE_INT (TImode).
The calls in aarch64_function_value, arm_function_value,
aapcs_allocate_return_reg and mips_function_value_1 are handling
cases in which a big-endian target passes or returns values at
the most significant end of a register. In each case the ABI
constrains the size to a small amount and does not handle
non-power-of-2 sizes wider than a word.
The calls in c6x_expand_movmem, i386.c:emit_memset,
lm32_block_move_inline, microblaze_block_move_straight and
mips_block_move_straight are dealing with expansions of
block memory operations using register-wise operations,
and those registers must have non-BLK mode.
The reason for using require () in ix86_expand_sse_cmp,
mips_expand_ins_as_unaligned_store, spu.c:adjust_operand and
spu_emit_branch_and_set is that we go on to emit non-call
instructions that use registers of that mode, which wouldn't
be valid for BLKmode.
2017-09-05 Richard Sandiford <richard.sandiford@linaro.org>
gcc/
* builtins.c (expand_builtin_powi): Use int_mode_for_size.
(get_builtin_sync_mode): Likewise.
(expand_ifn_atomic_compare_exchange): Likewise.
(expand_builtin_atomic_clear): Likewise.
(expand_builtin_atomic_test_and_set): Likewise.
(fold_builtin_atomic_always_lock_free): Likewise.
* calls.c (compute_argument_addresses): Likewise.
(emit_library_call_value_1): Likewise.
(store_one_arg): Likewise.
* combine.c (combine_instructions): Likewise.
* config/aarch64/aarch64.c (aarch64_function_value): Likewise.
* config/arm/arm.c (arm_function_value): Likewise.
(aapcs_allocate_return_reg): Likewise.
* config/c6x/c6x.c (c6x_expand_movmem): Likewise.
* config/i386/i386.c (construct_container): Likewise.
(ix86_gimplify_va_arg): Likewise.
(ix86_expand_sse_cmp): Likewise.
(emit_memmov): Likewise.
(emit_memset): Likewise.
(expand_small_movmem_or_setmem): Likewise.
(ix86_expand_pextr): Likewise.
(ix86_expand_pinsr): Likewise.
* config/lm32/lm32.c (lm32_block_move_inline): Likewise.
* config/microblaze/microblaze.c (microblaze_block_move_straight):
Likewise.
* config/mips/mips.c (mips_function_value_1) Likewise.
(mips_block_move_straight): Likewise.
(mips_expand_ins_as_unaligned_store): Likewise.
* config/powerpcspe/powerpcspe.c
(rs6000_darwin64_record_arg_advance_flush): Likewise.
(rs6000_darwin64_record_arg_flush): Likewise.
* config/rs6000/rs6000.c
(rs6000_darwin64_record_arg_advance_flush): Likewise.
(rs6000_darwin64_record_arg_flush): Likewise.
* config/sparc/sparc.c (sparc_function_arg_1): Likewise.
(sparc_function_value_1): Likewise.
* config/spu/spu.c (adjust_operand): Likewise.
(spu_emit_branch_or_set): Likewise.
(arith_immediate_p): Likewise.
* emit-rtl.c (gen_lowpart_common): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* function.c (assign_parm_setup_block): Likewise.
* gimple-ssa-store-merging.c (encode_tree_to_bitpos): Likewise.
* reload1.c (alter_reg): Likewise.
* stor-layout.c (mode_for_vector): Likewise.
(layout_type): Likewise.
gcc/ada/
* gcc-interface/utils2.c (build_load_modify_store):
Use int_mode_for_size.
From-SVN: r251726
2017-09-05 21:57:01 +02:00
|
|
|
|| !int_mode_for_size (bitlen, 0).exists ());
|
2019-05-06 23:50:14 +02:00
|
|
|
bool empty_ctor_p
|
|
|
|
= (TREE_CODE (expr) == CONSTRUCTOR
|
|
|
|
&& CONSTRUCTOR_NELTS (expr) == 0
|
|
|
|
&& TYPE_SIZE_UNIT (TREE_TYPE (expr))
|
|
|
|
&& tree_fits_uhwi_p (TYPE_SIZE_UNIT (TREE_TYPE (expr))));
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
if (!sub_byte_op_p)
|
2019-05-06 23:50:14 +02:00
|
|
|
{
|
|
|
|
if (first_byte >= total_bytes)
|
|
|
|
return false;
|
|
|
|
total_bytes -= first_byte;
|
|
|
|
if (empty_ctor_p)
|
|
|
|
{
|
|
|
|
unsigned HOST_WIDE_INT rhs_bytes
|
|
|
|
= tree_to_uhwi (TYPE_SIZE_UNIT (TREE_TYPE (expr)));
|
|
|
|
if (rhs_bytes > total_bytes)
|
|
|
|
return false;
|
|
|
|
memset (ptr + first_byte, '\0', rhs_bytes);
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
return native_encode_expr (expr, ptr + first_byte, total_bytes) != 0;
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
/* LITTLE-ENDIAN
|
|
|
|
We are writing a non byte-sized quantity or at a position that is not
|
|
|
|
at a byte boundary.
|
|
|
|
|--------|--------|--------| ptr + first_byte
|
|
|
|
^ ^
|
|
|
|
xxx xxxxxxxx xxx< bp>
|
|
|
|
|______EXPR____|
|
|
|
|
|
2016-11-22 11:15:43 +01:00
|
|
|
First native_encode_expr EXPR into a temporary buffer and shift each
|
2016-10-28 16:18:50 +02:00
|
|
|
byte in the buffer by 'bp' (carrying the bits over as necessary).
|
|
|
|
|00000000|00xxxxxx|xxxxxxxx| << bp = |000xxxxx|xxxxxxxx|xxx00000|
|
|
|
|
<------bitlen---->< bp>
|
|
|
|
Then we clear the destination bits:
|
|
|
|
|---00000|00000000|000-----| ptr + first_byte
|
|
|
|
<-------bitlen--->< bp>
|
|
|
|
|
|
|
|
Finally we ORR the bytes of the shifted EXPR into the cleared region:
|
|
|
|
|---xxxxx||xxxxxxxx||xxx-----| ptr + first_byte.
|
|
|
|
|
|
|
|
BIG-ENDIAN
|
|
|
|
We are writing a non byte-sized quantity or at a position that is not
|
|
|
|
at a byte boundary.
|
|
|
|
ptr + first_byte |--------|--------|--------|
|
|
|
|
^ ^
|
|
|
|
<bp >xxx xxxxxxxx xxx
|
|
|
|
|_____EXPR_____|
|
|
|
|
|
2016-11-22 11:15:43 +01:00
|
|
|
First native_encode_expr EXPR into a temporary buffer and shift each
|
2016-10-28 16:18:50 +02:00
|
|
|
byte in the buffer to the right by (carrying the bits over as necessary).
|
|
|
|
We shift by as much as needed to align the most significant bit of EXPR
|
|
|
|
with bitpos:
|
|
|
|
|00xxxxxx|xxxxxxxx| >> 3 = |00000xxx|xxxxxxxx|xxxxx000|
|
|
|
|
<---bitlen----> <bp ><-----bitlen----->
|
|
|
|
Then we clear the destination bits:
|
|
|
|
ptr + first_byte |-----000||00000000||00000---|
|
|
|
|
<bp ><-------bitlen----->
|
|
|
|
|
|
|
|
Finally we ORR the bytes of the shifted EXPR into the cleared region:
|
|
|
|
ptr + first_byte |---xxxxx||xxxxxxxx||xxx-----|.
|
|
|
|
The awkwardness comes from the fact that bitpos is counted from the
|
|
|
|
most significant bit of a byte. */
|
|
|
|
|
2017-11-01 12:49:34 +01:00
|
|
|
/* We must be dealing with fixed-size data at this point, since the
|
|
|
|
total size is also fixed. */
|
2019-05-06 23:50:14 +02:00
|
|
|
unsigned int byte_size;
|
|
|
|
if (empty_ctor_p)
|
|
|
|
{
|
|
|
|
unsigned HOST_WIDE_INT rhs_bytes
|
|
|
|
= tree_to_uhwi (TYPE_SIZE_UNIT (TREE_TYPE (expr)));
|
|
|
|
if (rhs_bytes > total_bytes)
|
|
|
|
return false;
|
|
|
|
byte_size = rhs_bytes;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
fixed_size_mode mode
|
|
|
|
= as_a <fixed_size_mode> (TYPE_MODE (TREE_TYPE (expr)));
|
2020-07-03 18:10:25 +02:00
|
|
|
byte_size
|
|
|
|
= mode == BLKmode
|
|
|
|
? tree_to_uhwi (TYPE_SIZE_UNIT (TREE_TYPE (expr)))
|
|
|
|
: GET_MODE_SIZE (mode);
|
2019-05-06 23:50:14 +02:00
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Allocate an extra byte so that we have space to shift into. */
|
2019-05-06 23:50:14 +02:00
|
|
|
byte_size++;
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned char *tmpbuf = XALLOCAVEC (unsigned char, byte_size);
|
2016-11-22 11:15:43 +01:00
|
|
|
memset (tmpbuf, '\0', byte_size);
|
2016-10-28 16:18:50 +02:00
|
|
|
/* The store detection code should only have allowed constants that are
|
2019-05-06 23:50:14 +02:00
|
|
|
accepted by native_encode_expr or empty ctors. */
|
|
|
|
if (!empty_ctor_p
|
|
|
|
&& native_encode_expr (expr, tmpbuf, byte_size - 1) == 0)
|
2016-10-28 16:18:50 +02:00
|
|
|
gcc_unreachable ();
|
|
|
|
|
|
|
|
/* The native_encode_expr machinery uses TYPE_MODE to determine how many
|
|
|
|
bytes to write. This means it can write more than
|
|
|
|
ROUND_UP (bitlen, BITS_PER_UNIT) / BITS_PER_UNIT bytes (for example
|
|
|
|
write 8 bytes for a bitlen of 40). Skip the bytes that are not within
|
|
|
|
bitlen and zero out the bits that are not relevant as well (that may
|
|
|
|
contain a sign bit due to sign-extension). */
|
|
|
|
unsigned int padding
|
|
|
|
= byte_size - ROUND_UP (bitlen, BITS_PER_UNIT) / BITS_PER_UNIT - 1;
|
2017-02-28 23:39:11 +01:00
|
|
|
/* On big-endian the padding is at the 'front' so just skip the initial
|
|
|
|
bytes. */
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
tmpbuf += padding;
|
|
|
|
|
|
|
|
byte_size -= padding;
|
|
|
|
|
|
|
|
if (bitlen % BITS_PER_UNIT != 0)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2016-11-02 10:28:35 +01:00
|
|
|
if (BYTES_BIG_ENDIAN)
|
2017-02-28 23:39:11 +01:00
|
|
|
clear_bit_region_be (tmpbuf, BITS_PER_UNIT - 1,
|
|
|
|
BITS_PER_UNIT - (bitlen % BITS_PER_UNIT));
|
|
|
|
else
|
|
|
|
clear_bit_region (tmpbuf, bitlen,
|
|
|
|
byte_size * BITS_PER_UNIT - bitlen);
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
2017-02-28 23:39:11 +01:00
|
|
|
/* Left shifting relies on the last byte being clear if bitlen is
|
|
|
|
a multiple of BITS_PER_UNIT, which might not be clear if
|
|
|
|
there are padding bytes. */
|
|
|
|
else if (!BYTES_BIG_ENDIAN)
|
|
|
|
tmpbuf[byte_size - 1] = '\0';
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
/* Clear the bit region in PTR where the bits from TMPBUF will be
|
2016-11-22 11:15:43 +01:00
|
|
|
inserted into. */
|
2016-10-28 16:18:50 +02:00
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
clear_bit_region_be (ptr + first_byte,
|
|
|
|
BITS_PER_UNIT - 1 - (bitpos % BITS_PER_UNIT), bitlen);
|
|
|
|
else
|
|
|
|
clear_bit_region (ptr + first_byte, bitpos % BITS_PER_UNIT, bitlen);
|
|
|
|
|
|
|
|
int shift_amnt;
|
|
|
|
int bitlen_mod = bitlen % BITS_PER_UNIT;
|
|
|
|
int bitpos_mod = bitpos % BITS_PER_UNIT;
|
|
|
|
|
|
|
|
bool skip_byte = false;
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
{
|
|
|
|
/* BITPOS and BITLEN are exactly aligned and no shifting
|
|
|
|
is necessary. */
|
|
|
|
if (bitpos_mod + bitlen_mod == BITS_PER_UNIT
|
|
|
|
|| (bitpos_mod == 0 && bitlen_mod == 0))
|
|
|
|
shift_amnt = 0;
|
|
|
|
/* |. . . . . . . .|
|
|
|
|
<bp > <blen >.
|
|
|
|
We always shift right for BYTES_BIG_ENDIAN so shift the beginning
|
|
|
|
of the value until it aligns with 'bp' in the next byte over. */
|
|
|
|
else if (bitpos_mod + bitlen_mod < BITS_PER_UNIT)
|
|
|
|
{
|
|
|
|
shift_amnt = bitlen_mod + bitpos_mod;
|
|
|
|
skip_byte = bitlen_mod != 0;
|
|
|
|
}
|
|
|
|
/* |. . . . . . . .|
|
|
|
|
<----bp--->
|
|
|
|
<---blen---->.
|
|
|
|
Shift the value right within the same byte so it aligns with 'bp'. */
|
|
|
|
else
|
|
|
|
shift_amnt = bitlen_mod + bitpos_mod - BITS_PER_UNIT;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
shift_amnt = bitpos % BITS_PER_UNIT;
|
|
|
|
|
|
|
|
/* Create the shifted version of EXPR. */
|
|
|
|
if (!BYTES_BIG_ENDIAN)
|
2016-11-22 11:15:43 +01:00
|
|
|
{
|
sccvn: Handle bitfields in vn_reference_lookup_3 [PR93582]
The following patch is first step towards fixing PR93582.
vn_reference_lookup_3 right now punts on anything that isn't byte aligned,
so to be able to lookup a constant bitfield store, one needs to use
the exact same COMPONENT_REF, otherwise it isn't found.
This patch lifts up that that restriction if the bits to be loaded are
covered by a single store of a constant (keeps the restriction so far
for the multiple store case, can tweak that incrementally, but I think
for bisection etc. it is worth to do it one step at a time).
2020-02-13 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93582
* fold-const.h (shift_bytes_in_array_left,
shift_bytes_in_array_right): Declare.
* fold-const.c (shift_bytes_in_array_left,
shift_bytes_in_array_right): New function, moved from
gimple-ssa-store-merging.c, no longer static.
* gimple-ssa-store-merging.c (shift_bytes_in_array): Move
to gimple-ssa-store-merging.c and rename to shift_bytes_in_array_left.
(shift_bytes_in_array_right): Move to gimple-ssa-store-merging.c.
(encode_tree_to_bitpos): Use shift_bytes_in_array_left instead of
shift_bytes_in_array.
(verify_shift_bytes_in_array): Rename to ...
(verify_shift_bytes_in_array_left): ... this. Use
shift_bytes_in_array_left instead of shift_bytes_in_array.
(store_merging_c_tests): Call verify_shift_bytes_in_array_left
instead of verify_shift_bytes_in_array.
* tree-ssa-sccvn.c (vn_reference_lookup_3): For native_encode_expr
/ native_interpret_expr where the store covers all needed bits,
punt on PDP-endian, otherwise allow all involved offsets and sizes
not to be byte-aligned.
* gcc.dg/tree-ssa/pr93582-1.c: New test.
* gcc.dg/tree-ssa/pr93582-2.c: New test.
* gcc.dg/tree-ssa/pr93582-3.c: New test.
2020-02-13 10:04:11 +01:00
|
|
|
shift_bytes_in_array_left (tmpbuf, byte_size, shift_amnt);
|
2016-11-22 11:15:43 +01:00
|
|
|
if (shift_amnt == 0)
|
|
|
|
byte_size--;
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
else
|
|
|
|
{
|
|
|
|
gcc_assert (BYTES_BIG_ENDIAN);
|
|
|
|
shift_bytes_in_array_right (tmpbuf, byte_size, shift_amnt);
|
|
|
|
/* If shifting right forced us to move into the next byte skip the now
|
|
|
|
empty byte. */
|
|
|
|
if (skip_byte)
|
|
|
|
{
|
|
|
|
tmpbuf++;
|
|
|
|
byte_size--;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Insert the bits from TMPBUF. */
|
|
|
|
for (unsigned int i = 0; i < byte_size; i++)
|
|
|
|
ptr[first_byte + i] |= tmpbuf[i];
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Sorting function for store_immediate_info objects.
|
|
|
|
Sorts them by bitposition. */
|
|
|
|
|
|
|
|
static int
|
|
|
|
sort_by_bitpos (const void *x, const void *y)
|
|
|
|
{
|
|
|
|
store_immediate_info *const *tmp = (store_immediate_info * const *) x;
|
|
|
|
store_immediate_info *const *tmp2 = (store_immediate_info * const *) y;
|
|
|
|
|
2017-07-20 17:13:56 +02:00
|
|
|
if ((*tmp)->bitpos < (*tmp2)->bitpos)
|
2016-10-28 16:18:50 +02:00
|
|
|
return -1;
|
|
|
|
else if ((*tmp)->bitpos > (*tmp2)->bitpos)
|
|
|
|
return 1;
|
2017-07-20 17:13:56 +02:00
|
|
|
else
|
2017-09-13 15:49:50 +02:00
|
|
|
/* If they are the same let's use the order which is guaranteed to
|
|
|
|
be different. */
|
|
|
|
return (*tmp)->order - (*tmp2)->order;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Sorting function for store_immediate_info objects.
|
|
|
|
Sorts them by the order field. */
|
|
|
|
|
|
|
|
static int
|
|
|
|
sort_by_order (const void *x, const void *y)
|
|
|
|
{
|
|
|
|
store_immediate_info *const *tmp = (store_immediate_info * const *) x;
|
|
|
|
store_immediate_info *const *tmp2 = (store_immediate_info * const *) y;
|
|
|
|
|
|
|
|
if ((*tmp)->order < (*tmp2)->order)
|
|
|
|
return -1;
|
|
|
|
else if ((*tmp)->order > (*tmp2)->order)
|
|
|
|
return 1;
|
|
|
|
|
|
|
|
gcc_unreachable ();
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Initialize a merged_store_group object from a store_immediate_info
|
|
|
|
object. */
|
|
|
|
|
|
|
|
merged_store_group::merged_store_group (store_immediate_info *info)
|
|
|
|
{
|
|
|
|
start = info->bitpos;
|
|
|
|
width = info->bitsize;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
bitregion_start = info->bitregion_start;
|
|
|
|
bitregion_end = info->bitregion_end;
|
2016-10-28 16:18:50 +02:00
|
|
|
/* VAL has memory allocated for it in apply_stores once the group
|
|
|
|
width has been finalized. */
|
|
|
|
val = NULL;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
mask = NULL;
|
2020-07-03 18:10:25 +02:00
|
|
|
bit_insertion = info->rhs_code == BIT_INSERT_EXPR;
|
|
|
|
string_concatenation = info->rhs_code == STRING_CST;
|
2018-11-05 11:28:19 +01:00
|
|
|
only_constants = info->rhs_code == INTEGER_CST;
|
2020-11-20 12:21:12 +01:00
|
|
|
consecutive = true;
|
2018-11-05 11:28:19 +01:00
|
|
|
first_nonmergeable_order = ~0U;
|
2019-10-02 15:35:40 +02:00
|
|
|
lp_nr = info->lp_nr;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT align_bitpos = 0;
|
|
|
|
get_object_alignment_1 (gimple_assign_lhs (info->stmt),
|
|
|
|
&align, &align_bitpos);
|
|
|
|
align_base = start - align_bitpos;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
for (int i = 0; i < 2; ++i)
|
|
|
|
{
|
|
|
|
store_operand_info &op = info->ops[i];
|
|
|
|
if (op.base_addr == NULL_TREE)
|
|
|
|
{
|
|
|
|
load_align[i] = 0;
|
|
|
|
load_align_base[i] = 0;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
get_object_alignment_1 (op.val, &load_align[i], &align_bitpos);
|
|
|
|
load_align_base[i] = op.bitpos - align_bitpos;
|
|
|
|
}
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
stores.create (1);
|
|
|
|
stores.safe_push (info);
|
|
|
|
last_stmt = info->stmt;
|
|
|
|
last_order = info->order;
|
|
|
|
first_stmt = last_stmt;
|
|
|
|
first_order = last_order;
|
|
|
|
buf_size = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
merged_store_group::~merged_store_group ()
|
|
|
|
{
|
|
|
|
if (val)
|
|
|
|
XDELETEVEC (val);
|
|
|
|
}
|
|
|
|
|
2018-06-05 00:12:51 +02:00
|
|
|
/* Return true if the store described by INFO can be merged into the group. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
merged_store_group::can_be_merged_into (store_immediate_info *info)
|
|
|
|
{
|
|
|
|
/* Do not merge bswap patterns. */
|
|
|
|
if (info->rhs_code == LROTATE_EXPR)
|
|
|
|
return false;
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (info->lp_nr != lp_nr)
|
|
|
|
return false;
|
|
|
|
|
2018-06-05 00:12:51 +02:00
|
|
|
/* The canonical case. */
|
|
|
|
if (info->rhs_code == stores[0]->rhs_code)
|
|
|
|
return true;
|
|
|
|
|
2020-07-03 18:10:25 +02:00
|
|
|
/* BIT_INSERT_EXPR is compatible with INTEGER_CST if no STRING_CST. */
|
2018-06-05 00:12:51 +02:00
|
|
|
if (info->rhs_code == BIT_INSERT_EXPR && stores[0]->rhs_code == INTEGER_CST)
|
2020-07-03 18:10:25 +02:00
|
|
|
return !string_concatenation;
|
2018-06-05 00:12:51 +02:00
|
|
|
|
|
|
|
if (stores[0]->rhs_code == BIT_INSERT_EXPR && info->rhs_code == INTEGER_CST)
|
2020-07-03 18:10:25 +02:00
|
|
|
return !string_concatenation;
|
2018-06-05 00:12:51 +02:00
|
|
|
|
2020-05-25 22:13:11 +02:00
|
|
|
/* We can turn MEM_REF into BIT_INSERT_EXPR for bit-field stores, but do it
|
|
|
|
only for small regions since this can generate a lot of instructions. */
|
2018-06-05 00:12:51 +02:00
|
|
|
if (info->rhs_code == MEM_REF
|
|
|
|
&& (stores[0]->rhs_code == INTEGER_CST
|
|
|
|
|| stores[0]->rhs_code == BIT_INSERT_EXPR)
|
|
|
|
&& info->bitregion_start == stores[0]->bitregion_start
|
2020-05-25 22:13:11 +02:00
|
|
|
&& info->bitregion_end == stores[0]->bitregion_end
|
2020-05-28 19:29:42 +02:00
|
|
|
&& info->bitregion_end - info->bitregion_start <= MAX_FIXED_MODE_SIZE)
|
2020-07-03 18:10:25 +02:00
|
|
|
return !string_concatenation;
|
2018-06-05 00:12:51 +02:00
|
|
|
|
|
|
|
if (stores[0]->rhs_code == MEM_REF
|
|
|
|
&& (info->rhs_code == INTEGER_CST
|
|
|
|
|| info->rhs_code == BIT_INSERT_EXPR)
|
|
|
|
&& info->bitregion_start == stores[0]->bitregion_start
|
2020-05-25 22:13:11 +02:00
|
|
|
&& info->bitregion_end == stores[0]->bitregion_end
|
2020-05-28 19:29:42 +02:00
|
|
|
&& info->bitregion_end - info->bitregion_start <= MAX_FIXED_MODE_SIZE)
|
2020-07-03 18:10:25 +02:00
|
|
|
return !string_concatenation;
|
|
|
|
|
|
|
|
/* STRING_CST is compatible with INTEGER_CST if no BIT_INSERT_EXPR. */
|
|
|
|
if (info->rhs_code == STRING_CST
|
|
|
|
&& stores[0]->rhs_code == INTEGER_CST
|
|
|
|
&& stores[0]->bitsize == CHAR_BIT)
|
|
|
|
return !bit_insertion;
|
|
|
|
|
|
|
|
if (stores[0]->rhs_code == STRING_CST
|
|
|
|
&& info->rhs_code == INTEGER_CST
|
|
|
|
&& info->bitsize == CHAR_BIT)
|
|
|
|
return !bit_insertion;
|
2018-06-05 00:12:51 +02:00
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* Helper method for merge_into and merge_overlapping to do
|
|
|
|
the common part. */
|
2018-06-05 00:12:51 +02:00
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
void
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
merged_store_group::do_merge (store_immediate_info *info)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
bitregion_start = MIN (bitregion_start, info->bitregion_start);
|
|
|
|
bitregion_end = MAX (bitregion_end, info->bitregion_end);
|
|
|
|
|
|
|
|
unsigned int this_align;
|
|
|
|
unsigned HOST_WIDE_INT align_bitpos = 0;
|
|
|
|
get_object_alignment_1 (gimple_assign_lhs (info->stmt),
|
|
|
|
&this_align, &align_bitpos);
|
|
|
|
if (this_align > align)
|
|
|
|
{
|
|
|
|
align = this_align;
|
|
|
|
align_base = info->bitpos - align_bitpos;
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
for (int i = 0; i < 2; ++i)
|
|
|
|
{
|
|
|
|
store_operand_info &op = info->ops[i];
|
|
|
|
if (!op.base_addr)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
get_object_alignment_1 (op.val, &this_align, &align_bitpos);
|
|
|
|
if (this_align > load_align[i])
|
|
|
|
{
|
|
|
|
load_align[i] = this_align;
|
|
|
|
load_align_base[i] = op.bitpos - align_bitpos;
|
|
|
|
}
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
gimple *stmt = info->stmt;
|
|
|
|
stores.safe_push (info);
|
|
|
|
if (info->order > last_order)
|
|
|
|
{
|
|
|
|
last_order = info->order;
|
|
|
|
last_stmt = stmt;
|
|
|
|
}
|
|
|
|
else if (info->order < first_order)
|
|
|
|
{
|
|
|
|
first_order = info->order;
|
|
|
|
first_stmt = stmt;
|
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
|
2020-11-20 12:21:12 +01:00
|
|
|
if (info->bitpos != start + width)
|
|
|
|
consecutive = false;
|
|
|
|
|
2020-07-03 18:10:25 +02:00
|
|
|
/* We need to use extraction if there is any bit-field. */
|
|
|
|
if (info->rhs_code == BIT_INSERT_EXPR)
|
|
|
|
{
|
|
|
|
bit_insertion = true;
|
|
|
|
gcc_assert (!string_concatenation);
|
|
|
|
}
|
|
|
|
|
2020-11-20 12:21:12 +01:00
|
|
|
/* We want to use concatenation if there is any string. */
|
2020-07-03 18:10:25 +02:00
|
|
|
if (info->rhs_code == STRING_CST)
|
|
|
|
{
|
|
|
|
string_concatenation = true;
|
|
|
|
gcc_assert (!bit_insertion);
|
|
|
|
}
|
|
|
|
|
2020-11-20 12:21:12 +01:00
|
|
|
/* But we cannot use it if we don't have consecutive stores. */
|
|
|
|
if (!consecutive)
|
|
|
|
string_concatenation = false;
|
|
|
|
|
2018-11-05 11:28:19 +01:00
|
|
|
if (info->rhs_code != INTEGER_CST)
|
|
|
|
only_constants = false;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* Merge a store recorded by INFO into this merged store.
|
|
|
|
The store is not overlapping with the existing recorded
|
|
|
|
stores. */
|
|
|
|
|
|
|
|
void
|
|
|
|
merged_store_group::merge_into (store_immediate_info *info)
|
|
|
|
{
|
2020-11-20 12:21:12 +01:00
|
|
|
do_merge (info);
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* Make sure we're inserting in the position we think we're inserting. */
|
|
|
|
gcc_assert (info->bitpos >= start + width
|
|
|
|
&& info->bitregion_start <= bitregion_end);
|
|
|
|
|
2018-02-22 09:28:42 +01:00
|
|
|
width = info->bitpos + info->bitsize - start;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Merge a store described by INFO into this merged store.
|
|
|
|
INFO overlaps in some way with the current store (i.e. it's not contiguous
|
|
|
|
which is handled by merged_store_group::merge_into). */
|
|
|
|
|
|
|
|
void
|
|
|
|
merged_store_group::merge_overlapping (store_immediate_info *info)
|
|
|
|
{
|
2020-11-20 12:21:12 +01:00
|
|
|
do_merge (info);
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* If the store extends the size of the group, extend the width. */
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (info->bitpos + info->bitsize > start + width)
|
2018-02-22 09:28:42 +01:00
|
|
|
width = info->bitpos + info->bitsize - start;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Go through all the recorded stores in this group in program order and
|
|
|
|
apply their values to the VAL byte array to create the final merged
|
|
|
|
value. Return true if the operation succeeded. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
merged_store_group::apply_stores ()
|
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
store_immediate_info *info;
|
|
|
|
unsigned int i;
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* Make sure we have more than one store in the group, otherwise we cannot
|
|
|
|
merge anything. */
|
|
|
|
if (bitregion_start % BITS_PER_UNIT != 0
|
|
|
|
|| bitregion_end % BITS_PER_UNIT != 0
|
2016-10-28 16:18:50 +02:00
|
|
|
|| stores.length () == 1)
|
|
|
|
return false;
|
|
|
|
|
2020-07-03 18:10:25 +02:00
|
|
|
buf_size = (bitregion_end - bitregion_start) / BITS_PER_UNIT;
|
|
|
|
|
|
|
|
/* Really do string concatenation for large strings only. */
|
|
|
|
if (buf_size <= MOVE_MAX)
|
|
|
|
string_concatenation = false;
|
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
/* Create a power-of-2-sized buffer for native_encode_expr. */
|
2020-07-03 18:10:25 +02:00
|
|
|
if (!string_concatenation)
|
|
|
|
buf_size = 1 << ceil_log2 (buf_size);
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
val = XNEWVEC (unsigned char, 2 * buf_size);
|
|
|
|
mask = val + buf_size;
|
|
|
|
memset (val, 0, buf_size);
|
|
|
|
memset (mask, ~0U, buf_size);
|
2016-10-28 16:18:50 +02:00
|
|
|
|
2020-07-03 18:10:25 +02:00
|
|
|
stores.qsort (sort_by_order);
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
FOR_EACH_VEC_ELT (stores, i, info)
|
|
|
|
{
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned int pos_in_buffer = info->bitpos - bitregion_start;
|
2018-06-01 23:56:17 +02:00
|
|
|
tree cst;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (info->ops[0].val && info->ops[0].base_addr == NULL_TREE)
|
|
|
|
cst = info->ops[0].val;
|
|
|
|
else if (info->ops[1].val && info->ops[1].base_addr == NULL_TREE)
|
|
|
|
cst = info->ops[1].val;
|
2018-06-01 23:56:17 +02:00
|
|
|
else
|
|
|
|
cst = NULL_TREE;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
bool ret = true;
|
2020-07-03 18:10:25 +02:00
|
|
|
if (cst && info->rhs_code != BIT_INSERT_EXPR)
|
|
|
|
ret = encode_tree_to_bitpos (cst, val, info->bitsize, pos_in_buffer,
|
|
|
|
buf_size);
|
2018-06-01 23:56:17 +02:00
|
|
|
unsigned char *m = mask + (pos_in_buffer / BITS_PER_UNIT);
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
clear_bit_region_be (m, (BITS_PER_UNIT - 1
|
|
|
|
- (pos_in_buffer % BITS_PER_UNIT)),
|
|
|
|
info->bitsize);
|
|
|
|
else
|
|
|
|
clear_bit_region (m, pos_in_buffer % BITS_PER_UNIT, info->bitsize);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (cst && dump_file && (dump_flags & TDF_DETAILS))
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
if (ret)
|
|
|
|
{
|
2018-06-01 23:56:17 +02:00
|
|
|
fputs ("After writing ", dump_file);
|
Convert dump and optgroup flags to enums
gcc/brig/ChangeLog:
* brigfrontend/brig-to-generic.cc
(brig_to_generic::write_globals): Use TDF_NONE rather than 0.
(dump_function): Likewise.
gcc/c-family/ChangeLog:
* c-pretty-print.c (c_pretty_printer::statement): Use TDF_NONE
rather than 0.
gcc/ChangeLog:
* cfg.c (debug): Use TDF_NONE rather than 0.
* cfghooks.c (debug): Likewise.
* dumpfile.c (DUMP_FILE_INFO): Likewise; also for OPTGROUP.
(struct dump_option_value_info): Convert to...
(struct kv_pair): ...this template type.
(dump_options): Convert to kv_pair<dump_flags_t>; use TDF_NONE
rather than 0.
(optinfo_verbosity_options): Likewise.
(optgroup_options): Convert to kv_pair<optgroup_flags_t>; use
OPTGROUP_NONE.
(gcc::dump_manager::dump_register): Use optgroup_flags_t rather
than int for "optgroup_flags" param.
(dump_generic_expr_loc): Use dump_flags_t rather than int for
"dump_kind" param.
(dump_dec): Likewise.
(dump_finish): Use TDF_NONE rather than 0.
(gcc::dump_manager::opt_info_enable_passes): Use optgroup_flags_t
rather than int for "optgroup_flags" param. Use TDF_NONE rather
than 0. Update for change to option_ptr.
(opt_info_switch_p_1): Convert "optgroup_flags" param from int *
to optgroup_flags_t *. Use TDF_NONE and OPTGROUP_NONE rather than
0. Update for changes to optinfo_verbosity_options and
optgroup_options.
(opt_info_switch_p): Convert optgroup_flags from int to
optgroup_flags_t.
(dump_basic_block): Use dump_flags_t rather than int
for "dump_kind" param.
* dumpfile.h (TDF_ADDRESS, TDF_SLIM, TDF_RAW, TDF_DETAILS,
TDF_STATS, TDF_BLOCKS, TDF_VOPS, TDF_LINENO, TDF_UID)
TDF_STMTADDR, TDF_GRAPH, TDF_MEMSYMS, TDF_RHS_ONLY, TDF_ASMNAME,
TDF_EH, TDF_NOUID, TDF_ALIAS, TDF_ENUMERATE_LOCALS, TDF_CSELIB,
TDF_SCEV, TDF_GIMPLE, TDF_FOLDING, MSG_OPTIMIZED_LOCATIONS,
MSG_MISSED_OPTIMIZATION, MSG_NOTE, MSG_ALL, TDF_COMPARE_DEBUG,
TDF_NONE): Convert from macros to...
(enum dump_flag): ...this new enum.
(dump_flags_t): Update to use enum.
(operator|, operator&, operator~, operator|=, operator&=):
Implement for dump_flags_t.
(OPTGROUP_NONE, OPTGROUP_IPA, OPTGROUP_LOOP, OPTGROUP_INLINE,
OPTGROUP_OMP, OPTGROUP_VEC, OPTGROUP_OTHER, OPTGROUP_ALL):
Convert from macros to...
(enum optgroup_flag): ...this new enum.
(optgroup_flags_t): New typedef.
(operator|, operator|=): Implement for optgroup_flags_t.
(struct dump_file_info): Convert field "alt_flags" to
dump_flags_t. Convert field "optgroup_flags" to
optgroup_flags_t.
(dump_basic_block): Use dump_flags_t rather than int for param.
(dump_generic_expr_loc): Likewise.
(dump_dec): Likewise.
(dump_register): Convert param "optgroup_flags" to
optgroup_flags_t.
(opt_info_enable_passes): Likewise.
* early-remat.c (early_remat::dump_edge_list): Use TDF_NONE rather
than 0.
* gimple-pretty-print.c (debug): Likewise.
* gimple-ssa-store-merging.c (bswap_replace): Likewise.
(merged_store_group::apply_stores): Likewise.
* gimple-ssa-strength-reduction.c (insert_initializers): Likewise.
* gimple.c (verify_gimple_pp): Likewise.
* graphite-poly.c (print_pbb_body): Likewise.
* passes.c (pass_manager::register_one_dump_file): Convert
local "optgroup_flags" to optgroup_flags_t.
* print-tree.c (print_node): Use TDF_NONE rather than 0.
(debug): Likewise.
(debug_body): Likewise.
* tree-pass.h (struct pass_data): Convert field "optgroup_flags"
to optgroup_flags_t.
* tree-pretty-print.c (print_struct_decl): Use TDF_NONE rather
than 0.
* tree-ssa-math-opts.c (convert_mult_to_fma_1): Likewise.
(convert_mult_to_fma): Likewise.
* tree-ssa-reassoc.c (undistribute_ops_list): Likewise.
* tree-ssa-sccvn.c (vn_eliminate): Likewise.
* tree-vect-data-refs.c (dump_lower_bound): Convert param
"dump_kind" to dump_flags_t.
From-SVN: r261325
2018-06-08 14:50:19 +02:00
|
|
|
print_generic_expr (dump_file, cst, TDF_NONE);
|
2016-10-28 16:18:50 +02:00
|
|
|
fprintf (dump_file, " of size " HOST_WIDE_INT_PRINT_DEC
|
2018-06-01 23:56:17 +02:00
|
|
|
" at position %d\n", info->bitsize, pos_in_buffer);
|
|
|
|
fputs (" the merged value contains ", dump_file);
|
2016-10-28 16:18:50 +02:00
|
|
|
dump_char_array (dump_file, val, buf_size);
|
2018-06-01 23:56:17 +02:00
|
|
|
fputs (" the merged mask contains ", dump_file);
|
|
|
|
dump_char_array (dump_file, mask, buf_size);
|
|
|
|
if (bit_insertion)
|
|
|
|
fputs (" bit insertion is required\n", dump_file);
|
2020-07-03 18:10:25 +02:00
|
|
|
if (string_concatenation)
|
|
|
|
fputs (" string concatenation is required\n", dump_file);
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
else
|
|
|
|
fprintf (dump_file, "Failed to merge stores\n");
|
2017-11-20 11:10:23 +01:00
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
if (!ret)
|
|
|
|
return false;
|
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
stores.qsort (sort_by_bitpos);
|
2016-10-28 16:18:50 +02:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Structure describing the store chain. */
|
|
|
|
|
2019-07-09 18:36:00 +02:00
|
|
|
class imm_store_chain_info
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2019-07-09 18:36:00 +02:00
|
|
|
public:
|
2017-03-17 00:31:01 +01:00
|
|
|
/* Doubly-linked list that imposes an order on chain processing.
|
|
|
|
PNXP (prev's next pointer) points to the head of a list, or to
|
|
|
|
the next field in the previous chain in the list.
|
|
|
|
See pass_store_merging::m_stores_head for more rationale. */
|
|
|
|
imm_store_chain_info *next, **pnxp;
|
2016-11-02 14:05:07 +01:00
|
|
|
tree base_addr;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
auto_vec<store_immediate_info *> m_store_info;
|
2016-10-28 16:18:50 +02:00
|
|
|
auto_vec<merged_store_group *> m_merged_store_groups;
|
|
|
|
|
2017-03-17 00:31:01 +01:00
|
|
|
imm_store_chain_info (imm_store_chain_info *&inspt, tree b_a)
|
|
|
|
: next (inspt), pnxp (&inspt), base_addr (b_a)
|
|
|
|
{
|
|
|
|
inspt = this;
|
|
|
|
if (next)
|
|
|
|
{
|
|
|
|
gcc_checking_assert (pnxp == next->pnxp);
|
|
|
|
next->pnxp = &next;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
~imm_store_chain_info ()
|
|
|
|
{
|
|
|
|
*pnxp = next;
|
|
|
|
if (next)
|
|
|
|
{
|
|
|
|
gcc_checking_assert (&next == next->pnxp);
|
|
|
|
next->pnxp = pnxp;
|
|
|
|
}
|
|
|
|
}
|
2016-11-02 14:05:07 +01:00
|
|
|
bool terminate_and_process_chain ();
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
bool try_coalesce_bswap (merged_store_group *, unsigned int, unsigned int,
|
|
|
|
unsigned int);
|
2016-10-28 16:18:50 +02:00
|
|
|
bool coalesce_immediate_stores ();
|
2016-11-02 14:05:07 +01:00
|
|
|
bool output_merged_store (merged_store_group *);
|
|
|
|
bool output_merged_stores ();
|
2016-10-28 16:18:50 +02:00
|
|
|
};
|
|
|
|
|
|
|
|
const pass_data pass_data_tree_store_merging = {
|
|
|
|
GIMPLE_PASS, /* type */
|
|
|
|
"store-merging", /* name */
|
|
|
|
OPTGROUP_NONE, /* optinfo_flags */
|
|
|
|
TV_GIMPLE_STORE_MERGING, /* tv_id */
|
|
|
|
PROP_ssa, /* properties_required */
|
|
|
|
0, /* properties_provided */
|
|
|
|
0, /* properties_destroyed */
|
|
|
|
0, /* todo_flags_start */
|
|
|
|
TODO_update_ssa, /* todo_flags_finish */
|
|
|
|
};
|
|
|
|
|
|
|
|
class pass_store_merging : public gimple_opt_pass
|
|
|
|
{
|
|
|
|
public:
|
|
|
|
pass_store_merging (gcc::context *ctxt)
|
2021-02-11 11:13:47 +01:00
|
|
|
: gimple_opt_pass (pass_data_tree_store_merging, ctxt), m_stores_head (),
|
|
|
|
m_n_chains (0), m_n_stores (0)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
/* Pass not supported for PDP-endian, nor for insane hosts or
|
|
|
|
target character sizes where native_{encode,interpret}_expr
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
doesn't work properly. */
|
2016-10-28 16:18:50 +02:00
|
|
|
virtual bool
|
|
|
|
gate (function *)
|
|
|
|
{
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
return flag_store_merging
|
2018-06-01 23:56:17 +02:00
|
|
|
&& BYTES_BIG_ENDIAN == WORDS_BIG_ENDIAN
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
&& CHAR_BIT == 8
|
|
|
|
&& BITS_PER_UNIT == 8;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
virtual unsigned int execute (function *);
|
|
|
|
|
|
|
|
private:
|
2019-07-09 20:32:49 +02:00
|
|
|
hash_map<tree_operand_hash, class imm_store_chain_info *> m_stores;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
2017-03-17 00:31:01 +01:00
|
|
|
/* Form a doubly-linked stack of the elements of m_stores, so that
|
|
|
|
we can iterate over them in a predictable way. Using this order
|
|
|
|
avoids extraneous differences in the compiler output just because
|
|
|
|
of tree pointer variations (e.g. different chains end up in
|
|
|
|
different positions of m_stores, so they are handled in different
|
|
|
|
orders, so they allocate or release SSA names in different
|
|
|
|
orders, and when they get reused, subsequent passes end up
|
|
|
|
getting different SSA names, which may ultimately change
|
|
|
|
decisions when going out of SSA). */
|
|
|
|
imm_store_chain_info *m_stores_head;
|
|
|
|
|
2021-02-11 11:13:47 +01:00
|
|
|
/* The number of store chains currently tracked. */
|
|
|
|
unsigned m_n_chains;
|
|
|
|
/* The number of stores currently tracked. */
|
|
|
|
unsigned m_n_stores;
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
bool process_store (gimple *);
|
|
|
|
bool terminate_and_process_chain (imm_store_chain_info *);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
bool terminate_all_aliasing_chains (imm_store_chain_info **, gimple *);
|
2019-10-02 15:35:40 +02:00
|
|
|
bool terminate_and_process_all_chains ();
|
2016-10-28 16:18:50 +02:00
|
|
|
}; // class pass_store_merging
|
|
|
|
|
|
|
|
/* Terminate and process all recorded chains. Return true if any changes
|
|
|
|
were made. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
pass_store_merging::terminate_and_process_all_chains ()
|
|
|
|
{
|
|
|
|
bool ret = false;
|
2017-03-17 00:31:01 +01:00
|
|
|
while (m_stores_head)
|
2019-10-02 15:35:40 +02:00
|
|
|
ret |= terminate_and_process_chain (m_stores_head);
|
2019-05-03 14:37:22 +02:00
|
|
|
gcc_assert (m_stores.is_empty ());
|
2016-10-28 16:18:50 +02:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
/* Terminate all chains that are affected by the statement STMT.
|
|
|
|
CHAIN_INFO is the chain we should ignore from the checks if
|
2019-10-02 15:35:40 +02:00
|
|
|
non-NULL. Return true if any changes were made. */
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
bool
|
2016-11-07 13:26:00 +01:00
|
|
|
pass_store_merging::terminate_all_aliasing_chains (imm_store_chain_info
|
2016-11-02 14:05:07 +01:00
|
|
|
**chain_info,
|
2016-10-28 16:18:50 +02:00
|
|
|
gimple *stmt)
|
|
|
|
{
|
|
|
|
bool ret = false;
|
|
|
|
|
|
|
|
/* If the statement doesn't touch memory it can't alias. */
|
|
|
|
if (!gimple_vuse (stmt))
|
|
|
|
return false;
|
|
|
|
|
2017-11-10 11:31:34 +01:00
|
|
|
tree store_lhs = gimple_store_p (stmt) ? gimple_get_lhs (stmt) : NULL_TREE;
|
2020-01-10 14:24:04 +01:00
|
|
|
ao_ref store_lhs_ref;
|
|
|
|
ao_ref_init (&store_lhs_ref, store_lhs);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
for (imm_store_chain_info *next = m_stores_head, *cur = next; cur; cur = next)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
next = cur->next;
|
|
|
|
|
|
|
|
/* We already checked all the stores in chain_info and terminated the
|
|
|
|
chain if necessary. Skip it here. */
|
|
|
|
if (chain_info && *chain_info == cur)
|
|
|
|
continue;
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
store_immediate_info *info;
|
|
|
|
unsigned int i;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
FOR_EACH_VEC_ELT (cur->m_store_info, i, info)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2017-11-10 11:31:34 +01:00
|
|
|
tree lhs = gimple_assign_lhs (info->stmt);
|
2020-01-10 14:24:04 +01:00
|
|
|
ao_ref lhs_ref;
|
|
|
|
ao_ref_init (&lhs_ref, lhs);
|
|
|
|
if (ref_maybe_used_by_stmt_p (stmt, &lhs_ref)
|
|
|
|
|| stmt_may_clobber_ref_p_1 (stmt, &lhs_ref)
|
|
|
|
|| (store_lhs && refs_may_alias_p_1 (&store_lhs_ref,
|
|
|
|
&lhs_ref, false)))
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
fprintf (dump_file, "stmt causes chain termination:\n");
|
|
|
|
print_gimple_stmt (dump_file, stmt, 0);
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
2019-10-02 15:35:40 +02:00
|
|
|
ret |= terminate_and_process_chain (cur);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
break;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Helper function. Terminate the recorded chain storing to base object
|
|
|
|
BASE. Return true if the merging and output was successful. The m_stores
|
|
|
|
entry is removed after the processing in any case. */
|
|
|
|
|
|
|
|
bool
|
2019-10-02 15:35:40 +02:00
|
|
|
pass_store_merging::terminate_and_process_chain (imm_store_chain_info *chain_info)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2021-02-11 11:13:47 +01:00
|
|
|
m_n_stores -= chain_info->m_store_info.length ();
|
|
|
|
m_n_chains--;
|
2016-11-02 14:05:07 +01:00
|
|
|
bool ret = chain_info->terminate_and_process_chain ();
|
|
|
|
m_stores.remove (chain_info->base_addr);
|
|
|
|
delete chain_info;
|
2016-10-28 16:18:50 +02:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Return true if stmts in between FIRST (inclusive) and LAST (exclusive)
|
2019-10-02 15:35:40 +02:00
|
|
|
may clobber REF. FIRST and LAST must have non-NULL vdef. We want to
|
|
|
|
be able to sink load of REF across stores between FIRST and LAST, up
|
|
|
|
to right before LAST. */
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
|
|
|
bool
|
|
|
|
stmts_may_clobber_ref_p (gimple *first, gimple *last, tree ref)
|
|
|
|
{
|
|
|
|
ao_ref r;
|
|
|
|
ao_ref_init (&r, ref);
|
|
|
|
unsigned int count = 0;
|
|
|
|
tree vop = gimple_vdef (last);
|
|
|
|
gimple *stmt;
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
/* Return true conservatively if the basic blocks are different. */
|
|
|
|
if (gimple_bb (first) != gimple_bb (last))
|
|
|
|
return true;
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
do
|
|
|
|
{
|
|
|
|
stmt = SSA_NAME_DEF_STMT (vop);
|
|
|
|
if (stmt_may_clobber_ref_p_1 (stmt, &r))
|
|
|
|
return true;
|
2017-11-20 11:10:23 +01:00
|
|
|
if (gimple_store_p (stmt)
|
|
|
|
&& refs_anti_dependent_p (ref, gimple_get_lhs (stmt)))
|
|
|
|
return true;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Avoid quadratic compile time by bounding the number of checks
|
|
|
|
we perform. */
|
|
|
|
if (++count > MAX_STORE_ALIAS_CHECKS)
|
|
|
|
return true;
|
|
|
|
vop = gimple_vuse (stmt);
|
|
|
|
}
|
|
|
|
while (stmt != first);
|
2019-10-02 15:35:40 +02:00
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Return true if INFO->ops[IDX] is mergeable with the
|
|
|
|
corresponding loads already in MERGED_STORE group.
|
|
|
|
BASE_ADDR is the base address of the whole store group. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
compatible_load_p (merged_store_group *merged_store,
|
|
|
|
store_immediate_info *info,
|
|
|
|
tree base_addr, int idx)
|
|
|
|
{
|
|
|
|
store_immediate_info *infof = merged_store->stores[0];
|
|
|
|
if (!info->ops[idx].base_addr
|
2017-12-20 13:56:50 +01:00
|
|
|
|| maybe_ne (info->ops[idx].bitpos - infof->ops[idx].bitpos,
|
|
|
|
info->bitpos - infof->bitpos)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|| !operand_equal_p (info->ops[idx].base_addr,
|
|
|
|
infof->ops[idx].base_addr, 0))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
store_immediate_info *infol = merged_store->stores.last ();
|
|
|
|
tree load_vuse = gimple_vuse (info->ops[idx].stmt);
|
|
|
|
/* In this case all vuses should be the same, e.g.
|
|
|
|
_1 = s.a; _2 = s.b; _3 = _1 | 1; t.a = _3; _4 = _2 | 2; t.b = _4;
|
|
|
|
or
|
|
|
|
_1 = s.a; _2 = s.b; t.a = _1; t.b = _2;
|
|
|
|
and we can emit the coalesced load next to any of those loads. */
|
|
|
|
if (gimple_vuse (infof->ops[idx].stmt) == load_vuse
|
|
|
|
&& gimple_vuse (infol->ops[idx].stmt) == load_vuse)
|
|
|
|
return true;
|
|
|
|
|
|
|
|
/* Otherwise, at least for now require that the load has the same
|
|
|
|
vuse as the store. See following examples. */
|
|
|
|
if (gimple_vuse (info->stmt) != load_vuse)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
if (gimple_vuse (infof->stmt) != gimple_vuse (infof->ops[idx].stmt)
|
|
|
|
|| (infof != infol
|
|
|
|
&& gimple_vuse (infol->stmt) != gimple_vuse (infol->ops[idx].stmt)))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
/* If the load is from the same location as the store, already
|
|
|
|
the construction of the immediate chain info guarantees no intervening
|
|
|
|
stores, so no further checks are needed. Example:
|
|
|
|
_1 = s.a; _2 = _1 & -7; s.a = _2; _3 = s.b; _4 = _3 & -7; s.b = _4; */
|
2017-12-20 13:56:50 +01:00
|
|
|
if (known_eq (info->ops[idx].bitpos, info->bitpos)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
&& operand_equal_p (info->ops[idx].base_addr, base_addr, 0))
|
|
|
|
return true;
|
|
|
|
|
|
|
|
/* Otherwise, we need to punt if any of the loads can be clobbered by any
|
|
|
|
of the stores in the group, or any other stores in between those.
|
|
|
|
Previous calls to compatible_load_p ensured that for all the
|
|
|
|
merged_store->stores IDX loads, no stmts starting with
|
|
|
|
merged_store->first_stmt and ending right before merged_store->last_stmt
|
|
|
|
clobbers those loads. */
|
|
|
|
gimple *first = merged_store->first_stmt;
|
|
|
|
gimple *last = merged_store->last_stmt;
|
|
|
|
/* The stores are sorted by increasing store bitpos, so if info->stmt store
|
|
|
|
comes before the so far first load, we'll be changing
|
|
|
|
merged_store->first_stmt. In that case we need to give up if
|
|
|
|
any of the earlier processed loads clobber with the stmts in the new
|
|
|
|
range. */
|
|
|
|
if (info->order < merged_store->first_order)
|
|
|
|
{
|
2021-06-12 05:49:22 +02:00
|
|
|
for (store_immediate_info *infoc : merged_store->stores)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (stmts_may_clobber_ref_p (info->stmt, first, infoc->ops[idx].val))
|
|
|
|
return false;
|
|
|
|
first = info->stmt;
|
|
|
|
}
|
|
|
|
/* Similarly, we could change merged_store->last_stmt, so ensure
|
|
|
|
in that case no stmts in the new range clobber any of the earlier
|
|
|
|
processed loads. */
|
|
|
|
else if (info->order > merged_store->last_order)
|
|
|
|
{
|
2021-06-12 05:49:22 +02:00
|
|
|
for (store_immediate_info *infoc : merged_store->stores)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (stmts_may_clobber_ref_p (last, info->stmt, infoc->ops[idx].val))
|
|
|
|
return false;
|
|
|
|
last = info->stmt;
|
|
|
|
}
|
|
|
|
/* And finally, we'd be adding a new load to the set, ensure it isn't
|
|
|
|
clobbered in the new range. */
|
|
|
|
if (stmts_may_clobber_ref_p (first, last, info->ops[idx].val))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
/* Otherwise, we are looking for:
|
|
|
|
_1 = s.a; _2 = _1 ^ 15; t.a = _2; _3 = s.b; _4 = _3 ^ 15; t.b = _4;
|
|
|
|
or
|
|
|
|
_1 = s.a; t.a = _1; _2 = s.b; t.b = _2; */
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
/* Add all refs loaded to compute VAL to REFS vector. */
|
|
|
|
|
|
|
|
void
|
|
|
|
gather_bswap_load_refs (vec<tree> *refs, tree val)
|
|
|
|
{
|
|
|
|
if (TREE_CODE (val) != SSA_NAME)
|
|
|
|
return;
|
|
|
|
|
|
|
|
gimple *stmt = SSA_NAME_DEF_STMT (val);
|
|
|
|
if (!is_gimple_assign (stmt))
|
|
|
|
return;
|
|
|
|
|
|
|
|
if (gimple_assign_load_p (stmt))
|
|
|
|
{
|
|
|
|
refs->safe_push (gimple_assign_rhs1 (stmt));
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
switch (gimple_assign_rhs_class (stmt))
|
|
|
|
{
|
|
|
|
case GIMPLE_BINARY_RHS:
|
|
|
|
gather_bswap_load_refs (refs, gimple_assign_rhs2 (stmt));
|
|
|
|
/* FALLTHRU */
|
|
|
|
case GIMPLE_UNARY_RHS:
|
|
|
|
gather_bswap_load_refs (refs, gimple_assign_rhs1 (stmt));
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
gcc_unreachable ();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-02-22 09:28:42 +01:00
|
|
|
/* Check if there are any stores in M_STORE_INFO after index I
|
|
|
|
(where M_STORE_INFO must be sorted by sort_by_bitpos) that overlap
|
|
|
|
a potential group ending with END that have their order
|
store-merging: Fix coalesce_immediate_stores [PR93820]
The following testcase is miscompiled in 8+.
The problem is that check_no_overlap has a special case for INTEGER_CST
marked stores (i.e. stores of constants), if both all currenly merged stores
and the one under consideration for merging with them are marked that way,
it anticipates that other INTEGER_CST marked stores that overlap with those
and precede those (have smaller info->order) could be merged with those and
doesn't punt for them.
In PR86844 and PR87859 fixes I've then added quite large code that is
performed after check_no_overlap and tries to find out if we need and can
merge further INTEGER_CST marked stores, or need to punt.
Unfortunately, that code is there only in the overlapping case code and
the testcase below shows that we really need it even in the adjacent store
case. After sort_by_bitpos we have:
bitpos width order rhs_code
96 32 3 INTEGER_CST
128 32 1 INTEGER_CST
128 128 2 INTEGER_CST
192 32 0 MEM_REF
Because of the missing PR86844/PR87859-ish code in the adjacent store
case, we merge the adjacent (memory wise) stores 96/32/3 and 128/32/1,
and then we consider the 128-bit store which is in program-order in between
them, but in this case we punt, because the merging would extend the
merged store region from bitpos 96 and 64-bits to bitpos 96 and 160-bits
and that has an overlap with an incompatible store (the MEM_REF one).
The problem is that we can't really punt this way, because the 128-bit
store is in between those two we've merged already, so either we manage
to merge even that one together with the others, or would need to avoid
already merging the 96/32/3 and 128/32/1 stores together.
Now, rather than copying around the PR86844/PR87859 code to the other spot,
we can actually just use the overlapping code, merge_overlapping is really
a superset of merge_into, so that is what the patch does. If doing
adjacent store merge for rhs_code other than INTEGER_CST, I believe the
current code is already fine, check_no_overlap in that case doesn't make
the exception and will punt if there is some earlier (smaller order)
non-mergeable overlapping store. There is just one case that could be
problematic, if the merged_store has BIT_INSERT_EXPRs in them and the
new store is a constant store (INTEGER_CST rhs_code), then check_no_overlap
would do the exception and still would allow the special case. But we
really shouldn't have the special case in that case, so this patch also
changes check_no_overlap to just have a bool whether we should have the
special case or not.
Note, as I said in the PR, for GCC11 we could consider performing some kind
of cheap DSE during the store merging (perhaps guarded with flag_tree_dse).
And another thing to consider is only consider as problematic non-mergeable
stores that not only have order smaller than last_order as currently, but
also have order larger than first_order, as in this testcase if we actually
ignored (not merged with anything at all) the 192/32/0 store, because it is
not in between the other stores we'd merge, it would be fine to merge the
other 3 stores, though of course the testcase can be easily adjusted by
putting the 192/32 store after the 128/32 store and then this patch would be
still needed. Though, I think I'd need more time thinking this over.
2020-02-26 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93820
* gimple-ssa-store-merging.c (check_no_overlap): Change RHS_CODE
argument to ALL_INTEGER_CST_P boolean.
(imm_store_chain_info::try_coalesce_bswap): Adjust caller.
(imm_store_chain_info::coalesce_immediate_stores): Likewise. Handle
adjacent INTEGER_CST store into merged_store->only_constants like
overlapping one.
* gcc.dg/pr93820.c: New test.
2020-02-26 09:33:48 +01:00
|
|
|
smaller than LAST_ORDER. ALL_INTEGER_CST_P is true if
|
|
|
|
all the stores already merged and the one under consideration
|
|
|
|
have rhs_code of INTEGER_CST. Return true if there are no such stores.
|
2018-02-22 09:28:42 +01:00
|
|
|
Consider:
|
|
|
|
MEM[(long long int *)p_28] = 0;
|
|
|
|
MEM[(long long int *)p_28 + 8B] = 0;
|
|
|
|
MEM[(long long int *)p_28 + 16B] = 0;
|
|
|
|
MEM[(long long int *)p_28 + 24B] = 0;
|
|
|
|
_129 = (int) _130;
|
|
|
|
MEM[(int *)p_28 + 8B] = _129;
|
|
|
|
MEM[(int *)p_28].a = -1;
|
|
|
|
We already have
|
|
|
|
MEM[(long long int *)p_28] = 0;
|
|
|
|
MEM[(int *)p_28].a = -1;
|
|
|
|
stmts in the current group and need to consider if it is safe to
|
|
|
|
add MEM[(long long int *)p_28 + 8B] = 0; store into the same group.
|
|
|
|
There is an overlap between that store and the MEM[(int *)p_28 + 8B] = _129;
|
|
|
|
store though, so if we add the MEM[(long long int *)p_28 + 8B] = 0;
|
|
|
|
into the group and merging of those 3 stores is successful, merged
|
|
|
|
stmts will be emitted at the latest store from that group, i.e.
|
|
|
|
LAST_ORDER, which is the MEM[(int *)p_28].a = -1; store.
|
|
|
|
The MEM[(int *)p_28 + 8B] = _129; store that originally follows
|
|
|
|
the MEM[(long long int *)p_28 + 8B] = 0; would now be before it,
|
|
|
|
so we need to refuse merging MEM[(long long int *)p_28 + 8B] = 0;
|
|
|
|
into the group. That way it will be its own store group and will
|
store-merging: Fix coalesce_immediate_stores [PR93820]
The following testcase is miscompiled in 8+.
The problem is that check_no_overlap has a special case for INTEGER_CST
marked stores (i.e. stores of constants), if both all currenly merged stores
and the one under consideration for merging with them are marked that way,
it anticipates that other INTEGER_CST marked stores that overlap with those
and precede those (have smaller info->order) could be merged with those and
doesn't punt for them.
In PR86844 and PR87859 fixes I've then added quite large code that is
performed after check_no_overlap and tries to find out if we need and can
merge further INTEGER_CST marked stores, or need to punt.
Unfortunately, that code is there only in the overlapping case code and
the testcase below shows that we really need it even in the adjacent store
case. After sort_by_bitpos we have:
bitpos width order rhs_code
96 32 3 INTEGER_CST
128 32 1 INTEGER_CST
128 128 2 INTEGER_CST
192 32 0 MEM_REF
Because of the missing PR86844/PR87859-ish code in the adjacent store
case, we merge the adjacent (memory wise) stores 96/32/3 and 128/32/1,
and then we consider the 128-bit store which is in program-order in between
them, but in this case we punt, because the merging would extend the
merged store region from bitpos 96 and 64-bits to bitpos 96 and 160-bits
and that has an overlap with an incompatible store (the MEM_REF one).
The problem is that we can't really punt this way, because the 128-bit
store is in between those two we've merged already, so either we manage
to merge even that one together with the others, or would need to avoid
already merging the 96/32/3 and 128/32/1 stores together.
Now, rather than copying around the PR86844/PR87859 code to the other spot,
we can actually just use the overlapping code, merge_overlapping is really
a superset of merge_into, so that is what the patch does. If doing
adjacent store merge for rhs_code other than INTEGER_CST, I believe the
current code is already fine, check_no_overlap in that case doesn't make
the exception and will punt if there is some earlier (smaller order)
non-mergeable overlapping store. There is just one case that could be
problematic, if the merged_store has BIT_INSERT_EXPRs in them and the
new store is a constant store (INTEGER_CST rhs_code), then check_no_overlap
would do the exception and still would allow the special case. But we
really shouldn't have the special case in that case, so this patch also
changes check_no_overlap to just have a bool whether we should have the
special case or not.
Note, as I said in the PR, for GCC11 we could consider performing some kind
of cheap DSE during the store merging (perhaps guarded with flag_tree_dse).
And another thing to consider is only consider as problematic non-mergeable
stores that not only have order smaller than last_order as currently, but
also have order larger than first_order, as in this testcase if we actually
ignored (not merged with anything at all) the 192/32/0 store, because it is
not in between the other stores we'd merge, it would be fine to merge the
other 3 stores, though of course the testcase can be easily adjusted by
putting the 192/32 store after the 128/32 store and then this patch would be
still needed. Though, I think I'd need more time thinking this over.
2020-02-26 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93820
* gimple-ssa-store-merging.c (check_no_overlap): Change RHS_CODE
argument to ALL_INTEGER_CST_P boolean.
(imm_store_chain_info::try_coalesce_bswap): Adjust caller.
(imm_store_chain_info::coalesce_immediate_stores): Likewise. Handle
adjacent INTEGER_CST store into merged_store->only_constants like
overlapping one.
* gcc.dg/pr93820.c: New test.
2020-02-26 09:33:48 +01:00
|
|
|
not be touched. If ALL_INTEGER_CST_P and there are overlapping
|
2018-02-22 09:28:42 +01:00
|
|
|
INTEGER_CST stores, those are mergeable using merge_overlapping,
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
so don't return false for those.
|
|
|
|
|
|
|
|
Similarly, check stores from FIRST_EARLIER (inclusive) to END_EARLIER
|
|
|
|
(exclusive), whether they don't overlap the bitrange START to END
|
|
|
|
and have order in between FIRST_ORDER and LAST_ORDER. This is to
|
|
|
|
prevent merging in cases like:
|
|
|
|
MEM <char[12]> [&b + 8B] = {};
|
|
|
|
MEM[(short *) &b] = 5;
|
|
|
|
_5 = *x_4(D);
|
|
|
|
MEM <long long unsigned int> [&b + 2B] = _5;
|
|
|
|
MEM[(char *)&b + 16B] = 88;
|
|
|
|
MEM[(int *)&b + 20B] = 1;
|
|
|
|
The = {} store comes in sort_by_bitpos before the = 88 store, and can't
|
|
|
|
be merged with it, because the = _5 store overlaps these and is in between
|
|
|
|
them in sort_by_order ordering. If it was merged, the merged store would
|
|
|
|
go after the = _5 store and thus change behavior. */
|
2018-02-22 09:28:42 +01:00
|
|
|
|
|
|
|
static bool
|
|
|
|
check_no_overlap (vec<store_immediate_info *> m_store_info, unsigned int i,
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
bool all_integer_cst_p, unsigned int first_order,
|
|
|
|
unsigned int last_order, unsigned HOST_WIDE_INT start,
|
|
|
|
unsigned HOST_WIDE_INT end, unsigned int first_earlier,
|
|
|
|
unsigned end_earlier)
|
2018-02-22 09:28:42 +01:00
|
|
|
{
|
|
|
|
unsigned int len = m_store_info.length ();
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
for (unsigned int j = first_earlier; j < end_earlier; j++)
|
|
|
|
{
|
|
|
|
store_immediate_info *info = m_store_info[j];
|
|
|
|
if (info->order > first_order
|
|
|
|
&& info->order < last_order
|
|
|
|
&& info->bitpos + info->bitsize > start)
|
|
|
|
return false;
|
|
|
|
}
|
2018-02-22 09:28:42 +01:00
|
|
|
for (++i; i < len; ++i)
|
|
|
|
{
|
|
|
|
store_immediate_info *info = m_store_info[i];
|
|
|
|
if (info->bitpos >= end)
|
|
|
|
break;
|
|
|
|
if (info->order < last_order
|
store-merging: Fix coalesce_immediate_stores [PR93820]
The following testcase is miscompiled in 8+.
The problem is that check_no_overlap has a special case for INTEGER_CST
marked stores (i.e. stores of constants), if both all currenly merged stores
and the one under consideration for merging with them are marked that way,
it anticipates that other INTEGER_CST marked stores that overlap with those
and precede those (have smaller info->order) could be merged with those and
doesn't punt for them.
In PR86844 and PR87859 fixes I've then added quite large code that is
performed after check_no_overlap and tries to find out if we need and can
merge further INTEGER_CST marked stores, or need to punt.
Unfortunately, that code is there only in the overlapping case code and
the testcase below shows that we really need it even in the adjacent store
case. After sort_by_bitpos we have:
bitpos width order rhs_code
96 32 3 INTEGER_CST
128 32 1 INTEGER_CST
128 128 2 INTEGER_CST
192 32 0 MEM_REF
Because of the missing PR86844/PR87859-ish code in the adjacent store
case, we merge the adjacent (memory wise) stores 96/32/3 and 128/32/1,
and then we consider the 128-bit store which is in program-order in between
them, but in this case we punt, because the merging would extend the
merged store region from bitpos 96 and 64-bits to bitpos 96 and 160-bits
and that has an overlap with an incompatible store (the MEM_REF one).
The problem is that we can't really punt this way, because the 128-bit
store is in between those two we've merged already, so either we manage
to merge even that one together with the others, or would need to avoid
already merging the 96/32/3 and 128/32/1 stores together.
Now, rather than copying around the PR86844/PR87859 code to the other spot,
we can actually just use the overlapping code, merge_overlapping is really
a superset of merge_into, so that is what the patch does. If doing
adjacent store merge for rhs_code other than INTEGER_CST, I believe the
current code is already fine, check_no_overlap in that case doesn't make
the exception and will punt if there is some earlier (smaller order)
non-mergeable overlapping store. There is just one case that could be
problematic, if the merged_store has BIT_INSERT_EXPRs in them and the
new store is a constant store (INTEGER_CST rhs_code), then check_no_overlap
would do the exception and still would allow the special case. But we
really shouldn't have the special case in that case, so this patch also
changes check_no_overlap to just have a bool whether we should have the
special case or not.
Note, as I said in the PR, for GCC11 we could consider performing some kind
of cheap DSE during the store merging (perhaps guarded with flag_tree_dse).
And another thing to consider is only consider as problematic non-mergeable
stores that not only have order smaller than last_order as currently, but
also have order larger than first_order, as in this testcase if we actually
ignored (not merged with anything at all) the 192/32/0 store, because it is
not in between the other stores we'd merge, it would be fine to merge the
other 3 stores, though of course the testcase can be easily adjusted by
putting the 192/32 store after the 128/32 store and then this patch would be
still needed. Though, I think I'd need more time thinking this over.
2020-02-26 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93820
* gimple-ssa-store-merging.c (check_no_overlap): Change RHS_CODE
argument to ALL_INTEGER_CST_P boolean.
(imm_store_chain_info::try_coalesce_bswap): Adjust caller.
(imm_store_chain_info::coalesce_immediate_stores): Likewise. Handle
adjacent INTEGER_CST store into merged_store->only_constants like
overlapping one.
* gcc.dg/pr93820.c: New test.
2020-02-26 09:33:48 +01:00
|
|
|
&& (!all_integer_cst_p || info->rhs_code != INTEGER_CST))
|
2018-02-22 09:28:42 +01:00
|
|
|
return false;
|
|
|
|
}
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
/* Return true if m_store_info[first] and at least one following store
|
|
|
|
form a group which store try_size bitsize value which is byte swapped
|
|
|
|
from a memory load or some value, or identity from some value.
|
|
|
|
This uses the bswap pass APIs. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
imm_store_chain_info::try_coalesce_bswap (merged_store_group *merged_store,
|
|
|
|
unsigned int first,
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
unsigned int try_size,
|
|
|
|
unsigned int first_earlier)
|
2017-11-20 11:10:23 +01:00
|
|
|
{
|
|
|
|
unsigned int len = m_store_info.length (), last = first;
|
|
|
|
unsigned HOST_WIDE_INT width = m_store_info[first]->bitsize;
|
|
|
|
if (width >= try_size)
|
|
|
|
return false;
|
|
|
|
for (unsigned int i = first + 1; i < len; ++i)
|
|
|
|
{
|
|
|
|
if (m_store_info[i]->bitpos != m_store_info[first]->bitpos + width
|
2020-04-23 22:25:04 +02:00
|
|
|
|| m_store_info[i]->lp_nr != merged_store->lp_nr
|
2017-11-20 11:10:23 +01:00
|
|
|
|| m_store_info[i]->ins_stmt == NULL)
|
|
|
|
return false;
|
|
|
|
width += m_store_info[i]->bitsize;
|
|
|
|
if (width >= try_size)
|
|
|
|
{
|
|
|
|
last = i;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (width != try_size)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
bool allow_unaligned
|
2019-11-12 11:08:40 +01:00
|
|
|
= !STRICT_ALIGNMENT && param_store_merging_allow_unaligned;
|
2017-11-20 11:10:23 +01:00
|
|
|
/* Punt if the combined store would not be aligned and we need alignment. */
|
|
|
|
if (!allow_unaligned)
|
|
|
|
{
|
|
|
|
unsigned int align = merged_store->align;
|
|
|
|
unsigned HOST_WIDE_INT align_base = merged_store->align_base;
|
|
|
|
for (unsigned int i = first + 1; i <= last; ++i)
|
|
|
|
{
|
|
|
|
unsigned int this_align;
|
|
|
|
unsigned HOST_WIDE_INT align_bitpos = 0;
|
|
|
|
get_object_alignment_1 (gimple_assign_lhs (m_store_info[i]->stmt),
|
|
|
|
&this_align, &align_bitpos);
|
|
|
|
if (this_align > align)
|
|
|
|
{
|
|
|
|
align = this_align;
|
|
|
|
align_base = m_store_info[i]->bitpos - align_bitpos;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
unsigned HOST_WIDE_INT align_bitpos
|
|
|
|
= (m_store_info[first]->bitpos - align_base) & (align - 1);
|
|
|
|
if (align_bitpos)
|
|
|
|
align = least_bit_hwi (align_bitpos);
|
|
|
|
if (align < try_size)
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
tree type;
|
|
|
|
switch (try_size)
|
|
|
|
{
|
|
|
|
case 16: type = uint16_type_node; break;
|
|
|
|
case 32: type = uint32_type_node; break;
|
|
|
|
case 64: type = uint64_type_node; break;
|
|
|
|
default: gcc_unreachable ();
|
|
|
|
}
|
|
|
|
struct symbolic_number n;
|
|
|
|
gimple *ins_stmt = NULL;
|
|
|
|
int vuse_store = -1;
|
|
|
|
unsigned int first_order = merged_store->first_order;
|
|
|
|
unsigned int last_order = merged_store->last_order;
|
|
|
|
gimple *first_stmt = merged_store->first_stmt;
|
|
|
|
gimple *last_stmt = merged_store->last_stmt;
|
2018-02-22 09:28:42 +01:00
|
|
|
unsigned HOST_WIDE_INT end = merged_store->start + merged_store->width;
|
2017-11-20 11:10:23 +01:00
|
|
|
store_immediate_info *infof = m_store_info[first];
|
|
|
|
|
|
|
|
for (unsigned int i = first; i <= last; ++i)
|
|
|
|
{
|
|
|
|
store_immediate_info *info = m_store_info[i];
|
|
|
|
struct symbolic_number this_n = info->n;
|
|
|
|
this_n.type = type;
|
|
|
|
if (!this_n.base_addr)
|
|
|
|
this_n.range = try_size / BITS_PER_UNIT;
|
2017-12-02 08:58:57 +01:00
|
|
|
else
|
|
|
|
/* Update vuse in case it has changed by output_merged_stores. */
|
|
|
|
this_n.vuse = gimple_vuse (info->ins_stmt);
|
2017-11-20 11:10:23 +01:00
|
|
|
unsigned int bitpos = info->bitpos - infof->bitpos;
|
|
|
|
if (!do_shift_rotate (LSHIFT_EXPR, &this_n,
|
|
|
|
BYTES_BIG_ENDIAN
|
|
|
|
? try_size - info->bitsize - bitpos
|
|
|
|
: bitpos))
|
|
|
|
return false;
|
2017-11-21 14:54:08 +01:00
|
|
|
if (this_n.base_addr && vuse_store)
|
2017-11-20 11:10:23 +01:00
|
|
|
{
|
|
|
|
unsigned int j;
|
|
|
|
for (j = first; j <= last; ++j)
|
|
|
|
if (this_n.vuse == gimple_vuse (m_store_info[j]->stmt))
|
|
|
|
break;
|
|
|
|
if (j > last)
|
|
|
|
{
|
|
|
|
if (vuse_store == 1)
|
|
|
|
return false;
|
|
|
|
vuse_store = 0;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (i == first)
|
|
|
|
{
|
|
|
|
n = this_n;
|
|
|
|
ins_stmt = info->ins_stmt;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2018-02-22 09:28:42 +01:00
|
|
|
if (n.base_addr && n.vuse != this_n.vuse)
|
2017-11-20 11:10:23 +01:00
|
|
|
{
|
2018-02-22 09:28:42 +01:00
|
|
|
if (vuse_store == 0)
|
|
|
|
return false;
|
|
|
|
vuse_store = 1;
|
2017-11-20 11:10:23 +01:00
|
|
|
}
|
2018-02-22 09:28:42 +01:00
|
|
|
if (info->order > last_order)
|
|
|
|
{
|
|
|
|
last_order = info->order;
|
|
|
|
last_stmt = info->stmt;
|
|
|
|
}
|
|
|
|
else if (info->order < first_order)
|
|
|
|
{
|
|
|
|
first_order = info->order;
|
|
|
|
first_stmt = info->stmt;
|
|
|
|
}
|
|
|
|
end = MAX (end, info->bitpos + info->bitsize);
|
2017-11-20 11:10:23 +01:00
|
|
|
|
|
|
|
ins_stmt = perform_symbolic_merge (ins_stmt, &n, info->ins_stmt,
|
|
|
|
&this_n, &n);
|
|
|
|
if (ins_stmt == NULL)
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
uint64_t cmpxchg, cmpnop;
|
|
|
|
find_bswap_or_nop_finalize (&n, &cmpxchg, &cmpnop);
|
|
|
|
|
|
|
|
/* A complete byte swap should make the symbolic number to start with
|
|
|
|
the largest digit in the highest order byte. Unchanged symbolic
|
|
|
|
number indicates a read with same endianness as target architecture. */
|
|
|
|
if (n.n != cmpnop && n.n != cmpxchg)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
if (n.base_addr == NULL_TREE && !is_gimple_val (n.src))
|
|
|
|
return false;
|
|
|
|
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
if (!check_no_overlap (m_store_info, last, false, first_order, last_order,
|
|
|
|
merged_store->start, end, first_earlier, first))
|
2018-02-22 09:28:42 +01:00
|
|
|
return false;
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
/* Don't handle memory copy this way if normal non-bswap processing
|
|
|
|
would handle it too. */
|
|
|
|
if (n.n == cmpnop && (unsigned) n.n_ops == last - first + 1)
|
|
|
|
{
|
|
|
|
unsigned int i;
|
|
|
|
for (i = first; i <= last; ++i)
|
|
|
|
if (m_store_info[i]->rhs_code != MEM_REF)
|
|
|
|
break;
|
|
|
|
if (i == last + 1)
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (n.n == cmpxchg)
|
|
|
|
switch (try_size)
|
|
|
|
{
|
|
|
|
case 16:
|
|
|
|
/* Will emit LROTATE_EXPR. */
|
|
|
|
break;
|
|
|
|
case 32:
|
|
|
|
if (builtin_decl_explicit_p (BUILT_IN_BSWAP32)
|
|
|
|
&& optab_handler (bswap_optab, SImode) != CODE_FOR_nothing)
|
|
|
|
break;
|
|
|
|
return false;
|
|
|
|
case 64:
|
|
|
|
if (builtin_decl_explicit_p (BUILT_IN_BSWAP64)
|
|
|
|
&& optab_handler (bswap_optab, DImode) != CODE_FOR_nothing)
|
|
|
|
break;
|
|
|
|
return false;
|
|
|
|
default:
|
|
|
|
gcc_unreachable ();
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!allow_unaligned && n.base_addr)
|
|
|
|
{
|
|
|
|
unsigned int align = get_object_alignment (n.src);
|
|
|
|
if (align < try_size)
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* If each load has vuse of the corresponding store, need to verify
|
|
|
|
the loads can be sunk right before the last store. */
|
|
|
|
if (vuse_store == 1)
|
|
|
|
{
|
|
|
|
auto_vec<tree, 64> refs;
|
|
|
|
for (unsigned int i = first; i <= last; ++i)
|
|
|
|
gather_bswap_load_refs (&refs,
|
|
|
|
gimple_assign_rhs1 (m_store_info[i]->stmt));
|
|
|
|
|
2021-06-12 05:49:22 +02:00
|
|
|
for (tree ref : refs)
|
2017-11-20 11:10:23 +01:00
|
|
|
if (stmts_may_clobber_ref_p (first_stmt, last_stmt, ref))
|
|
|
|
return false;
|
|
|
|
n.vuse = NULL_TREE;
|
|
|
|
}
|
|
|
|
|
|
|
|
infof->n = n;
|
|
|
|
infof->ins_stmt = ins_stmt;
|
|
|
|
for (unsigned int i = first; i <= last; ++i)
|
|
|
|
{
|
|
|
|
m_store_info[i]->rhs_code = n.n == cmpxchg ? LROTATE_EXPR : NOP_EXPR;
|
|
|
|
m_store_info[i]->ops[0].base_addr = NULL_TREE;
|
|
|
|
m_store_info[i]->ops[1].base_addr = NULL_TREE;
|
|
|
|
if (i != first)
|
|
|
|
merged_store->merge_into (m_store_info[i]);
|
|
|
|
}
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Go through the candidate stores recorded in m_store_info and merge them
|
|
|
|
into merged_store_group objects recorded into m_merged_store_groups
|
|
|
|
representing the widened stores. Return true if coalescing was successful
|
|
|
|
and the number of widened stores is fewer than the original number
|
|
|
|
of stores. */
|
|
|
|
|
|
|
|
bool
|
|
|
|
imm_store_chain_info::coalesce_immediate_stores ()
|
|
|
|
{
|
|
|
|
/* Anything less can't be processed. */
|
|
|
|
if (m_store_info.length () < 2)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
2018-06-01 23:56:17 +02:00
|
|
|
fprintf (dump_file, "Attempting to coalesce %u stores in chain\n",
|
2016-10-28 16:18:50 +02:00
|
|
|
m_store_info.length ());
|
|
|
|
|
|
|
|
store_immediate_info *info;
|
2017-11-20 11:10:23 +01:00
|
|
|
unsigned int i, ignore = 0;
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
unsigned int first_earlier = 0;
|
|
|
|
unsigned int end_earlier = 0;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
/* Order the stores by the bitposition they write to. */
|
|
|
|
m_store_info.qsort (sort_by_bitpos);
|
|
|
|
|
|
|
|
info = m_store_info[0];
|
|
|
|
merged_store_group *merged_store = new merged_store_group (info);
|
2018-06-01 23:56:17 +02:00
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fputs ("New store group\n", dump_file);
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
FOR_EACH_VEC_ELT (m_store_info, i, info)
|
|
|
|
{
|
2019-05-06 23:50:14 +02:00
|
|
|
unsigned HOST_WIDE_INT new_bitregion_start, new_bitregion_end;
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (i <= ignore)
|
2018-06-01 23:56:17 +02:00
|
|
|
goto done;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
while (first_earlier < end_earlier
|
|
|
|
&& (m_store_info[first_earlier]->bitpos
|
|
|
|
+ m_store_info[first_earlier]->bitsize
|
|
|
|
<= merged_store->start))
|
|
|
|
first_earlier++;
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
/* First try to handle group of stores like:
|
|
|
|
p[0] = data >> 24;
|
|
|
|
p[1] = data >> 16;
|
|
|
|
p[2] = data >> 8;
|
|
|
|
p[3] = data;
|
|
|
|
using the bswap framework. */
|
|
|
|
if (info->bitpos == merged_store->start + merged_store->width
|
|
|
|
&& merged_store->stores.length () == 1
|
|
|
|
&& merged_store->stores[0]->ins_stmt != NULL
|
2020-04-23 22:25:04 +02:00
|
|
|
&& info->lp_nr == merged_store->lp_nr
|
2017-11-20 11:10:23 +01:00
|
|
|
&& info->ins_stmt != NULL)
|
|
|
|
{
|
|
|
|
unsigned int try_size;
|
|
|
|
for (try_size = 64; try_size >= 16; try_size >>= 1)
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
if (try_coalesce_bswap (merged_store, i - 1, try_size,
|
|
|
|
first_earlier))
|
2017-11-20 11:10:23 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
if (try_size >= 16)
|
|
|
|
{
|
|
|
|
ignore = i + merged_store->stores.length () - 1;
|
|
|
|
m_merged_store_groups.safe_push (merged_store);
|
|
|
|
if (ignore < m_store_info.length ())
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
{
|
|
|
|
merged_store = new merged_store_group (m_store_info[ignore]);
|
|
|
|
end_earlier = ignore;
|
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
else
|
|
|
|
merged_store = NULL;
|
2018-06-01 23:56:17 +02:00
|
|
|
goto done;
|
2017-11-20 11:10:23 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-05-06 23:50:14 +02:00
|
|
|
new_bitregion_start
|
|
|
|
= MIN (merged_store->bitregion_start, info->bitregion_start);
|
|
|
|
new_bitregion_end
|
|
|
|
= MAX (merged_store->bitregion_end, info->bitregion_end);
|
|
|
|
|
|
|
|
if (info->order >= merged_store->first_nonmergeable_order
|
|
|
|
|| (((new_bitregion_end - new_bitregion_start + 1) / BITS_PER_UNIT)
|
2019-11-12 11:08:40 +01:00
|
|
|
> (unsigned) param_store_merging_max_size))
|
2018-11-05 11:28:19 +01:00
|
|
|
;
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* |---store 1---|
|
|
|
|
|---store 2---|
|
2017-11-20 11:10:23 +01:00
|
|
|
Overlapping stores. */
|
2018-11-05 11:28:19 +01:00
|
|
|
else if (IN_RANGE (info->bitpos, merged_store->start,
|
store-merging: Fix coalesce_immediate_stores [PR93820]
The following testcase is miscompiled in 8+.
The problem is that check_no_overlap has a special case for INTEGER_CST
marked stores (i.e. stores of constants), if both all currenly merged stores
and the one under consideration for merging with them are marked that way,
it anticipates that other INTEGER_CST marked stores that overlap with those
and precede those (have smaller info->order) could be merged with those and
doesn't punt for them.
In PR86844 and PR87859 fixes I've then added quite large code that is
performed after check_no_overlap and tries to find out if we need and can
merge further INTEGER_CST marked stores, or need to punt.
Unfortunately, that code is there only in the overlapping case code and
the testcase below shows that we really need it even in the adjacent store
case. After sort_by_bitpos we have:
bitpos width order rhs_code
96 32 3 INTEGER_CST
128 32 1 INTEGER_CST
128 128 2 INTEGER_CST
192 32 0 MEM_REF
Because of the missing PR86844/PR87859-ish code in the adjacent store
case, we merge the adjacent (memory wise) stores 96/32/3 and 128/32/1,
and then we consider the 128-bit store which is in program-order in between
them, but in this case we punt, because the merging would extend the
merged store region from bitpos 96 and 64-bits to bitpos 96 and 160-bits
and that has an overlap with an incompatible store (the MEM_REF one).
The problem is that we can't really punt this way, because the 128-bit
store is in between those two we've merged already, so either we manage
to merge even that one together with the others, or would need to avoid
already merging the 96/32/3 and 128/32/1 stores together.
Now, rather than copying around the PR86844/PR87859 code to the other spot,
we can actually just use the overlapping code, merge_overlapping is really
a superset of merge_into, so that is what the patch does. If doing
adjacent store merge for rhs_code other than INTEGER_CST, I believe the
current code is already fine, check_no_overlap in that case doesn't make
the exception and will punt if there is some earlier (smaller order)
non-mergeable overlapping store. There is just one case that could be
problematic, if the merged_store has BIT_INSERT_EXPRs in them and the
new store is a constant store (INTEGER_CST rhs_code), then check_no_overlap
would do the exception and still would allow the special case. But we
really shouldn't have the special case in that case, so this patch also
changes check_no_overlap to just have a bool whether we should have the
special case or not.
Note, as I said in the PR, for GCC11 we could consider performing some kind
of cheap DSE during the store merging (perhaps guarded with flag_tree_dse).
And another thing to consider is only consider as problematic non-mergeable
stores that not only have order smaller than last_order as currently, but
also have order larger than first_order, as in this testcase if we actually
ignored (not merged with anything at all) the 192/32/0 store, because it is
not in between the other stores we'd merge, it would be fine to merge the
other 3 stores, though of course the testcase can be easily adjusted by
putting the 192/32 store after the 128/32 store and then this patch would be
still needed. Though, I think I'd need more time thinking this over.
2020-02-26 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93820
* gimple-ssa-store-merging.c (check_no_overlap): Change RHS_CODE
argument to ALL_INTEGER_CST_P boolean.
(imm_store_chain_info::try_coalesce_bswap): Adjust caller.
(imm_store_chain_info::coalesce_immediate_stores): Likewise. Handle
adjacent INTEGER_CST store into merged_store->only_constants like
overlapping one.
* gcc.dg/pr93820.c: New test.
2020-02-26 09:33:48 +01:00
|
|
|
merged_store->start + merged_store->width - 1)
|
|
|
|
/* |---store 1---||---store 2---|
|
|
|
|
Handle also the consecutive INTEGER_CST stores case here,
|
|
|
|
as we have here the code to deal with overlaps. */
|
|
|
|
|| (info->bitregion_start <= merged_store->bitregion_end
|
|
|
|
&& info->rhs_code == INTEGER_CST
|
|
|
|
&& merged_store->only_constants
|
|
|
|
&& merged_store->can_be_merged_into (info)))
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Only allow overlapping stores of constants. */
|
2019-10-02 15:35:40 +02:00
|
|
|
if (info->rhs_code == INTEGER_CST
|
|
|
|
&& merged_store->only_constants
|
|
|
|
&& info->lp_nr == merged_store->lp_nr)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
unsigned int first_order
|
|
|
|
= MIN (merged_store->first_order, info->order);
|
2018-09-12 11:25:07 +02:00
|
|
|
unsigned int last_order
|
|
|
|
= MAX (merged_store->last_order, info->order);
|
|
|
|
unsigned HOST_WIDE_INT end
|
|
|
|
= MAX (merged_store->start + merged_store->width,
|
|
|
|
info->bitpos + info->bitsize);
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
if (check_no_overlap (m_store_info, i, true, first_order,
|
|
|
|
last_order, merged_store->start, end,
|
|
|
|
first_earlier, end_earlier))
|
2018-09-12 11:25:07 +02:00
|
|
|
{
|
|
|
|
/* check_no_overlap call above made sure there are no
|
|
|
|
overlapping stores with non-INTEGER_CST rhs_code
|
|
|
|
in between the first and last of the stores we've
|
|
|
|
just merged. If there are any INTEGER_CST rhs_code
|
|
|
|
stores in between, we need to merge_overlapping them
|
|
|
|
even if in the sort_by_bitpos order there are other
|
|
|
|
overlapping stores in between. Keep those stores as is.
|
|
|
|
Example:
|
|
|
|
MEM[(int *)p_28] = 0;
|
|
|
|
MEM[(char *)p_28 + 3B] = 1;
|
|
|
|
MEM[(char *)p_28 + 1B] = 2;
|
|
|
|
MEM[(char *)p_28 + 2B] = MEM[(char *)p_28 + 6B];
|
|
|
|
We can't merge the zero store with the store of two and
|
|
|
|
not merge anything else, because the store of one is
|
|
|
|
in the original order in between those two, but in
|
|
|
|
store_by_bitpos order it comes after the last store that
|
|
|
|
we can't merge with them. We can merge the first 3 stores
|
|
|
|
and keep the last store as is though. */
|
2018-11-05 11:28:19 +01:00
|
|
|
unsigned int len = m_store_info.length ();
|
|
|
|
unsigned int try_order = last_order;
|
|
|
|
unsigned int first_nonmergeable_order;
|
|
|
|
unsigned int k;
|
|
|
|
bool last_iter = false;
|
|
|
|
int attempts = 0;
|
|
|
|
do
|
2018-09-12 11:25:07 +02:00
|
|
|
{
|
2018-11-05 11:28:19 +01:00
|
|
|
unsigned int max_order = 0;
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
unsigned int min_order = first_order;
|
2018-11-05 11:28:19 +01:00
|
|
|
unsigned first_nonmergeable_int_order = ~0U;
|
|
|
|
unsigned HOST_WIDE_INT this_end = end;
|
|
|
|
k = i;
|
|
|
|
first_nonmergeable_order = ~0U;
|
|
|
|
for (unsigned int j = i + 1; j < len; ++j)
|
2018-09-12 11:25:07 +02:00
|
|
|
{
|
2018-11-05 11:28:19 +01:00
|
|
|
store_immediate_info *info2 = m_store_info[j];
|
|
|
|
if (info2->bitpos >= this_end)
|
|
|
|
break;
|
|
|
|
if (info2->order < try_order)
|
2018-09-12 11:25:07 +02:00
|
|
|
{
|
2020-03-20 09:33:38 +01:00
|
|
|
if (info2->rhs_code != INTEGER_CST
|
|
|
|
|| info2->lp_nr != merged_store->lp_nr)
|
2018-11-05 11:28:19 +01:00
|
|
|
{
|
|
|
|
/* Normally check_no_overlap makes sure this
|
|
|
|
doesn't happen, but if end grows below,
|
|
|
|
then we need to process more stores than
|
|
|
|
check_no_overlap verified. Example:
|
|
|
|
MEM[(int *)p_5] = 0;
|
|
|
|
MEM[(short *)p_5 + 3B] = 1;
|
|
|
|
MEM[(char *)p_5 + 4B] = _9;
|
|
|
|
MEM[(char *)p_5 + 2B] = 2; */
|
|
|
|
k = 0;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
k = j;
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
min_order = MIN (min_order, info2->order);
|
2018-11-05 11:28:19 +01:00
|
|
|
this_end = MAX (this_end,
|
|
|
|
info2->bitpos + info2->bitsize);
|
2018-09-12 11:25:07 +02:00
|
|
|
}
|
2018-11-05 11:28:19 +01:00
|
|
|
else if (info2->rhs_code == INTEGER_CST
|
2020-03-20 09:33:38 +01:00
|
|
|
&& info2->lp_nr == merged_store->lp_nr
|
2018-11-05 11:28:19 +01:00
|
|
|
&& !last_iter)
|
|
|
|
{
|
|
|
|
max_order = MAX (max_order, info2->order + 1);
|
|
|
|
first_nonmergeable_int_order
|
|
|
|
= MIN (first_nonmergeable_int_order,
|
|
|
|
info2->order);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
first_nonmergeable_order
|
|
|
|
= MIN (first_nonmergeable_order, info2->order);
|
2018-09-12 11:25:07 +02:00
|
|
|
}
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
if (k > i
|
|
|
|
&& !check_no_overlap (m_store_info, len - 1, true,
|
|
|
|
min_order, try_order,
|
|
|
|
merged_store->start, this_end,
|
|
|
|
first_earlier, end_earlier))
|
|
|
|
k = 0;
|
2018-11-05 11:28:19 +01:00
|
|
|
if (k == 0)
|
|
|
|
{
|
|
|
|
if (last_order == try_order)
|
|
|
|
break;
|
|
|
|
/* If this failed, but only because we grew
|
|
|
|
try_order, retry with the last working one,
|
|
|
|
so that we merge at least something. */
|
|
|
|
try_order = last_order;
|
|
|
|
last_iter = true;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
last_order = try_order;
|
|
|
|
/* Retry with a larger try_order to see if we could
|
|
|
|
merge some further INTEGER_CST stores. */
|
|
|
|
if (max_order
|
|
|
|
&& (first_nonmergeable_int_order
|
|
|
|
< first_nonmergeable_order))
|
|
|
|
{
|
|
|
|
try_order = MIN (max_order,
|
|
|
|
first_nonmergeable_order);
|
|
|
|
try_order
|
|
|
|
= MIN (try_order,
|
|
|
|
merged_store->first_nonmergeable_order);
|
|
|
|
if (try_order > last_order && ++attempts < 16)
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
first_nonmergeable_order
|
|
|
|
= MIN (first_nonmergeable_order,
|
|
|
|
first_nonmergeable_int_order);
|
|
|
|
end = this_end;
|
|
|
|
break;
|
2018-09-12 11:25:07 +02:00
|
|
|
}
|
2018-11-05 11:28:19 +01:00
|
|
|
while (1);
|
2018-09-12 11:25:07 +02:00
|
|
|
|
|
|
|
if (k != 0)
|
|
|
|
{
|
|
|
|
merged_store->merge_overlapping (info);
|
|
|
|
|
2018-11-05 11:28:19 +01:00
|
|
|
merged_store->first_nonmergeable_order
|
|
|
|
= MIN (merged_store->first_nonmergeable_order,
|
|
|
|
first_nonmergeable_order);
|
|
|
|
|
2018-09-12 11:25:07 +02:00
|
|
|
for (unsigned int j = i + 1; j <= k; j++)
|
|
|
|
{
|
|
|
|
store_immediate_info *info2 = m_store_info[j];
|
|
|
|
gcc_assert (info2->bitpos < end);
|
|
|
|
if (info2->order < last_order)
|
|
|
|
{
|
|
|
|
gcc_assert (info2->rhs_code == INTEGER_CST);
|
2018-11-05 11:28:19 +01:00
|
|
|
if (info != info2)
|
|
|
|
merged_store->merge_overlapping (info2);
|
2018-09-12 11:25:07 +02:00
|
|
|
}
|
|
|
|
/* Other stores are kept and not merged in any
|
|
|
|
way. */
|
|
|
|
}
|
|
|
|
ignore = k;
|
|
|
|
goto done;
|
|
|
|
}
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* |---store 1---||---store 2---|
|
|
|
|
This store is consecutive to the previous one.
|
|
|
|
Merge it into the current store group. There can be gaps in between
|
|
|
|
the stores, but there can't be gaps in between bitregions. */
|
2018-06-01 23:56:17 +02:00
|
|
|
else if (info->bitregion_start <= merged_store->bitregion_end
|
2018-06-05 00:12:51 +02:00
|
|
|
&& merged_store->can_be_merged_into (info))
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
store_immediate_info *infof = merged_store->stores[0];
|
|
|
|
|
|
|
|
/* All the rhs_code ops that take 2 operands are commutative,
|
|
|
|
swap the operands if it could make the operands compatible. */
|
|
|
|
if (infof->ops[0].base_addr
|
|
|
|
&& infof->ops[1].base_addr
|
|
|
|
&& info->ops[0].base_addr
|
|
|
|
&& info->ops[1].base_addr
|
2017-12-20 13:56:50 +01:00
|
|
|
&& known_eq (info->ops[1].bitpos - infof->ops[0].bitpos,
|
|
|
|
info->bitpos - infof->bitpos)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
&& operand_equal_p (info->ops[1].base_addr,
|
|
|
|
infof->ops[0].base_addr, 0))
|
2017-11-10 15:43:13 +01:00
|
|
|
{
|
|
|
|
std::swap (info->ops[0], info->ops[1]);
|
|
|
|
info->ops_swapped_p = true;
|
|
|
|
}
|
store-merging: Fix coalesce_immediate_stores [PR93820]
The following testcase is miscompiled in 8+.
The problem is that check_no_overlap has a special case for INTEGER_CST
marked stores (i.e. stores of constants), if both all currenly merged stores
and the one under consideration for merging with them are marked that way,
it anticipates that other INTEGER_CST marked stores that overlap with those
and precede those (have smaller info->order) could be merged with those and
doesn't punt for them.
In PR86844 and PR87859 fixes I've then added quite large code that is
performed after check_no_overlap and tries to find out if we need and can
merge further INTEGER_CST marked stores, or need to punt.
Unfortunately, that code is there only in the overlapping case code and
the testcase below shows that we really need it even in the adjacent store
case. After sort_by_bitpos we have:
bitpos width order rhs_code
96 32 3 INTEGER_CST
128 32 1 INTEGER_CST
128 128 2 INTEGER_CST
192 32 0 MEM_REF
Because of the missing PR86844/PR87859-ish code in the adjacent store
case, we merge the adjacent (memory wise) stores 96/32/3 and 128/32/1,
and then we consider the 128-bit store which is in program-order in between
them, but in this case we punt, because the merging would extend the
merged store region from bitpos 96 and 64-bits to bitpos 96 and 160-bits
and that has an overlap with an incompatible store (the MEM_REF one).
The problem is that we can't really punt this way, because the 128-bit
store is in between those two we've merged already, so either we manage
to merge even that one together with the others, or would need to avoid
already merging the 96/32/3 and 128/32/1 stores together.
Now, rather than copying around the PR86844/PR87859 code to the other spot,
we can actually just use the overlapping code, merge_overlapping is really
a superset of merge_into, so that is what the patch does. If doing
adjacent store merge for rhs_code other than INTEGER_CST, I believe the
current code is already fine, check_no_overlap in that case doesn't make
the exception and will punt if there is some earlier (smaller order)
non-mergeable overlapping store. There is just one case that could be
problematic, if the merged_store has BIT_INSERT_EXPRs in them and the
new store is a constant store (INTEGER_CST rhs_code), then check_no_overlap
would do the exception and still would allow the special case. But we
really shouldn't have the special case in that case, so this patch also
changes check_no_overlap to just have a bool whether we should have the
special case or not.
Note, as I said in the PR, for GCC11 we could consider performing some kind
of cheap DSE during the store merging (perhaps guarded with flag_tree_dse).
And another thing to consider is only consider as problematic non-mergeable
stores that not only have order smaller than last_order as currently, but
also have order larger than first_order, as in this testcase if we actually
ignored (not merged with anything at all) the 192/32/0 store, because it is
not in between the other stores we'd merge, it would be fine to merge the
other 3 stores, though of course the testcase can be easily adjusted by
putting the 192/32 store after the 128/32 store and then this patch would be
still needed. Though, I think I'd need more time thinking this over.
2020-02-26 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93820
* gimple-ssa-store-merging.c (check_no_overlap): Change RHS_CODE
argument to ALL_INTEGER_CST_P boolean.
(imm_store_chain_info::try_coalesce_bswap): Adjust caller.
(imm_store_chain_info::coalesce_immediate_stores): Likewise. Handle
adjacent INTEGER_CST store into merged_store->only_constants like
overlapping one.
* gcc.dg/pr93820.c: New test.
2020-02-26 09:33:48 +01:00
|
|
|
if (check_no_overlap (m_store_info, i, false,
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
MIN (merged_store->first_order, info->order),
|
2018-07-12 09:39:33 +02:00
|
|
|
MAX (merged_store->last_order, info->order),
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
merged_store->start,
|
2018-07-12 09:39:33 +02:00
|
|
|
MAX (merged_store->start + merged_store->width,
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
info->bitpos + info->bitsize),
|
|
|
|
first_earlier, end_earlier))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
2018-06-05 00:12:51 +02:00
|
|
|
/* Turn MEM_REF into BIT_INSERT_EXPR for bit-field stores. */
|
|
|
|
if (info->rhs_code == MEM_REF && infof->rhs_code != MEM_REF)
|
|
|
|
{
|
|
|
|
info->rhs_code = BIT_INSERT_EXPR;
|
|
|
|
info->ops[0].val = gimple_assign_rhs1 (info->stmt);
|
|
|
|
info->ops[0].base_addr = NULL_TREE;
|
|
|
|
}
|
|
|
|
else if (infof->rhs_code == MEM_REF && info->rhs_code != MEM_REF)
|
|
|
|
{
|
2021-06-12 05:49:22 +02:00
|
|
|
for (store_immediate_info *infoj : merged_store->stores)
|
2018-06-05 00:12:51 +02:00
|
|
|
{
|
|
|
|
infoj->rhs_code = BIT_INSERT_EXPR;
|
|
|
|
infoj->ops[0].val = gimple_assign_rhs1 (infoj->stmt);
|
|
|
|
infoj->ops[0].base_addr = NULL_TREE;
|
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
merged_store->bit_insertion = true;
|
2018-06-05 00:12:51 +02:00
|
|
|
}
|
|
|
|
if ((infof->ops[0].base_addr
|
|
|
|
? compatible_load_p (merged_store, info, base_addr, 0)
|
|
|
|
: !info->ops[0].base_addr)
|
|
|
|
&& (infof->ops[1].base_addr
|
|
|
|
? compatible_load_p (merged_store, info, base_addr, 1)
|
|
|
|
: !info->ops[1].base_addr))
|
|
|
|
{
|
|
|
|
merged_store->merge_into (info);
|
|
|
|
goto done;
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* |---store 1---| <gap> |---store 2---|.
|
|
|
|
Gap between stores or the rhs not compatible. Start a new group. */
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Try to apply all the stores recorded for the group to determine
|
|
|
|
the bitpattern they write and discard it if that fails.
|
|
|
|
This will also reject single-store groups. */
|
2018-06-01 23:56:17 +02:00
|
|
|
if (merged_store->apply_stores ())
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
m_merged_store_groups.safe_push (merged_store);
|
2018-06-01 23:56:17 +02:00
|
|
|
else
|
|
|
|
delete merged_store;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
merged_store = new merged_store_group (info);
|
store-merging: Consider also overlapping stores earlier in the by bitpos sorting [PR97053]
As the testcases show, if we have something like:
MEM <char[12]> [&b + 8B] = {};
MEM[(short *) &b] = 5;
_5 = *x_4(D);
MEM <long long unsigned int> [&b + 2B] = _5;
MEM[(char *)&b + 16B] = 88;
MEM[(int *)&b + 20B] = 1;
then in sort_by_bitpos the stores are almost like in the given order,
except the first store is after the = _5; store.
We can't coalesce the = 5; store with = _5;, because the latter is MEM_REF,
while the former INTEGER_CST, and we can't coalesce the = _5 store with
the = {} store because the former is MEM_REF, the latter INTEGER_CST.
But we happily coalesce the remaining 3 stores, which is wrong, because the
= _5; store overlaps those and is in between them in the program order.
We already have code to deal with similar cases in check_no_overlap, but we
deal only with the following stores in sort_by_bitpos order, not the earlier
ones.
The following patch checks also the earlier ones. In coalesce_immediate_stores
it computes the first one that needs to be checked (all the ones whose
bitpos + bitsize is smaller or equal to merged_store->start don't need to be
checked and don't need to be checked even for any following attempts because
of the sort_by_bitpos sorting) and the end of that (that is the first store
in the merged_store).
2020-09-16 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/97053
* gimple-ssa-store-merging.c (check_no_overlap): Add FIRST_ORDER,
START, FIRST_EARLIER and LAST_EARLIER arguments. Return false if
any stores between FIRST_EARLIER inclusive and LAST_EARLIER exclusive
has order in between FIRST_ORDER and LAST_ORDER and overlaps the to
be merged store.
(imm_store_chain_info::try_coalesce_bswap): Add FIRST_EARLIER argument.
Adjust check_no_overlap caller.
(imm_store_chain_info::coalesce_immediate_stores): Add first_earlier
and last_earlier variables, adjust them during iterations. Adjust
check_no_overlap callers, call check_no_overlap even when extending
overlapping stores by extra INTEGER_CST stores.
* gcc.dg/store_merging_31.c: New test.
* gcc.dg/store_merging_32.c: New test.
2020-09-16 09:42:33 +02:00
|
|
|
end_earlier = i;
|
2018-06-01 23:56:17 +02:00
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fputs ("New store group\n", dump_file);
|
|
|
|
|
|
|
|
done:
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
{
|
|
|
|
fprintf (dump_file, "Store %u:\nbitsize:" HOST_WIDE_INT_PRINT_DEC
|
|
|
|
" bitpos:" HOST_WIDE_INT_PRINT_DEC " val:",
|
|
|
|
i, info->bitsize, info->bitpos);
|
|
|
|
print_generic_expr (dump_file, gimple_assign_rhs1 (info->stmt));
|
|
|
|
fputc ('\n', dump_file);
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* Record or discard the last store group. */
|
2017-11-20 11:10:23 +01:00
|
|
|
if (merged_store)
|
|
|
|
{
|
2018-06-01 23:56:17 +02:00
|
|
|
if (merged_store->apply_stores ())
|
2017-11-20 11:10:23 +01:00
|
|
|
m_merged_store_groups.safe_push (merged_store);
|
2018-06-01 23:56:17 +02:00
|
|
|
else
|
|
|
|
delete merged_store;
|
2017-11-20 11:10:23 +01:00
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
gcc_assert (m_merged_store_groups.length () <= m_store_info.length ());
|
2018-06-01 23:56:17 +02:00
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
bool success
|
|
|
|
= !m_merged_store_groups.is_empty ()
|
|
|
|
&& m_merged_store_groups.length () < m_store_info.length ();
|
|
|
|
|
|
|
|
if (success && dump_file)
|
2018-06-01 23:56:17 +02:00
|
|
|
fprintf (dump_file, "Coalescing successful!\nMerged into %u stores\n",
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
m_merged_store_groups.length ());
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
return success;
|
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Return the type to use for the merged stores or loads described by STMTS.
|
|
|
|
This is needed to get the alias sets right. If IS_LOAD, look for rhs,
|
|
|
|
otherwise lhs. Additionally set *CLIQUEP and *BASEP to MR_DEPENDENCE_*
|
|
|
|
of the MEM_REFs if any. */
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
static tree
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
get_alias_type_for_stmts (vec<gimple *> &stmts, bool is_load,
|
|
|
|
unsigned short *cliquep, unsigned short *basep)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
gimple *stmt;
|
|
|
|
unsigned int i;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
tree type = NULL_TREE;
|
|
|
|
tree ret = NULL_TREE;
|
|
|
|
*cliquep = 0;
|
|
|
|
*basep = 0;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
FOR_EACH_VEC_ELT (stmts, i, stmt)
|
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
tree ref = is_load ? gimple_assign_rhs1 (stmt)
|
|
|
|
: gimple_assign_lhs (stmt);
|
|
|
|
tree type1 = reference_alias_ptr_type (ref);
|
|
|
|
tree base = get_base_address (ref);
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (i == 0)
|
|
|
|
{
|
|
|
|
if (TREE_CODE (base) == MEM_REF)
|
|
|
|
{
|
|
|
|
*cliquep = MR_DEPENDENCE_CLIQUE (base);
|
|
|
|
*basep = MR_DEPENDENCE_BASE (base);
|
|
|
|
}
|
|
|
|
ret = type = type1;
|
|
|
|
continue;
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
if (!alias_ptr_types_compatible_p (type, type1))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
ret = ptr_type_node;
|
|
|
|
if (TREE_CODE (base) != MEM_REF
|
|
|
|
|| *cliquep != MR_DEPENDENCE_CLIQUE (base)
|
|
|
|
|| *basep != MR_DEPENDENCE_BASE (base))
|
|
|
|
{
|
|
|
|
*cliquep = 0;
|
|
|
|
*basep = 0;
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return ret;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Return the location_t information we can find among the statements
|
|
|
|
in STMTS. */
|
|
|
|
|
|
|
|
static location_t
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
get_location_for_stmts (vec<gimple *> &stmts)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2021-06-12 05:49:22 +02:00
|
|
|
for (gimple *stmt : stmts)
|
2016-10-28 16:18:50 +02:00
|
|
|
if (gimple_has_location (stmt))
|
|
|
|
return gimple_location (stmt);
|
|
|
|
|
|
|
|
return UNKNOWN_LOCATION;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Used to decribe a store resulting from splitting a wide store in smaller
|
|
|
|
regularly-sized stores in split_group. */
|
|
|
|
|
2019-07-09 18:36:00 +02:00
|
|
|
class split_store
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2019-07-09 18:36:00 +02:00
|
|
|
public:
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT bytepos;
|
|
|
|
unsigned HOST_WIDE_INT size;
|
|
|
|
unsigned HOST_WIDE_INT align;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
auto_vec<store_immediate_info *> orig_stores;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* True if there is a single orig stmt covering the whole split store. */
|
|
|
|
bool orig;
|
2016-10-28 16:18:50 +02:00
|
|
|
split_store (unsigned HOST_WIDE_INT, unsigned HOST_WIDE_INT,
|
|
|
|
unsigned HOST_WIDE_INT);
|
|
|
|
};
|
|
|
|
|
|
|
|
/* Simple constructor. */
|
|
|
|
|
|
|
|
split_store::split_store (unsigned HOST_WIDE_INT bp,
|
|
|
|
unsigned HOST_WIDE_INT sz,
|
|
|
|
unsigned HOST_WIDE_INT al)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
: bytepos (bp), size (sz), align (al), orig (false)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
orig_stores.create (0);
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Record all stores in GROUP that write to the region starting at BITPOS and
|
|
|
|
is of size BITSIZE. Record infos for such statements in STORES if
|
|
|
|
non-NULL. The stores in GROUP must be sorted by bitposition. Return INFO
|
2019-11-08 11:53:50 +01:00
|
|
|
if there is exactly one original store in the range (in that case ignore
|
|
|
|
clobber stmts, unless there are only clobber stmts). */
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
static store_immediate_info *
|
2019-07-09 20:32:49 +02:00
|
|
|
find_constituent_stores (class merged_store_group *group,
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
vec<store_immediate_info *> *stores,
|
|
|
|
unsigned int *first,
|
|
|
|
unsigned HOST_WIDE_INT bitpos,
|
|
|
|
unsigned HOST_WIDE_INT bitsize)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
store_immediate_info *info, *ret = NULL;
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned int i;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
bool second = false;
|
|
|
|
bool update_first = true;
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT end = bitpos + bitsize;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
for (i = *first; group->stores.iterate (i, &info); ++i)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
unsigned HOST_WIDE_INT stmt_start = info->bitpos;
|
|
|
|
unsigned HOST_WIDE_INT stmt_end = stmt_start + info->bitsize;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (stmt_end <= bitpos)
|
|
|
|
{
|
|
|
|
/* BITPOS passed to this function never decreases from within the
|
|
|
|
same split_group call, so optimize and don't scan info records
|
|
|
|
which are known to end before or at BITPOS next time.
|
|
|
|
Only do it if all stores before this one also pass this. */
|
|
|
|
if (update_first)
|
|
|
|
*first = i + 1;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
update_first = false;
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* The stores in GROUP are ordered by bitposition so if we're past
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
the region for this group return early. */
|
|
|
|
if (stmt_start >= end)
|
|
|
|
return ret;
|
|
|
|
|
2019-11-08 11:53:50 +01:00
|
|
|
if (gimple_clobber_p (info->stmt))
|
|
|
|
{
|
|
|
|
if (stores)
|
|
|
|
stores->safe_push (info);
|
|
|
|
if (ret == NULL)
|
|
|
|
ret = info;
|
|
|
|
continue;
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (stores)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
stores->safe_push (info);
|
2019-11-08 11:53:50 +01:00
|
|
|
if (ret && !gimple_clobber_p (ret->stmt))
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
|
|
|
ret = NULL;
|
|
|
|
second = true;
|
|
|
|
}
|
|
|
|
}
|
2019-11-08 11:53:50 +01:00
|
|
|
else if (ret && !gimple_clobber_p (ret->stmt))
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
return NULL;
|
|
|
|
if (!second)
|
|
|
|
ret = info;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
return ret;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
2017-11-09 14:08:41 +01:00
|
|
|
/* Return how many SSA_NAMEs used to compute value to store in the INFO
|
|
|
|
store have multiple uses. If any SSA_NAME has multiple uses, also
|
|
|
|
count statements needed to compute it. */
|
|
|
|
|
|
|
|
static unsigned
|
|
|
|
count_multiple_uses (store_immediate_info *info)
|
|
|
|
{
|
|
|
|
gimple *stmt = info->stmt;
|
|
|
|
unsigned ret = 0;
|
|
|
|
switch (info->rhs_code)
|
|
|
|
{
|
|
|
|
case INTEGER_CST:
|
2020-07-03 18:10:25 +02:00
|
|
|
case STRING_CST:
|
2017-11-09 14:08:41 +01:00
|
|
|
return 0;
|
|
|
|
case BIT_AND_EXPR:
|
|
|
|
case BIT_IOR_EXPR:
|
|
|
|
case BIT_XOR_EXPR:
|
2017-11-09 20:24:59 +01:00
|
|
|
if (info->bit_not_p)
|
|
|
|
{
|
|
|
|
if (!has_single_use (gimple_assign_rhs1 (stmt)))
|
|
|
|
ret = 1; /* Fall through below to return
|
|
|
|
the BIT_NOT_EXPR stmt and then
|
|
|
|
BIT_{AND,IOR,XOR}_EXPR and anything it
|
|
|
|
uses. */
|
|
|
|
else
|
|
|
|
/* stmt is after this the BIT_NOT_EXPR. */
|
|
|
|
stmt = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (stmt));
|
|
|
|
}
|
2017-11-09 14:08:41 +01:00
|
|
|
if (!has_single_use (gimple_assign_rhs1 (stmt)))
|
|
|
|
{
|
|
|
|
ret += 1 + info->ops[0].bit_not_p;
|
|
|
|
if (info->ops[1].base_addr)
|
|
|
|
ret += 1 + info->ops[1].bit_not_p;
|
|
|
|
return ret + 1;
|
|
|
|
}
|
|
|
|
stmt = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (stmt));
|
|
|
|
/* stmt is now the BIT_*_EXPR. */
|
|
|
|
if (!has_single_use (gimple_assign_rhs1 (stmt)))
|
2017-11-10 15:43:13 +01:00
|
|
|
ret += 1 + info->ops[info->ops_swapped_p].bit_not_p;
|
|
|
|
else if (info->ops[info->ops_swapped_p].bit_not_p)
|
2017-11-09 14:08:41 +01:00
|
|
|
{
|
|
|
|
gimple *stmt2 = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (stmt));
|
|
|
|
if (!has_single_use (gimple_assign_rhs1 (stmt2)))
|
|
|
|
++ret;
|
|
|
|
}
|
|
|
|
if (info->ops[1].base_addr == NULL_TREE)
|
2017-11-10 15:43:13 +01:00
|
|
|
{
|
|
|
|
gcc_checking_assert (!info->ops_swapped_p);
|
|
|
|
return ret;
|
|
|
|
}
|
2017-11-09 14:08:41 +01:00
|
|
|
if (!has_single_use (gimple_assign_rhs2 (stmt)))
|
2017-11-10 15:43:13 +01:00
|
|
|
ret += 1 + info->ops[1 - info->ops_swapped_p].bit_not_p;
|
|
|
|
else if (info->ops[1 - info->ops_swapped_p].bit_not_p)
|
2017-11-09 14:08:41 +01:00
|
|
|
{
|
|
|
|
gimple *stmt2 = SSA_NAME_DEF_STMT (gimple_assign_rhs2 (stmt));
|
|
|
|
if (!has_single_use (gimple_assign_rhs1 (stmt2)))
|
|
|
|
++ret;
|
|
|
|
}
|
|
|
|
return ret;
|
|
|
|
case MEM_REF:
|
|
|
|
if (!has_single_use (gimple_assign_rhs1 (stmt)))
|
|
|
|
return 1 + info->ops[0].bit_not_p;
|
|
|
|
else if (info->ops[0].bit_not_p)
|
|
|
|
{
|
|
|
|
stmt = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (stmt));
|
|
|
|
if (!has_single_use (gimple_assign_rhs1 (stmt)))
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
return 0;
|
2018-06-01 23:56:17 +02:00
|
|
|
case BIT_INSERT_EXPR:
|
|
|
|
return has_single_use (gimple_assign_rhs1 (stmt)) ? 0 : 1;
|
2017-11-09 14:08:41 +01:00
|
|
|
default:
|
|
|
|
gcc_unreachable ();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Split a merged store described by GROUP by populating the SPLIT_STORES
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
vector (if non-NULL) with split_store structs describing the byte offset
|
|
|
|
(from the base), the bit size and alignment of each store as well as the
|
|
|
|
original statements involved in each such split group.
|
2016-10-28 16:18:50 +02:00
|
|
|
This is to separate the splitting strategy from the statement
|
|
|
|
building/emission/linking done in output_merged_store.
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
Return number of new stores.
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
If ALLOW_UNALIGNED_STORE is false, then all stores must be aligned.
|
|
|
|
If ALLOW_UNALIGNED_LOAD is false, then all loads must be aligned.
|
2019-05-06 23:50:14 +02:00
|
|
|
BZERO_FIRST may be true only when the first store covers the whole group
|
|
|
|
and clears it; if BZERO_FIRST is true, keep that first store in the set
|
|
|
|
unmodified and emit further stores for the overrides only.
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
If SPLIT_STORES is NULL, it is just a dry run to count number of
|
|
|
|
new stores. */
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
static unsigned int
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
split_group (merged_store_group *group, bool allow_unaligned_store,
|
2019-05-06 23:50:14 +02:00
|
|
|
bool allow_unaligned_load, bool bzero_first,
|
2019-07-09 20:32:49 +02:00
|
|
|
vec<split_store *> *split_stores,
|
2017-11-09 14:08:41 +01:00
|
|
|
unsigned *total_orig,
|
|
|
|
unsigned *total_new)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT pos = group->bitregion_start;
|
|
|
|
unsigned HOST_WIDE_INT size = group->bitregion_end - pos;
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT bytepos = pos / BITS_PER_UNIT;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned HOST_WIDE_INT group_align = group->align;
|
|
|
|
unsigned HOST_WIDE_INT align_base = group->align_base;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
unsigned HOST_WIDE_INT group_load_align = group_align;
|
2017-11-09 14:08:41 +01:00
|
|
|
bool any_orig = false;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
gcc_assert ((size % BITS_PER_UNIT == 0) && (pos % BITS_PER_UNIT == 0));
|
|
|
|
|
2020-07-03 18:10:25 +02:00
|
|
|
/* For bswap framework using sets of stores, all the checking has been done
|
|
|
|
earlier in try_coalesce_bswap and the result always needs to be emitted
|
|
|
|
as a single store. Likewise for string concatenation, */
|
2017-11-20 11:10:23 +01:00
|
|
|
if (group->stores[0]->rhs_code == LROTATE_EXPR
|
2020-07-03 18:10:25 +02:00
|
|
|
|| group->stores[0]->rhs_code == NOP_EXPR
|
|
|
|
|| group->string_concatenation)
|
2017-11-20 11:10:23 +01:00
|
|
|
{
|
2019-05-06 23:50:14 +02:00
|
|
|
gcc_assert (!bzero_first);
|
2017-11-20 11:10:23 +01:00
|
|
|
if (total_orig)
|
|
|
|
{
|
|
|
|
/* Avoid the old/new stmt count heuristics. It should be
|
|
|
|
always beneficial. */
|
|
|
|
total_new[0] = 1;
|
|
|
|
total_orig[0] = 2;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (split_stores)
|
|
|
|
{
|
|
|
|
unsigned HOST_WIDE_INT align_bitpos
|
|
|
|
= (group->start - align_base) & (group_align - 1);
|
|
|
|
unsigned HOST_WIDE_INT align = group_align;
|
|
|
|
if (align_bitpos)
|
|
|
|
align = least_bit_hwi (align_bitpos);
|
|
|
|
bytepos = group->start / BITS_PER_UNIT;
|
2019-07-09 20:32:49 +02:00
|
|
|
split_store *store
|
2017-11-20 11:10:23 +01:00
|
|
|
= new split_store (bytepos, group->width, align);
|
|
|
|
unsigned int first = 0;
|
|
|
|
find_constituent_stores (group, &store->orig_stores,
|
|
|
|
&first, group->start, group->width);
|
|
|
|
split_stores->safe_push (store);
|
|
|
|
}
|
|
|
|
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned int ret = 0, first = 0;
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT try_pos = bytepos;
|
|
|
|
|
2017-11-09 14:08:41 +01:00
|
|
|
if (total_orig)
|
|
|
|
{
|
|
|
|
unsigned int i;
|
|
|
|
store_immediate_info *info = group->stores[0];
|
|
|
|
|
|
|
|
total_new[0] = 0;
|
|
|
|
total_orig[0] = 1; /* The orig store. */
|
|
|
|
info = group->stores[0];
|
|
|
|
if (info->ops[0].base_addr)
|
2017-11-13 11:26:13 +01:00
|
|
|
total_orig[0]++;
|
2017-11-09 14:08:41 +01:00
|
|
|
if (info->ops[1].base_addr)
|
2017-11-13 11:26:13 +01:00
|
|
|
total_orig[0]++;
|
2017-11-09 14:08:41 +01:00
|
|
|
switch (info->rhs_code)
|
|
|
|
{
|
|
|
|
case BIT_AND_EXPR:
|
|
|
|
case BIT_IOR_EXPR:
|
|
|
|
case BIT_XOR_EXPR:
|
|
|
|
total_orig[0]++; /* The orig BIT_*_EXPR stmt. */
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
total_orig[0] *= group->stores.length ();
|
|
|
|
|
|
|
|
FOR_EACH_VEC_ELT (group->stores, i, info)
|
2017-11-13 11:26:13 +01:00
|
|
|
{
|
|
|
|
total_new[0] += count_multiple_uses (info);
|
|
|
|
total_orig[0] += (info->bit_not_p
|
|
|
|
+ info->ops[0].bit_not_p
|
|
|
|
+ info->ops[1].bit_not_p);
|
|
|
|
}
|
2017-11-09 14:08:41 +01:00
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (!allow_unaligned_load)
|
|
|
|
for (int i = 0; i < 2; ++i)
|
|
|
|
if (group->load_align[i])
|
|
|
|
group_load_align = MIN (group_load_align, group->load_align[i]);
|
|
|
|
|
2019-05-06 23:50:14 +02:00
|
|
|
if (bzero_first)
|
|
|
|
{
|
2019-11-08 11:53:50 +01:00
|
|
|
store_immediate_info *gstore;
|
|
|
|
FOR_EACH_VEC_ELT (group->stores, first, gstore)
|
|
|
|
if (!gimple_clobber_p (gstore->stmt))
|
|
|
|
break;
|
|
|
|
++first;
|
2019-05-06 23:50:14 +02:00
|
|
|
ret = 1;
|
|
|
|
if (split_stores)
|
|
|
|
{
|
2019-07-09 20:32:49 +02:00
|
|
|
split_store *store
|
2019-11-08 11:53:50 +01:00
|
|
|
= new split_store (bytepos, gstore->bitsize, align_base);
|
|
|
|
store->orig_stores.safe_push (gstore);
|
2019-05-06 23:50:14 +02:00
|
|
|
store->orig = true;
|
|
|
|
any_orig = true;
|
|
|
|
split_stores->safe_push (store);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
while (size > 0)
|
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if ((allow_unaligned_store || group_align <= BITS_PER_UNIT)
|
2019-05-06 23:50:14 +02:00
|
|
|
&& (group->mask[try_pos - bytepos] == (unsigned char) ~0U
|
|
|
|
|| (bzero_first && group->val[try_pos - bytepos] == 0)))
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
|
|
|
/* Skip padding bytes. */
|
|
|
|
++try_pos;
|
|
|
|
size -= BITS_PER_UNIT;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned HOST_WIDE_INT try_bitpos = try_pos * BITS_PER_UNIT;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
unsigned int try_size = MAX_STORE_BITSIZE, nonmasked;
|
|
|
|
unsigned HOST_WIDE_INT align_bitpos
|
|
|
|
= (try_bitpos - align_base) & (group_align - 1);
|
|
|
|
unsigned HOST_WIDE_INT align = group_align;
|
2019-11-08 11:53:50 +01:00
|
|
|
bool found_orig = false;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (align_bitpos)
|
|
|
|
align = least_bit_hwi (align_bitpos);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (!allow_unaligned_store)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
try_size = MIN (try_size, align);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (!allow_unaligned_load)
|
|
|
|
{
|
|
|
|
/* If we can't do or don't want to do unaligned stores
|
|
|
|
as well as loads, we need to take the loads into account
|
|
|
|
as well. */
|
|
|
|
unsigned HOST_WIDE_INT load_align = group_load_align;
|
|
|
|
align_bitpos = (try_bitpos - align_base) & (load_align - 1);
|
|
|
|
if (align_bitpos)
|
|
|
|
load_align = least_bit_hwi (align_bitpos);
|
|
|
|
for (int i = 0; i < 2; ++i)
|
|
|
|
if (group->load_align[i])
|
|
|
|
{
|
2017-12-20 13:56:50 +01:00
|
|
|
align_bitpos
|
|
|
|
= known_alignment (try_bitpos
|
|
|
|
- group->stores[0]->bitpos
|
|
|
|
+ group->stores[0]->ops[i].bitpos
|
|
|
|
- group->load_align_base[i]);
|
|
|
|
if (align_bitpos & (group_load_align - 1))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
unsigned HOST_WIDE_INT a = least_bit_hwi (align_bitpos);
|
|
|
|
load_align = MIN (load_align, a);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
try_size = MIN (try_size, load_align);
|
|
|
|
}
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
store_immediate_info *info
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
= find_constituent_stores (group, NULL, &first, try_bitpos, try_size);
|
2019-11-08 11:53:50 +01:00
|
|
|
if (info && !gimple_clobber_p (info->stmt))
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
|
|
|
/* If there is just one original statement for the range, see if
|
|
|
|
we can just reuse the original store which could be even larger
|
|
|
|
than try_size. */
|
|
|
|
unsigned HOST_WIDE_INT stmt_end
|
|
|
|
= ROUND_UP (info->bitpos + info->bitsize, BITS_PER_UNIT);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
info = find_constituent_stores (group, NULL, &first, try_bitpos,
|
|
|
|
stmt_end - try_bitpos);
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (info && info->bitpos >= try_bitpos)
|
|
|
|
{
|
2019-11-08 11:53:50 +01:00
|
|
|
store_immediate_info *info2 = NULL;
|
|
|
|
unsigned int first_copy = first;
|
|
|
|
if (info->bitpos > try_bitpos
|
|
|
|
&& stmt_end - try_bitpos <= try_size)
|
|
|
|
{
|
|
|
|
info2 = find_constituent_stores (group, NULL, &first_copy,
|
|
|
|
try_bitpos,
|
|
|
|
info->bitpos - try_bitpos);
|
|
|
|
gcc_assert (info2 == NULL || gimple_clobber_p (info2->stmt));
|
|
|
|
}
|
|
|
|
if (info2 == NULL && stmt_end - try_bitpos < try_size)
|
|
|
|
{
|
|
|
|
info2 = find_constituent_stores (group, NULL, &first_copy,
|
|
|
|
stmt_end,
|
|
|
|
(try_bitpos + try_size)
|
|
|
|
- stmt_end);
|
|
|
|
gcc_assert (info2 == NULL || gimple_clobber_p (info2->stmt));
|
|
|
|
}
|
|
|
|
if (info2 == NULL)
|
|
|
|
{
|
|
|
|
try_size = stmt_end - try_bitpos;
|
|
|
|
found_orig = true;
|
|
|
|
goto found;
|
|
|
|
}
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
}
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* Approximate store bitsize for the case when there are no padding
|
|
|
|
bits. */
|
|
|
|
while (try_size > size)
|
|
|
|
try_size /= 2;
|
|
|
|
/* Now look for whole padding bytes at the end of that bitsize. */
|
|
|
|
for (nonmasked = try_size / BITS_PER_UNIT; nonmasked > 0; --nonmasked)
|
|
|
|
if (group->mask[try_pos - bytepos + nonmasked - 1]
|
2019-05-06 23:50:14 +02:00
|
|
|
!= (unsigned char) ~0U
|
|
|
|
&& (!bzero_first
|
|
|
|
|| group->val[try_pos - bytepos + nonmasked - 1] != 0))
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
break;
|
2019-11-08 11:53:50 +01:00
|
|
|
if (nonmasked == 0 || (info && gimple_clobber_p (info->stmt)))
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
|
|
|
/* If entire try_size range is padding, skip it. */
|
|
|
|
try_pos += try_size / BITS_PER_UNIT;
|
|
|
|
size -= try_size;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
/* Otherwise try to decrease try_size if second half, last 3 quarters
|
|
|
|
etc. are padding. */
|
|
|
|
nonmasked *= BITS_PER_UNIT;
|
|
|
|
while (nonmasked <= try_size / 2)
|
|
|
|
try_size /= 2;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (!allow_unaligned_store && group_align > BITS_PER_UNIT)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
|
|
|
/* Now look for whole padding bytes at the start of that bitsize. */
|
|
|
|
unsigned int try_bytesize = try_size / BITS_PER_UNIT, masked;
|
|
|
|
for (masked = 0; masked < try_bytesize; ++masked)
|
2019-05-06 23:50:14 +02:00
|
|
|
if (group->mask[try_pos - bytepos + masked] != (unsigned char) ~0U
|
|
|
|
&& (!bzero_first
|
|
|
|
|| group->val[try_pos - bytepos + masked] != 0))
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
break;
|
|
|
|
masked *= BITS_PER_UNIT;
|
|
|
|
gcc_assert (masked < try_size);
|
|
|
|
if (masked >= try_size / 2)
|
|
|
|
{
|
|
|
|
while (masked >= try_size / 2)
|
|
|
|
{
|
|
|
|
try_size /= 2;
|
|
|
|
try_pos += try_size / BITS_PER_UNIT;
|
|
|
|
size -= try_size;
|
|
|
|
masked -= try_size;
|
|
|
|
}
|
|
|
|
/* Need to recompute the alignment, so just retry at the new
|
|
|
|
position. */
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
found:
|
|
|
|
++ret;
|
2016-10-28 16:18:50 +02:00
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (split_stores)
|
|
|
|
{
|
2019-07-09 20:32:49 +02:00
|
|
|
split_store *store
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
= new split_store (try_pos, try_size, align);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
info = find_constituent_stores (group, &store->orig_stores,
|
|
|
|
&first, try_bitpos, try_size);
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (info
|
2019-11-08 11:53:50 +01:00
|
|
|
&& !gimple_clobber_p (info->stmt)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
&& info->bitpos >= try_bitpos
|
2019-11-08 11:53:50 +01:00
|
|
|
&& info->bitpos + info->bitsize <= try_bitpos + try_size
|
|
|
|
&& (store->orig_stores.length () == 1
|
|
|
|
|| found_orig
|
|
|
|
|| (info->bitpos == try_bitpos
|
|
|
|
&& (info->bitpos + info->bitsize
|
|
|
|
== try_bitpos + try_size))))
|
2017-11-09 14:08:41 +01:00
|
|
|
{
|
|
|
|
store->orig = true;
|
|
|
|
any_orig = true;
|
|
|
|
}
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
split_stores->safe_push (store);
|
|
|
|
}
|
|
|
|
|
|
|
|
try_pos += try_size / BITS_PER_UNIT;
|
2016-10-28 16:18:50 +02:00
|
|
|
size -= try_size;
|
|
|
|
}
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
|
2017-11-09 14:08:41 +01:00
|
|
|
if (total_orig)
|
|
|
|
{
|
2017-11-13 11:26:13 +01:00
|
|
|
unsigned int i;
|
2019-07-09 20:32:49 +02:00
|
|
|
split_store *store;
|
2017-11-09 14:08:41 +01:00
|
|
|
/* If we are reusing some original stores and any of the
|
|
|
|
original SSA_NAMEs had multiple uses, we need to subtract
|
|
|
|
those now before we add the new ones. */
|
|
|
|
if (total_new[0] && any_orig)
|
|
|
|
{
|
|
|
|
FOR_EACH_VEC_ELT (*split_stores, i, store)
|
|
|
|
if (store->orig)
|
|
|
|
total_new[0] -= count_multiple_uses (store->orig_stores[0]);
|
|
|
|
}
|
|
|
|
total_new[0] += ret; /* The new store. */
|
|
|
|
store_immediate_info *info = group->stores[0];
|
|
|
|
if (info->ops[0].base_addr)
|
2017-11-13 11:26:13 +01:00
|
|
|
total_new[0] += ret;
|
2017-11-09 14:08:41 +01:00
|
|
|
if (info->ops[1].base_addr)
|
2017-11-13 11:26:13 +01:00
|
|
|
total_new[0] += ret;
|
2017-11-09 14:08:41 +01:00
|
|
|
switch (info->rhs_code)
|
|
|
|
{
|
|
|
|
case BIT_AND_EXPR:
|
|
|
|
case BIT_IOR_EXPR:
|
|
|
|
case BIT_XOR_EXPR:
|
|
|
|
total_new[0] += ret; /* The new BIT_*_EXPR stmt. */
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
break;
|
|
|
|
}
|
2017-11-13 11:26:13 +01:00
|
|
|
FOR_EACH_VEC_ELT (*split_stores, i, store)
|
|
|
|
{
|
|
|
|
unsigned int j;
|
|
|
|
bool bit_not_p[3] = { false, false, false };
|
|
|
|
/* If all orig_stores have certain bit_not_p set, then
|
|
|
|
we'd use a BIT_NOT_EXPR stmt and need to account for it.
|
|
|
|
If some orig_stores have certain bit_not_p set, then
|
|
|
|
we'd use a BIT_XOR_EXPR with a mask and need to account for
|
|
|
|
it. */
|
|
|
|
FOR_EACH_VEC_ELT (store->orig_stores, j, info)
|
|
|
|
{
|
|
|
|
if (info->ops[0].bit_not_p)
|
|
|
|
bit_not_p[0] = true;
|
|
|
|
if (info->ops[1].bit_not_p)
|
|
|
|
bit_not_p[1] = true;
|
|
|
|
if (info->bit_not_p)
|
|
|
|
bit_not_p[2] = true;
|
|
|
|
}
|
|
|
|
total_new[0] += bit_not_p[0] + bit_not_p[1] + bit_not_p[2];
|
|
|
|
}
|
|
|
|
|
2017-11-09 14:08:41 +01:00
|
|
|
}
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
return ret;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
2017-11-13 11:26:13 +01:00
|
|
|
/* Return the operation through which the operand IDX (if < 2) or
|
|
|
|
result (IDX == 2) should be inverted. If NOP_EXPR, no inversion
|
|
|
|
is done, if BIT_NOT_EXPR, all bits are inverted, if BIT_XOR_EXPR,
|
|
|
|
the bits should be xored with mask. */
|
|
|
|
|
|
|
|
static enum tree_code
|
|
|
|
invert_op (split_store *split_store, int idx, tree int_type, tree &mask)
|
|
|
|
{
|
|
|
|
unsigned int i;
|
|
|
|
store_immediate_info *info;
|
|
|
|
unsigned int cnt = 0;
|
2018-03-21 21:20:40 +01:00
|
|
|
bool any_paddings = false;
|
2017-11-13 11:26:13 +01:00
|
|
|
FOR_EACH_VEC_ELT (split_store->orig_stores, i, info)
|
|
|
|
{
|
|
|
|
bool bit_not_p = idx < 2 ? info->ops[idx].bit_not_p : info->bit_not_p;
|
|
|
|
if (bit_not_p)
|
2018-03-21 21:20:40 +01:00
|
|
|
{
|
|
|
|
++cnt;
|
|
|
|
tree lhs = gimple_assign_lhs (info->stmt);
|
|
|
|
if (INTEGRAL_TYPE_P (TREE_TYPE (lhs))
|
|
|
|
&& TYPE_PRECISION (TREE_TYPE (lhs)) < info->bitsize)
|
|
|
|
any_paddings = true;
|
|
|
|
}
|
2017-11-13 11:26:13 +01:00
|
|
|
}
|
|
|
|
mask = NULL_TREE;
|
|
|
|
if (cnt == 0)
|
|
|
|
return NOP_EXPR;
|
2018-03-21 21:20:40 +01:00
|
|
|
if (cnt == split_store->orig_stores.length () && !any_paddings)
|
2017-11-13 11:26:13 +01:00
|
|
|
return BIT_NOT_EXPR;
|
|
|
|
|
|
|
|
unsigned HOST_WIDE_INT try_bitpos = split_store->bytepos * BITS_PER_UNIT;
|
|
|
|
unsigned buf_size = split_store->size / BITS_PER_UNIT;
|
|
|
|
unsigned char *buf
|
|
|
|
= XALLOCAVEC (unsigned char, buf_size);
|
|
|
|
memset (buf, ~0U, buf_size);
|
|
|
|
FOR_EACH_VEC_ELT (split_store->orig_stores, i, info)
|
|
|
|
{
|
|
|
|
bool bit_not_p = idx < 2 ? info->ops[idx].bit_not_p : info->bit_not_p;
|
|
|
|
if (!bit_not_p)
|
|
|
|
continue;
|
|
|
|
/* Clear regions with bit_not_p and invert afterwards, rather than
|
|
|
|
clear regions with !bit_not_p, so that gaps in between stores aren't
|
|
|
|
set in the mask. */
|
|
|
|
unsigned HOST_WIDE_INT bitsize = info->bitsize;
|
2018-03-21 21:20:40 +01:00
|
|
|
unsigned HOST_WIDE_INT prec = bitsize;
|
2017-11-13 11:26:13 +01:00
|
|
|
unsigned int pos_in_buffer = 0;
|
2018-03-21 21:20:40 +01:00
|
|
|
if (any_paddings)
|
|
|
|
{
|
|
|
|
tree lhs = gimple_assign_lhs (info->stmt);
|
|
|
|
if (INTEGRAL_TYPE_P (TREE_TYPE (lhs))
|
|
|
|
&& TYPE_PRECISION (TREE_TYPE (lhs)) < bitsize)
|
|
|
|
prec = TYPE_PRECISION (TREE_TYPE (lhs));
|
|
|
|
}
|
2017-11-13 11:26:13 +01:00
|
|
|
if (info->bitpos < try_bitpos)
|
|
|
|
{
|
|
|
|
gcc_assert (info->bitpos + bitsize > try_bitpos);
|
2018-03-21 21:20:40 +01:00
|
|
|
if (!BYTES_BIG_ENDIAN)
|
|
|
|
{
|
|
|
|
if (prec <= try_bitpos - info->bitpos)
|
|
|
|
continue;
|
|
|
|
prec -= try_bitpos - info->bitpos;
|
|
|
|
}
|
|
|
|
bitsize -= try_bitpos - info->bitpos;
|
|
|
|
if (BYTES_BIG_ENDIAN && prec > bitsize)
|
|
|
|
prec = bitsize;
|
2017-11-13 11:26:13 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
pos_in_buffer = info->bitpos - try_bitpos;
|
2018-03-21 21:20:40 +01:00
|
|
|
if (prec < bitsize)
|
|
|
|
{
|
|
|
|
/* If this is a bool inversion, invert just the least significant
|
|
|
|
prec bits rather than all bits of it. */
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
{
|
|
|
|
pos_in_buffer += bitsize - prec;
|
|
|
|
if (pos_in_buffer >= split_store->size)
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
bitsize = prec;
|
|
|
|
}
|
2017-11-13 11:26:13 +01:00
|
|
|
if (pos_in_buffer + bitsize > split_store->size)
|
|
|
|
bitsize = split_store->size - pos_in_buffer;
|
|
|
|
unsigned char *p = buf + (pos_in_buffer / BITS_PER_UNIT);
|
|
|
|
if (BYTES_BIG_ENDIAN)
|
|
|
|
clear_bit_region_be (p, (BITS_PER_UNIT - 1
|
|
|
|
- (pos_in_buffer % BITS_PER_UNIT)), bitsize);
|
|
|
|
else
|
|
|
|
clear_bit_region (p, pos_in_buffer % BITS_PER_UNIT, bitsize);
|
|
|
|
}
|
|
|
|
for (unsigned int i = 0; i < buf_size; ++i)
|
|
|
|
buf[i] = ~buf[i];
|
|
|
|
mask = native_interpret_expr (int_type, buf, buf_size);
|
|
|
|
return BIT_XOR_EXPR;
|
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Given a merged store group GROUP output the widened version of it.
|
|
|
|
The store chain is against the base object BASE.
|
|
|
|
Try store sizes of at most MAX_STORE_BITSIZE bits wide and don't output
|
|
|
|
unaligned stores for STRICT_ALIGNMENT targets or if it's too expensive.
|
|
|
|
Make sure that the number of statements output is less than the number of
|
|
|
|
original statements. If a better sequence is possible emit it and
|
|
|
|
return true. */
|
|
|
|
|
|
|
|
bool
|
2016-11-02 14:05:07 +01:00
|
|
|
imm_store_chain_info::output_merged_store (merged_store_group *group)
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
const unsigned HOST_WIDE_INT start_byte_pos
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
= group->bitregion_start / BITS_PER_UNIT;
|
2016-10-28 16:18:50 +02:00
|
|
|
unsigned int orig_num_stmts = group->stores.length ();
|
|
|
|
if (orig_num_stmts < 2)
|
|
|
|
return false;
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
bool allow_unaligned_store
|
2019-11-12 11:08:40 +01:00
|
|
|
= !STRICT_ALIGNMENT && param_store_merging_allow_unaligned;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
bool allow_unaligned_load = allow_unaligned_store;
|
2019-05-06 23:50:14 +02:00
|
|
|
bool bzero_first = false;
|
2019-11-08 11:53:50 +01:00
|
|
|
store_immediate_info *store;
|
|
|
|
unsigned int num_clobber_stmts = 0;
|
|
|
|
if (group->stores[0]->rhs_code == INTEGER_CST)
|
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
unsigned int i;
|
2019-11-08 11:53:50 +01:00
|
|
|
FOR_EACH_VEC_ELT (group->stores, i, store)
|
|
|
|
if (gimple_clobber_p (store->stmt))
|
|
|
|
num_clobber_stmts++;
|
|
|
|
else if (TREE_CODE (gimple_assign_rhs1 (store->stmt)) == CONSTRUCTOR
|
|
|
|
&& CONSTRUCTOR_NELTS (gimple_assign_rhs1 (store->stmt)) == 0
|
|
|
|
&& group->start == store->bitpos
|
|
|
|
&& group->width == store->bitsize
|
|
|
|
&& (group->start % BITS_PER_UNIT) == 0
|
|
|
|
&& (group->width % BITS_PER_UNIT) == 0)
|
|
|
|
{
|
|
|
|
bzero_first = true;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
break;
|
|
|
|
FOR_EACH_VEC_ELT_FROM (group->stores, i, store, i)
|
|
|
|
if (gimple_clobber_p (store->stmt))
|
|
|
|
num_clobber_stmts++;
|
|
|
|
if (num_clobber_stmts == orig_num_stmts)
|
|
|
|
return false;
|
|
|
|
orig_num_stmts -= num_clobber_stmts;
|
|
|
|
}
|
2019-05-06 23:50:14 +02:00
|
|
|
if (allow_unaligned_store || bzero_first)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
|
|
|
/* If unaligned stores are allowed, see how many stores we'd emit
|
|
|
|
for unaligned and how many stores we'd emit for aligned stores.
|
2019-05-06 23:50:14 +02:00
|
|
|
Only use unaligned stores if it allows fewer stores than aligned.
|
|
|
|
Similarly, if there is a whole region clear first, prefer expanding
|
|
|
|
it together compared to expanding clear first followed by merged
|
|
|
|
further stores. */
|
2020-10-05 09:09:41 +02:00
|
|
|
unsigned cnt[4] = { ~0U, ~0U, ~0U, ~0U };
|
2019-05-06 23:50:14 +02:00
|
|
|
int pass_min = 0;
|
|
|
|
for (int pass = 0; pass < 4; ++pass)
|
|
|
|
{
|
|
|
|
if (!allow_unaligned_store && (pass & 1) != 0)
|
|
|
|
continue;
|
|
|
|
if (!bzero_first && (pass & 2) != 0)
|
|
|
|
continue;
|
|
|
|
cnt[pass] = split_group (group, (pass & 1) != 0,
|
|
|
|
allow_unaligned_load, (pass & 2) != 0,
|
|
|
|
NULL, NULL, NULL);
|
|
|
|
if (cnt[pass] < cnt[pass_min])
|
|
|
|
pass_min = pass;
|
|
|
|
}
|
|
|
|
if ((pass_min & 1) == 0)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
allow_unaligned_store = false;
|
2019-05-06 23:50:14 +02:00
|
|
|
if ((pass_min & 2) == 0)
|
|
|
|
bzero_first = false;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
|
|
|
|
auto_vec<class split_store *, 32> split_stores;
|
|
|
|
split_store *split_store;
|
|
|
|
unsigned total_orig, total_new, i;
|
2019-05-06 23:50:14 +02:00
|
|
|
split_group (group, allow_unaligned_store, allow_unaligned_load, bzero_first,
|
2017-11-09 14:08:41 +01:00
|
|
|
&split_stores, &total_orig, &total_new);
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
|
2019-11-08 11:53:50 +01:00
|
|
|
/* Determine if there is a clobber covering the whole group at the start,
|
|
|
|
followed by proposed split stores that cover the whole group. In that
|
|
|
|
case, prefer the transformation even if
|
|
|
|
split_stores.length () == orig_num_stmts. */
|
|
|
|
bool clobber_first = false;
|
|
|
|
if (num_clobber_stmts
|
|
|
|
&& gimple_clobber_p (group->stores[0]->stmt)
|
|
|
|
&& group->start == group->stores[0]->bitpos
|
|
|
|
&& group->width == group->stores[0]->bitsize
|
|
|
|
&& (group->start % BITS_PER_UNIT) == 0
|
|
|
|
&& (group->width % BITS_PER_UNIT) == 0)
|
|
|
|
{
|
|
|
|
clobber_first = true;
|
|
|
|
unsigned HOST_WIDE_INT pos = group->start / BITS_PER_UNIT;
|
|
|
|
FOR_EACH_VEC_ELT (split_stores, i, split_store)
|
|
|
|
if (split_store->bytepos != pos)
|
|
|
|
{
|
|
|
|
clobber_first = false;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
pos += split_store->size / BITS_PER_UNIT;
|
|
|
|
if (pos != (group->start + group->width) / BITS_PER_UNIT)
|
|
|
|
clobber_first = false;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (split_stores.length () >= orig_num_stmts + clobber_first)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
{
|
2019-11-08 11:53:50 +01:00
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
/* We didn't manage to reduce the number of statements. Bail out. */
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
2017-11-09 14:08:41 +01:00
|
|
|
fprintf (dump_file, "Exceeded original number of stmts (%u)."
|
|
|
|
" Not profitable to emit new sequence.\n",
|
|
|
|
orig_num_stmts);
|
2018-05-04 15:33:15 +02:00
|
|
|
FOR_EACH_VEC_ELT (split_stores, i, split_store)
|
|
|
|
delete split_store;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
return false;
|
|
|
|
}
|
2017-11-09 14:08:41 +01:00
|
|
|
if (total_orig <= total_new)
|
|
|
|
{
|
|
|
|
/* If number of estimated new statements is above estimated original
|
|
|
|
statements, bail out too. */
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fprintf (dump_file, "Estimated number of original stmts (%u)"
|
|
|
|
" not larger than estimated number of new"
|
|
|
|
" stmts (%u).\n",
|
|
|
|
total_orig, total_new);
|
2018-05-04 15:33:15 +02:00
|
|
|
FOR_EACH_VEC_ELT (split_stores, i, split_store)
|
|
|
|
delete split_store;
|
2017-11-20 11:10:23 +01:00
|
|
|
return false;
|
2017-11-09 14:08:41 +01:00
|
|
|
}
|
2019-11-08 11:53:50 +01:00
|
|
|
if (group->stores[0]->rhs_code == INTEGER_CST)
|
|
|
|
{
|
|
|
|
bool all_orig = true;
|
|
|
|
FOR_EACH_VEC_ELT (split_stores, i, split_store)
|
|
|
|
if (!split_store->orig)
|
|
|
|
{
|
|
|
|
all_orig = false;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
if (all_orig)
|
|
|
|
{
|
|
|
|
unsigned int cnt = split_stores.length ();
|
|
|
|
store_immediate_info *store;
|
|
|
|
FOR_EACH_VEC_ELT (group->stores, i, store)
|
|
|
|
if (gimple_clobber_p (store->stmt))
|
|
|
|
++cnt;
|
|
|
|
/* Punt if we wouldn't make any real changes, i.e. keep all
|
|
|
|
orig stmts + all clobbers. */
|
|
|
|
if (cnt == group->stores.length ())
|
|
|
|
{
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fprintf (dump_file, "Exceeded original number of stmts (%u)."
|
|
|
|
" Not profitable to emit new sequence.\n",
|
|
|
|
orig_num_stmts);
|
|
|
|
FOR_EACH_VEC_ELT (split_stores, i, split_store)
|
|
|
|
delete split_store;
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
gimple_stmt_iterator last_gsi = gsi_for_stmt (group->last_stmt);
|
|
|
|
gimple_seq seq = NULL;
|
|
|
|
tree last_vdef, new_vuse;
|
|
|
|
last_vdef = gimple_vdef (group->last_stmt);
|
|
|
|
new_vuse = gimple_vuse (group->last_stmt);
|
2017-11-20 11:10:23 +01:00
|
|
|
tree bswap_res = NULL_TREE;
|
|
|
|
|
2019-11-08 11:53:50 +01:00
|
|
|
/* Clobbers are not removed. */
|
|
|
|
if (gimple_clobber_p (group->last_stmt))
|
|
|
|
{
|
|
|
|
new_vuse = make_ssa_name (gimple_vop (cfun), group->last_stmt);
|
|
|
|
gimple_set_vdef (group->last_stmt, new_vuse);
|
|
|
|
}
|
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (group->stores[0]->rhs_code == LROTATE_EXPR
|
|
|
|
|| group->stores[0]->rhs_code == NOP_EXPR)
|
|
|
|
{
|
|
|
|
tree fndecl = NULL_TREE, bswap_type = NULL_TREE, load_type;
|
|
|
|
gimple *ins_stmt = group->stores[0]->ins_stmt;
|
|
|
|
struct symbolic_number *n = &group->stores[0]->n;
|
|
|
|
bool bswap = group->stores[0]->rhs_code == LROTATE_EXPR;
|
|
|
|
|
|
|
|
switch (n->range)
|
|
|
|
{
|
|
|
|
case 16:
|
|
|
|
load_type = bswap_type = uint16_type_node;
|
|
|
|
break;
|
|
|
|
case 32:
|
|
|
|
load_type = uint32_type_node;
|
|
|
|
if (bswap)
|
|
|
|
{
|
|
|
|
fndecl = builtin_decl_explicit (BUILT_IN_BSWAP32);
|
|
|
|
bswap_type = TREE_VALUE (TYPE_ARG_TYPES (TREE_TYPE (fndecl)));
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
case 64:
|
|
|
|
load_type = uint64_type_node;
|
|
|
|
if (bswap)
|
|
|
|
{
|
|
|
|
fndecl = builtin_decl_explicit (BUILT_IN_BSWAP64);
|
|
|
|
bswap_type = TREE_VALUE (TYPE_ARG_TYPES (TREE_TYPE (fndecl)));
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
gcc_unreachable ();
|
|
|
|
}
|
|
|
|
|
|
|
|
/* If the loads have each vuse of the corresponding store,
|
|
|
|
we've checked the aliasing already in try_coalesce_bswap and
|
|
|
|
we want to sink the need load into seq. So need to use new_vuse
|
|
|
|
on the load. */
|
2017-12-02 08:58:57 +01:00
|
|
|
if (n->base_addr)
|
2017-11-20 11:10:23 +01:00
|
|
|
{
|
2017-12-02 08:58:57 +01:00
|
|
|
if (n->vuse == NULL)
|
|
|
|
{
|
|
|
|
n->vuse = new_vuse;
|
|
|
|
ins_stmt = NULL;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
/* Update vuse in case it has changed by output_merged_stores. */
|
|
|
|
n->vuse = gimple_vuse (ins_stmt);
|
2017-11-20 11:10:23 +01:00
|
|
|
}
|
|
|
|
bswap_res = bswap_replace (gsi_start (seq), ins_stmt, fndecl,
|
|
|
|
bswap_type, load_type, n, bswap);
|
|
|
|
gcc_assert (bswap_res);
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
gimple *stmt = NULL;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
auto_vec<gimple *, 32> orig_stmts;
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple_seq this_seq;
|
|
|
|
tree addr = force_gimple_operand_1 (unshare_expr (base_addr), &this_seq,
|
2016-11-02 16:00:48 +01:00
|
|
|
is_gimple_mem_ref_addr, NULL_TREE);
|
2017-11-20 11:10:23 +01:00
|
|
|
gimple_seq_add_seq_without_update (&seq, this_seq);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
|
|
|
tree load_addr[2] = { NULL_TREE, NULL_TREE };
|
|
|
|
gimple_seq load_seq[2] = { NULL, NULL };
|
|
|
|
gimple_stmt_iterator load_gsi[2] = { gsi_none (), gsi_none () };
|
|
|
|
for (int j = 0; j < 2; ++j)
|
|
|
|
{
|
|
|
|
store_operand_info &op = group->stores[0]->ops[j];
|
|
|
|
if (op.base_addr == NULL_TREE)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
store_immediate_info *infol = group->stores.last ();
|
|
|
|
if (gimple_vuse (op.stmt) == gimple_vuse (infol->ops[j].stmt))
|
|
|
|
{
|
2017-11-21 09:41:47 +01:00
|
|
|
/* We can't pick the location randomly; while we've verified
|
|
|
|
all the loads have the same vuse, they can be still in different
|
|
|
|
basic blocks and we need to pick the one from the last bb:
|
|
|
|
int x = q[0];
|
|
|
|
if (x == N) return;
|
|
|
|
int y = q[1];
|
|
|
|
p[0] = x;
|
|
|
|
p[1] = y;
|
|
|
|
otherwise if we put the wider load at the q[0] load, we might
|
|
|
|
segfault if q[1] is not mapped. */
|
|
|
|
basic_block bb = gimple_bb (op.stmt);
|
|
|
|
gimple *ostmt = op.stmt;
|
|
|
|
store_immediate_info *info;
|
|
|
|
FOR_EACH_VEC_ELT (group->stores, i, info)
|
|
|
|
{
|
|
|
|
gimple *tstmt = info->ops[j].stmt;
|
|
|
|
basic_block tbb = gimple_bb (tstmt);
|
|
|
|
if (dominated_by_p (CDI_DOMINATORS, tbb, bb))
|
|
|
|
{
|
|
|
|
ostmt = tstmt;
|
|
|
|
bb = tbb;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
load_gsi[j] = gsi_for_stmt (ostmt);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
load_addr[j]
|
|
|
|
= force_gimple_operand_1 (unshare_expr (op.base_addr),
|
|
|
|
&load_seq[j], is_gimple_mem_ref_addr,
|
|
|
|
NULL_TREE);
|
|
|
|
}
|
|
|
|
else if (operand_equal_p (base_addr, op.base_addr, 0))
|
|
|
|
load_addr[j] = addr;
|
|
|
|
else
|
2017-11-06 17:29:11 +01:00
|
|
|
{
|
|
|
|
load_addr[j]
|
|
|
|
= force_gimple_operand_1 (unshare_expr (op.base_addr),
|
|
|
|
&this_seq, is_gimple_mem_ref_addr,
|
|
|
|
NULL_TREE);
|
|
|
|
gimple_seq_add_seq_without_update (&seq, this_seq);
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
FOR_EACH_VEC_ELT (split_stores, i, split_store)
|
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
const unsigned HOST_WIDE_INT try_size = split_store->size;
|
|
|
|
const unsigned HOST_WIDE_INT try_pos = split_store->bytepos;
|
|
|
|
const unsigned HOST_WIDE_INT try_bitpos = try_pos * BITS_PER_UNIT;
|
|
|
|
const unsigned HOST_WIDE_INT try_align = split_store->align;
|
|
|
|
const unsigned HOST_WIDE_INT try_offset = try_pos - start_byte_pos;
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
tree dest, src;
|
|
|
|
location_t loc;
|
2020-07-03 18:10:25 +02:00
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (split_store->orig)
|
|
|
|
{
|
2019-11-08 11:53:50 +01:00
|
|
|
/* If there is just a single non-clobber constituent store
|
|
|
|
which covers the whole area, just reuse the lhs and rhs. */
|
|
|
|
gimple *orig_stmt = NULL;
|
|
|
|
store_immediate_info *store;
|
|
|
|
unsigned int j;
|
|
|
|
FOR_EACH_VEC_ELT (split_store->orig_stores, j, store)
|
|
|
|
if (!gimple_clobber_p (store->stmt))
|
|
|
|
{
|
|
|
|
orig_stmt = store->stmt;
|
|
|
|
break;
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
dest = gimple_assign_lhs (orig_stmt);
|
|
|
|
src = gimple_assign_rhs1 (orig_stmt);
|
|
|
|
loc = gimple_location (orig_stmt);
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
store_immediate_info *info;
|
|
|
|
unsigned short clique, base;
|
|
|
|
unsigned int k;
|
|
|
|
FOR_EACH_VEC_ELT (split_store->orig_stores, k, info)
|
|
|
|
orig_stmts.safe_push (info->stmt);
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
tree offset_type
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
= get_alias_type_for_stmts (orig_stmts, false, &clique, &base);
|
2020-07-03 18:10:25 +02:00
|
|
|
tree dest_type;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
loc = get_location_for_stmts (orig_stmts);
|
|
|
|
orig_stmts.truncate (0);
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
|
2020-07-03 18:10:25 +02:00
|
|
|
if (group->string_concatenation)
|
|
|
|
dest_type
|
|
|
|
= build_array_type_nelts (char_type_node,
|
|
|
|
try_size / BITS_PER_UNIT);
|
|
|
|
else
|
|
|
|
{
|
|
|
|
dest_type = build_nonstandard_integer_type (try_size, UNSIGNED);
|
|
|
|
dest_type = build_aligned_type (dest_type, try_align);
|
|
|
|
}
|
|
|
|
dest = fold_build2 (MEM_REF, dest_type, addr,
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
build_int_cst (offset_type, try_pos));
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (TREE_CODE (dest) == MEM_REF)
|
|
|
|
{
|
|
|
|
MR_DEPENDENCE_CLIQUE (dest) = clique;
|
|
|
|
MR_DEPENDENCE_BASE (dest) = base;
|
|
|
|
}
|
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
tree mask;
|
2020-07-03 18:10:25 +02:00
|
|
|
if (bswap_res || group->string_concatenation)
|
2018-06-01 23:56:17 +02:00
|
|
|
mask = integer_zero_node;
|
|
|
|
else
|
2020-07-03 18:10:25 +02:00
|
|
|
mask = native_interpret_expr (dest_type,
|
|
|
|
group->mask + try_offset,
|
2017-11-20 11:10:23 +01:00
|
|
|
group->buf_size);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
|
|
|
tree ops[2];
|
|
|
|
for (int j = 0;
|
|
|
|
j < 1 + (split_store->orig_stores[0]->ops[1].val != NULL_TREE);
|
|
|
|
++j)
|
|
|
|
{
|
|
|
|
store_operand_info &op = split_store->orig_stores[0]->ops[j];
|
2017-11-20 11:10:23 +01:00
|
|
|
if (bswap_res)
|
|
|
|
ops[j] = bswap_res;
|
2020-07-03 18:10:25 +02:00
|
|
|
else if (group->string_concatenation)
|
|
|
|
{
|
|
|
|
ops[j] = build_string (try_size / BITS_PER_UNIT,
|
|
|
|
(const char *) group->val + try_offset);
|
|
|
|
TREE_TYPE (ops[j]) = dest_type;
|
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
else if (op.base_addr)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
FOR_EACH_VEC_ELT (split_store->orig_stores, k, info)
|
|
|
|
orig_stmts.safe_push (info->ops[j].stmt);
|
|
|
|
|
|
|
|
offset_type = get_alias_type_for_stmts (orig_stmts, true,
|
|
|
|
&clique, &base);
|
|
|
|
location_t load_loc = get_location_for_stmts (orig_stmts);
|
|
|
|
orig_stmts.truncate (0);
|
|
|
|
|
|
|
|
unsigned HOST_WIDE_INT load_align = group->load_align[j];
|
|
|
|
unsigned HOST_WIDE_INT align_bitpos
|
2018-06-01 23:56:17 +02:00
|
|
|
= known_alignment (try_bitpos
|
2017-12-20 13:56:50 +01:00
|
|
|
- split_store->orig_stores[0]->bitpos
|
|
|
|
+ op.bitpos);
|
|
|
|
if (align_bitpos & (load_align - 1))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
load_align = least_bit_hwi (align_bitpos);
|
|
|
|
|
|
|
|
tree load_int_type
|
|
|
|
= build_nonstandard_integer_type (try_size, UNSIGNED);
|
|
|
|
load_int_type
|
|
|
|
= build_aligned_type (load_int_type, load_align);
|
|
|
|
|
2017-12-20 13:56:50 +01:00
|
|
|
poly_uint64 load_pos
|
2018-06-01 23:56:17 +02:00
|
|
|
= exact_div (try_bitpos
|
2017-12-20 13:56:50 +01:00
|
|
|
- split_store->orig_stores[0]->bitpos
|
|
|
|
+ op.bitpos,
|
|
|
|
BITS_PER_UNIT);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
ops[j] = fold_build2 (MEM_REF, load_int_type, load_addr[j],
|
|
|
|
build_int_cst (offset_type, load_pos));
|
|
|
|
if (TREE_CODE (ops[j]) == MEM_REF)
|
|
|
|
{
|
|
|
|
MR_DEPENDENCE_CLIQUE (ops[j]) = clique;
|
|
|
|
MR_DEPENDENCE_BASE (ops[j]) = base;
|
|
|
|
}
|
|
|
|
if (!integer_zerop (mask))
|
|
|
|
/* The load might load some bits (that will be masked off
|
|
|
|
later on) uninitialized, avoid -W*uninitialized
|
|
|
|
warnings in that case. */
|
|
|
|
TREE_NO_WARNING (ops[j]) = 1;
|
|
|
|
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type), ops[j]);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
gimple_set_location (stmt, load_loc);
|
|
|
|
if (gsi_bb (load_gsi[j]))
|
|
|
|
{
|
|
|
|
gimple_set_vuse (stmt, gimple_vuse (op.stmt));
|
|
|
|
gimple_seq_add_stmt_without_update (&load_seq[j], stmt);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
gimple_set_vuse (stmt, new_vuse);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
}
|
|
|
|
ops[j] = gimple_assign_lhs (stmt);
|
2017-11-13 11:26:13 +01:00
|
|
|
tree xor_mask;
|
|
|
|
enum tree_code inv_op
|
2020-07-03 18:10:25 +02:00
|
|
|
= invert_op (split_store, j, dest_type, xor_mask);
|
2017-11-13 11:26:13 +01:00
|
|
|
if (inv_op != NOP_EXPR)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type),
|
2017-11-13 11:26:13 +01:00
|
|
|
inv_op, ops[j], xor_mask);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
gimple_set_location (stmt, load_loc);
|
|
|
|
ops[j] = gimple_assign_lhs (stmt);
|
|
|
|
|
|
|
|
if (gsi_bb (load_gsi[j]))
|
|
|
|
gimple_seq_add_stmt_without_update (&load_seq[j],
|
|
|
|
stmt);
|
|
|
|
else
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
|
|
|
else
|
2020-07-03 18:10:25 +02:00
|
|
|
ops[j] = native_interpret_expr (dest_type,
|
|
|
|
group->val + try_offset,
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
group->buf_size);
|
|
|
|
}
|
|
|
|
|
|
|
|
switch (split_store->orig_stores[0]->rhs_code)
|
|
|
|
{
|
|
|
|
case BIT_AND_EXPR:
|
|
|
|
case BIT_IOR_EXPR:
|
|
|
|
case BIT_XOR_EXPR:
|
|
|
|
FOR_EACH_VEC_ELT (split_store->orig_stores, k, info)
|
|
|
|
{
|
|
|
|
tree rhs1 = gimple_assign_rhs1 (info->stmt);
|
|
|
|
orig_stmts.safe_push (SSA_NAME_DEF_STMT (rhs1));
|
|
|
|
}
|
|
|
|
location_t bit_loc;
|
|
|
|
bit_loc = get_location_for_stmts (orig_stmts);
|
|
|
|
orig_stmts.truncate (0);
|
|
|
|
|
|
|
|
stmt
|
2020-07-03 18:10:25 +02:00
|
|
|
= gimple_build_assign (make_ssa_name (dest_type),
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
split_store->orig_stores[0]->rhs_code,
|
|
|
|
ops[0], ops[1]);
|
|
|
|
gimple_set_location (stmt, bit_loc);
|
|
|
|
/* If there is just one load and there is a separate
|
|
|
|
load_seq[0], emit the bitwise op right after it. */
|
|
|
|
if (load_addr[1] == NULL_TREE && gsi_bb (load_gsi[0]))
|
|
|
|
gimple_seq_add_stmt_without_update (&load_seq[0], stmt);
|
|
|
|
/* Otherwise, if at least one load is in seq, we need to
|
|
|
|
emit the bitwise op right before the store. If there
|
|
|
|
are two loads and are emitted somewhere else, it would
|
|
|
|
be better to emit the bitwise op as early as possible;
|
|
|
|
we don't track where that would be possible right now
|
|
|
|
though. */
|
|
|
|
else
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
src = gimple_assign_lhs (stmt);
|
2017-11-13 11:26:13 +01:00
|
|
|
tree xor_mask;
|
|
|
|
enum tree_code inv_op;
|
2020-07-03 18:10:25 +02:00
|
|
|
inv_op = invert_op (split_store, 2, dest_type, xor_mask);
|
2017-11-13 11:26:13 +01:00
|
|
|
if (inv_op != NOP_EXPR)
|
2017-11-09 20:24:59 +01:00
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type),
|
2017-11-13 11:26:13 +01:00
|
|
|
inv_op, src, xor_mask);
|
2017-11-09 20:24:59 +01:00
|
|
|
gimple_set_location (stmt, bit_loc);
|
|
|
|
if (load_addr[1] == NULL_TREE && gsi_bb (load_gsi[0]))
|
|
|
|
gimple_seq_add_stmt_without_update (&load_seq[0], stmt);
|
|
|
|
else
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
src = gimple_assign_lhs (stmt);
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
break;
|
2017-11-20 11:10:23 +01:00
|
|
|
case LROTATE_EXPR:
|
|
|
|
case NOP_EXPR:
|
|
|
|
src = ops[0];
|
|
|
|
if (!is_gimple_val (src))
|
|
|
|
{
|
|
|
|
stmt = gimple_build_assign (make_ssa_name (TREE_TYPE (src)),
|
|
|
|
src);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
src = gimple_assign_lhs (stmt);
|
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
if (!useless_type_conversion_p (dest_type, TREE_TYPE (src)))
|
2017-11-20 11:10:23 +01:00
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type),
|
2017-11-20 11:10:23 +01:00
|
|
|
NOP_EXPR, src);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
src = gimple_assign_lhs (stmt);
|
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
inv_op = invert_op (split_store, 2, dest_type, xor_mask);
|
2018-01-16 09:53:09 +01:00
|
|
|
if (inv_op != NOP_EXPR)
|
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type),
|
2018-01-16 09:53:09 +01:00
|
|
|
inv_op, src, xor_mask);
|
|
|
|
gimple_set_location (stmt, loc);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
src = gimple_assign_lhs (stmt);
|
|
|
|
}
|
2017-11-20 11:10:23 +01:00
|
|
|
break;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
default:
|
|
|
|
src = ops[0];
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
2018-06-01 23:56:17 +02:00
|
|
|
/* If bit insertion is required, we use the source as an accumulator
|
|
|
|
into which the successive bit-field values are manually inserted.
|
|
|
|
FIXME: perhaps use BIT_INSERT_EXPR instead in some cases? */
|
|
|
|
if (group->bit_insertion)
|
|
|
|
FOR_EACH_VEC_ELT (split_store->orig_stores, k, info)
|
|
|
|
if (info->rhs_code == BIT_INSERT_EXPR
|
|
|
|
&& info->bitpos < try_bitpos + try_size
|
|
|
|
&& info->bitpos + info->bitsize > try_bitpos)
|
|
|
|
{
|
|
|
|
/* Mask, truncate, convert to final type, shift and ior into
|
|
|
|
the accumulator. Note that every step can be a no-op. */
|
|
|
|
const HOST_WIDE_INT start_gap = info->bitpos - try_bitpos;
|
|
|
|
const HOST_WIDE_INT end_gap
|
|
|
|
= (try_bitpos + try_size) - (info->bitpos + info->bitsize);
|
|
|
|
tree tem = info->ops[0].val;
|
2020-05-25 22:13:11 +02:00
|
|
|
if (!INTEGRAL_TYPE_P (TREE_TYPE (tem)))
|
|
|
|
{
|
|
|
|
const unsigned HOST_WIDE_INT size
|
|
|
|
= tree_to_uhwi (TYPE_SIZE (TREE_TYPE (tem)));
|
|
|
|
tree integer_type
|
|
|
|
= build_nonstandard_integer_type (size, UNSIGNED);
|
|
|
|
tem = gimple_build (&seq, loc, VIEW_CONVERT_EXPR,
|
|
|
|
integer_type, tem);
|
|
|
|
}
|
2018-06-03 13:51:10 +02:00
|
|
|
if (TYPE_PRECISION (TREE_TYPE (tem)) <= info->bitsize)
|
|
|
|
{
|
|
|
|
tree bitfield_type
|
|
|
|
= build_nonstandard_integer_type (info->bitsize,
|
|
|
|
UNSIGNED);
|
|
|
|
tem = gimple_convert (&seq, loc, bitfield_type, tem);
|
|
|
|
}
|
|
|
|
else if ((BYTES_BIG_ENDIAN ? start_gap : end_gap) > 0)
|
2018-06-01 23:56:17 +02:00
|
|
|
{
|
|
|
|
const unsigned HOST_WIDE_INT imask
|
|
|
|
= (HOST_WIDE_INT_1U << info->bitsize) - 1;
|
|
|
|
tem = gimple_build (&seq, loc,
|
|
|
|
BIT_AND_EXPR, TREE_TYPE (tem), tem,
|
|
|
|
build_int_cst (TREE_TYPE (tem),
|
|
|
|
imask));
|
|
|
|
}
|
|
|
|
const HOST_WIDE_INT shift
|
|
|
|
= (BYTES_BIG_ENDIAN ? end_gap : start_gap);
|
|
|
|
if (shift < 0)
|
|
|
|
tem = gimple_build (&seq, loc,
|
|
|
|
RSHIFT_EXPR, TREE_TYPE (tem), tem,
|
|
|
|
build_int_cst (NULL_TREE, -shift));
|
2020-07-03 18:10:25 +02:00
|
|
|
tem = gimple_convert (&seq, loc, dest_type, tem);
|
2018-06-01 23:56:17 +02:00
|
|
|
if (shift > 0)
|
|
|
|
tem = gimple_build (&seq, loc,
|
2020-07-03 18:10:25 +02:00
|
|
|
LSHIFT_EXPR, dest_type, tem,
|
2018-06-01 23:56:17 +02:00
|
|
|
build_int_cst (NULL_TREE, shift));
|
|
|
|
src = gimple_build (&seq, loc,
|
2020-07-03 18:10:25 +02:00
|
|
|
BIT_IOR_EXPR, dest_type, tem, src);
|
2018-06-01 23:56:17 +02:00
|
|
|
}
|
|
|
|
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
if (!integer_zerop (mask))
|
|
|
|
{
|
2020-07-03 18:10:25 +02:00
|
|
|
tree tem = make_ssa_name (dest_type);
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
tree load_src = unshare_expr (dest);
|
|
|
|
/* The load might load some or all bits uninitialized,
|
|
|
|
avoid -W*uninitialized warnings in that case.
|
|
|
|
As optimization, it would be nice if all the bits are
|
|
|
|
provably uninitialized (no stores at all yet or previous
|
|
|
|
store a CLOBBER) we'd optimize away the load and replace
|
|
|
|
it e.g. with 0. */
|
|
|
|
TREE_NO_WARNING (load_src) = 1;
|
|
|
|
stmt = gimple_build_assign (tem, load_src);
|
|
|
|
gimple_set_location (stmt, loc);
|
|
|
|
gimple_set_vuse (stmt, new_vuse);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
|
|
|
|
/* FIXME: If there is a single chunk of zero bits in mask,
|
|
|
|
perhaps use BIT_INSERT_EXPR instead? */
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type),
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
BIT_AND_EXPR, tem, mask);
|
|
|
|
gimple_set_location (stmt, loc);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
tem = gimple_assign_lhs (stmt);
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (TREE_CODE (src) == INTEGER_CST)
|
2020-07-03 18:10:25 +02:00
|
|
|
src = wide_int_to_tree (dest_type,
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
wi::bit_and_not (wi::to_wide (src),
|
|
|
|
wi::to_wide (mask)));
|
|
|
|
else
|
|
|
|
{
|
|
|
|
tree nmask
|
2020-07-03 18:10:25 +02:00
|
|
|
= wide_int_to_tree (dest_type,
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
wi::bit_not (wi::to_wide (mask)));
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type),
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
BIT_AND_EXPR, src, nmask);
|
|
|
|
gimple_set_location (stmt, loc);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
src = gimple_assign_lhs (stmt);
|
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
stmt = gimple_build_assign (make_ssa_name (dest_type),
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
BIT_IOR_EXPR, tem, src);
|
|
|
|
gimple_set_location (stmt, loc);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
src = gimple_assign_lhs (stmt);
|
|
|
|
}
|
|
|
|
}
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
stmt = gimple_build_assign (dest, src);
|
|
|
|
gimple_set_location (stmt, loc);
|
|
|
|
gimple_set_vuse (stmt, new_vuse);
|
|
|
|
gimple_seq_add_stmt_without_update (&seq, stmt);
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (group->lp_nr && stmt_could_throw_p (cfun, stmt))
|
|
|
|
add_stmt_to_eh_lp (stmt, group->lp_nr);
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
tree new_vdef;
|
|
|
|
if (i < split_stores.length () - 1)
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
new_vdef = make_ssa_name (gimple_vop (cfun), stmt);
|
2016-10-28 16:18:50 +02:00
|
|
|
else
|
|
|
|
new_vdef = last_vdef;
|
|
|
|
|
|
|
|
gimple_set_vdef (stmt, new_vdef);
|
|
|
|
SSA_NAME_DEF_STMT (new_vdef) = stmt;
|
|
|
|
new_vuse = new_vdef;
|
|
|
|
}
|
|
|
|
|
|
|
|
FOR_EACH_VEC_ELT (split_stores, i, split_store)
|
|
|
|
delete split_store;
|
|
|
|
|
|
|
|
gcc_assert (seq);
|
|
|
|
if (dump_file)
|
|
|
|
{
|
|
|
|
fprintf (dump_file,
|
2018-06-01 23:56:17 +02:00
|
|
|
"New sequence of %u stores to replace old one of %u stores\n",
|
re PR middle-end/22141 (Missing optimization when storing structures)
PR middle-end/22141
* gimple-ssa-store-merging.c: Include rtl.h and expr.h.
(struct store_immediate_info): Add bitregion_start and bitregion_end
fields.
(store_immediate_info::store_immediate_info): Add brs and bre
arguments and initialize bitregion_{start,end} from those.
(struct merged_store_group): Add bitregion_start, bitregion_end,
align_base and mask fields. Drop unnecessary struct keyword from
struct store_immediate_info. Add do_merge method.
(clear_bit_region_be): Use memset instead of loop storing zeros.
(merged_store_group::do_merge): New method.
(merged_store_group::merge_into): Use do_merge. Allow gaps in between
stores as long as the surrounding bitregions have no gaps.
(merged_store_group::merge_overlapping): Use do_merge.
(merged_store_group::apply_stores): Test that bitregion_{start,end}
is byte aligned, rather than requiring that start and width are
byte aligned. Drop unnecessary struct keyword from
struct store_immediate_info. Allocate and populate also mask array.
Make start of the arrays relative to bitregion_start rather than
start and size them according to bitregion_{end,start} difference.
(struct imm_store_chain_info): Drop unnecessary struct keyword from
struct store_immediate_info.
(pass_store_merging::gate): Punt if BITS_PER_UNIT or CHAR_BIT is not 8.
(pass_store_merging::terminate_all_aliasing_chains): Drop unnecessary
struct keyword from struct store_immediate_info.
(imm_store_chain_info::coalesce_immediate_stores): Allow gaps in
between stores as long as the surrounding bitregions have no gaps.
Formatting fixes.
(struct split_store): Add orig non-static data member.
(split_store::split_store): Initialize orig to false.
(find_constituent_stmts): Return store_immediate_info *, non-NULL
if there is exactly a single original stmt. Change stmts argument
to pointer from reference, if NULL, don't push anything to it. Add
first argument, use it to optimize skipping over orig stmts that
are known to be before bitpos already. Simplify.
(split_group): Return unsigned int count how many stores are or
would be needed rather than a bool. Add allow_unaligned argument.
Change split_stores argument from reference to pointer, if NULL,
only do a dry run computing how many stores would be produced.
Rewritten algorithm to use both alignment and misalign if
!allow_unaligned and handle bitfield stores with gaps.
(imm_store_chain_info::output_merged_store): Set start_byte_pos
from bitregion_start instead of start. Compute allow_unaligned
here, if true, do 2 split_group dry runs to compute which one
produces fewer stores and prefer aligned if equal. Punt if
new count is bigger or equal than original before emitting any
statements, rather than during that. Remove no longer needed
new_ssa_names tracking. Replace num_stmts with
split_stores.length (). Use 32-bit stack allocated entries
in split_stores auto_vec. Try to reuse original store lhs/rhs1
if possible. Handle bitfields with gaps.
(pass_store_merging::execute): Ignore bitsize == 0 stores.
Compute bitregion_{start,end} for the stores and construct
store_immediate_info with that. Formatting fixes.
* gcc.dg/store_merging_10.c: New test.
* gcc.dg/store_merging_11.c: New test.
* gcc.dg/store_merging_12.c: New test.
* g++.dg/pr71694.C: Add -fno-store-merging to dg-options.
From-SVN: r254213
2017-10-30 12:04:49 +01:00
|
|
|
split_stores.length (), orig_num_stmts);
|
2016-10-28 16:18:50 +02:00
|
|
|
if (dump_flags & TDF_DETAILS)
|
|
|
|
print_gimple_seq (dump_file, seq, 0, TDF_VOPS | TDF_MEMSYMS);
|
|
|
|
}
|
2019-10-02 15:35:40 +02:00
|
|
|
|
2019-11-08 11:53:50 +01:00
|
|
|
if (gimple_clobber_p (group->last_stmt))
|
|
|
|
update_stmt (group->last_stmt);
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (group->lp_nr > 0)
|
|
|
|
{
|
|
|
|
/* We're going to insert a sequence of (potentially) throwing stores
|
|
|
|
into an active EH region. This means that we're going to create
|
|
|
|
new basic blocks with EH edges pointing to the post landing pad
|
|
|
|
and, therefore, to have to update its PHI nodes, if any. For the
|
|
|
|
virtual PHI node, we're going to use the VDEFs created above, but
|
|
|
|
for the other nodes, we need to record the original reaching defs. */
|
|
|
|
eh_landing_pad lp = get_eh_landing_pad_from_number (group->lp_nr);
|
|
|
|
basic_block lp_bb = label_to_block (cfun, lp->post_landing_pad);
|
|
|
|
basic_block last_bb = gimple_bb (group->last_stmt);
|
|
|
|
edge last_edge = find_edge (last_bb, lp_bb);
|
|
|
|
auto_vec<tree, 16> last_defs;
|
|
|
|
gphi_iterator gpi;
|
|
|
|
for (gpi = gsi_start_phis (lp_bb); !gsi_end_p (gpi); gsi_next (&gpi))
|
|
|
|
{
|
|
|
|
gphi *phi = gpi.phi ();
|
|
|
|
tree last_def;
|
|
|
|
if (virtual_operand_p (gimple_phi_result (phi)))
|
|
|
|
last_def = NULL_TREE;
|
|
|
|
else
|
|
|
|
last_def = gimple_phi_arg_def (phi, last_edge->dest_idx);
|
|
|
|
last_defs.safe_push (last_def);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Do the insertion. Then, if new basic blocks have been created in the
|
|
|
|
process, rewind the chain of VDEFs create above to walk the new basic
|
|
|
|
blocks and update the corresponding arguments of the PHI nodes. */
|
|
|
|
update_modified_stmts (seq);
|
|
|
|
if (gimple_find_sub_bbs (seq, &last_gsi))
|
|
|
|
while (last_vdef != gimple_vuse (group->last_stmt))
|
|
|
|
{
|
|
|
|
gimple *stmt = SSA_NAME_DEF_STMT (last_vdef);
|
|
|
|
if (stmt_could_throw_p (cfun, stmt))
|
|
|
|
{
|
|
|
|
edge new_edge = find_edge (gimple_bb (stmt), lp_bb);
|
|
|
|
unsigned int i;
|
|
|
|
for (gpi = gsi_start_phis (lp_bb), i = 0;
|
|
|
|
!gsi_end_p (gpi);
|
|
|
|
gsi_next (&gpi), i++)
|
|
|
|
{
|
|
|
|
gphi *phi = gpi.phi ();
|
|
|
|
tree new_def;
|
|
|
|
if (virtual_operand_p (gimple_phi_result (phi)))
|
|
|
|
new_def = last_vdef;
|
|
|
|
else
|
|
|
|
new_def = last_defs[i];
|
|
|
|
add_phi_arg (phi, new_def, new_edge, UNKNOWN_LOCATION);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
last_vdef = gimple_vuse (stmt);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
else
|
|
|
|
gsi_insert_seq_after (&last_gsi, seq, GSI_SAME_STMT);
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
for (int j = 0; j < 2; ++j)
|
|
|
|
if (load_seq[j])
|
|
|
|
gsi_insert_seq_after (&load_gsi[j], load_seq[j], GSI_SAME_STMT);
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Process the merged_store_group objects created in the coalescing phase.
|
|
|
|
The stores are all against the base object BASE.
|
|
|
|
Try to output the widened stores and delete the original statements if
|
|
|
|
successful. Return true iff any changes were made. */
|
|
|
|
|
|
|
|
bool
|
2016-11-02 14:05:07 +01:00
|
|
|
imm_store_chain_info::output_merged_stores ()
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
unsigned int i;
|
|
|
|
merged_store_group *merged_store;
|
|
|
|
bool ret = false;
|
|
|
|
FOR_EACH_VEC_ELT (m_merged_store_groups, i, merged_store)
|
|
|
|
{
|
2019-09-18 11:03:14 +02:00
|
|
|
if (dbg_cnt (store_merging)
|
|
|
|
&& output_merged_store (merged_store))
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
unsigned int j;
|
|
|
|
store_immediate_info *store;
|
|
|
|
FOR_EACH_VEC_ELT (merged_store->stores, j, store)
|
|
|
|
{
|
|
|
|
gimple *stmt = store->stmt;
|
|
|
|
gimple_stmt_iterator gsi = gsi_for_stmt (stmt);
|
2019-11-08 11:53:50 +01:00
|
|
|
/* Don't remove clobbers, they are still useful even if
|
|
|
|
everything is overwritten afterwards. */
|
|
|
|
if (gimple_clobber_p (stmt))
|
|
|
|
continue;
|
2016-10-28 16:18:50 +02:00
|
|
|
gsi_remove (&gsi, true);
|
2019-10-02 15:35:40 +02:00
|
|
|
if (store->lp_nr)
|
|
|
|
remove_stmt_from_eh_lp (stmt);
|
2016-10-28 16:18:50 +02:00
|
|
|
if (stmt != merged_store->last_stmt)
|
|
|
|
{
|
|
|
|
unlink_stmt_vdef (stmt);
|
|
|
|
release_defs (stmt);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
ret = true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (ret && dump_file)
|
|
|
|
fprintf (dump_file, "Merging successful!\n");
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Coalesce the store_immediate_info objects recorded against the base object
|
|
|
|
BASE in the first phase and output them.
|
|
|
|
Delete the allocated structures.
|
|
|
|
Return true if any changes were made. */
|
|
|
|
|
|
|
|
bool
|
2016-11-02 14:05:07 +01:00
|
|
|
imm_store_chain_info::terminate_and_process_chain ()
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
2021-02-11 11:13:47 +01:00
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fprintf (dump_file, "Terminating chain with %u stores\n",
|
|
|
|
m_store_info.length ());
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Process store chain. */
|
|
|
|
bool ret = false;
|
|
|
|
if (m_store_info.length () > 1)
|
|
|
|
{
|
|
|
|
ret = coalesce_immediate_stores ();
|
|
|
|
if (ret)
|
2016-11-02 14:05:07 +01:00
|
|
|
ret = output_merged_stores ();
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Delete all the entries we allocated ourselves. */
|
|
|
|
store_immediate_info *info;
|
|
|
|
unsigned int i;
|
|
|
|
FOR_EACH_VEC_ELT (m_store_info, i, info)
|
|
|
|
delete info;
|
|
|
|
|
|
|
|
merged_store_group *merged_info;
|
|
|
|
FOR_EACH_VEC_ELT (m_merged_store_groups, i, merged_info)
|
|
|
|
delete merged_info;
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Return true iff LHS is a destination potentially interesting for
|
|
|
|
store merging. In practice these are the codes that get_inner_reference
|
|
|
|
can process. */
|
|
|
|
|
|
|
|
static bool
|
|
|
|
lhs_valid_for_store_merging_p (tree lhs)
|
|
|
|
{
|
2019-10-02 15:35:40 +02:00
|
|
|
if (DECL_P (lhs))
|
2016-10-28 16:18:50 +02:00
|
|
|
return true;
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
switch (TREE_CODE (lhs))
|
|
|
|
{
|
|
|
|
case ARRAY_REF:
|
|
|
|
case ARRAY_RANGE_REF:
|
|
|
|
case BIT_FIELD_REF:
|
|
|
|
case COMPONENT_REF:
|
|
|
|
case MEM_REF:
|
2020-07-03 18:10:25 +02:00
|
|
|
case VIEW_CONVERT_EXPR:
|
2019-10-02 15:35:40 +02:00
|
|
|
return true;
|
|
|
|
default:
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
gcc_unreachable ();
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Return true if the tree RHS is a constant we want to consider
|
|
|
|
during store merging. In practice accept all codes that
|
|
|
|
native_encode_expr accepts. */
|
|
|
|
|
|
|
|
static bool
|
|
|
|
rhs_valid_for_store_merging_p (tree rhs)
|
|
|
|
{
|
poly_int: GET_MODE_SIZE
This patch changes GET_MODE_SIZE from unsigned short to poly_uint16.
The non-mechanical parts were handled by previous patches.
2018-01-03 Richard Sandiford <richard.sandiford@linaro.org>
Alan Hayward <alan.hayward@arm.com>
David Sherwood <david.sherwood@arm.com>
gcc/
* machmode.h (mode_size): Change from unsigned short to
poly_uint16_pod.
(mode_to_bytes): Return a poly_uint16 rather than an unsigned short.
(GET_MODE_SIZE): Return a constant if ONLY_FIXED_SIZE_MODES,
or if measurement_type is not polynomial.
(fixed_size_mode::includes_p): Check for constant-sized modes.
* genmodes.c (emit_mode_size_inline): Make mode_size_inline
return a poly_uint16 rather than an unsigned short.
(emit_mode_size): Change the type of mode_size from unsigned short
to poly_uint16_pod. Use ZERO_COEFFS for the initializer.
(emit_mode_adjustments): Cope with polynomial vector sizes.
* lto-streamer-in.c (lto_input_mode_table): Use bp_unpack_poly_value
for GET_MODE_SIZE.
* lto-streamer-out.c (lto_write_mode_table): Use bp_pack_poly_value
for GET_MODE_SIZE.
* auto-inc-dec.c (try_merge): Treat GET_MODE_SIZE as polynomial.
* builtins.c (expand_ifn_atomic_compare_exchange_into_call): Likewise.
* caller-save.c (setup_save_areas): Likewise.
(replace_reg_with_saved_mem): Likewise.
* calls.c (emit_library_call_value_1): Likewise.
* combine-stack-adj.c (combine_stack_adjustments_for_block): Likewise.
* combine.c (simplify_set, make_extraction, simplify_shift_const_1)
(gen_lowpart_for_combine): Likewise.
* convert.c (convert_to_integer_1): Likewise.
* cse.c (equiv_constant, cse_insn): Likewise.
* cselib.c (autoinc_split, cselib_hash_rtx): Likewise.
(cselib_subst_to_values): Likewise.
* dce.c (word_dce_process_block): Likewise.
* df-problems.c (df_word_lr_mark_ref): Likewise.
* dwarf2cfi.c (init_one_dwarf_reg_size): Likewise.
* dwarf2out.c (multiple_reg_loc_descriptor, mem_loc_descriptor)
(concat_loc_descriptor, concatn_loc_descriptor, loc_descriptor)
(rtl_for_decl_location): Likewise.
* emit-rtl.c (gen_highpart, widen_memory_access): Likewise.
* expmed.c (extract_bit_field_1, extract_integral_bit_field): Likewise.
* expr.c (emit_group_load_1, clear_storage_hints): Likewise.
(emit_move_complex, emit_move_multi_word, emit_push_insn): Likewise.
(expand_expr_real_1): Likewise.
* function.c (assign_parm_setup_block_p, assign_parm_setup_block)
(pad_below): Likewise.
* gimple-fold.c (optimize_atomic_compare_exchange_p): Likewise.
* gimple-ssa-store-merging.c (rhs_valid_for_store_merging_p): Likewise.
* ira.c (get_subreg_tracking_sizes): Likewise.
* ira-build.c (ira_create_allocno_objects): Likewise.
* ira-color.c (coalesced_pseudo_reg_slot_compare): Likewise.
(ira_sort_regnos_for_alter_reg): Likewise.
* ira-costs.c (record_operand_costs): Likewise.
* lower-subreg.c (interesting_mode_p, simplify_gen_subreg_concatn)
(resolve_simple_move): Likewise.
* lra-constraints.c (get_reload_reg, operands_match_p): Likewise.
(process_addr_reg, simplify_operand_subreg, curr_insn_transform)
(lra_constraints): Likewise.
(CONST_POOL_OK_P): Reject variable-sized modes.
* lra-spills.c (slot, assign_mem_slot, pseudo_reg_slot_compare)
(add_pseudo_to_slot, lra_spill): Likewise.
* omp-low.c (omp_clause_aligned_alignment): Likewise.
* optabs-query.c (get_best_extraction_insn): Likewise.
* optabs-tree.c (expand_vec_cond_expr_p): Likewise.
* optabs.c (expand_vec_perm_var, expand_vec_cond_expr): Likewise.
(expand_mult_highpart, valid_multiword_target_p): Likewise.
* recog.c (offsettable_address_addr_space_p): Likewise.
* regcprop.c (maybe_mode_change): Likewise.
* reginfo.c (choose_hard_reg_mode, record_subregs_of_mode): Likewise.
* regrename.c (build_def_use): Likewise.
* regstat.c (dump_reg_info): Likewise.
* reload.c (complex_word_subreg_p, push_reload, find_dummy_reload)
(find_reloads, find_reloads_subreg_address): Likewise.
* reload1.c (eliminate_regs_1): Likewise.
* rtlanal.c (for_each_inc_dec_find_inc_dec, rtx_cost): Likewise.
* simplify-rtx.c (avoid_constant_pool_reference): Likewise.
(simplify_binary_operation_1, simplify_subreg): Likewise.
* targhooks.c (default_function_arg_padding): Likewise.
(default_hard_regno_nregs, default_class_max_nregs): Likewise.
* tree-cfg.c (verify_gimple_assign_binary): Likewise.
(verify_gimple_assign_ternary): Likewise.
* tree-inline.c (estimate_move_cost): Likewise.
* tree-ssa-forwprop.c (simplify_vector_constructor): Likewise.
* tree-ssa-loop-ivopts.c (add_autoinc_candidates): Likewise.
(get_address_cost_ainc): Likewise.
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Likewise.
(vect_supportable_dr_alignment): Likewise.
* tree-vect-loop.c (vect_determine_vectorization_factor): Likewise.
(vectorizable_reduction): Likewise.
* tree-vect-stmts.c (vectorizable_assignment, vectorizable_shift)
(vectorizable_operation, vectorizable_load): Likewise.
* tree.c (build_same_sized_truth_vector_type): Likewise.
* valtrack.c (cleanup_auto_inc_dec): Likewise.
* var-tracking.c (emit_note_insn_var_location): Likewise.
* config/arc/arc.h (ASM_OUTPUT_CASE_END): Use as_a <scalar_int_mode>.
(ADDR_VEC_ALIGN): Likewise.
Co-Authored-By: Alan Hayward <alan.hayward@arm.com>
Co-Authored-By: David Sherwood <david.sherwood@arm.com>
From-SVN: r256201
2018-01-03 22:42:52 +01:00
|
|
|
unsigned HOST_WIDE_INT size;
|
2019-05-06 23:50:14 +02:00
|
|
|
if (TREE_CODE (rhs) == CONSTRUCTOR
|
|
|
|
&& CONSTRUCTOR_NELTS (rhs) == 0
|
|
|
|
&& TYPE_SIZE_UNIT (TREE_TYPE (rhs))
|
|
|
|
&& tree_fits_uhwi_p (TYPE_SIZE_UNIT (TREE_TYPE (rhs))))
|
|
|
|
return true;
|
poly_int: GET_MODE_SIZE
This patch changes GET_MODE_SIZE from unsigned short to poly_uint16.
The non-mechanical parts were handled by previous patches.
2018-01-03 Richard Sandiford <richard.sandiford@linaro.org>
Alan Hayward <alan.hayward@arm.com>
David Sherwood <david.sherwood@arm.com>
gcc/
* machmode.h (mode_size): Change from unsigned short to
poly_uint16_pod.
(mode_to_bytes): Return a poly_uint16 rather than an unsigned short.
(GET_MODE_SIZE): Return a constant if ONLY_FIXED_SIZE_MODES,
or if measurement_type is not polynomial.
(fixed_size_mode::includes_p): Check for constant-sized modes.
* genmodes.c (emit_mode_size_inline): Make mode_size_inline
return a poly_uint16 rather than an unsigned short.
(emit_mode_size): Change the type of mode_size from unsigned short
to poly_uint16_pod. Use ZERO_COEFFS for the initializer.
(emit_mode_adjustments): Cope with polynomial vector sizes.
* lto-streamer-in.c (lto_input_mode_table): Use bp_unpack_poly_value
for GET_MODE_SIZE.
* lto-streamer-out.c (lto_write_mode_table): Use bp_pack_poly_value
for GET_MODE_SIZE.
* auto-inc-dec.c (try_merge): Treat GET_MODE_SIZE as polynomial.
* builtins.c (expand_ifn_atomic_compare_exchange_into_call): Likewise.
* caller-save.c (setup_save_areas): Likewise.
(replace_reg_with_saved_mem): Likewise.
* calls.c (emit_library_call_value_1): Likewise.
* combine-stack-adj.c (combine_stack_adjustments_for_block): Likewise.
* combine.c (simplify_set, make_extraction, simplify_shift_const_1)
(gen_lowpart_for_combine): Likewise.
* convert.c (convert_to_integer_1): Likewise.
* cse.c (equiv_constant, cse_insn): Likewise.
* cselib.c (autoinc_split, cselib_hash_rtx): Likewise.
(cselib_subst_to_values): Likewise.
* dce.c (word_dce_process_block): Likewise.
* df-problems.c (df_word_lr_mark_ref): Likewise.
* dwarf2cfi.c (init_one_dwarf_reg_size): Likewise.
* dwarf2out.c (multiple_reg_loc_descriptor, mem_loc_descriptor)
(concat_loc_descriptor, concatn_loc_descriptor, loc_descriptor)
(rtl_for_decl_location): Likewise.
* emit-rtl.c (gen_highpart, widen_memory_access): Likewise.
* expmed.c (extract_bit_field_1, extract_integral_bit_field): Likewise.
* expr.c (emit_group_load_1, clear_storage_hints): Likewise.
(emit_move_complex, emit_move_multi_word, emit_push_insn): Likewise.
(expand_expr_real_1): Likewise.
* function.c (assign_parm_setup_block_p, assign_parm_setup_block)
(pad_below): Likewise.
* gimple-fold.c (optimize_atomic_compare_exchange_p): Likewise.
* gimple-ssa-store-merging.c (rhs_valid_for_store_merging_p): Likewise.
* ira.c (get_subreg_tracking_sizes): Likewise.
* ira-build.c (ira_create_allocno_objects): Likewise.
* ira-color.c (coalesced_pseudo_reg_slot_compare): Likewise.
(ira_sort_regnos_for_alter_reg): Likewise.
* ira-costs.c (record_operand_costs): Likewise.
* lower-subreg.c (interesting_mode_p, simplify_gen_subreg_concatn)
(resolve_simple_move): Likewise.
* lra-constraints.c (get_reload_reg, operands_match_p): Likewise.
(process_addr_reg, simplify_operand_subreg, curr_insn_transform)
(lra_constraints): Likewise.
(CONST_POOL_OK_P): Reject variable-sized modes.
* lra-spills.c (slot, assign_mem_slot, pseudo_reg_slot_compare)
(add_pseudo_to_slot, lra_spill): Likewise.
* omp-low.c (omp_clause_aligned_alignment): Likewise.
* optabs-query.c (get_best_extraction_insn): Likewise.
* optabs-tree.c (expand_vec_cond_expr_p): Likewise.
* optabs.c (expand_vec_perm_var, expand_vec_cond_expr): Likewise.
(expand_mult_highpart, valid_multiword_target_p): Likewise.
* recog.c (offsettable_address_addr_space_p): Likewise.
* regcprop.c (maybe_mode_change): Likewise.
* reginfo.c (choose_hard_reg_mode, record_subregs_of_mode): Likewise.
* regrename.c (build_def_use): Likewise.
* regstat.c (dump_reg_info): Likewise.
* reload.c (complex_word_subreg_p, push_reload, find_dummy_reload)
(find_reloads, find_reloads_subreg_address): Likewise.
* reload1.c (eliminate_regs_1): Likewise.
* rtlanal.c (for_each_inc_dec_find_inc_dec, rtx_cost): Likewise.
* simplify-rtx.c (avoid_constant_pool_reference): Likewise.
(simplify_binary_operation_1, simplify_subreg): Likewise.
* targhooks.c (default_function_arg_padding): Likewise.
(default_hard_regno_nregs, default_class_max_nregs): Likewise.
* tree-cfg.c (verify_gimple_assign_binary): Likewise.
(verify_gimple_assign_ternary): Likewise.
* tree-inline.c (estimate_move_cost): Likewise.
* tree-ssa-forwprop.c (simplify_vector_constructor): Likewise.
* tree-ssa-loop-ivopts.c (add_autoinc_candidates): Likewise.
(get_address_cost_ainc): Likewise.
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Likewise.
(vect_supportable_dr_alignment): Likewise.
* tree-vect-loop.c (vect_determine_vectorization_factor): Likewise.
(vectorizable_reduction): Likewise.
* tree-vect-stmts.c (vectorizable_assignment, vectorizable_shift)
(vectorizable_operation, vectorizable_load): Likewise.
* tree.c (build_same_sized_truth_vector_type): Likewise.
* valtrack.c (cleanup_auto_inc_dec): Likewise.
* var-tracking.c (emit_note_insn_var_location): Likewise.
* config/arc/arc.h (ASM_OUTPUT_CASE_END): Use as_a <scalar_int_mode>.
(ADDR_VEC_ALIGN): Likewise.
Co-Authored-By: Alan Hayward <alan.hayward@arm.com>
Co-Authored-By: David Sherwood <david.sherwood@arm.com>
From-SVN: r256201
2018-01-03 22:42:52 +01:00
|
|
|
return (GET_MODE_SIZE (TYPE_MODE (TREE_TYPE (rhs))).is_constant (&size)
|
|
|
|
&& native_encode_expr (rhs, NULL, size) != 0);
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
/* Adjust *PBITPOS, *PBITREGION_START and *PBITREGION_END by BYTE_OFF bytes
|
|
|
|
and return true on success or false on failure. */
|
|
|
|
|
|
|
|
static bool
|
|
|
|
adjust_bit_pos (poly_offset_int byte_off,
|
|
|
|
poly_int64 *pbitpos,
|
|
|
|
poly_uint64 *pbitregion_start,
|
|
|
|
poly_uint64 *pbitregion_end)
|
|
|
|
{
|
|
|
|
poly_offset_int bit_off = byte_off << LOG2_BITS_PER_UNIT;
|
|
|
|
bit_off += *pbitpos;
|
|
|
|
|
|
|
|
if (known_ge (bit_off, 0) && bit_off.to_shwi (pbitpos))
|
|
|
|
{
|
|
|
|
if (maybe_ne (*pbitregion_end, 0U))
|
|
|
|
{
|
|
|
|
bit_off = byte_off << LOG2_BITS_PER_UNIT;
|
|
|
|
bit_off += *pbitregion_start;
|
|
|
|
if (bit_off.to_uhwi (pbitregion_start))
|
|
|
|
{
|
|
|
|
bit_off = byte_off << LOG2_BITS_PER_UNIT;
|
|
|
|
bit_off += *pbitregion_end;
|
|
|
|
if (!bit_off.to_uhwi (pbitregion_end))
|
|
|
|
*pbitregion_end = 0;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
*pbitregion_end = 0;
|
|
|
|
}
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* If MEM is a memory reference usable for store merging (either as
|
|
|
|
store destination or for loads), return the non-NULL base_addr
|
|
|
|
and set *PBITSIZE, *PBITPOS, *PBITREGION_START and *PBITREGION_END.
|
|
|
|
Otherwise return NULL, *PBITPOS should be still valid even for that
|
|
|
|
case. */
|
|
|
|
|
|
|
|
static tree
|
2017-12-20 13:56:50 +01:00
|
|
|
mem_valid_for_store_merging (tree mem, poly_uint64 *pbitsize,
|
|
|
|
poly_uint64 *pbitpos,
|
|
|
|
poly_uint64 *pbitregion_start,
|
|
|
|
poly_uint64 *pbitregion_end)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
2017-12-20 13:56:50 +01:00
|
|
|
poly_int64 bitsize, bitpos;
|
|
|
|
poly_uint64 bitregion_start = 0, bitregion_end = 0;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
machine_mode mode;
|
|
|
|
int unsignedp = 0, reversep = 0, volatilep = 0;
|
|
|
|
tree offset;
|
|
|
|
tree base_addr = get_inner_reference (mem, &bitsize, &bitpos, &offset, &mode,
|
|
|
|
&unsignedp, &reversep, &volatilep);
|
|
|
|
*pbitsize = bitsize;
|
2017-12-20 13:56:50 +01:00
|
|
|
if (known_eq (bitsize, 0))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return NULL_TREE;
|
|
|
|
|
|
|
|
if (TREE_CODE (mem) == COMPONENT_REF
|
|
|
|
&& DECL_BIT_FIELD_TYPE (TREE_OPERAND (mem, 1)))
|
|
|
|
{
|
|
|
|
get_bit_range (&bitregion_start, &bitregion_end, mem, &bitpos, &offset);
|
2017-12-20 13:56:50 +01:00
|
|
|
if (maybe_ne (bitregion_end, 0U))
|
|
|
|
bitregion_end += 1;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if (reversep)
|
|
|
|
return NULL_TREE;
|
|
|
|
|
|
|
|
/* We do not want to rewrite TARGET_MEM_REFs. */
|
|
|
|
if (TREE_CODE (base_addr) == TARGET_MEM_REF)
|
|
|
|
return NULL_TREE;
|
|
|
|
/* In some cases get_inner_reference may return a
|
|
|
|
MEM_REF [ptr + byteoffset]. For the purposes of this pass
|
|
|
|
canonicalize the base_addr to MEM_REF [ptr] and take
|
|
|
|
byteoffset into account in the bitpos. This occurs in
|
|
|
|
PR 23684 and this way we can catch more chains. */
|
|
|
|
else if (TREE_CODE (base_addr) == MEM_REF)
|
|
|
|
{
|
2019-10-02 15:35:40 +02:00
|
|
|
if (!adjust_bit_pos (mem_ref_offset (base_addr), &bitpos,
|
|
|
|
&bitregion_start, &bitregion_end))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return NULL_TREE;
|
|
|
|
base_addr = TREE_OPERAND (base_addr, 0);
|
|
|
|
}
|
|
|
|
/* get_inner_reference returns the base object, get at its
|
|
|
|
address now. */
|
|
|
|
else
|
|
|
|
{
|
2017-12-20 13:56:50 +01:00
|
|
|
if (maybe_lt (bitpos, 0))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return NULL_TREE;
|
|
|
|
base_addr = build_fold_addr_expr (base_addr);
|
|
|
|
}
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (offset)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
/* If the access is variable offset then a base decl has to be
|
|
|
|
address-taken to be able to emit pointer-based stores to it.
|
|
|
|
??? We might be able to get away with re-using the original
|
|
|
|
base up to the first variable part and then wrapping that inside
|
|
|
|
a BIT_FIELD_REF. */
|
|
|
|
tree base = get_base_address (base_addr);
|
2019-10-02 15:35:40 +02:00
|
|
|
if (!base || (DECL_P (base) && !TREE_ADDRESSABLE (base)))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return NULL_TREE;
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
/* Similarly to above for the base, remove constant from the offset. */
|
|
|
|
if (TREE_CODE (offset) == PLUS_EXPR
|
|
|
|
&& TREE_CODE (TREE_OPERAND (offset, 1)) == INTEGER_CST
|
|
|
|
&& adjust_bit_pos (wi::to_poly_offset (TREE_OPERAND (offset, 1)),
|
|
|
|
&bitpos, &bitregion_start, &bitregion_end))
|
|
|
|
offset = TREE_OPERAND (offset, 0);
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
base_addr = build2 (POINTER_PLUS_EXPR, TREE_TYPE (base_addr),
|
|
|
|
base_addr, offset);
|
|
|
|
}
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (known_eq (bitregion_end, 0U))
|
|
|
|
{
|
|
|
|
bitregion_start = round_down_to_byte_boundary (bitpos);
|
|
|
|
bitregion_end = round_up_to_byte_boundary (bitpos + bitsize);
|
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
*pbitsize = bitsize;
|
|
|
|
*pbitpos = bitpos;
|
|
|
|
*pbitregion_start = bitregion_start;
|
|
|
|
*pbitregion_end = bitregion_end;
|
|
|
|
return base_addr;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Return true if STMT is a load that can be used for store merging.
|
|
|
|
In that case fill in *OP. BITSIZE, BITPOS, BITREGION_START and
|
|
|
|
BITREGION_END are properties of the corresponding store. */
|
|
|
|
|
|
|
|
static bool
|
|
|
|
handled_load (gimple *stmt, store_operand_info *op,
|
2017-12-20 13:56:50 +01:00
|
|
|
poly_uint64 bitsize, poly_uint64 bitpos,
|
|
|
|
poly_uint64 bitregion_start, poly_uint64 bitregion_end)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
if (!is_gimple_assign (stmt))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return false;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
if (gimple_assign_rhs_code (stmt) == BIT_NOT_EXPR)
|
|
|
|
{
|
|
|
|
tree rhs1 = gimple_assign_rhs1 (stmt);
|
|
|
|
if (TREE_CODE (rhs1) == SSA_NAME
|
|
|
|
&& handled_load (SSA_NAME_DEF_STMT (rhs1), op, bitsize, bitpos,
|
|
|
|
bitregion_start, bitregion_end))
|
|
|
|
{
|
2017-11-09 20:24:59 +01:00
|
|
|
/* Don't allow _1 = load; _2 = ~1; _3 = ~_2; which should have
|
|
|
|
been optimized earlier, but if allowed here, would confuse the
|
|
|
|
multiple uses counting. */
|
|
|
|
if (op->bit_not_p)
|
|
|
|
return false;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
op->bit_not_p = !op->bit_not_p;
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
if (gimple_vuse (stmt)
|
|
|
|
&& gimple_assign_load_p (stmt)
|
Add a fun parameter to three stmt_could_throw... functions
This long patch only does one simple thing, adds an explicit function
parameter to predicates stmt_could_throw_p, stmt_can_throw_external
and stmt_can_throw_internal.
My motivation was ability to use stmt_can_throw_external in IPA
analysis phase without the need to push cfun. As I have discovered,
we were already doing that in cgraph.c, which this patch avoids as
well. In the process, I had to add a struct function parameter to
stmt_could_throw_p and decided to also change the interface of
stmt_can_throw_internal just for the sake of some minimal consistency.
In the process I have discovered that calling method
cgraph_node::create_version_clone_with_body (used by ipa-split,
ipa-sra, OMP simd and multiple_target) leads to calls of
stmt_can_throw_external with NULL cfun. I have worked around this by
making stmt_can_throw_external and stmt_could_throw_p gracefully
accept NULL and just be pessimistic in that case. The problem with
fixing this in a better way is that struct function for the clone is
created after cloning edges where we attempt to push the yet not
existing cfun, and moving it before would require a bit of surgery in
tree-inline.c. A slightly hackish but simpler fix might be to
explicitely pass the "old" function to symbol_table::create_edge
because it should be just as good at that moment. In any event, that
is a topic for another patch.
I believe that currently we incorrectly use cfun in
maybe_clean_eh_stmt_fn and maybe_duplicate_eh_stmt_fn, both in
tree-eh.c, and so I have fixed these cases too. The bulk of other
changes is just mechanical adding of cfun to all users.
Bootstrapped and tested on x86_64-linux (also with extra NULLing and
restoring cfun to double check it is not used in a place I missed), OK
for trunk?
Thanks,
Martin
2018-10-22 Martin Jambor <mjambor@suse.cz>
* tree-eh.h (stmt_could_throw_p): Add function parameter.
(stmt_can_throw_external): Likewise.
(stmt_can_throw_internal): Likewise.
* tree-eh.c (lower_eh_constructs_2): Pass cfun to stmt_could_throw_p.
(lower_eh_constructs_2): Likewise.
(stmt_could_throw_p): Add fun parameter, use it instead of cfun.
(stmt_can_throw_external): Likewise.
(stmt_can_throw_internal): Likewise.
(maybe_clean_eh_stmt_fn): Pass cfun to stmt_could_throw_p.
(maybe_clean_or_replace_eh_stmt): Pass cfun to stmt_could_throw_p.
(maybe_duplicate_eh_stmt_fn): Pass new_fun to stmt_could_throw_p.
(maybe_duplicate_eh_stmt): Pass cfun to stmt_could_throw_p.
(pass_lower_eh_dispatch::execute): Pass cfun to
stmt_can_throw_external.
(cleanup_empty_eh): Likewise.
(verify_eh_edges): Pass cfun to stmt_could_throw_p.
* cgraph.c (cgraph_edge::set_call_stmt): Pass a function to
stmt_can_throw_external instead of pushing it to cfun.
(symbol_table::create_edge): Likewise.
* gimple-fold.c (fold_builtin_atomic_compare_exchange): Pass cfun to
stmt_can_throw_internal.
* gimple-ssa-evrp.c (evrp_dom_walker::before_dom_children): Pass cfun
to stmt_could_throw_p.
* gimple-ssa-store-merging.c (handled_load): Pass cfun to
stmt_can_throw_internal.
(pass_store_merging::execute): Likewise.
* gimple-ssa-strength-reduction.c
(find_candidates_dom_walker::before_dom_children): Pass cfun to
stmt_could_throw_p.
* gimplify-me.c (gimple_regimplify_operands): Pass cfun to
stmt_can_throw_internal.
* ipa-pure-const.c (check_call): Pass cfun to stmt_could_throw_p and
to stmt_can_throw_external.
(check_stmt): Pass cfun to stmt_could_throw_p.
(check_stmt): Pass cfun to stmt_can_throw_external.
(pass_nothrow::execute): Likewise.
* trans-mem.c (expand_call_tm): Pass cfun to stmt_can_throw_internal.
* tree-cfg.c (is_ctrl_altering_stmt): Pass cfun to
stmt_can_throw_internal.
(verify_gimple_in_cfg): Pass cfun to stmt_could_throw_p.
(stmt_can_terminate_bb_p): Pass cfun to stmt_can_throw_external.
(gimple_purge_dead_eh_edges): Pass cfun to stmt_can_throw_internal.
* tree-complex.c (expand_complex_libcall): Pass cfun to
stmt_could_throw_p and to stmt_can_throw_internal.
(expand_complex_multiplication): Pass cfun to stmt_can_throw_internal.
* tree-inline.c (copy_edges_for_bb): Likewise.
(maybe_move_debug_stmts_to_successors): Likewise.
* tree-outof-ssa.c (ssa_is_replaceable_p): Pass cfun to
stmt_could_throw_p.
* tree-parloops.c (oacc_entry_exit_ok_1): Likewise.
* tree-sra.c (scan_function): Pass cfun to stmt_can_throw_external.
* tree-ssa-alias.c (stmt_kills_ref_p): Pass cfun to
stmt_can_throw_internal.
* tree-ssa-ccp.c (optimize_atomic_bit_test_and): Likewise.
* tree-ssa-dce.c (mark_stmt_if_obviously_necessary): Pass cfun to
stmt_could_throw_p.
(mark_aliased_reaching_defs_necessary_1): Pass cfun to
stmt_can_throw_internal.
* tree-ssa-forwprop.c (pass_forwprop::execute): Likewise.
* tree-ssa-loop-im.c (movement_possibility): Pass cfun to
stmt_could_throw_p.
* tree-ssa-loop-ivopts.c (find_givs_in_stmt_scev): Likewise.
(add_autoinc_candidates): Pass cfun to stmt_can_throw_internal.
* tree-ssa-math-opts.c (pass_cse_reciprocals::execute): Likewise.
(convert_mult_to_fma_1): Likewise.
(convert_to_divmod): Likewise.
* tree-ssa-phiprop.c (propagate_with_phi): Likewise.
* tree-ssa-pre.c (compute_avail): Pass cfun to stmt_could_throw_p.
* tree-ssa-propagate.c
(substitute_and_fold_dom_walker::before_dom_children): Likewise.
* tree-ssa-reassoc.c (suitable_cond_bb): Likewise.
(maybe_optimize_range_tests): Likewise.
(linearize_expr_tree): Likewise.
(reassociate_bb): Likewise.
* tree-ssa-sccvn.c (copy_reference_ops_from_call): Likewise.
* tree-ssa-scopedtables.c (hashable_expr_equal_p): Likewise.
* tree-ssa-strlen.c (adjust_last_stmt): Likewise.
(handle_char_store): Likewise.
* tree-vect-data-refs.c (vect_find_stmt_data_reference): Pass cfun to
stmt_can_throw_internal.
* tree-vect-patterns.c (check_bool_pattern): Pass cfun to
stmt_could_throw_p.
* tree-vect-stmts.c (vect_finish_stmt_generation_1): Likewise.
(vectorizable_call): Pass cfun to stmt_can_throw_internal.
(vectorizable_simd_clone_call): Likewise.
* value-prof.c (gimple_ic): Pass cfun to stmt_could_throw_p.
(gimple_stringop_fixed_value): Likewise.
From-SVN: r265372
2018-10-22 10:27:50 +02:00
|
|
|
&& !stmt_can_throw_internal (cfun, stmt)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
&& !gimple_has_volatile_ops (stmt))
|
|
|
|
{
|
|
|
|
tree mem = gimple_assign_rhs1 (stmt);
|
|
|
|
op->base_addr
|
|
|
|
= mem_valid_for_store_merging (mem, &op->bitsize, &op->bitpos,
|
|
|
|
&op->bitregion_start,
|
|
|
|
&op->bitregion_end);
|
|
|
|
if (op->base_addr != NULL_TREE
|
2017-12-20 13:56:50 +01:00
|
|
|
&& known_eq (op->bitsize, bitsize)
|
|
|
|
&& multiple_p (op->bitpos - bitpos, BITS_PER_UNIT)
|
|
|
|
&& known_ge (op->bitpos - op->bitregion_start,
|
|
|
|
bitpos - bitregion_start)
|
|
|
|
&& known_ge (op->bitregion_end - op->bitpos,
|
|
|
|
bitregion_end - bitpos))
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
op->stmt = stmt;
|
|
|
|
op->val = mem;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
op->bit_not_p = false;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
/* Return the index number of the landing pad for STMT, if any. */
|
|
|
|
|
|
|
|
static int
|
|
|
|
lp_nr_for_store (gimple *stmt)
|
|
|
|
{
|
|
|
|
if (!cfun->can_throw_non_call_exceptions || !cfun->eh)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
if (!stmt_could_throw_p (cfun, stmt))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
return lookup_stmt_eh_lp (stmt);
|
|
|
|
}
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* Record the store STMT for store merging optimization if it can be
|
2019-10-02 15:35:40 +02:00
|
|
|
optimized. Return true if any changes were made. */
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
bool
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
pass_store_merging::process_store (gimple *stmt)
|
|
|
|
{
|
|
|
|
tree lhs = gimple_assign_lhs (stmt);
|
|
|
|
tree rhs = gimple_assign_rhs1 (stmt);
|
2020-05-05 19:44:19 +02:00
|
|
|
poly_uint64 bitsize, bitpos = 0;
|
|
|
|
poly_uint64 bitregion_start = 0, bitregion_end = 0;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
tree base_addr
|
|
|
|
= mem_valid_for_store_merging (lhs, &bitsize, &bitpos,
|
|
|
|
&bitregion_start, &bitregion_end);
|
2017-12-20 13:56:50 +01:00
|
|
|
if (known_eq (bitsize, 0U))
|
2019-10-02 15:35:40 +02:00
|
|
|
return false;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
|
|
|
bool invalid = (base_addr == NULL_TREE
|
2017-12-20 13:56:50 +01:00
|
|
|
|| (maybe_gt (bitsize,
|
|
|
|
(unsigned int) MAX_BITSIZE_MODE_ANY_INT)
|
2019-05-06 23:50:14 +02:00
|
|
|
&& TREE_CODE (rhs) != INTEGER_CST
|
|
|
|
&& (TREE_CODE (rhs) != CONSTRUCTOR
|
|
|
|
|| CONSTRUCTOR_NELTS (rhs) != 0)));
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
enum tree_code rhs_code = ERROR_MARK;
|
2017-11-09 20:24:59 +01:00
|
|
|
bool bit_not_p = false;
|
2017-11-20 11:10:23 +01:00
|
|
|
struct symbolic_number n;
|
|
|
|
gimple *ins_stmt = NULL;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
store_operand_info ops[2];
|
|
|
|
if (invalid)
|
|
|
|
;
|
2020-07-03 18:10:25 +02:00
|
|
|
else if (TREE_CODE (rhs) == STRING_CST)
|
|
|
|
{
|
|
|
|
rhs_code = STRING_CST;
|
|
|
|
ops[0].val = rhs;
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
else if (rhs_valid_for_store_merging_p (rhs))
|
|
|
|
{
|
|
|
|
rhs_code = INTEGER_CST;
|
|
|
|
ops[0].val = rhs;
|
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
else if (TREE_CODE (rhs) == SSA_NAME)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
gimple *def_stmt = SSA_NAME_DEF_STMT (rhs), *def_stmt1, *def_stmt2;
|
|
|
|
if (!is_gimple_assign (def_stmt))
|
|
|
|
invalid = true;
|
|
|
|
else if (handled_load (def_stmt, &ops[0], bitsize, bitpos,
|
|
|
|
bitregion_start, bitregion_end))
|
|
|
|
rhs_code = MEM_REF;
|
2017-11-09 20:24:59 +01:00
|
|
|
else if (gimple_assign_rhs_code (def_stmt) == BIT_NOT_EXPR)
|
|
|
|
{
|
|
|
|
tree rhs1 = gimple_assign_rhs1 (def_stmt);
|
|
|
|
if (TREE_CODE (rhs1) == SSA_NAME
|
|
|
|
&& is_gimple_assign (SSA_NAME_DEF_STMT (rhs1)))
|
|
|
|
{
|
|
|
|
bit_not_p = true;
|
|
|
|
def_stmt = SSA_NAME_DEF_STMT (rhs1);
|
|
|
|
}
|
|
|
|
}
|
2018-06-01 23:56:17 +02:00
|
|
|
|
2017-11-09 20:24:59 +01:00
|
|
|
if (rhs_code == ERROR_MARK && !invalid)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
switch ((rhs_code = gimple_assign_rhs_code (def_stmt)))
|
|
|
|
{
|
|
|
|
case BIT_AND_EXPR:
|
|
|
|
case BIT_IOR_EXPR:
|
|
|
|
case BIT_XOR_EXPR:
|
|
|
|
tree rhs1, rhs2;
|
|
|
|
rhs1 = gimple_assign_rhs1 (def_stmt);
|
|
|
|
rhs2 = gimple_assign_rhs2 (def_stmt);
|
|
|
|
invalid = true;
|
2017-11-09 14:08:41 +01:00
|
|
|
if (TREE_CODE (rhs1) != SSA_NAME)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
break;
|
|
|
|
def_stmt1 = SSA_NAME_DEF_STMT (rhs1);
|
|
|
|
if (!is_gimple_assign (def_stmt1)
|
|
|
|
|| !handled_load (def_stmt1, &ops[0], bitsize, bitpos,
|
|
|
|
bitregion_start, bitregion_end))
|
|
|
|
break;
|
|
|
|
if (rhs_valid_for_store_merging_p (rhs2))
|
|
|
|
ops[1].val = rhs2;
|
2017-11-09 14:08:41 +01:00
|
|
|
else if (TREE_CODE (rhs2) != SSA_NAME)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
break;
|
|
|
|
else
|
|
|
|
{
|
|
|
|
def_stmt2 = SSA_NAME_DEF_STMT (rhs2);
|
|
|
|
if (!is_gimple_assign (def_stmt2))
|
|
|
|
break;
|
|
|
|
else if (!handled_load (def_stmt2, &ops[1], bitsize, bitpos,
|
|
|
|
bitregion_start, bitregion_end))
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
invalid = false;
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
invalid = true;
|
|
|
|
break;
|
|
|
|
}
|
2018-06-01 23:56:17 +02:00
|
|
|
|
2017-12-20 13:56:50 +01:00
|
|
|
unsigned HOST_WIDE_INT const_bitsize;
|
|
|
|
if (bitsize.is_constant (&const_bitsize)
|
2018-06-01 23:56:17 +02:00
|
|
|
&& (const_bitsize % BITS_PER_UNIT) == 0
|
2017-12-20 13:56:50 +01:00
|
|
|
&& const_bitsize <= 64
|
2018-06-01 23:56:17 +02:00
|
|
|
&& multiple_p (bitpos, BITS_PER_UNIT))
|
2017-11-20 11:10:23 +01:00
|
|
|
{
|
|
|
|
ins_stmt = find_bswap_or_nop_1 (def_stmt, &n, 12);
|
|
|
|
if (ins_stmt)
|
|
|
|
{
|
|
|
|
uint64_t nn = n.n;
|
|
|
|
for (unsigned HOST_WIDE_INT i = 0;
|
2017-12-20 13:56:50 +01:00
|
|
|
i < const_bitsize;
|
|
|
|
i += BITS_PER_UNIT, nn >>= BITS_PER_MARKER)
|
2017-11-20 11:10:23 +01:00
|
|
|
if ((nn & MARKER_MASK) == 0
|
|
|
|
|| (nn & MARKER_MASK) == MARKER_BYTE_UNKNOWN)
|
|
|
|
{
|
|
|
|
ins_stmt = NULL;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
if (ins_stmt)
|
|
|
|
{
|
|
|
|
if (invalid)
|
|
|
|
{
|
|
|
|
rhs_code = LROTATE_EXPR;
|
|
|
|
ops[0].base_addr = NULL_TREE;
|
|
|
|
ops[1].base_addr = NULL_TREE;
|
|
|
|
}
|
|
|
|
invalid = false;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2018-06-01 23:56:17 +02:00
|
|
|
|
|
|
|
if (invalid
|
|
|
|
&& bitsize.is_constant (&const_bitsize)
|
|
|
|
&& ((const_bitsize % BITS_PER_UNIT) != 0
|
|
|
|
|| !multiple_p (bitpos, BITS_PER_UNIT))
|
2020-05-25 22:13:11 +02:00
|
|
|
&& const_bitsize <= MAX_FIXED_MODE_SIZE)
|
2018-06-01 23:56:17 +02:00
|
|
|
{
|
2018-06-03 13:51:10 +02:00
|
|
|
/* Bypass a conversion to the bit-field type. */
|
2018-06-06 15:13:24 +02:00
|
|
|
if (!bit_not_p
|
|
|
|
&& is_gimple_assign (def_stmt)
|
|
|
|
&& CONVERT_EXPR_CODE_P (rhs_code))
|
2018-06-01 23:56:17 +02:00
|
|
|
{
|
|
|
|
tree rhs1 = gimple_assign_rhs1 (def_stmt);
|
|
|
|
if (TREE_CODE (rhs1) == SSA_NAME
|
2018-06-03 13:51:10 +02:00
|
|
|
&& INTEGRAL_TYPE_P (TREE_TYPE (rhs1)))
|
2018-06-01 23:56:17 +02:00
|
|
|
rhs = rhs1;
|
|
|
|
}
|
|
|
|
rhs_code = BIT_INSERT_EXPR;
|
2018-06-06 15:13:24 +02:00
|
|
|
bit_not_p = false;
|
2018-06-01 23:56:17 +02:00
|
|
|
ops[0].val = rhs;
|
|
|
|
ops[0].base_addr = NULL_TREE;
|
|
|
|
ops[1].base_addr = NULL_TREE;
|
|
|
|
invalid = false;
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
2020-07-03 18:10:25 +02:00
|
|
|
else
|
|
|
|
invalid = true;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
2017-12-20 13:56:50 +01:00
|
|
|
unsigned HOST_WIDE_INT const_bitsize, const_bitpos;
|
|
|
|
unsigned HOST_WIDE_INT const_bitregion_start, const_bitregion_end;
|
|
|
|
if (invalid
|
|
|
|
|| !bitsize.is_constant (&const_bitsize)
|
|
|
|
|| !bitpos.is_constant (&const_bitpos)
|
|
|
|
|| !bitregion_start.is_constant (&const_bitregion_start)
|
|
|
|
|| !bitregion_end.is_constant (&const_bitregion_end))
|
2019-10-02 15:35:40 +02:00
|
|
|
return terminate_all_aliasing_chains (NULL, stmt);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
if (!ins_stmt)
|
|
|
|
memset (&n, 0, sizeof (n));
|
|
|
|
|
2019-07-09 20:32:49 +02:00
|
|
|
class imm_store_chain_info **chain_info = NULL;
|
2019-10-02 15:35:40 +02:00
|
|
|
bool ret = false;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c (struct store_operand_info): Add bit_not_p
data member.
(store_operand_info::store_operand_info): Initialize it to false.
(pass_store_merging::terminate_all_aliasing_chains): Rewritten to use
ref_maybe_used_by_stmt_p and stmt_may_clobber_ref_p on lhs of each
store in the group, and if chain_info is non-NULL, to ignore altogether
that chain.
(compatible_load_p): Fail if bit_not_p does not match.
(imm_store_chain_info::output_merged_store): Handle bit_not_p loads.
(handled_load): Fill in bit_not_p. Handle BIT_NOT_EXPR.
(pass_store_merging::process_store): Adjust
terminate_all_aliasing_chains calls to pass NULL in all current spots,
call terminate_all_aliasing_chains newly when adding a store into
a chain with non-NULL chain_info.
* gcc.dg/store_merging_2.c: Expect 3 store mergings instead of 2.
* gcc.dg/store_merging_13.c (f7, f8, f9, f10, f11, f12, f13): New
functions.
(main): Test also those. Expect 13 store mergings instead of 6.
* gcc.dg/store_merging_14.c (f7, f8, f9): New functions.
(main): Test also those. Expect 9 store mergings instead of 6.
From-SVN: r254536
2017-11-08 16:46:58 +01:00
|
|
|
if (base_addr)
|
|
|
|
chain_info = m_stores.get (base_addr);
|
|
|
|
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
store_immediate_info *info;
|
|
|
|
if (chain_info)
|
|
|
|
{
|
|
|
|
unsigned int ord = (*chain_info)->m_store_info.length ();
|
2017-12-20 13:56:50 +01:00
|
|
|
info = new store_immediate_info (const_bitsize, const_bitpos,
|
|
|
|
const_bitregion_start,
|
|
|
|
const_bitregion_end,
|
|
|
|
stmt, ord, rhs_code, n, ins_stmt,
|
2019-10-02 15:35:40 +02:00
|
|
|
bit_not_p, lp_nr_for_store (stmt),
|
|
|
|
ops[0], ops[1]);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
{
|
|
|
|
fprintf (dump_file, "Recording immediate store from stmt:\n");
|
|
|
|
print_gimple_stmt (dump_file, stmt, 0);
|
|
|
|
}
|
|
|
|
(*chain_info)->m_store_info.safe_push (info);
|
2021-02-11 11:13:47 +01:00
|
|
|
m_n_stores++;
|
2019-10-02 15:35:40 +02:00
|
|
|
ret |= terminate_all_aliasing_chains (chain_info, stmt);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
/* If we reach the limit of stores to merge in a chain terminate and
|
|
|
|
process the chain now. */
|
|
|
|
if ((*chain_info)->m_store_info.length ()
|
2019-11-12 11:08:40 +01:00
|
|
|
== (unsigned int) param_max_stores_to_merge)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fprintf (dump_file,
|
|
|
|
"Reached maximum number of statements to merge:\n");
|
2019-10-02 15:35:40 +02:00
|
|
|
ret |= terminate_and_process_chain (*chain_info);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
|
|
|
}
|
2021-02-11 11:13:47 +01:00
|
|
|
else
|
|
|
|
{
|
|
|
|
/* Store aliases any existing chain? */
|
|
|
|
ret |= terminate_all_aliasing_chains (NULL, stmt);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
|
2021-02-11 11:13:47 +01:00
|
|
|
/* Start a new chain. */
|
|
|
|
class imm_store_chain_info *new_chain
|
|
|
|
= new imm_store_chain_info (m_stores_head, base_addr);
|
|
|
|
info = new store_immediate_info (const_bitsize, const_bitpos,
|
|
|
|
const_bitregion_start,
|
|
|
|
const_bitregion_end,
|
|
|
|
stmt, 0, rhs_code, n, ins_stmt,
|
|
|
|
bit_not_p, lp_nr_for_store (stmt),
|
|
|
|
ops[0], ops[1]);
|
|
|
|
new_chain->m_store_info.safe_push (info);
|
|
|
|
m_n_stores++;
|
|
|
|
m_stores.put (base_addr, new_chain);
|
|
|
|
m_n_chains++;
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
{
|
|
|
|
fprintf (dump_file, "Starting active chain number %u with statement:\n",
|
|
|
|
m_n_chains);
|
|
|
|
print_gimple_stmt (dump_file, stmt, 0);
|
|
|
|
fprintf (dump_file, "The base object is:\n");
|
|
|
|
print_generic_expr (dump_file, base_addr);
|
|
|
|
fprintf (dump_file, "\n");
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Prune oldest chains so that after adding the chain or store above
|
|
|
|
we're again within the limits set by the params. */
|
|
|
|
if (m_n_chains > (unsigned)param_max_store_chains_to_track
|
|
|
|
|| m_n_stores > (unsigned)param_max_stores_to_track)
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
{
|
2021-02-11 11:13:47 +01:00
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fprintf (dump_file, "Too many chains (%u > %d) or stores (%u > %d), "
|
|
|
|
"terminating oldest chain(s).\n", m_n_chains,
|
|
|
|
param_max_store_chains_to_track, m_n_stores,
|
|
|
|
param_max_stores_to_track);
|
|
|
|
imm_store_chain_info **e = &m_stores_head;
|
|
|
|
unsigned idx = 0;
|
|
|
|
unsigned n_stores = 0;
|
|
|
|
while (*e)
|
|
|
|
{
|
|
|
|
if (idx >= (unsigned)param_max_store_chains_to_track
|
|
|
|
|| (n_stores + (*e)->m_store_info.length ()
|
|
|
|
> (unsigned)param_max_stores_to_track))
|
2021-02-22 08:36:55 +01:00
|
|
|
ret |= terminate_and_process_chain (*e);
|
2021-02-11 11:13:47 +01:00
|
|
|
else
|
|
|
|
{
|
|
|
|
n_stores += (*e)->m_store_info.length ();
|
|
|
|
e = &(*e)->next;
|
|
|
|
++idx;
|
|
|
|
}
|
|
|
|
}
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
2021-02-11 11:13:47 +01:00
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Return true if STMT is a store valid for store merging. */
|
|
|
|
|
|
|
|
static bool
|
|
|
|
store_valid_for_store_merging_p (gimple *stmt)
|
|
|
|
{
|
|
|
|
return gimple_assign_single_p (stmt)
|
|
|
|
&& gimple_vdef (stmt)
|
|
|
|
&& lhs_valid_for_store_merging_p (gimple_assign_lhs (stmt))
|
2019-11-08 11:53:50 +01:00
|
|
|
&& (!gimple_has_volatile_ops (stmt) || gimple_clobber_p (stmt));
|
2019-10-02 15:35:40 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
enum basic_block_status { BB_INVALID, BB_VALID, BB_EXTENDED_VALID };
|
|
|
|
|
|
|
|
/* Return the status of basic block BB wrt store merging. */
|
|
|
|
|
|
|
|
static enum basic_block_status
|
|
|
|
get_status_for_store_merging (basic_block bb)
|
|
|
|
{
|
|
|
|
unsigned int num_statements = 0;
|
2021-01-05 16:16:06 +01:00
|
|
|
unsigned int num_constructors = 0;
|
2019-10-02 15:35:40 +02:00
|
|
|
gimple_stmt_iterator gsi;
|
|
|
|
edge e;
|
|
|
|
|
|
|
|
for (gsi = gsi_after_labels (bb); !gsi_end_p (gsi); gsi_next (&gsi))
|
|
|
|
{
|
|
|
|
gimple *stmt = gsi_stmt (gsi);
|
|
|
|
|
|
|
|
if (is_gimple_debug (stmt))
|
|
|
|
continue;
|
|
|
|
|
|
|
|
if (store_valid_for_store_merging_p (stmt) && ++num_statements >= 2)
|
|
|
|
break;
|
2021-01-05 16:16:06 +01:00
|
|
|
|
|
|
|
if (is_gimple_assign (stmt)
|
|
|
|
&& gimple_assign_rhs_code (stmt) == CONSTRUCTOR)
|
|
|
|
{
|
|
|
|
tree rhs = gimple_assign_rhs1 (stmt);
|
|
|
|
if (VECTOR_TYPE_P (TREE_TYPE (rhs))
|
|
|
|
&& INTEGRAL_TYPE_P (TREE_TYPE (TREE_TYPE (rhs)))
|
|
|
|
&& gimple_assign_lhs (stmt) != NULL_TREE)
|
|
|
|
{
|
|
|
|
HOST_WIDE_INT sz
|
|
|
|
= int_size_in_bytes (TREE_TYPE (rhs)) * BITS_PER_UNIT;
|
|
|
|
if (sz == 16 || sz == 32 || sz == 64)
|
|
|
|
{
|
|
|
|
num_constructors = 1;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2019-10-02 15:35:40 +02:00
|
|
|
}
|
|
|
|
|
2021-01-05 16:16:06 +01:00
|
|
|
if (num_statements == 0 && num_constructors == 0)
|
2019-10-02 15:35:40 +02:00
|
|
|
return BB_INVALID;
|
|
|
|
|
|
|
|
if (cfun->can_throw_non_call_exceptions && cfun->eh
|
|
|
|
&& store_valid_for_store_merging_p (gimple_seq_last_stmt (bb_seq (bb)))
|
|
|
|
&& (e = find_fallthru_edge (bb->succs))
|
|
|
|
&& e->dest == bb->next_bb)
|
|
|
|
return BB_EXTENDED_VALID;
|
|
|
|
|
2021-01-05 16:16:06 +01:00
|
|
|
return (num_statements >= 2 || num_constructors) ? BB_VALID : BB_INVALID;
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
/* Entry point for the pass. Go over each basic block recording chains of
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
immediate stores. Upon encountering a terminating statement (as defined
|
|
|
|
by stmt_terminates_chain_p) process the recorded stores and emit the widened
|
|
|
|
variants. */
|
2016-10-28 16:18:50 +02:00
|
|
|
|
|
|
|
unsigned int
|
|
|
|
pass_store_merging::execute (function *fun)
|
|
|
|
{
|
|
|
|
basic_block bb;
|
|
|
|
hash_set<gimple *> orig_stmts;
|
2019-10-02 15:35:40 +02:00
|
|
|
bool changed = false, open_chains = false;
|
|
|
|
|
|
|
|
/* If the function can throw and catch non-call exceptions, we'll be trying
|
|
|
|
to merge stores across different basic blocks so we need to first unsplit
|
|
|
|
the EH edges in order to streamline the CFG of the function. */
|
|
|
|
if (cfun->can_throw_non_call_exceptions && cfun->eh)
|
|
|
|
unsplit_eh_edges ();
|
2016-10-28 16:18:50 +02:00
|
|
|
|
2017-11-20 11:10:23 +01:00
|
|
|
calculate_dominance_info (CDI_DOMINATORS);
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
FOR_EACH_BB_FN (bb, fun)
|
|
|
|
{
|
2019-10-02 15:35:40 +02:00
|
|
|
const basic_block_status bb_status = get_status_for_store_merging (bb);
|
2016-10-28 16:18:50 +02:00
|
|
|
gimple_stmt_iterator gsi;
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (open_chains && (bb_status == BB_INVALID || !single_pred_p (bb)))
|
|
|
|
{
|
|
|
|
changed |= terminate_and_process_all_chains ();
|
|
|
|
open_chains = false;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (bb_status == BB_INVALID)
|
2016-10-28 16:18:50 +02:00
|
|
|
continue;
|
|
|
|
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fprintf (dump_file, "Processing basic block <%d>:\n", bb->index);
|
|
|
|
|
2021-01-05 16:16:06 +01:00
|
|
|
for (gsi = gsi_after_labels (bb); !gsi_end_p (gsi); )
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
gimple *stmt = gsi_stmt (gsi);
|
2021-01-05 16:16:06 +01:00
|
|
|
gsi_next (&gsi);
|
2016-10-28 16:18:50 +02:00
|
|
|
|
2017-03-17 00:31:01 +01:00
|
|
|
if (is_gimple_debug (stmt))
|
|
|
|
continue;
|
|
|
|
|
2019-11-08 11:53:50 +01:00
|
|
|
if (gimple_has_volatile_ops (stmt) && !gimple_clobber_p (stmt))
|
2016-10-28 16:18:50 +02:00
|
|
|
{
|
|
|
|
/* Terminate all chains. */
|
|
|
|
if (dump_file && (dump_flags & TDF_DETAILS))
|
|
|
|
fprintf (dump_file, "Volatile access terminates "
|
|
|
|
"all chains\n");
|
2019-10-02 15:35:40 +02:00
|
|
|
changed |= terminate_and_process_all_chains ();
|
|
|
|
open_chains = false;
|
2016-10-28 16:18:50 +02:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
2021-01-05 16:16:06 +01:00
|
|
|
if (is_gimple_assign (stmt)
|
|
|
|
&& gimple_assign_rhs_code (stmt) == CONSTRUCTOR
|
|
|
|
&& maybe_optimize_vector_constructor (stmt))
|
|
|
|
continue;
|
|
|
|
|
2019-10-02 15:35:40 +02:00
|
|
|
if (store_valid_for_store_merging_p (stmt))
|
|
|
|
changed |= process_store (stmt);
|
re PR tree-optimization/78821 (GCC7: Copying whole 32 bits structure field by field not optimised into copying whole 32 bits at once)
PR tree-optimization/78821
* gimple-ssa-store-merging.c: Update the file comment.
(MAX_STORE_ALIAS_CHECKS): Define.
(struct store_operand_info): New type.
(store_operand_info::store_operand_info): New constructor.
(struct store_immediate_info): Add rhs_code and ops data members.
(store_immediate_info::store_immediate_info): Add rhscode, op0r
and op1r arguments to the ctor, initialize corresponding data members.
(struct merged_store_group): Add load_align_base and load_align
data members.
(merged_store_group::merged_store_group): Initialize them.
(merged_store_group::do_merge): Update them.
(merged_store_group::apply_stores): Pick the constant for
encode_tree_to_bitpos from one of the two operands, or skip
encode_tree_to_bitpos if neither operand is a constant.
(class pass_store_merging): Add process_store method decl. Remove
bool argument from terminate_all_aliasing_chains method decl.
(pass_store_merging::terminate_all_aliasing_chains): Remove
var_offset_p argument and corresponding handling.
(stmts_may_clobber_ref_p): New function.
(compatible_load_p): New function.
(imm_store_chain_info::coalesce_immediate_stores): Terminate group
if there is overlap and rhs_code is not INTEGER_CST. For
non-overlapping stores terminate group if rhs is not mergeable.
(get_alias_type_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &. Add IS_LOAD, CLIQUEP and
BASEP arguments. If IS_LOAD is true, look at rhs1 of the stmts
instead of lhs. Compute *CLIQUEP and *BASEP in addition to the
alias type.
(get_location_for_stmts): Change first argument from
auto_vec<gimple *> & to vec<gimple *> &.
(struct split_store): Remove orig_stmts data member, add orig_stores.
(split_store::split_store): Create orig_stores rather than orig_stmts.
(find_constituent_stmts): Renamed to ...
(find_constituent_stores): ... this. Change second argument from
vec<gimple *> * to vec<store_immediate_info *> *, push pointers
to info structures rather than the statements.
(split_group): Rename ALLOW_UNALIGNED argument to
ALLOW_UNALIGNED_STORE, add ALLOW_UNALIGNED_LOAD argument and handle
it. Adjust find_constituent_stores caller.
(imm_store_chain_info::output_merged_store): Handle rhs_code other
than INTEGER_CST, adjust split_group, get_alias_type_for_stmts and
get_location_for_stmts callers. Set MR_DEPENDENCE_CLIQUE and
MR_DEPENDENCE_BASE on the MEM_REFs if they are the same in all stores.
(mem_valid_for_store_merging): New function.
(handled_load): New function.
(pass_store_merging::process_store): New method.
(pass_store_merging::execute): Use process_store method. Adjust
terminate_all_aliasing_chains caller.
* gcc.dg/store_merging_13.c: New test.
* gcc.dg/store_merging_14.c: New test.
From-SVN: r254391
2017-11-03 20:08:25 +01:00
|
|
|
else
|
2019-10-02 15:35:40 +02:00
|
|
|
changed |= terminate_all_aliasing_chains (NULL, stmt);
|
|
|
|
}
|
|
|
|
|
|
|
|
if (bb_status == BB_EXTENDED_VALID)
|
|
|
|
open_chains = true;
|
|
|
|
else
|
|
|
|
{
|
|
|
|
changed |= terminate_and_process_all_chains ();
|
|
|
|
open_chains = false;
|
2016-10-28 16:18:50 +02:00
|
|
|
}
|
|
|
|
}
|
2019-10-02 15:35:40 +02:00
|
|
|
|
|
|
|
if (open_chains)
|
|
|
|
changed |= terminate_and_process_all_chains ();
|
|
|
|
|
|
|
|
/* If the function can throw and catch non-call exceptions and something
|
|
|
|
changed during the pass, then the CFG has (very likely) changed too. */
|
|
|
|
if (cfun->can_throw_non_call_exceptions && cfun->eh && changed)
|
|
|
|
{
|
|
|
|
free_dominance_info (CDI_DOMINATORS);
|
|
|
|
return TODO_cleanup_cfg;
|
|
|
|
}
|
|
|
|
|
2016-10-28 16:18:50 +02:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
} // anon namespace
|
|
|
|
|
|
|
|
/* Construct and return a store merging pass object. */
|
|
|
|
|
|
|
|
gimple_opt_pass *
|
|
|
|
make_pass_store_merging (gcc::context *ctxt)
|
|
|
|
{
|
|
|
|
return new pass_store_merging (ctxt);
|
|
|
|
}
|
2016-11-08 17:04:20 +01:00
|
|
|
|
|
|
|
#if CHECKING_P
|
|
|
|
|
|
|
|
namespace selftest {
|
|
|
|
|
|
|
|
/* Selftests for store merging helpers. */
|
|
|
|
|
|
|
|
/* Assert that all elements of the byte arrays X and Y, both of length N
|
|
|
|
are equal. */
|
|
|
|
|
|
|
|
static void
|
|
|
|
verify_array_eq (unsigned char *x, unsigned char *y, unsigned int n)
|
|
|
|
{
|
|
|
|
for (unsigned int i = 0; i < n; i++)
|
|
|
|
{
|
|
|
|
if (x[i] != y[i])
|
|
|
|
{
|
|
|
|
fprintf (stderr, "Arrays do not match. X:\n");
|
|
|
|
dump_char_array (stderr, x, n);
|
|
|
|
fprintf (stderr, "Y:\n");
|
|
|
|
dump_char_array (stderr, y, n);
|
|
|
|
}
|
|
|
|
ASSERT_EQ (x[i], y[i]);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
sccvn: Handle bitfields in vn_reference_lookup_3 [PR93582]
The following patch is first step towards fixing PR93582.
vn_reference_lookup_3 right now punts on anything that isn't byte aligned,
so to be able to lookup a constant bitfield store, one needs to use
the exact same COMPONENT_REF, otherwise it isn't found.
This patch lifts up that that restriction if the bits to be loaded are
covered by a single store of a constant (keeps the restriction so far
for the multiple store case, can tweak that incrementally, but I think
for bisection etc. it is worth to do it one step at a time).
2020-02-13 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93582
* fold-const.h (shift_bytes_in_array_left,
shift_bytes_in_array_right): Declare.
* fold-const.c (shift_bytes_in_array_left,
shift_bytes_in_array_right): New function, moved from
gimple-ssa-store-merging.c, no longer static.
* gimple-ssa-store-merging.c (shift_bytes_in_array): Move
to gimple-ssa-store-merging.c and rename to shift_bytes_in_array_left.
(shift_bytes_in_array_right): Move to gimple-ssa-store-merging.c.
(encode_tree_to_bitpos): Use shift_bytes_in_array_left instead of
shift_bytes_in_array.
(verify_shift_bytes_in_array): Rename to ...
(verify_shift_bytes_in_array_left): ... this. Use
shift_bytes_in_array_left instead of shift_bytes_in_array.
(store_merging_c_tests): Call verify_shift_bytes_in_array_left
instead of verify_shift_bytes_in_array.
* tree-ssa-sccvn.c (vn_reference_lookup_3): For native_encode_expr
/ native_interpret_expr where the store covers all needed bits,
punt on PDP-endian, otherwise allow all involved offsets and sizes
not to be byte-aligned.
* gcc.dg/tree-ssa/pr93582-1.c: New test.
* gcc.dg/tree-ssa/pr93582-2.c: New test.
* gcc.dg/tree-ssa/pr93582-3.c: New test.
2020-02-13 10:04:11 +01:00
|
|
|
/* Test shift_bytes_in_array_left and that it carries bits across between
|
2016-11-08 17:04:20 +01:00
|
|
|
bytes correctly. */
|
|
|
|
|
|
|
|
static void
|
sccvn: Handle bitfields in vn_reference_lookup_3 [PR93582]
The following patch is first step towards fixing PR93582.
vn_reference_lookup_3 right now punts on anything that isn't byte aligned,
so to be able to lookup a constant bitfield store, one needs to use
the exact same COMPONENT_REF, otherwise it isn't found.
This patch lifts up that that restriction if the bits to be loaded are
covered by a single store of a constant (keeps the restriction so far
for the multiple store case, can tweak that incrementally, but I think
for bisection etc. it is worth to do it one step at a time).
2020-02-13 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93582
* fold-const.h (shift_bytes_in_array_left,
shift_bytes_in_array_right): Declare.
* fold-const.c (shift_bytes_in_array_left,
shift_bytes_in_array_right): New function, moved from
gimple-ssa-store-merging.c, no longer static.
* gimple-ssa-store-merging.c (shift_bytes_in_array): Move
to gimple-ssa-store-merging.c and rename to shift_bytes_in_array_left.
(shift_bytes_in_array_right): Move to gimple-ssa-store-merging.c.
(encode_tree_to_bitpos): Use shift_bytes_in_array_left instead of
shift_bytes_in_array.
(verify_shift_bytes_in_array): Rename to ...
(verify_shift_bytes_in_array_left): ... this. Use
shift_bytes_in_array_left instead of shift_bytes_in_array.
(store_merging_c_tests): Call verify_shift_bytes_in_array_left
instead of verify_shift_bytes_in_array.
* tree-ssa-sccvn.c (vn_reference_lookup_3): For native_encode_expr
/ native_interpret_expr where the store covers all needed bits,
punt on PDP-endian, otherwise allow all involved offsets and sizes
not to be byte-aligned.
* gcc.dg/tree-ssa/pr93582-1.c: New test.
* gcc.dg/tree-ssa/pr93582-2.c: New test.
* gcc.dg/tree-ssa/pr93582-3.c: New test.
2020-02-13 10:04:11 +01:00
|
|
|
verify_shift_bytes_in_array_left (void)
|
2016-11-08 17:04:20 +01:00
|
|
|
{
|
|
|
|
/* byte 1 | byte 0
|
|
|
|
00011111 | 11100000. */
|
|
|
|
unsigned char orig[2] = { 0xe0, 0x1f };
|
|
|
|
unsigned char in[2];
|
|
|
|
memcpy (in, orig, sizeof orig);
|
|
|
|
|
|
|
|
unsigned char expected[2] = { 0x80, 0x7f };
|
sccvn: Handle bitfields in vn_reference_lookup_3 [PR93582]
The following patch is first step towards fixing PR93582.
vn_reference_lookup_3 right now punts on anything that isn't byte aligned,
so to be able to lookup a constant bitfield store, one needs to use
the exact same COMPONENT_REF, otherwise it isn't found.
This patch lifts up that that restriction if the bits to be loaded are
covered by a single store of a constant (keeps the restriction so far
for the multiple store case, can tweak that incrementally, but I think
for bisection etc. it is worth to do it one step at a time).
2020-02-13 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93582
* fold-const.h (shift_bytes_in_array_left,
shift_bytes_in_array_right): Declare.
* fold-const.c (shift_bytes_in_array_left,
shift_bytes_in_array_right): New function, moved from
gimple-ssa-store-merging.c, no longer static.
* gimple-ssa-store-merging.c (shift_bytes_in_array): Move
to gimple-ssa-store-merging.c and rename to shift_bytes_in_array_left.
(shift_bytes_in_array_right): Move to gimple-ssa-store-merging.c.
(encode_tree_to_bitpos): Use shift_bytes_in_array_left instead of
shift_bytes_in_array.
(verify_shift_bytes_in_array): Rename to ...
(verify_shift_bytes_in_array_left): ... this. Use
shift_bytes_in_array_left instead of shift_bytes_in_array.
(store_merging_c_tests): Call verify_shift_bytes_in_array_left
instead of verify_shift_bytes_in_array.
* tree-ssa-sccvn.c (vn_reference_lookup_3): For native_encode_expr
/ native_interpret_expr where the store covers all needed bits,
punt on PDP-endian, otherwise allow all involved offsets and sizes
not to be byte-aligned.
* gcc.dg/tree-ssa/pr93582-1.c: New test.
* gcc.dg/tree-ssa/pr93582-2.c: New test.
* gcc.dg/tree-ssa/pr93582-3.c: New test.
2020-02-13 10:04:11 +01:00
|
|
|
shift_bytes_in_array_left (in, sizeof (in), 2);
|
2016-11-08 17:04:20 +01:00
|
|
|
verify_array_eq (in, expected, sizeof (in));
|
|
|
|
|
|
|
|
memcpy (in, orig, sizeof orig);
|
|
|
|
memcpy (expected, orig, sizeof orig);
|
|
|
|
/* Check that shifting by zero doesn't change anything. */
|
sccvn: Handle bitfields in vn_reference_lookup_3 [PR93582]
The following patch is first step towards fixing PR93582.
vn_reference_lookup_3 right now punts on anything that isn't byte aligned,
so to be able to lookup a constant bitfield store, one needs to use
the exact same COMPONENT_REF, otherwise it isn't found.
This patch lifts up that that restriction if the bits to be loaded are
covered by a single store of a constant (keeps the restriction so far
for the multiple store case, can tweak that incrementally, but I think
for bisection etc. it is worth to do it one step at a time).
2020-02-13 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93582
* fold-const.h (shift_bytes_in_array_left,
shift_bytes_in_array_right): Declare.
* fold-const.c (shift_bytes_in_array_left,
shift_bytes_in_array_right): New function, moved from
gimple-ssa-store-merging.c, no longer static.
* gimple-ssa-store-merging.c (shift_bytes_in_array): Move
to gimple-ssa-store-merging.c and rename to shift_bytes_in_array_left.
(shift_bytes_in_array_right): Move to gimple-ssa-store-merging.c.
(encode_tree_to_bitpos): Use shift_bytes_in_array_left instead of
shift_bytes_in_array.
(verify_shift_bytes_in_array): Rename to ...
(verify_shift_bytes_in_array_left): ... this. Use
shift_bytes_in_array_left instead of shift_bytes_in_array.
(store_merging_c_tests): Call verify_shift_bytes_in_array_left
instead of verify_shift_bytes_in_array.
* tree-ssa-sccvn.c (vn_reference_lookup_3): For native_encode_expr
/ native_interpret_expr where the store covers all needed bits,
punt on PDP-endian, otherwise allow all involved offsets and sizes
not to be byte-aligned.
* gcc.dg/tree-ssa/pr93582-1.c: New test.
* gcc.dg/tree-ssa/pr93582-2.c: New test.
* gcc.dg/tree-ssa/pr93582-3.c: New test.
2020-02-13 10:04:11 +01:00
|
|
|
shift_bytes_in_array_left (in, sizeof (in), 0);
|
2016-11-08 17:04:20 +01:00
|
|
|
verify_array_eq (in, expected, sizeof (in));
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Test shift_bytes_in_array_right and that it carries bits across between
|
|
|
|
bytes correctly. */
|
|
|
|
|
|
|
|
static void
|
|
|
|
verify_shift_bytes_in_array_right (void)
|
|
|
|
{
|
|
|
|
/* byte 1 | byte 0
|
|
|
|
00011111 | 11100000. */
|
|
|
|
unsigned char orig[2] = { 0x1f, 0xe0};
|
|
|
|
unsigned char in[2];
|
|
|
|
memcpy (in, orig, sizeof orig);
|
|
|
|
unsigned char expected[2] = { 0x07, 0xf8};
|
|
|
|
shift_bytes_in_array_right (in, sizeof (in), 2);
|
|
|
|
verify_array_eq (in, expected, sizeof (in));
|
|
|
|
|
|
|
|
memcpy (in, orig, sizeof orig);
|
|
|
|
memcpy (expected, orig, sizeof orig);
|
|
|
|
/* Check that shifting by zero doesn't change anything. */
|
|
|
|
shift_bytes_in_array_right (in, sizeof (in), 0);
|
|
|
|
verify_array_eq (in, expected, sizeof (in));
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Test clear_bit_region that it clears exactly the bits asked and
|
|
|
|
nothing more. */
|
|
|
|
|
|
|
|
static void
|
|
|
|
verify_clear_bit_region (void)
|
|
|
|
{
|
|
|
|
/* Start with all bits set and test clearing various patterns in them. */
|
|
|
|
unsigned char orig[3] = { 0xff, 0xff, 0xff};
|
|
|
|
unsigned char in[3];
|
|
|
|
unsigned char expected[3];
|
|
|
|
memcpy (in, orig, sizeof in);
|
|
|
|
|
|
|
|
/* Check zeroing out all the bits. */
|
|
|
|
clear_bit_region (in, 0, 3 * BITS_PER_UNIT);
|
|
|
|
expected[0] = expected[1] = expected[2] = 0;
|
|
|
|
verify_array_eq (in, expected, sizeof in);
|
|
|
|
|
|
|
|
memcpy (in, orig, sizeof in);
|
|
|
|
/* Leave the first and last bits intact. */
|
|
|
|
clear_bit_region (in, 1, 3 * BITS_PER_UNIT - 2);
|
|
|
|
expected[0] = 0x1;
|
|
|
|
expected[1] = 0;
|
|
|
|
expected[2] = 0x80;
|
|
|
|
verify_array_eq (in, expected, sizeof in);
|
|
|
|
}
|
|
|
|
|
2019-11-08 11:53:50 +01:00
|
|
|
/* Test clear_bit_region_be that it clears exactly the bits asked and
|
2016-11-08 17:04:20 +01:00
|
|
|
nothing more. */
|
|
|
|
|
|
|
|
static void
|
|
|
|
verify_clear_bit_region_be (void)
|
|
|
|
{
|
|
|
|
/* Start with all bits set and test clearing various patterns in them. */
|
|
|
|
unsigned char orig[3] = { 0xff, 0xff, 0xff};
|
|
|
|
unsigned char in[3];
|
|
|
|
unsigned char expected[3];
|
|
|
|
memcpy (in, orig, sizeof in);
|
|
|
|
|
|
|
|
/* Check zeroing out all the bits. */
|
|
|
|
clear_bit_region_be (in, BITS_PER_UNIT - 1, 3 * BITS_PER_UNIT);
|
|
|
|
expected[0] = expected[1] = expected[2] = 0;
|
|
|
|
verify_array_eq (in, expected, sizeof in);
|
|
|
|
|
|
|
|
memcpy (in, orig, sizeof in);
|
|
|
|
/* Leave the first and last bits intact. */
|
|
|
|
clear_bit_region_be (in, BITS_PER_UNIT - 2, 3 * BITS_PER_UNIT - 2);
|
|
|
|
expected[0] = 0x80;
|
|
|
|
expected[1] = 0;
|
|
|
|
expected[2] = 0x1;
|
|
|
|
verify_array_eq (in, expected, sizeof in);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/* Run all of the selftests within this file. */
|
|
|
|
|
|
|
|
void
|
|
|
|
store_merging_c_tests (void)
|
|
|
|
{
|
sccvn: Handle bitfields in vn_reference_lookup_3 [PR93582]
The following patch is first step towards fixing PR93582.
vn_reference_lookup_3 right now punts on anything that isn't byte aligned,
so to be able to lookup a constant bitfield store, one needs to use
the exact same COMPONENT_REF, otherwise it isn't found.
This patch lifts up that that restriction if the bits to be loaded are
covered by a single store of a constant (keeps the restriction so far
for the multiple store case, can tweak that incrementally, but I think
for bisection etc. it is worth to do it one step at a time).
2020-02-13 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/93582
* fold-const.h (shift_bytes_in_array_left,
shift_bytes_in_array_right): Declare.
* fold-const.c (shift_bytes_in_array_left,
shift_bytes_in_array_right): New function, moved from
gimple-ssa-store-merging.c, no longer static.
* gimple-ssa-store-merging.c (shift_bytes_in_array): Move
to gimple-ssa-store-merging.c and rename to shift_bytes_in_array_left.
(shift_bytes_in_array_right): Move to gimple-ssa-store-merging.c.
(encode_tree_to_bitpos): Use shift_bytes_in_array_left instead of
shift_bytes_in_array.
(verify_shift_bytes_in_array): Rename to ...
(verify_shift_bytes_in_array_left): ... this. Use
shift_bytes_in_array_left instead of shift_bytes_in_array.
(store_merging_c_tests): Call verify_shift_bytes_in_array_left
instead of verify_shift_bytes_in_array.
* tree-ssa-sccvn.c (vn_reference_lookup_3): For native_encode_expr
/ native_interpret_expr where the store covers all needed bits,
punt on PDP-endian, otherwise allow all involved offsets and sizes
not to be byte-aligned.
* gcc.dg/tree-ssa/pr93582-1.c: New test.
* gcc.dg/tree-ssa/pr93582-2.c: New test.
* gcc.dg/tree-ssa/pr93582-3.c: New test.
2020-02-13 10:04:11 +01:00
|
|
|
verify_shift_bytes_in_array_left ();
|
2016-11-08 17:04:20 +01:00
|
|
|
verify_shift_bytes_in_array_right ();
|
|
|
|
verify_clear_bit_region ();
|
|
|
|
verify_clear_bit_region_be ();
|
|
|
|
}
|
|
|
|
|
|
|
|
} // namespace selftest
|
|
|
|
#endif /* CHECKING_P. */
|