PR tree-optimization/77536
* tree-ssa-loop-manip.c (niter_for_unrolled_loop): New function.
(tree_transform_and_unroll_loop): Use above function to compute the
estimated niter of unrolled loop and use it when scaling profile.
Also use count info rather than frequency if it's non-zero.
* tree-ssa-loop-manip.h niter_for_unrolled_loop(): New declaration.
* tree-vect-loop.c (scale_profile_for_vect_loop): New function.
(vect_transform_loop): Call above function.
gcc/testsuite
* gcc.dg/vect/pr79347.c: Revise testing string.
From-SVN: r245754
PR tree-optimization/79284
* tree-vectorizer.h (VECT_SCALAR_BOOLEAN_TYPE_P): Define.
* tree-vect-stmts.c (vect_get_vec_def_for_operand,
vectorizable_mask_load_store, vectorizable_operation,
vect_is_simple_cond, get_same_sized_vectype): Use it instead
of comparing TREE_CODE of a type against BOOLEAN_TYPE.
* tree-vect-patterns.c (check_bool_pattern, search_type_for_mask_1,
vect_recog_bool_pattern, vect_recog_mask_conversion_pattern): Likewise.
* tree-vect-slp.c (vect_get_constant_vectors): Likewise.
* tree-vect-loop.c (vect_determine_vectorization_factor): Likewise.
Remove redundant gimple_code (stmt) == GIMPLE_ASSIGN test after
is_gimple_assign (stmt). Replace another such test with
is_gimple_assign (stmt).
testsuite/
* gcc.c-torture/compile/pr79284.c: New test.
From-SVN: r245214
PR tree-optimization/79340
* tree-vect-loop.c (vectorizable_reduction): Release
vec_defs elements after safe_splicing them into other vectors.
Formatting fixes.
From-SVN: r245171
2017-02-01 Andrew Pinski <apinski@cavium.com>
* tree-vect-loop.c (vect_compute_single_scalar_iteration_cost): Pass
stmt_info to record_stmt_cost.
(vect_get_known_peeling_cost): Pass stmt_info if known to
record_stmt_cost.
* config/aarch64/aarch64-protos.h (cpu_vector_cost): Split
cpu_vector_cost field into
scalar_int_stmt_cost and scalar_fp_stmt_cost. Split vec_stmt_cost
field into vec_int_stmt_cost and vec_fp_stmt_cost.
* config/aarch64/aarch64.c (generic_vector_cost): Update for the
splitting of scalar_stmt_cost and vec_stmt_cost.
(thunderx_vector_cost): Likewise.
(cortexa57_vector_cost): LIkewise.
(exynosm1_vector_cost): Likewise.
(xgene1_vector_cost): Likewise.
(thunderx2t99_vector_cost): Improve after the splitting of the two
fields.
(aarch64_builtin_vectorization_cost): Update for the splitting of
scalar_stmt_cost and vec_stmt_cost.
From-SVN: r245101
Building with the bootstrap-O1 configuration option fails to compile a
number of files due to AFAICT false-positive warnings about uses of
uninitialized variables.
This patch adds dummy initializers to silence them all.
for gcc/ChangeLog
* multiple_target.c (create_dispatcher_calls): Init e_next.
* tree-ssa-loop-split.c (split_loop): Init border.
* tree-vect-loop.c (vect_determine_vectorization_factor): Init
scalar_type.
From-SVN: r244086
2016-12-06 Jakub Jelinek <jakub@redhat.com>
PR tree-optimization/78675
* tree-vect-loop.c (vectorizable_live_operation): For
VECTOR_BOOLEAN_TYPE_P vectype use integral type with bitsize precision
instead of TREE_TYPE (vectype) for the BIT_FIELD_REF.
* gcc.c-torture/execute/pr78675.c: New test.
* gcc.target/i386/pr78675-1.c: New test.
* gcc.target/i386/pr78675-2.c: New test.
From-SVN: r243283
gcc/
2016-11-16 Yuri Rumyantsev <ysrumyan@gmail.com>
* params.def (PARAM_VECT_EPILOGUES_NOMASK): New.
* tree-if-conv.c (tree_if_conversion): Make public.
* * tree-if-conv.h: New file.
* tree-vect-data-refs.c (vect_analyze_data_ref_dependences) Avoid
dynamic alias checks for epilogues.
* tree-vect-loop-manip.c (vect_do_peeling): Return created epilog.
* tree-vect-loop.c: include tree-if-conv.h.
(new_loop_vec_info): Add zeroing orig_loop_info field.
(vect_analyze_loop_2): Don't try to enhance alignment for epilogues.
(vect_analyze_loop): Add argument ORIG_LOOP_INFO which is not NULL
if epilogue is vectorized, set up orig_loop_info field of loop_vinfo
using passed argument.
(vect_transform_loop): Check if created epilogue should be returned
for further vectorization with less vf. If-convert epilogue if
required. Print vectorization success for epilogue.
* tree-vectorizer.c (vectorize_loops): Add epilogue vectorization
if it is required, pass loop_vinfo produced during vectorization of
loop body to vect_analyze_loop.
* tree-vectorizer.h (struct _loop_vec_info): Add new field
orig_loop_info.
(LOOP_VINFO_ORIG_LOOP_INFO): New.
(LOOP_VINFO_EPILOGUE_P): New.
(LOOP_VINFO_ORIG_VECT_FACTOR): New.
(vect_do_peeling): Change prototype to return epilogue.
(vect_analyze_loop): Add argument of loop_vec_info type.
(vect_transform_loop): Return created loop.
gcc/testsuite/
2016-11-16 Yuri Rumyantsev <ysrumyan@gmail.com>
* lib/target-supports.exp (check_avx2_hw_available): New.
(check_effective_target_avx2_runtime): New.
* gcc.dg/vect/vect-tail-nomask-1.c: New test.
From-SVN: r242501
vect_transform_loop has to reduce three iteration counts by
the vectorisation factor: nb_iterations_upper_bound,
nb_iterations_likely_upper_bound and nb_iterations_estimate.
All three are latch execution counts rather than loop body
execution counts. The calculations were taking that into
account for the first two, but not for nb_iterations_estimate.
This patch updates the way the calculations are done to fix
this and to add a bit more commentary about what is going on.
gcc/
2016-11-15 Richard Sandiford <richard.sandiford@arm.com>
Alan Hayward <alan.hayward@arm.com>
David Sherwood <david.sherwood@arm.com>
* tree-vect-loop.c (vect_transform_loop): Protect the updates of
all three iteration counts with an any_* test. Use a single update
for each count. Fix the calculation of nb_iterations_estimate.
Co-Authored-By: Alan Hayward <alan.hayward@arm.com>
Co-Authored-By: David Sherwood <david.sherwood@arm.com>
From-SVN: r242475
* tree-vect-loop.c (loop_niters_no_overflow): New func.
(vect_transform_loop): Call loop_niters_no_overflow. Pass the
no-overflow information to vect_do_peeling_for_loop_bound and
vect_gen_vector_loop_niters.
From-SVN: r241102
* tree-vect-loop.c (vect_analyze_loop_2): Check and skip loop if it
has no enough iterations for LOOP_VINFO_PEELING_FOR_GAPS.
gcc/testsuite
* gcc.dg/vect/vect-98.c: Refine test case.
* gcc.dg/vect/vect-strided-a-u8-i8-gap2.c: Increase niters.
* gcc.dg/vect/vect-strided-u8-i8-gap2.c: Ditto.
* gcc.dg/vect/vect-strided-u8-i8-gap4.c: Ditto.
From-SVN: r241097
PR tree-optimization/77503
* tree-vect-loop.c (vectorizable_reduction): Record reduction
code for CONST_COND_REDUCTION at analysis stage and use it at
transform stage.
* tree-vectorizer.h (struct _stmt_vec_info): New field.
(STMT_VINFO_VEC_CONST_COND_REDUC_CODE): New macro.
* tree-vect-stmts.c (new_stmt_vec_info): Initialize above new
field.
gcc/testsuite
* gcc.dg/vect/pr77503.c: New test.
From-SVN: r240166
2016-08-23 Richard Biener <rguenther@suse.de>
PR tree-optimization/77286
* tree-vect-loop.c (vect_analyze_loop_form_1): Do not modify
the CFG here.
(vect_transform_loop): Split exit edges of loop and scalar
loop if required and at the appropriate time.
From-SVN: r239700
PR tree-optimization/69848
* tree-vectorizer.h (enum vect_def_type): New condition reduction
type CONST_COND_REDUCTION.
* tree-vect-loop.c (vectorizable_reduction): Support new condition
reudction type CONST_COND_REDUCTION.
gcc/testsuite
PR tree-optimization/69848
* gcc.dg/vect/vect-pr69848.c: New test.
From-SVN: r239416
PR tree-optimization/57558
* tree-vect-loop-manip.c (vect_create_cond_for_niters_checks): New
function.
(vect_loop_versioning): Support versioning with niter assumptions.
* tree-vect-loop.c (tree-ssa-loop.h): Include header file.
(vect_get_loop_niters): New parameter. Reimplement to support
assumptions in loop niter info.
(vect_analyze_loop_form_1, vect_analyze_loop_form): Ditto.
(new_loop_vec_info): Init LOOP_VINFO_NITERS_ASSUMPTIONS.
(vect_estimate_min_profitable_iters): Use LOOP_REQUIRES_VERSIONING.
Support loop versioning for niters.
* tree-vectorizer.c (tree-ssa-loop-niter.h): Include header file.
(vect_free_loop_info_assumptions): New function.
(vectorize_loops): Free loop niter info for loops with flag
LOOP_F_ASSUMPTIONS set if vectorization failed.
* tree-vectorizer.h (struct _loop_vec_info): New field
num_iters_assumptions.
(LOOP_VINFO_NITERS_ASSUMPTIONS): New macro.
(LOOP_REQUIRES_VERSIONING_FOR_NITERS): New macro.
(LOOP_REQUIRES_VERSIONING): New macro.
(vect_free_loop_info_assumptions): New decl.
gcc/testsuite
PR tree-optimization/57558
* gcc.dg/vect/pr57558-1.c: New test.
* gcc.dg/vect/pr57558-2.c: New test.
From-SVN: r238877
* tree-vect-data-refs.c (vect_no_alias_p): New function.
(vect_prune_runtime_alias_test_list): Call vect_no_alias_p to
resolve alias checks which are known at compilation time.
Truncate vector LOOP_VINFO_MAY_ALIAS_DDRS(loop_vinfo) if all
alias checks are resolved. Move dump info for too many runtime
alias checks to here...
* tree-vect-loop.c (vect_analyze_loop_2): ...From here.
gcc/testsuite
* gcc.dg/vect/vect-35-big-array.c: Refine comment and test.
* gcc.dg/vect/vect-35.c: Ditto.
* gcc.dg/vect/vect-alias-check-2.c: New test.
From-SVN: r238301
This patch moves the fix for PR65518 to the code that checks whether
load-and-permute operations are supported. If the group size is
greater than the vectorisation factor, it would still be possible
to fall back to elementwise loads (as for strided groups) rather
than fail vectorisation entirely.
Tested on aarch64-linux-gnu and x86_64-linux-gnu.
gcc/
* tree-vectorizer.h (vect_grouped_load_supported): Add a
single_element_p parameter.
* tree-vect-data-refs.c (vect_grouped_load_supported): Likewise.
Check the PR65518 case here rather than in vectorizable_load.
* tree-vect-loop.c (vect_analyze_loop_2): Update call accordignly.
* tree-vect-stmts.c (vectorizable_load): Likewise.
From-SVN: r238037
gcc/
2016-06-29 Yuri Rumyantsev <ysrumyan@gmail.com>
PR tree-optimization/70729
* tree-ssa-loop-im.c (ref_indep_loop_p_1): Consider memory reference as
independent in loops having positive safelen value.
* tree-vect-loop.c (vect_transform_loop): Clear-up safelen value since
it may be not valid after vectorization.
gcc/testsuite/
2016-06-29 Yuri Rumyantsev <ysrumyan@gmail.com>
PR tree-optimization/70729
* g++.dg/vect/pr70729.cc: New test.
From-SVN: r237844
2016-06-03 Alan Hayward <alan.hayward@arm.com>
[3/3] No need to vectorize simple only-live stmts
gcc/
* tree-vect-stmts.c (vect_stmt_relevant_p): Do not vectorize non live
relevant stmts which are simple and invariant.
* tree-vect-loop.c (vectorizable_live_operation): Check relevance
instead of simple and invariant
testsuite/
* gcc.dg/vect/vect-live-slp-5.c: Remove dg check.
From-SVN: r237065
2016-06-03 Alan Hayward <alan.hayward@arm.com>
[2/3] Vectorize inductions that are live after the loop
gcc/
* tree-vect-loop.c (vect_analyze_loop_operations): Allow live stmts.
(vectorizable_reduction): Check for new relevant state.
(vectorizable_live_operation): vectorize live stmts using
BIT_FIELD_REF. Remove special case for gimple assigns stmts.
* tree-vect-stmts.c (is_simple_and_all_uses_invariant): New function.
(vect_stmt_relevant_p): Check for stmts which are only used live.
(process_use): Use of a stmt does not inherit it's live value.
(vect_mark_stmts_to_be_vectorized): Simplify relevance inheritance.
(vect_analyze_stmt): Check for new relevant state.
* tree-vectorizer.h (vect_relevant): New entry for a stmt which is used
outside the loop, but not inside it.
testsuite/
* gcc.dg/tree-ssa/pr64183.c: Ensure test does not vectorize.
* testsuite/gcc.dg/vect/no-scevccp-vect-iv-2.c: Remove xfail.
* gcc.dg/vect/vect-live-1.c: New test.
* gcc.dg/vect/vect-live-2.c: New test.
* gcc.dg/vect/vect-live-3.c: New test.
* gcc.dg/vect/vect-live-4.c: New test.
* gcc.dg/vect/vect-live-5.c: New test.
* gcc.dg/vect/vect-live-slp-1.c: New test.
* gcc.dg/vect/vect-live-slp-2.c: New test.
* gcc.dg/vect/vect-live-slp-3.c: New test.
From-SVN: r237064
The vectorizable_* routines had many instances of:
slp_node || PURE_SLP_STMT (stmt_info)
which gives the misleading impression that we can have
!slp_node && PURE_SLP_STMT (stmt_info). In this context
it's really enough to test slp_node on its own.
There are three cases:
loop vectorisation only:
vectorizable_foo called only with !slp_node
pure SLP:
vectorizable_foo called only with slp_node
hybrid SLP:
(e.g. a vector that's used in SLP statements and also in a reduction)
- vectorizable_foo called once with slp_node for the SLP uses.
- vectorizable_foo called once with !slp_node for the non-SLP uses.
Hybrid SLP isn't possible for stores, so I added an explicit assert
for that.
I also made vectorizable_comparison static, to make it obvious that
no other callers outside tree-vect-stmts.c could use it with the
!slp && PURE_SLP_STMT combination.
Tested on aarch64-linux-gnu and x86_64-linux-gnu.
gcc/
* tree-vectorizer.h (vectorizable_comparison): Delete.
* tree-vect-loop.c (vectorizable_reduction): Remove redundant
PURE_SLP_STMT check.
* tree-vect-stmts.c (vectorizable_call): Likewise.
(vectorizable_simd_clone_call): Likewise.
(vectorizable_conversion): Likewise.
(vectorizable_assignment): Likewise.
(vectorizable_shift): Likewise.
(vectorizable_operation): Likewise.
(vectorizable_load): Likewise.
(vectorizable_condition): Likewise.
(vectorizable_store): Likewise. Assert that we don't have
hybrid SLP.
(vectorizable_comparison): Make static. Remove redundant
PURE_SLP_STMT check.
(vect_transform_stmt): Assert that we always have an slp_node
if PURE_SLP_STMT.
From-SVN: r236642
PR tree-optimization/70043
* tree-vect-loop.c (optimize_mask_stores): Move iterator to
previous statement if we see a debug statement.
* gfortran.dg/vect/pr70043.f90: New test.
From-SVN: r233934
Bin Cheng
Jakub Jelinek
Jakub Jelinek
Andrew Pinski
Alexandre Oliva
Yuri Rumyantsev
Richard Sandiford
Marek Polacek
Richard Biener
Alan Hayward
Ilya Enkovich
Eric Botcazou
Jan Hubicka
Martin Liska
Martin Liska