tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
/* Vectorizer
|
Make-lang.in, [...]: Update copyright years.
* ada/gcc-interface/Make-lang.in, alias.c, attribs.c, auto-inc-dec.c,
basic-block.h, bb-reorder.c, calls.c, c-common.c, cgraph.h,
collect2.h, config/alpha/alpha.c, config/alpha/alpha.md,
config/alpha/predicates.md, config/arm/arm.md,
config/arm/lib1funcs.asm, config/arm/neon-schedgen.ml,
config/avr/avr.c, config/avr/avr.md, config/bfin/bfin.c,
config/darwin9.h, config/darwin.c, config/darwin.h,
config/h8300/h8300.c, config/i386/cpuid.h, config/i386/cygming.h,
config/i386/cygwin.h, config/i386/mingw32.h, config/i386/msformat-c.c,
config/i386/sol2-10.h, config/i386/xopintrin.h, config/ia64/ia64.c,
config/ia64/ia64.md, config/ia64/sync.md, config/mep/mep.c,
config/mips/mips.md, config/mn10300/mn10300.c,
config/mn10300/mn10300.h, config/pa/pa.c, config/pa/pa.md,
config/rs6000/aix.h, config/rs6000/dfp.md,
config/rs6000/rs6000-builtin.def, config/rs6000/rs6000-c.c,
config/rs6000/vector.md, config/rtems.h, config/rx/rx.md,
config/s390/s390.md, config/sol2-c.c, config/sparc/sol2-bi.h,
config/sparc/sol2-gas.h, config/sparc/sparc.h, config/sparc/sparc.md,
config/sparc/sparc-protos.h, config/spu/spu.c, config/spu/spu-c.c,
config/t-darwin, convert.c, c.opt, c-opts.c, cp/Make-lang.in,
c-pretty-print.c, c-typeck.c, df-core.c, df-scan.c, diagnostic.c,
diagnostic.h, doc/cppopts.texi, doc/cpp.texi, doc/extend.texi,
doc/gimple.texi, doc/languages.texi, doc/plugins.texi, doc/rtl.texi,
doc/standards.texi, doc/tree-ssa.texi, doc/trouble.texi, dominance.c,
fold-const.c, fortran/Make-lang.in, fwprop.c, gcc-plugin.h,
gensupport.c, gimple.h, gimple-iterator.c, graphite.c,
graphite-clast-to-gimple.c, graphite-clast-to-gimple.h,
graphite-dependences.c, graphite-poly.c, graphite-poly.h,
graphite-ppl.c, graphite-ppl.h, graphite-scop-detection.c,
graphite-sese-to-poly.c, graphite-sese-to-poly.h, ifcvt.c, intl.c,
intl.h, ipa.c, ipa-cp.c, ipa-inline.c, ipa-prop.c, ipa-prop.h,
ipa-pure-const.c, ipa-reference.c, ipa-type-escape.c, ira-color.c,
ira-conflicts.c, ira-lives.c, java/Make-lang.in, lambda-code.c,
loop-invariant.c, lto/Make-lang.in, lto-streamer.h, lto-streamer-in.c,
objc/Make-lang.in, objcp/Make-lang.in, omp-low.c, optc-gen.awk,
opt-functions.awk, opth-gen.awk, params.def, passes.c,
postreload-gcse.c, print-tree.c, recog.c, regrename.c, reload.h,
rtl.def, sched-int.h, sched-rgn.c, sel-sched-dump.c, sese.c, sese.h,
store-motion.c, stor-layout.c, tree-cfgcleanup.c, tree-chrec.c,
tree-complex.c, tree-data-ref.c, tree.def, tree-eh.c, tree-flow.h,
tree-flow-inline.h, tree.h, tree-loop-distribution.c, tree-outof-ssa.c,
tree-parloops.c, tree-pass.h, tree-predcom.c, tree-profile.c,
tree-scalar-evolution.c, tree-ssa-address.c, tree-ssa-alias.c,
tree-ssa-coalesce.c, tree-ssa-copy.c, tree-ssa-dce.c, tree-ssa-dom.c,
tree-ssa-dse.c, tree-ssa-loop-im.c, tree-ssa-loop-ivcanon.c,
tree-ssa-loop-manip.c, tree-ssa-math-opts.c, tree-ssa-operands.c,
tree-ssa-pre.c, tree-ssa-sccvn.c, tree-ssa-structalias.c,
tree-ssa-uncprop.c, tree-tailcall.c, tree-vect-data-refs.c,
tree-vect-loop.c, tree-vectorizer.h, tree-vect-slp.c, tree-vrp.c,
unwind-dw2-fde-darwin.c, varpool.c: Update copyright years.
From-SVN: r157950
2010-04-02 21:54:46 +02:00
|
|
|
Copyright (C) 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010
|
|
|
|
Free Software Foundation, Inc.
|
2004-08-17 18:17:14 +02:00
|
|
|
Contributed by Dorit Naishlos <dorit@il.ibm.com>
|
|
|
|
|
|
|
|
This file is part of GCC.
|
|
|
|
|
|
|
|
GCC is free software; you can redistribute it and/or modify it under
|
|
|
|
the terms of the GNU General Public License as published by the Free
|
2007-07-26 10:37:01 +02:00
|
|
|
Software Foundation; either version 3, or (at your option) any later
|
2004-08-17 18:17:14 +02:00
|
|
|
version.
|
|
|
|
|
|
|
|
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
|
|
|
|
WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
|
|
|
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
|
|
|
for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License
|
2007-07-26 10:37:01 +02:00
|
|
|
along with GCC; see the file COPYING3. If not see
|
|
|
|
<http://www.gnu.org/licenses/>. */
|
2004-08-17 18:17:14 +02:00
|
|
|
|
|
|
|
#ifndef GCC_TREE_VECTORIZER_H
|
|
|
|
#define GCC_TREE_VECTORIZER_H
|
|
|
|
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
#include "tree-data-ref.h"
|
|
|
|
|
2008-02-26 17:05:24 +01:00
|
|
|
typedef source_location LOC;
|
|
|
|
#define UNKNOWN_LOC UNKNOWN_LOCATION
|
|
|
|
#define EXPR_LOC(e) EXPR_LOCATION(e)
|
|
|
|
#define LOC_FILE(l) LOCATION_FILE (l)
|
|
|
|
#define LOC_LINE(l) LOCATION_LINE (l)
|
2005-02-03 17:22:22 +01:00
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/* Used for naming of new temporaries. */
|
|
|
|
enum vect_var_kind {
|
|
|
|
vect_simple_var,
|
tree.def (REDUC_MAX_EXPR, [...]): New tree-codes.
* tree.def (REDUC_MAX_EXPR, REDUC_MIN_EXPR, REDUC_PLUS_EXPR): New
tree-codes.
* optabs.h (OTI_reduc_smax, OTI_reduc_umax, OTI_reduc_smin,
OTI_reduc_umin, OTI_reduc_plus): New optabs for reduction.
(reduc_smax_optab, reduc_umax_optab, reduc_smin_optab, reduc_umin_optab,
reduc_plus_optab): New optabs for reduction.
* expr.c (expand_expr_real_1): Handle new tree-codes.
* tree-inline.c (estimate_num_insns_1): Handle new tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio, op_symbol): Handle
new tree-codes.
* optabs.c (optab_for_tree_code): Handle new tree-codes.
(init_optabs): Initialize new optabs.
* genopinit.c (optabs): Define handlers for new optabs.
* tree-vect-analyze.c (vect_analyze_operations): Fail vectorization in
case of a phi that is marked as relevant. Call vectorizable_reduction.
(vect_mark_relevant): Phis may be marked as relevant.
(vect_mark_stmts_to_be_vectorized): The use corresponding to the
reduction variable in a reduction stmt does not mark its defining phi
as relevant. Update documentation accordingly.
(vect_can_advance_ivs_p): Skip reduction phis.
* tree-vect-transform.c (vect_get_vec_def_for_operand): Takes
additional argument. Handle reduction.
(vect_create_destination_var): Update call to vect_get_new_vect_var.
Handle non-vector argument.
(get_initial_def_for_reduction): New function.
(vect_create_epilog_for_reduction): New function.
(vectorizable_reduction): New function.
(vect_get_new_vect_var): Handle new vect_var_kind.
(vectorizable_assignment, vectorizable_operation, vectorizable_store,
vectorizable_condition): Update call to vect_get_new_vect_var.
(vect_transform_stmt): Call vectorizable_reduction.
(vect_update_ivs_after_vectorizer): Skip reduction phis.
(vect_transform_loop): Skip if stmt is both not relevant and not live.
* tree-vectorizer.c (reduction_code_for_scalar_code): New function.
(vect_is_simple_reduction): Was empty - added implementation.
* tree-vectorizer.h (vect_scalar_var): New enum vect_var_kind value.
(reduc_vec_info_type): New enum vect_def_type value.
* config/rs6000/altivec.md (reduc_smax_v4si, reduc_smax_v4sf,
reduc_umax_v4si, reduc_smin_v4si, reduc_umin_v4sf, reduc_smin_v4sf,
reduc_plus_v4si, reduc_plus_v4sf): New define_expands.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Remove
ENABLE_CHECKING around gcc_assert.
* tree-vect-transform.c (vect_do_peeling_for_loop_bound,
(vect_do_peeling_for_alignment, vect_transform_loop,
vect_get_vec_def_for_operand): Likewise.
From-SVN: r101155
2005-06-18 15:18:52 +02:00
|
|
|
vect_pointer_var,
|
|
|
|
vect_scalar_var
|
2004-08-17 18:17:14 +02:00
|
|
|
};
|
|
|
|
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
/* Defines type of operation. */
|
2004-08-17 18:17:14 +02:00
|
|
|
enum operation_type {
|
|
|
|
unary_op = 1,
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
binary_op,
|
|
|
|
ternary_op
|
2004-08-17 18:17:14 +02:00
|
|
|
};
|
|
|
|
|
2004-11-04 06:26:53 +01:00
|
|
|
/* Define type of available alignment support. */
|
|
|
|
enum dr_alignment_support {
|
|
|
|
dr_unaligned_unsupported,
|
|
|
|
dr_unaligned_supported,
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
dr_explicit_realign,
|
|
|
|
dr_explicit_realign_optimized,
|
2004-11-04 06:26:53 +01:00
|
|
|
dr_aligned
|
|
|
|
};
|
|
|
|
|
2005-06-12 16:03:01 +02:00
|
|
|
/* Define type of def-use cross-iteration cycle. */
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
enum vect_def_type {
|
2009-04-27 22:25:48 +02:00
|
|
|
vect_uninitialized_def = 0,
|
tree-vect-loop-manip.c (vect_create_cond_for_alias_checks): Use REPORT_VECTORIZED_LOCATIONS instead REPORT_VECTORIZED_LOOPS.
* tree-vect-loop-manip.c (vect_create_cond_for_alias_checks):
Use REPORT_VECTORIZED_LOCATIONS instead
REPORT_VECTORIZED_LOOPS.
* tree-vectorizer.c (vect_verbosity_level): Make static.
(vect_loop_location): Rename to vect_location.
(vect_set_verbosity_level): Update comment.
(vect_set_dump_settings): Use REPORT_VECTORIZED_LOCATIONS
and vect_location.
(vectorize_loops): Fix comment. Use REPORT_VECTORIZED_LOCATIONS
and vect_location. Use REPORT_UNVECTORIZED_LOCATIONS
instead REPORT_UNVECTORIZED_LOOPS.
* tree-vectorizer.h (enum vect_def_type): Rename vect_invariant_def and
vect_loop_def to vect_external_def and vect_internal_def.
(enum verbosity_levels): Rename REPORT_VECTORIZED_LOOPS
and REPORT_UNVECTORIZED_LOOPS to
REPORT_VECTORIZED_LOCATIONS and
REPORT_UNVECTORIZED_LOCATIONS.
(enum vect_relevant): Update comment. Rename vect_unused_in_loop
and vect_used_in_loop and to vect_unused_in_scope and
vect_used_in_scope.
(STMT_VINFO_RELEVANT_P): Use vect_unused_in_scope.
(vect_verbosity_level): Remove declaration.
(vect_analyze_operations): Likewise.
(vect_analyze_stmt): Declare.
* tree-vect-loop.c (vect_determine_vectorization_factor): Use
REPORT_UNVECTORIZED_LOCATIONS.
(vect_get_loop_niters): Fix indentation.
(vect_analyze_loop_form): Use REPORT_UNVECTORIZED_LOCATIONS.
(vect_analyze_loop_operations): New function.
(vect_analyze_loop): Call vect_analyze_loop_operations instead of
vect_analyze_operations.
(vect_is_simple_reduction): Use new names.
(vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-vect-data-refs.c (vect_check_interleaving): Add a return value to
specify whether the data references can be a part of interleaving chain.
(vect_analyze_data_ref_dependence): Use new names.
(vect_analyze_data_refs_alignment, vect_analyze_data_refs): Likewise.
(vect_create_addr_base_for_vector_ref): Remove redundant code.
* tree-vect-patterns.c (widened_name_p): Use new names.
(vect_recog_dot_prod_pattern): Likewise.
* tree-vect-stmts.c (vect_stmt_relevant_p): Use new names.
(process_use, vect_mark_stmts_to_be_vectorized,
vect_model_simple_cost, vect_model_store_cost,
vect_get_vec_def_for_operand, vect_get_vec_def_for_stmt_copy,
vectorizable_call, vectorizable_conversion, vectorizable_assignment,
vectorizable_operation, vectorizable_type_demotion,
vectorizable_type_promotion, vectorizable_store, vectorizable_load,
vectorizable_condition): Likewise.
(vect_analyze_operations): Split into vect_analyze_loop_operations
and ...
(vect_analyze_stmt): ... new function.
(new_stmt_vec_info): Use new names.
(vect_is_simple_use): Use new names and fix comment.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Use new names.
(vect_build_slp_tree, vect_analyze_slp, vect_schedule_slp): Likewise.
From-SVN: r146875
2009-04-28 10:08:25 +02:00
|
|
|
vect_constant_def = 1,
|
|
|
|
vect_external_def,
|
|
|
|
vect_internal_def,
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
vect_induction_def,
|
|
|
|
vect_reduction_def,
|
2009-07-12 09:09:07 +02:00
|
|
|
vect_double_reduction_def,
|
2009-06-16 09:21:12 +02:00
|
|
|
vect_nested_cycle,
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
vect_unknown_def_type
|
|
|
|
};
|
|
|
|
|
2010-02-16 12:35:03 +01:00
|
|
|
#define VECTORIZABLE_CYCLE_DEF(D) (((D) == vect_reduction_def) \
|
|
|
|
|| ((D) == vect_double_reduction_def) \
|
|
|
|
|| ((D) == vect_nested_cycle))
|
|
|
|
|
2005-02-07 11:07:07 +01:00
|
|
|
/* Define verbosity levels. */
|
|
|
|
enum verbosity_levels {
|
|
|
|
REPORT_NONE,
|
tree-vect-loop-manip.c (vect_create_cond_for_alias_checks): Use REPORT_VECTORIZED_LOCATIONS instead REPORT_VECTORIZED_LOOPS.
* tree-vect-loop-manip.c (vect_create_cond_for_alias_checks):
Use REPORT_VECTORIZED_LOCATIONS instead
REPORT_VECTORIZED_LOOPS.
* tree-vectorizer.c (vect_verbosity_level): Make static.
(vect_loop_location): Rename to vect_location.
(vect_set_verbosity_level): Update comment.
(vect_set_dump_settings): Use REPORT_VECTORIZED_LOCATIONS
and vect_location.
(vectorize_loops): Fix comment. Use REPORT_VECTORIZED_LOCATIONS
and vect_location. Use REPORT_UNVECTORIZED_LOCATIONS
instead REPORT_UNVECTORIZED_LOOPS.
* tree-vectorizer.h (enum vect_def_type): Rename vect_invariant_def and
vect_loop_def to vect_external_def and vect_internal_def.
(enum verbosity_levels): Rename REPORT_VECTORIZED_LOOPS
and REPORT_UNVECTORIZED_LOOPS to
REPORT_VECTORIZED_LOCATIONS and
REPORT_UNVECTORIZED_LOCATIONS.
(enum vect_relevant): Update comment. Rename vect_unused_in_loop
and vect_used_in_loop and to vect_unused_in_scope and
vect_used_in_scope.
(STMT_VINFO_RELEVANT_P): Use vect_unused_in_scope.
(vect_verbosity_level): Remove declaration.
(vect_analyze_operations): Likewise.
(vect_analyze_stmt): Declare.
* tree-vect-loop.c (vect_determine_vectorization_factor): Use
REPORT_UNVECTORIZED_LOCATIONS.
(vect_get_loop_niters): Fix indentation.
(vect_analyze_loop_form): Use REPORT_UNVECTORIZED_LOCATIONS.
(vect_analyze_loop_operations): New function.
(vect_analyze_loop): Call vect_analyze_loop_operations instead of
vect_analyze_operations.
(vect_is_simple_reduction): Use new names.
(vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-vect-data-refs.c (vect_check_interleaving): Add a return value to
specify whether the data references can be a part of interleaving chain.
(vect_analyze_data_ref_dependence): Use new names.
(vect_analyze_data_refs_alignment, vect_analyze_data_refs): Likewise.
(vect_create_addr_base_for_vector_ref): Remove redundant code.
* tree-vect-patterns.c (widened_name_p): Use new names.
(vect_recog_dot_prod_pattern): Likewise.
* tree-vect-stmts.c (vect_stmt_relevant_p): Use new names.
(process_use, vect_mark_stmts_to_be_vectorized,
vect_model_simple_cost, vect_model_store_cost,
vect_get_vec_def_for_operand, vect_get_vec_def_for_stmt_copy,
vectorizable_call, vectorizable_conversion, vectorizable_assignment,
vectorizable_operation, vectorizable_type_demotion,
vectorizable_type_promotion, vectorizable_store, vectorizable_load,
vectorizable_condition): Likewise.
(vect_analyze_operations): Split into vect_analyze_loop_operations
and ...
(vect_analyze_stmt): ... new function.
(new_stmt_vec_info): Use new names.
(vect_is_simple_use): Use new names and fix comment.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Use new names.
(vect_build_slp_tree, vect_analyze_slp, vect_schedule_slp): Likewise.
From-SVN: r146875
2009-04-28 10:08:25 +02:00
|
|
|
REPORT_VECTORIZED_LOCATIONS,
|
|
|
|
REPORT_UNVECTORIZED_LOCATIONS,
|
2007-12-17 21:54:01 +01:00
|
|
|
REPORT_COST,
|
2005-02-07 11:07:07 +01:00
|
|
|
REPORT_ALIGNMENT,
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
REPORT_DR_DETAILS,
|
2005-02-07 11:07:07 +01:00
|
|
|
REPORT_BAD_FORM_LOOPS,
|
|
|
|
REPORT_OUTER_LOOPS,
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
REPORT_SLP,
|
2005-02-07 11:07:07 +01:00
|
|
|
REPORT_DETAILS,
|
|
|
|
/* New verbosity levels should be added before this one. */
|
|
|
|
MAX_VERBOSITY_LEVEL
|
|
|
|
};
|
|
|
|
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
/************************************************************************
|
|
|
|
SLP
|
|
|
|
************************************************************************/
|
|
|
|
|
|
|
|
/* A computation tree of an SLP instance. Each node corresponds to a group of
|
|
|
|
stmts to be packed in a SIMD stmt. */
|
|
|
|
typedef struct _slp_tree {
|
|
|
|
/* Only binary and unary operations are supported. LEFT child corresponds to
|
|
|
|
the first operand and RIGHT child to the second if the operation is
|
|
|
|
binary. */
|
|
|
|
struct _slp_tree *left;
|
|
|
|
struct _slp_tree *right;
|
|
|
|
/* A group of scalar stmts to be vectorized together. */
|
2008-07-28 16:33:56 +02:00
|
|
|
VEC (gimple, heap) *stmts;
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
/* Vectorized stmt/s. */
|
2008-07-28 16:33:56 +02:00
|
|
|
VEC (gimple, heap) *vec_stmts;
|
2009-11-25 11:55:54 +01:00
|
|
|
/* Number of vector stmts that are created to replace the group of scalar
|
|
|
|
stmts. It is calculated during the transformation phase as the number of
|
|
|
|
scalar elements in one scalar iteration (GROUP_SIZE) multiplied by VF
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
divided by vector size. */
|
|
|
|
unsigned int vec_stmts_size;
|
|
|
|
/* Vectorization costs associated with SLP node. */
|
|
|
|
struct
|
|
|
|
{
|
|
|
|
int outside_of_loop; /* Statements generated outside loop. */
|
|
|
|
int inside_of_loop; /* Statements generated inside loop. */
|
|
|
|
} cost;
|
|
|
|
} *slp_tree;
|
|
|
|
|
2008-08-28 13:11:14 +02:00
|
|
|
DEF_VEC_P(slp_tree);
|
|
|
|
DEF_VEC_ALLOC_P(slp_tree, heap);
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
|
|
|
|
/* SLP instance is a sequence of stmts in a loop that can be packed into
|
|
|
|
SIMD stmts. */
|
|
|
|
typedef struct _slp_instance {
|
|
|
|
/* The root of SLP tree. */
|
|
|
|
slp_tree root;
|
|
|
|
|
|
|
|
/* Size of groups of scalar stmts that will be replaced by SIMD stmt/s. */
|
|
|
|
unsigned int group_size;
|
|
|
|
|
|
|
|
/* The unrolling factor required to vectorized this SLP instance. */
|
|
|
|
unsigned int unrolling_factor;
|
|
|
|
|
|
|
|
/* Vectorization costs associated with SLP instance. */
|
2009-11-25 11:55:54 +01:00
|
|
|
struct
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
{
|
|
|
|
int outside_of_loop; /* Statements generated outside loop. */
|
|
|
|
int inside_of_loop; /* Statements generated inside loop. */
|
|
|
|
} cost;
|
2008-08-28 13:11:14 +02:00
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
/* Loads permutation relatively to the stores, NULL if there is no
|
2008-08-28 13:11:14 +02:00
|
|
|
permutation. */
|
|
|
|
VEC (int, heap) *load_permutation;
|
|
|
|
|
|
|
|
/* The group of nodes that contain loads of this SLP instance. */
|
|
|
|
VEC (slp_tree, heap) *loads;
|
2008-09-22 09:55:39 +02:00
|
|
|
|
|
|
|
/* The first scalar load of the instance. The created vector loads will be
|
|
|
|
inserted before this statement. */
|
|
|
|
gimple first_load;
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
} *slp_instance;
|
|
|
|
|
|
|
|
DEF_VEC_P(slp_instance);
|
|
|
|
DEF_VEC_ALLOC_P(slp_instance, heap);
|
|
|
|
|
|
|
|
/* Access Functions. */
|
|
|
|
#define SLP_INSTANCE_TREE(S) (S)->root
|
|
|
|
#define SLP_INSTANCE_GROUP_SIZE(S) (S)->group_size
|
|
|
|
#define SLP_INSTANCE_UNROLLING_FACTOR(S) (S)->unrolling_factor
|
|
|
|
#define SLP_INSTANCE_OUTSIDE_OF_LOOP_COST(S) (S)->cost.outside_of_loop
|
|
|
|
#define SLP_INSTANCE_INSIDE_OF_LOOP_COST(S) (S)->cost.inside_of_loop
|
2008-08-28 13:11:14 +02:00
|
|
|
#define SLP_INSTANCE_LOAD_PERMUTATION(S) (S)->load_permutation
|
|
|
|
#define SLP_INSTANCE_LOADS(S) (S)->loads
|
2008-09-22 09:55:39 +02:00
|
|
|
#define SLP_INSTANCE_FIRST_LOAD_STMT(S) (S)->first_load
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
|
|
|
|
#define SLP_TREE_LEFT(S) (S)->left
|
|
|
|
#define SLP_TREE_RIGHT(S) (S)->right
|
|
|
|
#define SLP_TREE_SCALAR_STMTS(S) (S)->stmts
|
|
|
|
#define SLP_TREE_VEC_STMTS(S) (S)->vec_stmts
|
|
|
|
#define SLP_TREE_NUMBER_OF_VEC_STMTS(S) (S)->vec_stmts_size
|
|
|
|
#define SLP_TREE_OUTSIDE_OF_LOOP_COST(S) (S)->cost.outside_of_loop
|
|
|
|
#define SLP_TREE_INSIDE_OF_LOOP_COST(S) (S)->cost.inside_of_loop
|
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Info on vectorized loops. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
typedef struct _loop_vec_info {
|
|
|
|
|
|
|
|
/* The loop to which this info struct refers to. */
|
|
|
|
struct loop *loop;
|
|
|
|
|
|
|
|
/* The loop basic blocks. */
|
|
|
|
basic_block *bbs;
|
|
|
|
|
|
|
|
/* Number of iterations. */
|
|
|
|
tree num_iters;
|
2007-12-10 23:49:43 +01:00
|
|
|
tree num_iters_unchanged;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2007-07-05 15:15:20 +02:00
|
|
|
/* Minimum number of iterations below which vectorization is expected to
|
2009-11-25 11:55:54 +01:00
|
|
|
not be profitable (as estimated by the cost model).
|
2007-07-05 15:15:20 +02:00
|
|
|
-1 indicates that vectorization will not be profitable.
|
|
|
|
FORNOW: This field is an int. Will be a tree in the future, to represent
|
2009-11-25 11:55:54 +01:00
|
|
|
values unknown at compile time. */
|
|
|
|
int min_profitable_iters;
|
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
/* Is the loop vectorizable? */
|
|
|
|
bool vectorizable;
|
|
|
|
|
|
|
|
/* Unrolling factor */
|
|
|
|
int vectorization_factor;
|
|
|
|
|
2010-04-26 20:21:17 +02:00
|
|
|
/* The loop location in the source. */
|
|
|
|
LOC loop_line_number;
|
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
/* Unknown DRs according to which loop was peeled. */
|
|
|
|
struct data_reference *unaligned_dr;
|
|
|
|
|
2005-03-15 19:33:09 +01:00
|
|
|
/* peeling_for_alignment indicates whether peeling for alignment will take
|
|
|
|
place, and what the peeling factor should be:
|
|
|
|
peeling_for_alignment = X means:
|
|
|
|
If X=0: Peeling for alignment will not be applied.
|
|
|
|
If X>0: Peel first X iterations.
|
|
|
|
If X=-1: Generate a runtime test to calculate the number of iterations
|
|
|
|
to be peeled, using the dataref recorded in the field
|
|
|
|
unaligned_dr. */
|
|
|
|
int peeling_for_alignment;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2005-09-06 12:59:58 +02:00
|
|
|
/* The mask used to check the alignment of pointers or arrays. */
|
|
|
|
int ptr_mask;
|
|
|
|
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
/* All data references in the loop. */
|
tree-loop-linear.c: Don't include varray.h.
* tree-loop-linear.c: Don't include varray.h.
(gather_interchange_stats, try_interchange_loops,
linear_transform_loops): Use VEC instead of VARRAY.
* lambda-mat.c: Don't include varray.h.
* tree-chrec.c: Same.
* lambda-trans.c: Same.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info): Use
VEC instead of VARRAY.
* tree-vectorizer.h: Idem.
* tree-data-ref.c (dump_data_references,
dump_data_dependence_relations, dump_dist_dir_vectors, dump_ddrs,
initialize_data_dependence_relation, finalize_ddr_dependent,
compute_all_dependences, find_data_references_in_loop,
compute_data_dependences_for_loop, analyze_all_data_dependences,
free_dependence_relation, free_dependence_relations,
free_data_refs): Idem.
* tree-data-ref.h (data_reference_p, subscript_p): New.
(data_dependence_relation, DDR_SUBSCRIPT, DDR_NUM_SUBSCRIPTS): Use
VEC instead of VARRAY.
(DDR_SUBSCRIPTS_VECTOR_INIT): Removed.
(find_data_references_in_loop, compute_data_dependences_for_loop,
dump_ddrs, dump_dist_dir_vectors, dump_data_references,
dump_data_dependence_relations, free_dependence_relations,
free_data_refs): Adjust declaration.
(lambda_transform_legal_p): Move declaration here...
* tree-vect-analyze.c (vect_analyze_data_ref_dependences,
vect_compute_data_refs_alignment, vect_verify_datarefs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_data_ref_accesses,
vect_analyze_data_refs): Use VEC instead of VARRAY.
* lambda.h (lambda_transform_legal_p): ...from here.
* lambda-code.c (lambda_transform_legal_p): Use VEC instead of VARRAY.
* tree-vect-transform.c (vect_update_inits_of_drs): Idem.
* Makefile.in (tree-loop-linear.o, lambda-mat.o, lambda-trans.o,
tree-chrec.o): Don't depend on VARRAY_H.
From-SVN: r112437
2006-03-28 06:19:26 +02:00
|
|
|
VEC (data_reference_p, heap) *datarefs;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
/* All data dependences in the loop. */
|
tree-loop-linear.c: Don't include varray.h.
* tree-loop-linear.c: Don't include varray.h.
(gather_interchange_stats, try_interchange_loops,
linear_transform_loops): Use VEC instead of VARRAY.
* lambda-mat.c: Don't include varray.h.
* tree-chrec.c: Same.
* lambda-trans.c: Same.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info): Use
VEC instead of VARRAY.
* tree-vectorizer.h: Idem.
* tree-data-ref.c (dump_data_references,
dump_data_dependence_relations, dump_dist_dir_vectors, dump_ddrs,
initialize_data_dependence_relation, finalize_ddr_dependent,
compute_all_dependences, find_data_references_in_loop,
compute_data_dependences_for_loop, analyze_all_data_dependences,
free_dependence_relation, free_dependence_relations,
free_data_refs): Idem.
* tree-data-ref.h (data_reference_p, subscript_p): New.
(data_dependence_relation, DDR_SUBSCRIPT, DDR_NUM_SUBSCRIPTS): Use
VEC instead of VARRAY.
(DDR_SUBSCRIPTS_VECTOR_INIT): Removed.
(find_data_references_in_loop, compute_data_dependences_for_loop,
dump_ddrs, dump_dist_dir_vectors, dump_data_references,
dump_data_dependence_relations, free_dependence_relations,
free_data_refs): Adjust declaration.
(lambda_transform_legal_p): Move declaration here...
* tree-vect-analyze.c (vect_analyze_data_ref_dependences,
vect_compute_data_refs_alignment, vect_verify_datarefs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_data_ref_accesses,
vect_analyze_data_refs): Use VEC instead of VARRAY.
* lambda.h (lambda_transform_legal_p): ...from here.
* lambda-code.c (lambda_transform_legal_p): Use VEC instead of VARRAY.
* tree-vect-transform.c (vect_update_inits_of_drs): Idem.
* Makefile.in (tree-loop-linear.o, lambda-mat.o, lambda-trans.o,
tree-chrec.o): Don't depend on VARRAY_H.
From-SVN: r112437
2006-03-28 06:19:26 +02:00
|
|
|
VEC (ddr_p, heap) *ddrs;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
tree-vectorizer.c (new_loop_vec_info): Initialize new field.
gcc/ChangeLog
* tree-vectorizer.c (new_loop_vec_info): Initialize new
field.
(destroy_loop_vec_info): Add call to VEC_free.
* tree-vectorizer.h (may_alias_ddrs): Define.
(LOOP_VINFO_MAY_ALIAS_DDRS): Define.
* tree-vect-analyze.c (vect_analyze_data_ref_dependence):
Change reporting to dump.
(vect_is_duplicate_ddr): New.
(vect_mark_for_runtime_alias_test): New.
(vect_analyze_data_ref_dependences) Add call to
vect_mark_for_runtime_alias_test.
(vect_enhance_data_refs_alignment): Define local variable
vect_versioning_for_alias_required, don't perform
peeling for alignment if versioning for alias is
required.
(vect_enhance_data_refs_alignment): Use
PARAM_VECT_MAX_VERSION_FOR_ALIGNMENT_CHECKS instead of
PARAM_VECT_MAX_VERSION_CHECKS.
* tree-vect-transform.c
(vect_create_cond_for_alias_checks): New.
(vect_transform_loop): Add call to
vect_create_cond_for_alias_checks.
(vect_vfa_segment_size): New.
* params.def (PARAM_VECT_MAX_VERSION_FOR_ALIGNMENT_CHECKS):
Rename.
(PARAM_VECT_MAX_VERSION_FOR_ALIAS_CHECKS): Define.
* gcc/doc/invoke.texi
(vect-max-version-for-alignment-checks): Document.
(vect-max-version-for-alias-checks): Document.
(vect-max-version-checks): Remove.
gcc/testsuite/ChangeLog
* gcc.dg/vect/vect-vfa-01.c: New.
* gcc.dg/vect/vect-vfa-02.c: New.
* gcc.dg/vect/vect-vfa-03.c: New.
* gcc.dg/vect/vect-vfa-04.c: New.
* gcc.dg/vect/vect-102a.c, gcc.dg/vect/vect-51.c,
gcc.dg/vect/pr29145.c, gcc.dg/vect/vect-43.c,
gcc.dg/vect/vect-61.c, gcc.dg/vect/vect-53.c,
gcc.dg/vect/vect-45.c, gcc.dg/vect/vect-101.c,
gcc.dg/vect/vect-37.c, gcc.dg/vect/vect-79.c,
gcc.dg/vect/vect-102.c, gcc.dg/vect/vect-dv-2.c,
gcc.dg/vect/vect-57.c, gcc.dg/vect/vect-49.c,
gfortran.dg/vect/pr19049.f90: Rename to start with
prefix no-vfa-.
* gcc.dg/vect/vect.exp: Disable versioning for alias
when test starts with no-vfa-.
* gfortran.dg/vect/vect.exp: Likewise.
From-SVN: r127559
2007-08-16 16:20:39 +02:00
|
|
|
/* Data Dependence Relations defining address ranges that are candidates
|
|
|
|
for a run-time aliasing check. */
|
|
|
|
VEC (ddr_p, heap) *may_alias_ddrs;
|
|
|
|
|
2005-09-06 12:59:58 +02:00
|
|
|
/* Statements in the loop that have data references that are candidates for a
|
|
|
|
runtime (loop versioning) misalignment check. */
|
2008-07-28 16:33:56 +02:00
|
|
|
VEC(gimple,heap) *may_misalign_stmts;
|
2005-09-06 12:59:58 +02:00
|
|
|
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
/* All interleaving chains of stores in the loop, represented by the first
|
|
|
|
stmt in the chain. */
|
2008-07-28 16:33:56 +02:00
|
|
|
VEC(gimple, heap) *strided_stores;
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
|
|
|
|
/* All SLP instances in the loop. This is a subset of the set of STRIDED_STORES
|
|
|
|
of the loop. */
|
|
|
|
VEC(slp_instance, heap) *slp_instances;
|
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
/* The unrolling factor needed to SLP the loop. In case of that pure SLP is
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
applied to the loop, i.e., no unrolling is needed, this is 1. */
|
|
|
|
unsigned slp_unrolling_factor;
|
2010-04-19 11:10:45 +02:00
|
|
|
|
|
|
|
/* Reduction cycles detected in the loop. Used in loop-aware SLP. */
|
|
|
|
VEC (gimple, heap) *reductions;
|
2005-02-03 17:13:17 +01:00
|
|
|
} *loop_vec_info;
|
|
|
|
|
2005-09-06 12:59:58 +02:00
|
|
|
/* Access Functions. */
|
2009-06-01 10:15:01 +02:00
|
|
|
#define LOOP_VINFO_LOOP(L) (L)->loop
|
|
|
|
#define LOOP_VINFO_BBS(L) (L)->bbs
|
|
|
|
#define LOOP_VINFO_NITERS(L) (L)->num_iters
|
2007-12-06 17:18:55 +01:00
|
|
|
/* Since LOOP_VINFO_NITERS can change after prologue peeling
|
|
|
|
retain total unchanged scalar loop iterations for cost model. */
|
2009-06-01 10:15:01 +02:00
|
|
|
#define LOOP_VINFO_NITERS_UNCHANGED(L) (L)->num_iters_unchanged
|
|
|
|
#define LOOP_VINFO_COST_MODEL_MIN_ITERS(L) (L)->min_profitable_iters
|
|
|
|
#define LOOP_VINFO_VECTORIZABLE_P(L) (L)->vectorizable
|
|
|
|
#define LOOP_VINFO_VECT_FACTOR(L) (L)->vectorization_factor
|
|
|
|
#define LOOP_VINFO_PTR_MASK(L) (L)->ptr_mask
|
|
|
|
#define LOOP_VINFO_DATAREFS(L) (L)->datarefs
|
|
|
|
#define LOOP_VINFO_DDRS(L) (L)->ddrs
|
|
|
|
#define LOOP_VINFO_INT_NITERS(L) (TREE_INT_CST_LOW ((L)->num_iters))
|
|
|
|
#define LOOP_PEELING_FOR_ALIGNMENT(L) (L)->peeling_for_alignment
|
|
|
|
#define LOOP_VINFO_UNALIGNED_DR(L) (L)->unaligned_dr
|
|
|
|
#define LOOP_VINFO_MAY_MISALIGN_STMTS(L) (L)->may_misalign_stmts
|
|
|
|
#define LOOP_VINFO_LOC(L) (L)->loop_line_number
|
|
|
|
#define LOOP_VINFO_MAY_ALIAS_DDRS(L) (L)->may_alias_ddrs
|
|
|
|
#define LOOP_VINFO_STRIDED_STORES(L) (L)->strided_stores
|
|
|
|
#define LOOP_VINFO_SLP_INSTANCES(L) (L)->slp_instances
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
#define LOOP_VINFO_SLP_UNROLLING_FACTOR(L) (L)->slp_unrolling_factor
|
2010-04-19 11:10:45 +02:00
|
|
|
#define LOOP_VINFO_REDUCTIONS(L) (L)->reductions
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2009-06-01 10:15:01 +02:00
|
|
|
#define LOOP_REQUIRES_VERSIONING_FOR_ALIGNMENT(L) \
|
|
|
|
VEC_length (gimple, (L)->may_misalign_stmts) > 0
|
|
|
|
#define LOOP_REQUIRES_VERSIONING_FOR_ALIAS(L) \
|
|
|
|
VEC_length (ddr_p, (L)->may_alias_ddrs) > 0
|
|
|
|
|
2007-04-18 19:43:27 +02:00
|
|
|
#define NITERS_KNOWN_P(n) \
|
|
|
|
(host_integerp ((n),0) \
|
|
|
|
&& TREE_INT_CST_LOW ((n)) > 0)
|
|
|
|
|
2009-06-01 10:15:01 +02:00
|
|
|
#define LOOP_VINFO_NITERS_KNOWN_P(L) \
|
2007-04-18 19:43:27 +02:00
|
|
|
NITERS_KNOWN_P((L)->num_iters)
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2007-08-19 11:39:50 +02:00
|
|
|
static inline loop_vec_info
|
|
|
|
loop_vec_info_for_loop (struct loop *loop)
|
|
|
|
{
|
|
|
|
return (loop_vec_info) loop->aux;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline bool
|
2008-07-28 16:33:56 +02:00
|
|
|
nested_in_vect_loop_p (struct loop *loop, gimple stmt)
|
2007-08-19 11:39:50 +02:00
|
|
|
{
|
2009-11-25 11:55:54 +01:00
|
|
|
return (loop->inner
|
2008-07-28 16:33:56 +02:00
|
|
|
&& (loop->inner == (gimple_bb (stmt))->loop_father));
|
2007-08-19 11:39:50 +02:00
|
|
|
}
|
|
|
|
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
typedef struct _bb_vec_info {
|
|
|
|
|
|
|
|
basic_block bb;
|
2009-11-25 11:55:54 +01:00
|
|
|
/* All interleaving chains of stores in the basic block, represented by the
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
first stmt in the chain. */
|
|
|
|
VEC(gimple, heap) *strided_stores;
|
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
/* All SLP instances in the basic block. This is a subset of the set of
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
STRIDED_STORES of the basic block. */
|
|
|
|
VEC(slp_instance, heap) *slp_instances;
|
|
|
|
|
|
|
|
/* All data references in the basic block. */
|
|
|
|
VEC (data_reference_p, heap) *datarefs;
|
|
|
|
|
|
|
|
/* All data dependences in the basic block. */
|
|
|
|
VEC (ddr_p, heap) *ddrs;
|
|
|
|
} *bb_vec_info;
|
|
|
|
|
|
|
|
#define BB_VINFO_BB(B) (B)->bb
|
|
|
|
#define BB_VINFO_STRIDED_STORES(B) (B)->strided_stores
|
|
|
|
#define BB_VINFO_SLP_INSTANCES(B) (B)->slp_instances
|
|
|
|
#define BB_VINFO_DATAREFS(B) (B)->datarefs
|
|
|
|
#define BB_VINFO_DDRS(B) (B)->ddrs
|
|
|
|
|
|
|
|
static inline bb_vec_info
|
|
|
|
vec_info_for_bb (basic_block bb)
|
|
|
|
{
|
|
|
|
return (bb_vec_info) bb->aux;
|
|
|
|
}
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Info on vectorized defs. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
enum stmt_vec_info_type {
|
|
|
|
undef_vec_info_type = 0,
|
|
|
|
load_vec_info_type,
|
|
|
|
store_vec_info_type,
|
|
|
|
op_vec_info_type,
|
2006-11-27 13:41:51 +01:00
|
|
|
call_vec_info_type,
|
2005-04-12 03:35:54 +02:00
|
|
|
assignment_vec_info_type,
|
tree.def (REDUC_MAX_EXPR, [...]): New tree-codes.
* tree.def (REDUC_MAX_EXPR, REDUC_MIN_EXPR, REDUC_PLUS_EXPR): New
tree-codes.
* optabs.h (OTI_reduc_smax, OTI_reduc_umax, OTI_reduc_smin,
OTI_reduc_umin, OTI_reduc_plus): New optabs for reduction.
(reduc_smax_optab, reduc_umax_optab, reduc_smin_optab, reduc_umin_optab,
reduc_plus_optab): New optabs for reduction.
* expr.c (expand_expr_real_1): Handle new tree-codes.
* tree-inline.c (estimate_num_insns_1): Handle new tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio, op_symbol): Handle
new tree-codes.
* optabs.c (optab_for_tree_code): Handle new tree-codes.
(init_optabs): Initialize new optabs.
* genopinit.c (optabs): Define handlers for new optabs.
* tree-vect-analyze.c (vect_analyze_operations): Fail vectorization in
case of a phi that is marked as relevant. Call vectorizable_reduction.
(vect_mark_relevant): Phis may be marked as relevant.
(vect_mark_stmts_to_be_vectorized): The use corresponding to the
reduction variable in a reduction stmt does not mark its defining phi
as relevant. Update documentation accordingly.
(vect_can_advance_ivs_p): Skip reduction phis.
* tree-vect-transform.c (vect_get_vec_def_for_operand): Takes
additional argument. Handle reduction.
(vect_create_destination_var): Update call to vect_get_new_vect_var.
Handle non-vector argument.
(get_initial_def_for_reduction): New function.
(vect_create_epilog_for_reduction): New function.
(vectorizable_reduction): New function.
(vect_get_new_vect_var): Handle new vect_var_kind.
(vectorizable_assignment, vectorizable_operation, vectorizable_store,
vectorizable_condition): Update call to vect_get_new_vect_var.
(vect_transform_stmt): Call vectorizable_reduction.
(vect_update_ivs_after_vectorizer): Skip reduction phis.
(vect_transform_loop): Skip if stmt is both not relevant and not live.
* tree-vectorizer.c (reduction_code_for_scalar_code): New function.
(vect_is_simple_reduction): Was empty - added implementation.
* tree-vectorizer.h (vect_scalar_var): New enum vect_var_kind value.
(reduc_vec_info_type): New enum vect_def_type value.
* config/rs6000/altivec.md (reduc_smax_v4si, reduc_smax_v4sf,
reduc_umax_v4si, reduc_smin_v4si, reduc_umin_v4sf, reduc_smin_v4sf,
reduc_plus_v4si, reduc_plus_v4sf): New define_expands.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Remove
ENABLE_CHECKING around gcc_assert.
* tree-vect-transform.c (vect_do_peeling_for_loop_bound,
(vect_do_peeling_for_alignment, vect_transform_loop,
vect_get_vec_def_for_operand): Likewise.
From-SVN: r101155
2005-06-18 15:18:52 +02:00
|
|
|
condition_vec_info_type,
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
reduc_vec_info_type,
|
2007-04-17 09:31:45 +02:00
|
|
|
induc_vec_info_type,
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
type_promotion_vec_info_type,
|
2007-02-11 12:46:07 +01:00
|
|
|
type_demotion_vec_info_type,
|
2007-08-19 11:39:50 +02:00
|
|
|
type_conversion_vec_info_type,
|
|
|
|
loop_exit_ctrl_vec_info_type
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
};
|
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
/* Indicates whether/how a variable is used in the scope of loop/basic
|
tree-vect-loop-manip.c (vect_create_cond_for_alias_checks): Use REPORT_VECTORIZED_LOCATIONS instead REPORT_VECTORIZED_LOOPS.
* tree-vect-loop-manip.c (vect_create_cond_for_alias_checks):
Use REPORT_VECTORIZED_LOCATIONS instead
REPORT_VECTORIZED_LOOPS.
* tree-vectorizer.c (vect_verbosity_level): Make static.
(vect_loop_location): Rename to vect_location.
(vect_set_verbosity_level): Update comment.
(vect_set_dump_settings): Use REPORT_VECTORIZED_LOCATIONS
and vect_location.
(vectorize_loops): Fix comment. Use REPORT_VECTORIZED_LOCATIONS
and vect_location. Use REPORT_UNVECTORIZED_LOCATIONS
instead REPORT_UNVECTORIZED_LOOPS.
* tree-vectorizer.h (enum vect_def_type): Rename vect_invariant_def and
vect_loop_def to vect_external_def and vect_internal_def.
(enum verbosity_levels): Rename REPORT_VECTORIZED_LOOPS
and REPORT_UNVECTORIZED_LOOPS to
REPORT_VECTORIZED_LOCATIONS and
REPORT_UNVECTORIZED_LOCATIONS.
(enum vect_relevant): Update comment. Rename vect_unused_in_loop
and vect_used_in_loop and to vect_unused_in_scope and
vect_used_in_scope.
(STMT_VINFO_RELEVANT_P): Use vect_unused_in_scope.
(vect_verbosity_level): Remove declaration.
(vect_analyze_operations): Likewise.
(vect_analyze_stmt): Declare.
* tree-vect-loop.c (vect_determine_vectorization_factor): Use
REPORT_UNVECTORIZED_LOCATIONS.
(vect_get_loop_niters): Fix indentation.
(vect_analyze_loop_form): Use REPORT_UNVECTORIZED_LOCATIONS.
(vect_analyze_loop_operations): New function.
(vect_analyze_loop): Call vect_analyze_loop_operations instead of
vect_analyze_operations.
(vect_is_simple_reduction): Use new names.
(vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-vect-data-refs.c (vect_check_interleaving): Add a return value to
specify whether the data references can be a part of interleaving chain.
(vect_analyze_data_ref_dependence): Use new names.
(vect_analyze_data_refs_alignment, vect_analyze_data_refs): Likewise.
(vect_create_addr_base_for_vector_ref): Remove redundant code.
* tree-vect-patterns.c (widened_name_p): Use new names.
(vect_recog_dot_prod_pattern): Likewise.
* tree-vect-stmts.c (vect_stmt_relevant_p): Use new names.
(process_use, vect_mark_stmts_to_be_vectorized,
vect_model_simple_cost, vect_model_store_cost,
vect_get_vec_def_for_operand, vect_get_vec_def_for_stmt_copy,
vectorizable_call, vectorizable_conversion, vectorizable_assignment,
vectorizable_operation, vectorizable_type_demotion,
vectorizable_type_promotion, vectorizable_store, vectorizable_load,
vectorizable_condition): Likewise.
(vect_analyze_operations): Split into vect_analyze_loop_operations
and ...
(vect_analyze_stmt): ... new function.
(new_stmt_vec_info): Use new names.
(vect_is_simple_use): Use new names and fix comment.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Use new names.
(vect_build_slp_tree, vect_analyze_slp, vect_schedule_slp): Likewise.
From-SVN: r146875
2009-04-28 10:08:25 +02:00
|
|
|
block. */
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
enum vect_relevant {
|
tree-vect-loop-manip.c (vect_create_cond_for_alias_checks): Use REPORT_VECTORIZED_LOCATIONS instead REPORT_VECTORIZED_LOOPS.
* tree-vect-loop-manip.c (vect_create_cond_for_alias_checks):
Use REPORT_VECTORIZED_LOCATIONS instead
REPORT_VECTORIZED_LOOPS.
* tree-vectorizer.c (vect_verbosity_level): Make static.
(vect_loop_location): Rename to vect_location.
(vect_set_verbosity_level): Update comment.
(vect_set_dump_settings): Use REPORT_VECTORIZED_LOCATIONS
and vect_location.
(vectorize_loops): Fix comment. Use REPORT_VECTORIZED_LOCATIONS
and vect_location. Use REPORT_UNVECTORIZED_LOCATIONS
instead REPORT_UNVECTORIZED_LOOPS.
* tree-vectorizer.h (enum vect_def_type): Rename vect_invariant_def and
vect_loop_def to vect_external_def and vect_internal_def.
(enum verbosity_levels): Rename REPORT_VECTORIZED_LOOPS
and REPORT_UNVECTORIZED_LOOPS to
REPORT_VECTORIZED_LOCATIONS and
REPORT_UNVECTORIZED_LOCATIONS.
(enum vect_relevant): Update comment. Rename vect_unused_in_loop
and vect_used_in_loop and to vect_unused_in_scope and
vect_used_in_scope.
(STMT_VINFO_RELEVANT_P): Use vect_unused_in_scope.
(vect_verbosity_level): Remove declaration.
(vect_analyze_operations): Likewise.
(vect_analyze_stmt): Declare.
* tree-vect-loop.c (vect_determine_vectorization_factor): Use
REPORT_UNVECTORIZED_LOCATIONS.
(vect_get_loop_niters): Fix indentation.
(vect_analyze_loop_form): Use REPORT_UNVECTORIZED_LOCATIONS.
(vect_analyze_loop_operations): New function.
(vect_analyze_loop): Call vect_analyze_loop_operations instead of
vect_analyze_operations.
(vect_is_simple_reduction): Use new names.
(vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-vect-data-refs.c (vect_check_interleaving): Add a return value to
specify whether the data references can be a part of interleaving chain.
(vect_analyze_data_ref_dependence): Use new names.
(vect_analyze_data_refs_alignment, vect_analyze_data_refs): Likewise.
(vect_create_addr_base_for_vector_ref): Remove redundant code.
* tree-vect-patterns.c (widened_name_p): Use new names.
(vect_recog_dot_prod_pattern): Likewise.
* tree-vect-stmts.c (vect_stmt_relevant_p): Use new names.
(process_use, vect_mark_stmts_to_be_vectorized,
vect_model_simple_cost, vect_model_store_cost,
vect_get_vec_def_for_operand, vect_get_vec_def_for_stmt_copy,
vectorizable_call, vectorizable_conversion, vectorizable_assignment,
vectorizable_operation, vectorizable_type_demotion,
vectorizable_type_promotion, vectorizable_store, vectorizable_load,
vectorizable_condition): Likewise.
(vect_analyze_operations): Split into vect_analyze_loop_operations
and ...
(vect_analyze_stmt): ... new function.
(new_stmt_vec_info): Use new names.
(vect_is_simple_use): Use new names and fix comment.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Use new names.
(vect_build_slp_tree, vect_analyze_slp, vect_schedule_slp): Likewise.
From-SVN: r146875
2009-04-28 10:08:25 +02:00
|
|
|
vect_unused_in_scope = 0,
|
2009-06-16 09:21:12 +02:00
|
|
|
/* The def is in the inner loop, and the use is in the outer loop, and the
|
|
|
|
use is a reduction stmt. */
|
2007-08-19 11:39:50 +02:00
|
|
|
vect_used_in_outer_by_reduction,
|
2009-06-16 09:21:12 +02:00
|
|
|
/* The def is in the inner loop, and the use is in the outer loop (and is
|
|
|
|
not part of reduction). */
|
2007-08-19 11:39:50 +02:00
|
|
|
vect_used_in_outer,
|
2007-02-22 09:16:18 +01:00
|
|
|
|
|
|
|
/* defs that feed computations that end up (only) in a reduction. These
|
2009-11-25 11:55:54 +01:00
|
|
|
defs may be used by non-reduction stmts, but eventually, any
|
|
|
|
computations/values that are affected by these defs are used to compute
|
|
|
|
a reduction (i.e. don't get stored to memory, for example). We use this
|
|
|
|
to identify computations that we can change the order in which they are
|
2007-02-22 09:16:18 +01:00
|
|
|
computed. */
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
vect_used_by_reduction,
|
2007-02-22 09:16:18 +01:00
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
vect_used_in_scope
|
2004-08-17 18:17:14 +02:00
|
|
|
};
|
|
|
|
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
/* The type of vectorization that can be applied to the stmt: regular loop-based
|
|
|
|
vectorization; pure SLP - the stmt is a part of SLP instances and does not
|
|
|
|
have uses outside SLP instances; or hybrid SLP and loop-based - the stmt is
|
|
|
|
a part of SLP instance and also must be loop-based vectorized, since it has
|
2009-11-25 11:55:54 +01:00
|
|
|
uses outside SLP sequences.
|
|
|
|
|
|
|
|
In the loop context the meanings of pure and hybrid SLP are slightly
|
|
|
|
different. By saying that pure SLP is applied to the loop, we mean that we
|
|
|
|
exploit only intra-iteration parallelism in the loop; i.e., the loop can be
|
|
|
|
vectorized without doing any conceptual unrolling, cause we don't pack
|
|
|
|
together stmts from different iterations, only within a single iteration.
|
|
|
|
Loop hybrid SLP means that we exploit both intra-iteration and
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
inter-iteration parallelism (e.g., number of elements in the vector is 4
|
2009-11-25 11:55:54 +01:00
|
|
|
and the slp-group-size is 2, in which case we don't have enough parallelism
|
|
|
|
within an iteration, so we obtain the rest of the parallelism from subsequent
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
iterations by unrolling the loop by 2). */
|
2009-11-25 11:55:54 +01:00
|
|
|
enum slp_vect_type {
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
loop_vect = 0,
|
|
|
|
pure_slp,
|
|
|
|
hybrid
|
|
|
|
};
|
|
|
|
|
|
|
|
|
2005-06-10 16:52:01 +02:00
|
|
|
typedef struct data_reference *dr_p;
|
|
|
|
DEF_VEC_P(dr_p);
|
|
|
|
DEF_VEC_ALLOC_P(dr_p,heap);
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
typedef struct _stmt_vec_info {
|
|
|
|
|
|
|
|
enum stmt_vec_info_type type;
|
|
|
|
|
2010-04-26 20:21:17 +02:00
|
|
|
/* Indicates whether this stmts is part of a computation whose result is
|
|
|
|
used outside the loop. */
|
|
|
|
bool live;
|
|
|
|
|
|
|
|
/* Stmt is part of some pattern (computation idiom) */
|
|
|
|
bool in_pattern_p;
|
|
|
|
|
|
|
|
/* For loads only, if there is a store with the same location, this field is
|
|
|
|
TRUE. */
|
|
|
|
bool read_write_dep;
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/* The stmt to which this info struct refers to. */
|
2008-07-28 16:33:56 +02:00
|
|
|
gimple stmt;
|
2004-08-17 18:17:14 +02:00
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
/* The loop_vec_info with respect to which STMT is vectorized. */
|
|
|
|
loop_vec_info loop_vinfo;
|
2004-08-17 18:17:14 +02:00
|
|
|
|
2010-04-09 12:40:14 +02:00
|
|
|
/* The vector type to be used for the LHS of this statement. */
|
2004-08-17 18:17:14 +02:00
|
|
|
tree vectype;
|
|
|
|
|
|
|
|
/* The vectorized version of the stmt. */
|
2008-07-28 16:33:56 +02:00
|
|
|
gimple vectorized_stmt;
|
2004-08-17 18:17:14 +02:00
|
|
|
|
|
|
|
|
|
|
|
/** The following is relevant only for stmts that contain a non-scalar
|
2009-11-25 11:55:54 +01:00
|
|
|
data-ref (array/pointer/struct access). A GIMPLE stmt is expected to have
|
2004-08-17 18:17:14 +02:00
|
|
|
at most one such data-ref. **/
|
|
|
|
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
/* Information about the data-ref (access function, etc),
|
|
|
|
relative to the inner-most containing loop. */
|
2004-08-17 18:17:14 +02:00
|
|
|
struct data_reference *data_ref_info;
|
|
|
|
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
/* Information about the data-ref relative to this loop
|
|
|
|
nest (the loop that is being considered for vectorization). */
|
|
|
|
tree dr_base_address;
|
|
|
|
tree dr_init;
|
|
|
|
tree dr_offset;
|
|
|
|
tree dr_step;
|
|
|
|
tree dr_aligned_to;
|
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
/* Used for various bookkeeping purposes, generally holding a pointer to
|
|
|
|
some other stmt S that is in some way "related" to this stmt.
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
Current use of this field is:
|
2009-11-25 11:55:54 +01:00
|
|
|
If this stmt is part of a pattern (i.e. the field 'in_pattern_p' is
|
|
|
|
true): S is the "pattern stmt" that represents (and replaces) the
|
|
|
|
sequence of stmts that constitutes the pattern. Similarly, the
|
|
|
|
related_stmt of the "pattern stmt" points back to this stmt (which is
|
|
|
|
the last stmt in the original sequence of stmts that constitutes the
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
pattern). */
|
2008-07-28 16:33:56 +02:00
|
|
|
gimple related_stmt;
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
|
2005-06-10 16:52:01 +02:00
|
|
|
/* List of datarefs that are known to have the same alignment as the dataref
|
|
|
|
of this stmt. */
|
|
|
|
VEC(dr_p,heap) *same_align_refs;
|
|
|
|
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
/* Classify the def of this stmt. */
|
|
|
|
enum vect_def_type def_type;
|
|
|
|
|
2010-04-26 20:21:17 +02:00
|
|
|
/* Whether the stmt is SLPed, loop-based vectorized, or both. */
|
|
|
|
enum slp_vect_type slp_type;
|
|
|
|
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
/* Interleaving info. */
|
|
|
|
/* First data-ref in the interleaving group. */
|
2008-07-28 16:33:56 +02:00
|
|
|
gimple first_dr;
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
/* Pointer to the next data-ref in the group. */
|
2008-07-28 16:33:56 +02:00
|
|
|
gimple next_dr;
|
2010-04-26 20:21:17 +02:00
|
|
|
/* In case that two or more stmts share data-ref, this is the pointer to the
|
|
|
|
previously detected stmt with the same dr. */
|
|
|
|
gimple same_dr_stmt;
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
/* The size of the interleaving group. */
|
|
|
|
unsigned int size;
|
|
|
|
/* For stores, number of stores from this group seen. We vectorize the last
|
|
|
|
one. */
|
|
|
|
unsigned int store_count;
|
|
|
|
/* For loads only, the gap from the previous load. For consecutive loads, GAP
|
|
|
|
is 1. */
|
|
|
|
unsigned int gap;
|
2010-04-26 20:21:17 +02:00
|
|
|
|
|
|
|
/* Not all stmts in the loop need to be vectorized. e.g, the increment
|
|
|
|
of the loop induction variable and computation of array indexes. relevant
|
|
|
|
indicates whether the stmt needs to be vectorized. */
|
|
|
|
enum vect_relevant relevant;
|
2007-06-08 18:30:49 +02:00
|
|
|
|
|
|
|
/* Vectorization costs associated with statement. */
|
2009-11-25 11:55:54 +01:00
|
|
|
struct
|
2007-06-08 18:30:49 +02:00
|
|
|
{
|
|
|
|
int outside_of_loop; /* Statements generated outside loop. */
|
|
|
|
int inside_of_loop; /* Statements generated inside loop. */
|
|
|
|
} cost;
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
/* The bb_vec_info with respect to which STMT is vectorized. */
|
|
|
|
bb_vec_info bb_vinfo;
|
2010-04-26 08:39:27 +02:00
|
|
|
|
|
|
|
/* Is this statement vectorizable or should it be skipped in (partial)
|
|
|
|
vectorization. */
|
|
|
|
bool vectorizable;
|
2004-08-17 18:17:14 +02:00
|
|
|
} *stmt_vec_info;
|
|
|
|
|
|
|
|
/* Access Functions. */
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
#define STMT_VINFO_TYPE(S) (S)->type
|
|
|
|
#define STMT_VINFO_STMT(S) (S)->stmt
|
|
|
|
#define STMT_VINFO_LOOP_VINFO(S) (S)->loop_vinfo
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
#define STMT_VINFO_BB_VINFO(S) (S)->bb_vinfo
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
#define STMT_VINFO_RELEVANT(S) (S)->relevant
|
|
|
|
#define STMT_VINFO_LIVE_P(S) (S)->live
|
|
|
|
#define STMT_VINFO_VECTYPE(S) (S)->vectype
|
|
|
|
#define STMT_VINFO_VEC_STMT(S) (S)->vectorized_stmt
|
2010-04-26 08:39:27 +02:00
|
|
|
#define STMT_VINFO_VECTORIZABLE(S) (S)->vectorizable
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
#define STMT_VINFO_DATA_REF(S) (S)->data_ref_info
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
|
|
|
|
#define STMT_VINFO_DR_BASE_ADDRESS(S) (S)->dr_base_address
|
|
|
|
#define STMT_VINFO_DR_INIT(S) (S)->dr_init
|
|
|
|
#define STMT_VINFO_DR_OFFSET(S) (S)->dr_offset
|
|
|
|
#define STMT_VINFO_DR_STEP(S) (S)->dr_step
|
|
|
|
#define STMT_VINFO_DR_ALIGNED_TO(S) (S)->dr_aligned_to
|
|
|
|
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
#define STMT_VINFO_IN_PATTERN_P(S) (S)->in_pattern_p
|
|
|
|
#define STMT_VINFO_RELATED_STMT(S) (S)->related_stmt
|
|
|
|
#define STMT_VINFO_SAME_ALIGN_REFS(S) (S)->same_align_refs
|
|
|
|
#define STMT_VINFO_DEF_TYPE(S) (S)->def_type
|
|
|
|
#define STMT_VINFO_DR_GROUP_FIRST_DR(S) (S)->first_dr
|
|
|
|
#define STMT_VINFO_DR_GROUP_NEXT_DR(S) (S)->next_dr
|
|
|
|
#define STMT_VINFO_DR_GROUP_SIZE(S) (S)->size
|
|
|
|
#define STMT_VINFO_DR_GROUP_STORE_COUNT(S) (S)->store_count
|
|
|
|
#define STMT_VINFO_DR_GROUP_GAP(S) (S)->gap
|
|
|
|
#define STMT_VINFO_DR_GROUP_SAME_DR_STMT(S)(S)->same_dr_stmt
|
2007-01-21 10:22:56 +01:00
|
|
|
#define STMT_VINFO_DR_GROUP_READ_WRITE_DEPENDENCE(S) (S)->read_write_dep
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
#define STMT_VINFO_STRIDED_ACCESS(S) ((S)->first_dr != NULL)
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
|
|
|
|
#define DR_GROUP_FIRST_DR(S) (S)->first_dr
|
|
|
|
#define DR_GROUP_NEXT_DR(S) (S)->next_dr
|
|
|
|
#define DR_GROUP_SIZE(S) (S)->size
|
|
|
|
#define DR_GROUP_STORE_COUNT(S) (S)->store_count
|
|
|
|
#define DR_GROUP_GAP(S) (S)->gap
|
|
|
|
#define DR_GROUP_SAME_DR_STMT(S) (S)->same_dr_stmt
|
2007-01-21 10:22:56 +01:00
|
|
|
#define DR_GROUP_READ_WRITE_DEPENDENCE(S) (S)->read_write_dep
|
2004-08-17 18:17:14 +02:00
|
|
|
|
tree-vect-loop-manip.c (vect_create_cond_for_alias_checks): Use REPORT_VECTORIZED_LOCATIONS instead REPORT_VECTORIZED_LOOPS.
* tree-vect-loop-manip.c (vect_create_cond_for_alias_checks):
Use REPORT_VECTORIZED_LOCATIONS instead
REPORT_VECTORIZED_LOOPS.
* tree-vectorizer.c (vect_verbosity_level): Make static.
(vect_loop_location): Rename to vect_location.
(vect_set_verbosity_level): Update comment.
(vect_set_dump_settings): Use REPORT_VECTORIZED_LOCATIONS
and vect_location.
(vectorize_loops): Fix comment. Use REPORT_VECTORIZED_LOCATIONS
and vect_location. Use REPORT_UNVECTORIZED_LOCATIONS
instead REPORT_UNVECTORIZED_LOOPS.
* tree-vectorizer.h (enum vect_def_type): Rename vect_invariant_def and
vect_loop_def to vect_external_def and vect_internal_def.
(enum verbosity_levels): Rename REPORT_VECTORIZED_LOOPS
and REPORT_UNVECTORIZED_LOOPS to
REPORT_VECTORIZED_LOCATIONS and
REPORT_UNVECTORIZED_LOCATIONS.
(enum vect_relevant): Update comment. Rename vect_unused_in_loop
and vect_used_in_loop and to vect_unused_in_scope and
vect_used_in_scope.
(STMT_VINFO_RELEVANT_P): Use vect_unused_in_scope.
(vect_verbosity_level): Remove declaration.
(vect_analyze_operations): Likewise.
(vect_analyze_stmt): Declare.
* tree-vect-loop.c (vect_determine_vectorization_factor): Use
REPORT_UNVECTORIZED_LOCATIONS.
(vect_get_loop_niters): Fix indentation.
(vect_analyze_loop_form): Use REPORT_UNVECTORIZED_LOCATIONS.
(vect_analyze_loop_operations): New function.
(vect_analyze_loop): Call vect_analyze_loop_operations instead of
vect_analyze_operations.
(vect_is_simple_reduction): Use new names.
(vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-vect-data-refs.c (vect_check_interleaving): Add a return value to
specify whether the data references can be a part of interleaving chain.
(vect_analyze_data_ref_dependence): Use new names.
(vect_analyze_data_refs_alignment, vect_analyze_data_refs): Likewise.
(vect_create_addr_base_for_vector_ref): Remove redundant code.
* tree-vect-patterns.c (widened_name_p): Use new names.
(vect_recog_dot_prod_pattern): Likewise.
* tree-vect-stmts.c (vect_stmt_relevant_p): Use new names.
(process_use, vect_mark_stmts_to_be_vectorized,
vect_model_simple_cost, vect_model_store_cost,
vect_get_vec_def_for_operand, vect_get_vec_def_for_stmt_copy,
vectorizable_call, vectorizable_conversion, vectorizable_assignment,
vectorizable_operation, vectorizable_type_demotion,
vectorizable_type_promotion, vectorizable_store, vectorizable_load,
vectorizable_condition): Likewise.
(vect_analyze_operations): Split into vect_analyze_loop_operations
and ...
(vect_analyze_stmt): ... new function.
(new_stmt_vec_info): Use new names.
(vect_is_simple_use): Use new names and fix comment.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Use new names.
(vect_build_slp_tree, vect_analyze_slp, vect_schedule_slp): Likewise.
From-SVN: r146875
2009-04-28 10:08:25 +02:00
|
|
|
#define STMT_VINFO_RELEVANT_P(S) ((S)->relevant != vect_unused_in_scope)
|
2007-06-08 18:30:49 +02:00
|
|
|
#define STMT_VINFO_OUTSIDE_OF_LOOP_COST(S) (S)->cost.outside_of_loop
|
|
|
|
#define STMT_VINFO_INSIDE_OF_LOOP_COST(S) (S)->cost.inside_of_loop
|
|
|
|
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
#define HYBRID_SLP_STMT(S) ((S)->slp_type == hybrid)
|
|
|
|
#define PURE_SLP_STMT(S) ((S)->slp_type == pure_slp)
|
|
|
|
#define STMT_SLP_TYPE(S) (S)->slp_type
|
|
|
|
|
2008-08-19 10:31:41 +02:00
|
|
|
/* The maximum number of intermediate steps required in multi-step type
|
|
|
|
conversion. */
|
|
|
|
#define MAX_INTERM_CVT_STEPS 3
|
|
|
|
|
2010-04-13 11:48:26 +02:00
|
|
|
/* The maximum vectorization factor supported by any target (V32QI). */
|
|
|
|
#define MAX_VECTORIZATION_FACTOR 32
|
|
|
|
|
2008-07-28 16:33:56 +02:00
|
|
|
/* Avoid GTY(()) on stmt_vec_info. */
|
|
|
|
typedef void *vec_void_p;
|
|
|
|
DEF_VEC_P (vec_void_p);
|
|
|
|
DEF_VEC_ALLOC_P (vec_void_p, heap);
|
2004-08-17 18:17:14 +02:00
|
|
|
|
2008-07-28 16:33:56 +02:00
|
|
|
extern VEC(vec_void_p,heap) *stmt_vec_info_vec;
|
|
|
|
|
|
|
|
void init_stmt_vec_info_vec (void);
|
|
|
|
void free_stmt_vec_info_vec (void);
|
|
|
|
|
|
|
|
static inline stmt_vec_info
|
|
|
|
vinfo_for_stmt (gimple stmt)
|
2004-08-17 18:17:14 +02:00
|
|
|
{
|
2008-07-28 16:33:56 +02:00
|
|
|
unsigned int uid = gimple_uid (stmt);
|
|
|
|
if (uid == 0)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
return (stmt_vec_info) VEC_index (vec_void_p, stmt_vec_info_vec, uid - 1);
|
2004-08-17 18:17:14 +02:00
|
|
|
}
|
|
|
|
|
2008-07-28 16:33:56 +02:00
|
|
|
static inline void
|
|
|
|
set_vinfo_for_stmt (gimple stmt, stmt_vec_info info)
|
2004-08-17 18:17:14 +02:00
|
|
|
{
|
2008-07-28 16:33:56 +02:00
|
|
|
unsigned int uid = gimple_uid (stmt);
|
|
|
|
if (uid == 0)
|
|
|
|
{
|
cgraph.h (varpool_first_static_initializer, [...]): Make checking only when checking enabled.
* cgraph.h (varpool_first_static_initializer,
varpool_next_static_initializer): Make checking only when
checking enabled.
* tree-vectorizer.h (vinfo_for_stmt): Remove check.
(set_vinfo_for_stmt, get_earlier_stmt, is_loop_header_bb_p): Change
gcc_assert to gcc_checking_assert.
* tree-flow-inline.h (gimple_vop, get_var_ann, relink_imm_use, phi_nodes
set_phi_nodes, phi_arg_index_from_use, op_iter_next_use,
op_iter_next_def, op_iter_next_tree, op_iter_init, op_iter_init_use,
op_iter_init_phiuse, op_iter_init_phidef, array_ref_contains_indirect_ref,
ref_contains_array_ref): Use gcc_checking_assert.
* emit-rtl.h (set_first_insn, set_last_insn): Likewise.
* tree-ssa-live.h (var_to_partition, var_to_partition_to_var,
partition_is_global, live_on_entry, live_on_exit,
live_merge_and_clear): Likewise.
* system.h (gcc_checking_assert): New macro.
From-SVN: r160489
2010-06-09 17:41:23 +02:00
|
|
|
gcc_checking_assert (info);
|
2008-07-28 16:33:56 +02:00
|
|
|
uid = VEC_length (vec_void_p, stmt_vec_info_vec) + 1;
|
|
|
|
gimple_set_uid (stmt, uid);
|
|
|
|
VEC_safe_push (vec_void_p, heap, stmt_vec_info_vec, (vec_void_p) info);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
VEC_replace (vec_void_p, stmt_vec_info_vec, uid - 1, (vec_void_p) info);
|
2004-08-17 18:17:14 +02:00
|
|
|
}
|
|
|
|
|
2008-09-22 09:55:39 +02:00
|
|
|
static inline gimple
|
|
|
|
get_earlier_stmt (gimple stmt1, gimple stmt2)
|
|
|
|
{
|
|
|
|
unsigned int uid1, uid2;
|
|
|
|
|
|
|
|
if (stmt1 == NULL)
|
|
|
|
return stmt2;
|
|
|
|
|
|
|
|
if (stmt2 == NULL)
|
|
|
|
return stmt1;
|
|
|
|
|
|
|
|
uid1 = gimple_uid (stmt1);
|
|
|
|
uid2 = gimple_uid (stmt2);
|
|
|
|
|
|
|
|
if (uid1 == 0 || uid2 == 0)
|
|
|
|
return NULL;
|
|
|
|
|
cgraph.h (varpool_first_static_initializer, [...]): Make checking only when checking enabled.
* cgraph.h (varpool_first_static_initializer,
varpool_next_static_initializer): Make checking only when
checking enabled.
* tree-vectorizer.h (vinfo_for_stmt): Remove check.
(set_vinfo_for_stmt, get_earlier_stmt, is_loop_header_bb_p): Change
gcc_assert to gcc_checking_assert.
* tree-flow-inline.h (gimple_vop, get_var_ann, relink_imm_use, phi_nodes
set_phi_nodes, phi_arg_index_from_use, op_iter_next_use,
op_iter_next_def, op_iter_next_tree, op_iter_init, op_iter_init_use,
op_iter_init_phiuse, op_iter_init_phidef, array_ref_contains_indirect_ref,
ref_contains_array_ref): Use gcc_checking_assert.
* emit-rtl.h (set_first_insn, set_last_insn): Likewise.
* tree-ssa-live.h (var_to_partition, var_to_partition_to_var,
partition_is_global, live_on_entry, live_on_exit,
live_merge_and_clear): Likewise.
* system.h (gcc_checking_assert): New macro.
From-SVN: r160489
2010-06-09 17:41:23 +02:00
|
|
|
gcc_checking_assert (uid1 <= VEC_length (vec_void_p, stmt_vec_info_vec)
|
|
|
|
&& uid2 <= VEC_length (vec_void_p, stmt_vec_info_vec));
|
2008-09-22 09:55:39 +02:00
|
|
|
|
|
|
|
if (uid1 < uid2)
|
|
|
|
return stmt1;
|
|
|
|
else
|
|
|
|
return stmt2;
|
|
|
|
}
|
|
|
|
|
2007-01-16 09:26:03 +01:00
|
|
|
static inline bool
|
|
|
|
is_pattern_stmt_p (stmt_vec_info stmt_info)
|
|
|
|
{
|
2008-07-28 16:33:56 +02:00
|
|
|
gimple related_stmt;
|
2007-01-16 09:26:03 +01:00
|
|
|
stmt_vec_info related_stmt_info;
|
|
|
|
|
|
|
|
related_stmt = STMT_VINFO_RELATED_STMT (stmt_info);
|
|
|
|
if (related_stmt
|
|
|
|
&& (related_stmt_info = vinfo_for_stmt (related_stmt))
|
|
|
|
&& STMT_VINFO_IN_PATTERN_P (related_stmt_info))
|
|
|
|
return true;
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2007-08-19 11:39:50 +02:00
|
|
|
static inline bool
|
|
|
|
is_loop_header_bb_p (basic_block bb)
|
|
|
|
{
|
|
|
|
if (bb == (bb->loop_father)->header)
|
|
|
|
return true;
|
cgraph.h (varpool_first_static_initializer, [...]): Make checking only when checking enabled.
* cgraph.h (varpool_first_static_initializer,
varpool_next_static_initializer): Make checking only when
checking enabled.
* tree-vectorizer.h (vinfo_for_stmt): Remove check.
(set_vinfo_for_stmt, get_earlier_stmt, is_loop_header_bb_p): Change
gcc_assert to gcc_checking_assert.
* tree-flow-inline.h (gimple_vop, get_var_ann, relink_imm_use, phi_nodes
set_phi_nodes, phi_arg_index_from_use, op_iter_next_use,
op_iter_next_def, op_iter_next_tree, op_iter_init, op_iter_init_use,
op_iter_init_phiuse, op_iter_init_phidef, array_ref_contains_indirect_ref,
ref_contains_array_ref): Use gcc_checking_assert.
* emit-rtl.h (set_first_insn, set_last_insn): Likewise.
* tree-ssa-live.h (var_to_partition, var_to_partition_to_var,
partition_is_global, live_on_entry, live_on_exit,
live_merge_and_clear): Likewise.
* system.h (gcc_checking_assert): New macro.
From-SVN: r160489
2010-06-09 17:41:23 +02:00
|
|
|
gcc_checking_assert (EDGE_COUNT (bb->preds) == 1);
|
2007-08-19 11:39:50 +02:00
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
static inline void
|
|
|
|
stmt_vinfo_set_inside_of_loop_cost (stmt_vec_info stmt_info, slp_tree slp_node,
|
2007-09-09 13:22:24 +02:00
|
|
|
int cost)
|
|
|
|
{
|
|
|
|
if (slp_node)
|
|
|
|
SLP_TREE_INSIDE_OF_LOOP_COST (slp_node) = cost;
|
|
|
|
else
|
|
|
|
STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info) = cost;
|
2009-11-25 11:55:54 +01:00
|
|
|
}
|
2007-09-09 13:22:24 +02:00
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
static inline void
|
|
|
|
stmt_vinfo_set_outside_of_loop_cost (stmt_vec_info stmt_info, slp_tree slp_node,
|
2007-09-09 13:22:24 +02:00
|
|
|
int cost)
|
|
|
|
{
|
|
|
|
if (slp_node)
|
|
|
|
SLP_TREE_OUTSIDE_OF_LOOP_COST (slp_node) = cost;
|
|
|
|
else
|
|
|
|
STMT_VINFO_OUTSIDE_OF_LOOP_COST (stmt_info) = cost;
|
2009-11-25 11:55:54 +01:00
|
|
|
}
|
2007-09-09 13:22:24 +02:00
|
|
|
|
2008-08-19 10:31:41 +02:00
|
|
|
static inline int
|
|
|
|
vect_pow2 (int x)
|
|
|
|
{
|
|
|
|
int i, res = 1;
|
|
|
|
|
|
|
|
for (i = 0; i < x; i++)
|
|
|
|
res *= 2;
|
|
|
|
|
|
|
|
return res;
|
|
|
|
}
|
2007-09-09 13:22:24 +02:00
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Info on data references alignment. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
|
2005-03-15 19:33:09 +01:00
|
|
|
/* Reflects actual alignment of first access in the vectorized loop,
|
|
|
|
taking into account peeling/versioning if applied. */
|
tree-vectorizer.h (DR_MISALIGNMENT): Cast aux to integer.
* tree-vectorizer.h (DR_MISALIGNMENT): Cast aux to integer.
(SET_DR_MISALIGNMENT): New.
* tree-vect-analyze.c (vect_compute_data_ref_alignment,
vect_update_misalignment_for_peel, vect_enhance_data_refs_alignment):
Use SET_DR_MISALIGNMENT.
* tree-predcom.c (split_data_refs_to_components): Cast dr->aux from
pointer.
* tree-data-ref.c (create_data_ref, compute_all_dependences,
find_loop_nest): Export.
* tree-data-ref.h (struct data_reference): Change aux field to pointer.
(create_data_ref, compute_all_dependences, find_loop_nest): Declare.
* tree-ssa-loop-prefetch.c: Include tree-data-ref.h.
(L1_CACHE_SIZE_BYTES, L2_CACHE_SIZE_BYTES, NONTEMPORAL_FRACTION):
New macros.
(struct mem_ref): Add field reuse_distance.
(find_or_create_group, record_ref): Use XNEW instead of xcalloc.
Initialize reuse_distance field.
(issue_prefetch_ref): Select temporality of prefetch according to
reuse_distance.
(volume_of_references, volume_of_dist_vector, add_subscript_strides,
self_reuse_distance, determine_loop_nest_reuse): New functions.
(loop_prefetch_arrays): Call determine_loop_nest_reuse.
(tree_ssa_prefetch_arrays): Dump L2 cache size.
* Makefile.in (tree-ssa-loop-prefetch.o): Add TREE_DATA_REF_H
dependency.
* gcc.dg/tree-ssa/prefetch-6.c: New test.
From-SVN: r125172
2007-05-29 23:55:47 +02:00
|
|
|
#define DR_MISALIGNMENT(DR) ((int) (size_t) (DR)->aux)
|
|
|
|
#define SET_DR_MISALIGNMENT(DR, VAL) ((DR)->aux = (void *) (size_t) (VAL))
|
2004-08-17 18:17:14 +02:00
|
|
|
|
|
|
|
static inline bool
|
|
|
|
aligned_access_p (struct data_reference *data_ref_info)
|
|
|
|
{
|
|
|
|
return (DR_MISALIGNMENT (data_ref_info) == 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline bool
|
2005-03-15 19:33:09 +01:00
|
|
|
known_alignment_for_access_p (struct data_reference *data_ref_info)
|
2004-08-17 18:17:14 +02:00
|
|
|
{
|
2005-03-15 19:33:09 +01:00
|
|
|
return (DR_MISALIGNMENT (data_ref_info) != -1);
|
2004-08-17 18:17:14 +02:00
|
|
|
}
|
|
|
|
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/* vect_dump will be set to stderr or dump_file if exist. */
|
|
|
|
extern FILE *vect_dump;
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern LOC vect_loop_location;
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Function prototypes. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
|
2009-11-25 11:55:54 +01:00
|
|
|
/* Simple loop peeling and versioning utilities for vectorizer's purposes -
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
in tree-vect-loop-manip.c. */
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
extern void slpeel_make_loop_iterate_ntimes (struct loop *, tree);
|
alias.c (component_uses_parent_alias_set): Constify.
* alias.c (component_uses_parent_alias_set): Constify.
* alias.h (component_uses_parent_alias_set): Likewise.
* cfgrtl.c (print_rtl_with_bb): Likewise.
* double-int.c (tree_to_double_int, double_int_fits_to_tree_p,
mpz_get_double_int): Likewise.
* double-int.h (double_int_fits_to_tree_p, tree_to_double_int,
mpz_get_double_int): Likewise.
* expr.c (is_aligning_offset, undefined_operand_subword_p,
mostly_zeros_p, all_zeros_p, safe_from_p, is_aligning_offset):
Likewise.
* expr.h (safe_from_p): Likewise.
* gimple-low.c (try_catch_may_fallthru, block_may_fallthru):
Likewise.
* gimplify.c (should_carry_locus_p, zero_sized_field_decl,
zero_sized_type, goa_lhs_expr_p): Likewise.
* omp-low.c (is_variable_sized, use_pointer_for_field): Likewise.
* rtl.h (print_rtl_with_bb): Likewise.
* sched-vis.c (print_exp, print_value, print_pattern): Likewise.
* tree-cfg.c (const_first_stmt, const_last_stmt): New.
* tree-flow-inline.h (bb_stmt_list): Constify.
(cbsi_start, cbsi_last, cbsi_end_p, cbsi_next, cbsi_prev,
cbsi_stmt): New.
* tree-flow.h (const_block_stmt_iterator, cbsi_start, cbsi_last,
const_first_stmt, const_last_stmt): New.
(block_may_fallthru, empty_block_p): Constify.
* tree-iterator.c (EXPR_FIRST_BODY, EXPR_LAST_BODY,
EXPR_ONLY_BODY): New.
(expr_first, expr_last, expr_only): Use macro for body.
(const_expr_first, const_expr_last, const_expr_only): New.
* tree-iterator.h (const_tree_stmt_iterator, ctsi_start,
ctsi_last, ctsi_end_p, ctsi_one_before_end_p, ctsi_next,
ctsi_prev, ctsi_stmt): New.
* tree-scalar-evolution.c (get_loop_exit_condition): Constify.
* tree-scalar-evolution.h (get_loop_exit_condition): Likewise.
* tree-ssa-loop-niter.c (loop_only_exit_p,
derive_constant_upper_bound): Likewise.
* tree-ssa-phiopt.c (empty_block_p): Likewise.
* tree-ssa-threadupdate.c (redirection_block_p): Likewise.
* tree-vectorizer.c (slpeel_can_duplicate_loop_p): Likewise.
* tree-vectorizer.h (slpeel_can_duplicate_loop_p): Likewise.
* tree-vrp.c (vrp_bitmap_equal_p): Likewise.
* tree.c (get_type_static_bounds): Likewise.
* tree.h (const_expr_first, const_expr_last, const_expr_only): New.
(get_type_static_bounds): Constify.
From-SVN: r127483
2007-08-14 17:18:11 +02:00
|
|
|
extern bool slpeel_can_duplicate_loop_p (const struct loop *, const_edge);
|
2009-04-04 22:53:39 +02:00
|
|
|
extern void vect_loop_versioning (loop_vec_info, bool, tree *, gimple_seq *);
|
|
|
|
extern void vect_do_peeling_for_loop_bound (loop_vec_info, tree *,
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
tree, gimple_seq);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern void vect_do_peeling_for_alignment (loop_vec_info);
|
|
|
|
extern LOC find_loop_location (struct loop *);
|
|
|
|
extern bool vect_can_advance_ivs_p (loop_vec_info);
|
2004-08-17 18:17:14 +02:00
|
|
|
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
/* In tree-vect-stmts.c. */
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
extern tree get_vectype_for_scalar_type (tree);
|
2010-04-09 12:40:14 +02:00
|
|
|
extern tree get_same_sized_vectype (tree, tree);
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
extern bool vect_is_simple_use (tree, loop_vec_info, bb_vec_info, gimple *,
|
|
|
|
tree *, enum vect_def_type *);
|
2010-04-09 12:40:14 +02:00
|
|
|
extern bool vect_is_simple_use_1 (tree, loop_vec_info, bb_vec_info, gimple *,
|
|
|
|
tree *, enum vect_def_type *, tree *);
|
|
|
|
extern bool supportable_widening_operation (enum tree_code, gimple, tree, tree,
|
2009-11-25 11:55:54 +01:00
|
|
|
tree *, tree *, enum tree_code *,
|
|
|
|
enum tree_code *, int *,
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
VEC (tree, heap) **);
|
2010-04-09 12:40:14 +02:00
|
|
|
extern bool supportable_narrowing_operation (enum tree_code, tree, tree,
|
|
|
|
enum tree_code *,
|
|
|
|
int *, VEC (tree, heap) **);
|
2009-11-25 11:55:54 +01:00
|
|
|
extern stmt_vec_info new_stmt_vec_info (gimple stmt, loop_vec_info,
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
bb_vec_info);
|
2008-07-28 16:33:56 +02:00
|
|
|
extern void free_stmt_vec_info (gimple stmt);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern tree vectorizable_function (gimple, tree, tree);
|
|
|
|
extern void vect_model_simple_cost (stmt_vec_info, int, enum vect_def_type *,
|
|
|
|
slp_tree);
|
|
|
|
extern void vect_model_store_cost (stmt_vec_info, int, enum vect_def_type,
|
|
|
|
slp_tree);
|
|
|
|
extern void vect_model_load_cost (stmt_vec_info, int, slp_tree);
|
|
|
|
extern void vect_finish_stmt_generation (gimple, gimple,
|
|
|
|
gimple_stmt_iterator *);
|
|
|
|
extern bool vect_mark_stmts_to_be_vectorized (loop_vec_info);
|
|
|
|
extern int cost_for_stmt (gimple);
|
|
|
|
extern tree vect_get_vec_def_for_operand (tree, gimple, tree *);
|
|
|
|
extern tree vect_init_vector (gimple, tree, tree,
|
|
|
|
gimple_stmt_iterator *);
|
|
|
|
extern tree vect_get_vec_def_for_stmt_copy (enum vect_def_type, tree);
|
|
|
|
extern bool vect_transform_stmt (gimple, gimple_stmt_iterator *,
|
|
|
|
bool *, slp_tree, slp_instance);
|
|
|
|
extern void vect_remove_stores (gimple);
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
extern bool vect_analyze_stmt (gimple, bool *, slp_tree);
|
2009-11-25 11:55:54 +01:00
|
|
|
extern bool vectorizable_condition (gimple, gimple_stmt_iterator *, gimple *,
|
2009-07-20 13:59:10 +02:00
|
|
|
tree, int);
|
2009-11-25 11:55:54 +01:00
|
|
|
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
/* In tree-vect-data-refs.c. */
|
|
|
|
extern bool vect_can_force_dr_alignment_p (const_tree, unsigned int);
|
|
|
|
extern enum dr_alignment_support vect_supportable_dr_alignment
|
|
|
|
(struct data_reference *);
|
|
|
|
extern tree vect_get_smallest_scalar_type (gimple, HOST_WIDE_INT *,
|
|
|
|
HOST_WIDE_INT *);
|
2010-04-13 11:48:26 +02:00
|
|
|
extern bool vect_analyze_data_ref_dependences (loop_vec_info, bb_vec_info,
|
|
|
|
int *);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern bool vect_enhance_data_refs_alignment (loop_vec_info);
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
extern bool vect_analyze_data_refs_alignment (loop_vec_info, bb_vec_info);
|
|
|
|
extern bool vect_verify_datarefs_alignment (loop_vec_info, bb_vec_info);
|
|
|
|
extern bool vect_analyze_data_ref_accesses (loop_vec_info, bb_vec_info);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern bool vect_prune_runtime_alias_test_list (loop_vec_info);
|
2010-04-13 11:48:26 +02:00
|
|
|
extern bool vect_analyze_data_refs (loop_vec_info, bb_vec_info, int *);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern tree vect_create_data_ref_ptr (gimple, struct loop *, tree, tree *,
|
2009-11-25 11:55:54 +01:00
|
|
|
gimple *, bool, bool *);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern tree bump_vector_ptr (tree, gimple, gimple_stmt_iterator *, gimple, tree);
|
|
|
|
extern tree vect_create_destination_var (tree, tree);
|
|
|
|
extern bool vect_strided_store_supported (tree);
|
|
|
|
extern bool vect_strided_load_supported (tree);
|
|
|
|
extern bool vect_permute_store_chain (VEC(tree,heap) *,unsigned int, gimple,
|
|
|
|
gimple_stmt_iterator *, VEC(tree,heap) **);
|
|
|
|
extern tree vect_setup_realignment (gimple, gimple_stmt_iterator *, tree *,
|
2009-11-25 11:55:54 +01:00
|
|
|
enum dr_alignment_support, tree,
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
struct loop **);
|
|
|
|
extern bool vect_permute_load_chain (VEC(tree,heap) *,unsigned int, gimple,
|
|
|
|
gimple_stmt_iterator *, VEC(tree,heap) **);
|
|
|
|
extern bool vect_transform_strided_load (gimple, VEC(tree,heap) *, int,
|
|
|
|
gimple_stmt_iterator *);
|
|
|
|
extern int vect_get_place_in_interleaving_chain (gimple, gimple);
|
|
|
|
extern tree vect_get_new_vect_var (tree, enum vect_var_kind, const char *);
|
|
|
|
extern tree vect_create_addr_base_for_vector_ref (gimple, gimple_seq *,
|
|
|
|
tree, struct loop *);
|
|
|
|
|
|
|
|
/* In tree-vect-loop.c. */
|
|
|
|
/* FORNOW: Used in tree-parloops.c. */
|
|
|
|
extern void destroy_loop_vec_info (loop_vec_info, bool);
|
2010-05-10 16:31:37 +02:00
|
|
|
extern gimple vect_force_simple_reduction (loop_vec_info, gimple, bool, bool *);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
/* Drive for loop analysis stage. */
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
extern loop_vec_info vect_analyze_loop (struct loop *);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
/* Drive for loop transformation stage. */
|
|
|
|
extern void vect_transform_loop (loop_vec_info);
|
2007-09-23 Razya Ladelsky
Zdenek Dvorak
OMP_ATOMIC Changes,
Reduction support for automatic parallelization.
* expr.c (expand_expr_real_1): Add cases for OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* Makefile.in: Add dependencies to expr.o, tree-parloops.o, omp-low.o
* tree-pretty-print.c (dump_generic_node): Add OMP_ATOMIC_LOAD
and OMP_ATOMIC_STORE.
* tree.h (OMP_DIRECTIVE_P): Add OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* gimple-low.c (lower_stmt): Same.
* gimplify.c (gimplify_expr): Same.
(gimplify_omp_atomic_fetch_op, gimplify_omp_atomic_pipeline,
gimplify_omp_atomic_mutex): Remove.
(gimplify_omp_atomic): Change it to simply gimplify the
statement instead of expanding it.
* omp-low.c: Add includes to optabs.h, cfgloop.h.
(expand_omp_atomic, expand_omp_atomic_pipeline,
goa_stabilize_expr, expand_omp_atomic_mutex,
expand_omp_atomic_fetch_op): New functions to implement
expansion of OMP_ATOMIC.
(expand_omp, build_omp_regions_1): Add support for
OMP_ATOMIC_LOAD/OMP_ATOMIC_STORE.
* tree-cfg.c (make_edges): add case for OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-gimple.c (is_gimple_stmt): Add OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-parloops.c: add include to tree-vectorizer.h.
(reduction_info): New structure for reduction.
(reduction_list): New list to represent list of reductions
per loop.
(struct data_arg): New helper structure for reduction.
(reduction_info_hash, reduction_info_eq, reduction_phi,
initialize_reductions,
create_call_for_reduction, create_phi_for_local_result,
create_call_for_reduction_1, create_loads_for_reductions,
create_final_loads_for_reduction): New functions.
(loop_parallel_p): Identify reductions, add reduction_list parameter.
(separate_decls_in_loop_name): Support reduction variables.
(separate_decls_in_loop): Add reduction_list and ld_st_data arguments,
call create_loads_for_reduction for each reduction.
(canonicalize_loop_ivs): Identify reductions, add reduction_list
parameter.
(transform_to_exit_first_loop): Add reduction support, add
reduction_list parameter.
(gen_parallel_loop): Add reduction_list parameter. Add call
separate_decls_in_loop with
the new argument. Traverse reductions and call
initialize_reductions, create_call_for_reduction.
(parallelize_loops): Create and delete the reduction list.
(add_field_for_name): Change use of data parameter. Add fields for
reductions.
* tree-vectorizer.h (vect_analyze_loop_form): Add declaration.
* tree-vect-analyze.c (vect_analyze_loop_form): export it.
* tree.def: Add definitions for OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-inline.c (estimate_num_insns_1): add cases for
OMP_ATOMIC_LOAD, OMP_ATOMIC_STORE.
* tree-cfg.c (make_edges): Add OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-ssa-operands.c (get_addr_dereference_operands):
New function. Subroutine of get_indirect_ref_operands.
(get_indirect_ref_operands): Call get_addr_dereference_operands.
(get_expr_operands): Support OMP_ATOMIC_LOAD, OMP_ATOMIC_STORE.
From-SVN: r129716
2007-10-29 12:05:04 +01:00
|
|
|
extern loop_vec_info vect_analyze_loop_form (struct loop *);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern bool vectorizable_live_operation (gimple, gimple_stmt_iterator *,
|
|
|
|
gimple *);
|
2010-04-19 11:10:45 +02:00
|
|
|
extern bool vectorizable_reduction (gimple, gimple_stmt_iterator *, gimple *,
|
|
|
|
slp_tree);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern bool vectorizable_induction (gimple, gimple_stmt_iterator *, gimple *);
|
|
|
|
extern int vect_estimate_min_profitable_iters (loop_vec_info);
|
|
|
|
extern tree get_initial_def_for_reduction (gimple, tree, tree *);
|
|
|
|
extern int vect_min_worthwhile_factor (enum tree_code);
|
|
|
|
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
/* In tree-vect-slp.c. */
|
|
|
|
extern void vect_free_slp_instance (slp_instance);
|
|
|
|
extern bool vect_transform_slp_perm_load (gimple, VEC (tree, heap) *,
|
2009-11-25 11:55:54 +01:00
|
|
|
gimple_stmt_iterator *, int,
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
slp_instance, bool);
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
extern bool vect_schedule_slp (loop_vec_info, bb_vec_info);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern void vect_update_slp_costs_according_to_vf (loop_vec_info);
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
extern bool vect_analyze_slp (loop_vec_info, bb_vec_info);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
extern void vect_make_slp_decision (loop_vec_info);
|
|
|
|
extern void vect_detect_hybrid_slp (loop_vec_info);
|
|
|
|
extern void vect_get_slp_defs (slp_tree, VEC (tree,heap) **,
|
2010-04-19 11:10:45 +02:00
|
|
|
VEC (tree,heap) **, int);
|
passes.texi (Tree-SSA passes): Document SLP pass.
* doc/passes.texi (Tree-SSA passes): Document SLP pass.
* tree-pass.h (pass_slp_vectorize): New pass.
* params.h (SLP_MAX_INSNS_IN_BB): Define.
* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
* tree-vectorizer.c (timevar.h): Include.
(user_vect_verbosity_level): Declare.
(vect_location): Fix comment.
(vect_set_verbosity_level): Update user_vect_verbosity_level
instead of vect_verbosity_level.
(vect_set_dump_settings): Add an argument. Ignore user defined
verbosity if dump flags require higher level of verbosity. Print to
stderr only for loop vectorization.
(vectorize_loops): Update call to vect_set_dump_settings.
(execute_vect_slp): New function.
(gate_vect_slp): Likewise.
(struct gimple_opt_pass pass_slp_vectorize): New.
* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
access its members.
(vec_info_for_bb): New function.
(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
(VECTORIZATION_ENABLED): New macro.
(SLP_ENABLED, SLP_DISABLED): Likewise.
(vect_is_simple_use): Add bb_vec_info argument.
(new_stmt_vec_info, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
vect_analyze_data_ref_accesses, vect_analyze_data_refs,
vect_schedule_slp, vect_analyze_slp): Likewise.
(vect_analyze_stmt): Add slp_tree argument.
(find_bb_location): Declare.
(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
(vect_analyze_loop_operations, vect_analyze_loop,
get_initial_def_for_induction, vect_create_epilog_for_reduction,
vect_finalize_reduction, vectorizable_reduction,
vectorizable_live_operation, vect_transform_loop): Likewise.
* tree-data-ref.c (dr_analyze_innermost): Update comment,
skip evolution analysis if analyzing a basic block.
(dr_analyze_indices): Likewise.
(initialize_data_dependence_relation): Skip the test whether the
object is invariant for basic blocks.
(compute_all_dependences): Skip dependence analysis for data
references in basic blocks.
(find_data_references_in_stmt): Don't fail in case of invariant
access in basic block.
(find_data_references_in_bb): New function.
(find_data_references_in_loop): Move code to
find_data_references_in_bb and add a call to it.
(compute_data_dependences_for_bb): New function.
* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
that STEP is 0.
(vect_analyze_data_ref_dependence): Check for interleaving in case of
unknown dependence in basic block and fail in case of dependence in
basic block.
(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_compute_data_ref_alignment): Check if it is loop vectorization
before calling nested_in_vect_loop_p.
(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Adjust function calls.
(vect_analyze_data_refs_alignment): Likewise.
(vect_analyze_group_access): Fix printing. Skip different checks if
DR_STEP is 0. Keep strided stores either in loop or basic block
vectorization data structure. Fix indentation.
(vect_analyze_data_ref_access): Fix comments, allow zero step in
basic blocks.
(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
dependence instances from either loop or basic block vectorization
info.
(vect_analyze_data_refs): Update comment. Call
compute_data_dependences_for_bb to analyze basic blocks.
(vect_create_addr_base_for_vector_ref): Check for outer loop only in
case of loop vectorization. In case of basic block vectorization use
data-ref itself as a base.
(vect_create_data_ref_ptr): In case of basic block vectorization:
don't advance the pointer, add new statements before the current
statement. Adjust function calls.
(vect_supportable_dr_alignment): Support only aligned accesses in
basic block vectorization.
* common.opt (ftree-slp-vectorize): New flag.
* tree-vect-patterns.c (widened_name_p): Adjust function calls.
(vect_pattern_recog_1): Likewise.
* tree-vect-stmts.c (process_use): Likewise.
(vect_init_vector): Add new statements in the beginning of the basic
block in case of basic block SLP.
(vect_get_vec_def_for_operand): Adjust function calls.
(vect_finish_stmt_generation): Likewise.
(vectorizable_call): Add assert that it is loop vectorization, adjust
function calls.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_operation): In case of basic block SLP, take
vectorization factor from statement's type and skip the relevance
check. Adjust function calls.
(vectorizable_type_demotion): Add assert that it is loop
vectorization, adjust function calls.
(vectorizable_type_promotion): Likewise.
(vectorizable_store): Check for outer loop only in case of loop
vectorization. Adjust function calls. For basic blocks, skip the
relevance check and don't advance pointers.
(vectorizable_load): Likewise.
(vectorizable_condition): Add assert that it is loop vectorization,
adjust function calls.
(vect_analyze_stmt): Add argument. In case of basic block SLP, check
that it is not reduction, get vector type, call only supported
functions, skip loop specific parts.
(vect_transform_stmt): Check for outer loop only in case of loop
vectorization.
(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
(vect_is_simple_use): Fix comment, add new argument, fix conditions
for external definition.
* passes.c (pass_slp_vectorize): New pass.
* tree-vect-slp.c (find_bb_location): New function.
(vect_get_and_check_slp_defs): Add argument, adjust function calls,
check for patterns only in loops.
(vect_build_slp_tree): Add argument, adjust function calls, fail in
case of multiple types in basic block SLP.
(vect_mark_slp_stmts_relevant): New function.
(vect_supported_load_permutation_p): Fix comment.
(vect_analyze_slp_instance): Add argument. In case of basic block
SLP, take vectorization factor from statement's type, check that
unrolling factor is 1. Adjust function call. Save SLP instance in
either loop or basic block vectorization structure. Return FALSE,
if SLP failed.
(vect_analyze_slp): Add argument. Get strided stores groups from
either loop or basic block vectorization structure. Return FALSE
if basic block SLP failed.
(new_bb_vec_info): New function.
(destroy_bb_vec_info, vect_slp_analyze_node_operations,
vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
(vect_schedule_slp): Add argument. Get SLP instances from either
loop or basic block vectorization structure. Set vectorization factor
to be 1 for basic block SLP.
(vect_slp_transform_bb): New function.
* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.
From-SVN: r147829
2009-05-24 10:44:56 +02:00
|
|
|
extern LOC find_bb_location (basic_block);
|
|
|
|
extern bb_vec_info vect_slp_analyze_bb (basic_block);
|
|
|
|
extern void vect_slp_transform_bb (basic_block);
|
tree-vect-loop-manip.c: New file.
* tree-vect-loop-manip.c: New file.
* tree-vectorizer.c: Update documentation and included files.
(vect_loop_location): Make extern.
(rename_use_op): Move to tree-vect-loop-manip.c
(rename_variables_in_bb, rename_variables_in_loop,
slpeel_update_phis_for_duplicate_loop,
slpeel_update_phi_nodes_for_guard1,
slpeel_update_phi_nodes_for_guard2, slpeel_make_loop_iterate_ntimes,
slpeel_tree_duplicate_loop_to_edge_cfg, slpeel_add_loop_guard,
slpeel_can_duplicate_loop_p, slpeel_verify_cfg_after_peeling,
set_prologue_iterations, slpeel_tree_peel_loop_to_edge,
find_loop_location): Likewise.
(new_stmt_vec_info): Move to tree-vect-stmts.c.
(init_stmt_vec_info_vec, free_stmt_vec_info_vec, free_stmt_vec_info,
get_vectype_for_scalar_type, vect_is_simple_use,
supportable_widening_operation, supportable_narrowing_operation):
Likewise.
(bb_in_loop_p): Move to tree-vect-loop.c.
(new_loop_vec_info, destroy_loop_vec_info,
reduction_code_for_scalar_code, report_vect_op,
vect_is_simple_reduction, vect_is_simple_iv_evolution): Likewise.
(vect_can_force_dr_alignment_p): Move to tree-vect-data-refs.c.
(vect_supportable_dr_alignment): Likewise.
* tree-vectorizer.h (tree-data-ref.h): Include.
(vect_loop_location): Declare.
Reorganize function declarations according to the new file structure.
* tree-vect-loop.c: New file.
* tree-vect-analyze.c: Remove. Move functions to tree-vect-data-refs.c,
tree-vect-stmts.c, tree-vect-slp.c, tree-vect-loop.c.
* tree-vect-data-refs.c: New file.
* tree-vect-patterns.c (timevar.h): Don't include.
* tree-vect-stmts.c: New file.
* tree-vect-transform.c: Remove. Move functions to tree-vect-stmts.c,
tree-vect-slp.c, tree-vect-loop.c.
* Makefile.in (OBJS-common): Remove tree-vect-analyze.o and
tree-vect-transform.o. Add tree-vect-data-refs.o, tree-vect-stmts.o,
tree-vect-loop.o, tree-vect-loop-manip.o, tree-vect-slp.o.
(tree-vect-analyze.o): Remove.
(tree-vect-transform.o): Likewise.
(tree-vect-data-refs.o): Add rule.
(tree-vect-stmts.o, tree-vect-loop.o, tree-vect-loop-manip.o,
tree-vect-slp.o): Likewise.
(tree-vect-patterns.o): Remove redundant dependencies.
(tree-vectorizer.o): Likewise.
* tree-vect-slp.c: New file.
From-SVN: r145280
2009-03-30 09:22:04 +02:00
|
|
|
|
|
|
|
/* In tree-vect-patterns.c. */
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
/* Pattern recognition functions.
|
|
|
|
Additional pattern recognition functions can (and will) be added
|
|
|
|
in the future. */
|
2008-07-28 16:33:56 +02:00
|
|
|
typedef gimple (* vect_recog_func_ptr) (gimple, tree *, tree *);
|
2006-11-21 15:29:53 +01:00
|
|
|
#define NUM_PATTERNS 4
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
void vect_pattern_recog (loop_vec_info);
|
|
|
|
|
2009-07-08 13:35:18 +02:00
|
|
|
/* In tree-vectorizer.c. */
|
|
|
|
unsigned vectorize_loops (void);
|
|
|
|
/* Vectorization debug information */
|
tree-vectorizer.h (struct _loop_vec_info): Remove loop_line_number.
* tree-vectorizer.h (struct _loop_vec_info): Remove loop_line_number.
(LOOP_VINFO_LOC, LOOP_LOC): Remove.
* tree-vectorizer.c (vect_loop_location): New.
(vect_print_dump_info): Use vect_loop_location.
(new_loop_vec_info): Do not set LOOP_VINFO_LOC.
(vectorize_loops): Set vect_loop_location.
* tree-vect-analyze.c (vect_analyze_offset_expr,
vect_determin_vectorization_factor, vect_analyze_operations,
vect_analyze_scalar_cycles, vect_analyze_data_ref_dependence,
vect_analyze_data_ref_dependences, vect_compute_data_ref_alignment,
vect_analyze_data_refs_alignment, vect_analyze_data_ref_access,
vect_analyze_data_ref_accesses, vect_analyze_pointer_ref_access,
vect_object_analysis, vect_analyze_data_refs, vect_mark_relevant,
vect_stmt_relevant_p, vect_mark_stmts_to_be_vectorized,
vect_can_advance_ivs_p, vect_get_loop_niters, vect_analyze_loop_form,
vect_analyze_loop): Adjust vect_print_dump_info API.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref,
vect_create_data_ref_ptr, vect_init_vector, vect_get_vec_def_for_operand,
vect_finish_stmt_generation, vectorizable_assignment,
vectorizable_operation, vectorizable_store, vectorizable_load,
vectorizable_live_operation, vectorizable_condition, vect_transform_stmt,
vect_update_ivs_after_vectorizer, vect_do_peeling_for_loop_bound,
vect_gen_ninters_for_prolog_loop, vect_do_peeling_for_alignment,
vect_transform_loop): Same.
* tree-vectorizer.c (get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_reduction, vect_is_simple_iv_evolution, vectorize_loops):
Same.
From-SVN: r101649
2005-07-06 04:25:08 +02:00
|
|
|
extern bool vect_print_dump_info (enum verbosity_levels);
|
2004-08-17 18:17:14 +02:00
|
|
|
|
|
|
|
#endif /* GCC_TREE_VECTORIZER_H */
|