2004-08-17 18:17:14 +02:00
|
|
|
/* Loop Vectorization
|
2007-06-08 18:30:49 +02:00
|
|
|
Copyright (C) 2003, 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
|
2004-08-17 18:17:14 +02:00
|
|
|
Contributed by Dorit Naishlos <dorit@il.ibm.com>
|
|
|
|
|
|
|
|
This file is part of GCC.
|
|
|
|
|
|
|
|
GCC is free software; you can redistribute it and/or modify it under
|
|
|
|
the terms of the GNU General Public License as published by the Free
|
2007-07-26 10:37:01 +02:00
|
|
|
Software Foundation; either version 3, or (at your option) any later
|
2004-08-17 18:17:14 +02:00
|
|
|
version.
|
|
|
|
|
|
|
|
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
|
|
|
|
WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
|
|
|
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
|
|
|
for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License
|
2007-07-26 10:37:01 +02:00
|
|
|
along with GCC; see the file COPYING3. If not see
|
|
|
|
<http://www.gnu.org/licenses/>. */
|
2004-08-17 18:17:14 +02:00
|
|
|
|
|
|
|
#ifndef GCC_TREE_VECTORIZER_H
|
|
|
|
#define GCC_TREE_VECTORIZER_H
|
|
|
|
|
2005-02-03 17:22:22 +01:00
|
|
|
#ifdef USE_MAPPED_LOCATION
|
|
|
|
typedef source_location LOC;
|
|
|
|
#define UNKNOWN_LOC UNKNOWN_LOCATION
|
|
|
|
#define EXPR_LOC(e) EXPR_LOCATION(e)
|
|
|
|
#define LOC_FILE(l) LOCATION_FILE (l)
|
|
|
|
#define LOC_LINE(l) LOCATION_LINE (l)
|
|
|
|
#else
|
|
|
|
typedef source_locus LOC;
|
|
|
|
#define UNKNOWN_LOC NULL
|
|
|
|
#define EXPR_LOC(e) EXPR_LOCUS(e)
|
|
|
|
#define LOC_FILE(l) (l)->file
|
|
|
|
#define LOC_LINE(l) (l)->line
|
|
|
|
#endif
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/* Used for naming of new temporaries. */
|
|
|
|
enum vect_var_kind {
|
|
|
|
vect_simple_var,
|
tree.def (REDUC_MAX_EXPR, [...]): New tree-codes.
* tree.def (REDUC_MAX_EXPR, REDUC_MIN_EXPR, REDUC_PLUS_EXPR): New
tree-codes.
* optabs.h (OTI_reduc_smax, OTI_reduc_umax, OTI_reduc_smin,
OTI_reduc_umin, OTI_reduc_plus): New optabs for reduction.
(reduc_smax_optab, reduc_umax_optab, reduc_smin_optab, reduc_umin_optab,
reduc_plus_optab): New optabs for reduction.
* expr.c (expand_expr_real_1): Handle new tree-codes.
* tree-inline.c (estimate_num_insns_1): Handle new tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio, op_symbol): Handle
new tree-codes.
* optabs.c (optab_for_tree_code): Handle new tree-codes.
(init_optabs): Initialize new optabs.
* genopinit.c (optabs): Define handlers for new optabs.
* tree-vect-analyze.c (vect_analyze_operations): Fail vectorization in
case of a phi that is marked as relevant. Call vectorizable_reduction.
(vect_mark_relevant): Phis may be marked as relevant.
(vect_mark_stmts_to_be_vectorized): The use corresponding to the
reduction variable in a reduction stmt does not mark its defining phi
as relevant. Update documentation accordingly.
(vect_can_advance_ivs_p): Skip reduction phis.
* tree-vect-transform.c (vect_get_vec_def_for_operand): Takes
additional argument. Handle reduction.
(vect_create_destination_var): Update call to vect_get_new_vect_var.
Handle non-vector argument.
(get_initial_def_for_reduction): New function.
(vect_create_epilog_for_reduction): New function.
(vectorizable_reduction): New function.
(vect_get_new_vect_var): Handle new vect_var_kind.
(vectorizable_assignment, vectorizable_operation, vectorizable_store,
vectorizable_condition): Update call to vect_get_new_vect_var.
(vect_transform_stmt): Call vectorizable_reduction.
(vect_update_ivs_after_vectorizer): Skip reduction phis.
(vect_transform_loop): Skip if stmt is both not relevant and not live.
* tree-vectorizer.c (reduction_code_for_scalar_code): New function.
(vect_is_simple_reduction): Was empty - added implementation.
* tree-vectorizer.h (vect_scalar_var): New enum vect_var_kind value.
(reduc_vec_info_type): New enum vect_def_type value.
* config/rs6000/altivec.md (reduc_smax_v4si, reduc_smax_v4sf,
reduc_umax_v4si, reduc_smin_v4si, reduc_umin_v4sf, reduc_smin_v4sf,
reduc_plus_v4si, reduc_plus_v4sf): New define_expands.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Remove
ENABLE_CHECKING around gcc_assert.
* tree-vect-transform.c (vect_do_peeling_for_loop_bound,
(vect_do_peeling_for_alignment, vect_transform_loop,
vect_get_vec_def_for_operand): Likewise.
From-SVN: r101155
2005-06-18 15:18:52 +02:00
|
|
|
vect_pointer_var,
|
|
|
|
vect_scalar_var
|
2004-08-17 18:17:14 +02:00
|
|
|
};
|
|
|
|
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
/* Defines type of operation. */
|
2004-08-17 18:17:14 +02:00
|
|
|
enum operation_type {
|
|
|
|
unary_op = 1,
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
binary_op,
|
|
|
|
ternary_op
|
2004-08-17 18:17:14 +02:00
|
|
|
};
|
|
|
|
|
2004-11-04 06:26:53 +01:00
|
|
|
/* Define type of available alignment support. */
|
|
|
|
enum dr_alignment_support {
|
|
|
|
dr_unaligned_unsupported,
|
|
|
|
dr_unaligned_supported,
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
dr_explicit_realign,
|
|
|
|
dr_explicit_realign_optimized,
|
2004-11-04 06:26:53 +01:00
|
|
|
dr_aligned
|
|
|
|
};
|
|
|
|
|
2005-06-12 16:03:01 +02:00
|
|
|
/* Define type of def-use cross-iteration cycle. */
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
enum vect_def_type {
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
vect_constant_def = 1,
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
vect_invariant_def,
|
|
|
|
vect_loop_def,
|
|
|
|
vect_induction_def,
|
|
|
|
vect_reduction_def,
|
|
|
|
vect_unknown_def_type
|
|
|
|
};
|
|
|
|
|
2005-02-07 11:07:07 +01:00
|
|
|
/* Define verbosity levels. */
|
|
|
|
enum verbosity_levels {
|
|
|
|
REPORT_NONE,
|
|
|
|
REPORT_VECTORIZED_LOOPS,
|
|
|
|
REPORT_UNVECTORIZED_LOOPS,
|
2007-12-17 21:54:01 +01:00
|
|
|
REPORT_COST,
|
2005-02-07 11:07:07 +01:00
|
|
|
REPORT_ALIGNMENT,
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
REPORT_DR_DETAILS,
|
2005-02-07 11:07:07 +01:00
|
|
|
REPORT_BAD_FORM_LOOPS,
|
|
|
|
REPORT_OUTER_LOOPS,
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
REPORT_SLP,
|
2005-02-07 11:07:07 +01:00
|
|
|
REPORT_DETAILS,
|
|
|
|
/* New verbosity levels should be added before this one. */
|
|
|
|
MAX_VERBOSITY_LEVEL
|
|
|
|
};
|
|
|
|
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
/************************************************************************
|
|
|
|
SLP
|
|
|
|
************************************************************************/
|
|
|
|
|
|
|
|
/* A computation tree of an SLP instance. Each node corresponds to a group of
|
|
|
|
stmts to be packed in a SIMD stmt. */
|
|
|
|
typedef struct _slp_tree {
|
|
|
|
/* Only binary and unary operations are supported. LEFT child corresponds to
|
|
|
|
the first operand and RIGHT child to the second if the operation is
|
|
|
|
binary. */
|
|
|
|
struct _slp_tree *left;
|
|
|
|
struct _slp_tree *right;
|
|
|
|
/* A group of scalar stmts to be vectorized together. */
|
|
|
|
VEC (tree, heap) *stmts;
|
|
|
|
/* Vectorized stmt/s. */
|
|
|
|
VEC (tree, heap) *vec_stmts;
|
|
|
|
/* Number of vector stmts that are created to replace the group of scalar
|
|
|
|
stmts. It is calculated during the transformation phase as the number of
|
|
|
|
scalar elements in one scalar iteration (GROUP_SIZE) multiplied by VF
|
|
|
|
divided by vector size. */
|
|
|
|
unsigned int vec_stmts_size;
|
|
|
|
/* Vectorization costs associated with SLP node. */
|
|
|
|
struct
|
|
|
|
{
|
|
|
|
int outside_of_loop; /* Statements generated outside loop. */
|
|
|
|
int inside_of_loop; /* Statements generated inside loop. */
|
|
|
|
} cost;
|
|
|
|
} *slp_tree;
|
|
|
|
|
|
|
|
|
|
|
|
/* SLP instance is a sequence of stmts in a loop that can be packed into
|
|
|
|
SIMD stmts. */
|
|
|
|
typedef struct _slp_instance {
|
|
|
|
/* The root of SLP tree. */
|
|
|
|
slp_tree root;
|
|
|
|
|
|
|
|
/* Size of groups of scalar stmts that will be replaced by SIMD stmt/s. */
|
|
|
|
unsigned int group_size;
|
|
|
|
|
|
|
|
/* The unrolling factor required to vectorized this SLP instance. */
|
|
|
|
unsigned int unrolling_factor;
|
|
|
|
|
|
|
|
/* Vectorization costs associated with SLP instance. */
|
|
|
|
struct
|
|
|
|
{
|
|
|
|
int outside_of_loop; /* Statements generated outside loop. */
|
|
|
|
int inside_of_loop; /* Statements generated inside loop. */
|
|
|
|
} cost;
|
|
|
|
} *slp_instance;
|
|
|
|
|
|
|
|
DEF_VEC_P(slp_instance);
|
|
|
|
DEF_VEC_ALLOC_P(slp_instance, heap);
|
|
|
|
|
|
|
|
/* Access Functions. */
|
|
|
|
#define SLP_INSTANCE_TREE(S) (S)->root
|
|
|
|
#define SLP_INSTANCE_GROUP_SIZE(S) (S)->group_size
|
|
|
|
#define SLP_INSTANCE_UNROLLING_FACTOR(S) (S)->unrolling_factor
|
|
|
|
#define SLP_INSTANCE_OUTSIDE_OF_LOOP_COST(S) (S)->cost.outside_of_loop
|
|
|
|
#define SLP_INSTANCE_INSIDE_OF_LOOP_COST(S) (S)->cost.inside_of_loop
|
|
|
|
|
|
|
|
#define SLP_TREE_LEFT(S) (S)->left
|
|
|
|
#define SLP_TREE_RIGHT(S) (S)->right
|
|
|
|
#define SLP_TREE_SCALAR_STMTS(S) (S)->stmts
|
|
|
|
#define SLP_TREE_VEC_STMTS(S) (S)->vec_stmts
|
|
|
|
#define SLP_TREE_NUMBER_OF_VEC_STMTS(S) (S)->vec_stmts_size
|
|
|
|
#define SLP_TREE_OUTSIDE_OF_LOOP_COST(S) (S)->cost.outside_of_loop
|
|
|
|
#define SLP_TREE_INSIDE_OF_LOOP_COST(S) (S)->cost.inside_of_loop
|
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Info on vectorized loops. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
typedef struct _loop_vec_info {
|
|
|
|
|
|
|
|
/* The loop to which this info struct refers to. */
|
|
|
|
struct loop *loop;
|
|
|
|
|
|
|
|
/* The loop basic blocks. */
|
|
|
|
basic_block *bbs;
|
|
|
|
|
|
|
|
/* Number of iterations. */
|
|
|
|
tree num_iters;
|
2007-12-10 23:49:43 +01:00
|
|
|
tree num_iters_unchanged;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2007-07-05 15:15:20 +02:00
|
|
|
/* Minimum number of iterations below which vectorization is expected to
|
|
|
|
not be profitable (as estimated by the cost model).
|
|
|
|
-1 indicates that vectorization will not be profitable.
|
|
|
|
FORNOW: This field is an int. Will be a tree in the future, to represent
|
|
|
|
values unknown at compile time. */
|
|
|
|
int min_profitable_iters;
|
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
/* Is the loop vectorizable? */
|
|
|
|
bool vectorizable;
|
|
|
|
|
|
|
|
/* Unrolling factor */
|
|
|
|
int vectorization_factor;
|
|
|
|
|
|
|
|
/* Unknown DRs according to which loop was peeled. */
|
|
|
|
struct data_reference *unaligned_dr;
|
|
|
|
|
2005-03-15 19:33:09 +01:00
|
|
|
/* peeling_for_alignment indicates whether peeling for alignment will take
|
|
|
|
place, and what the peeling factor should be:
|
|
|
|
peeling_for_alignment = X means:
|
|
|
|
If X=0: Peeling for alignment will not be applied.
|
|
|
|
If X>0: Peel first X iterations.
|
|
|
|
If X=-1: Generate a runtime test to calculate the number of iterations
|
|
|
|
to be peeled, using the dataref recorded in the field
|
|
|
|
unaligned_dr. */
|
|
|
|
int peeling_for_alignment;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2005-09-06 12:59:58 +02:00
|
|
|
/* The mask used to check the alignment of pointers or arrays. */
|
|
|
|
int ptr_mask;
|
|
|
|
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
/* All data references in the loop. */
|
tree-loop-linear.c: Don't include varray.h.
* tree-loop-linear.c: Don't include varray.h.
(gather_interchange_stats, try_interchange_loops,
linear_transform_loops): Use VEC instead of VARRAY.
* lambda-mat.c: Don't include varray.h.
* tree-chrec.c: Same.
* lambda-trans.c: Same.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info): Use
VEC instead of VARRAY.
* tree-vectorizer.h: Idem.
* tree-data-ref.c (dump_data_references,
dump_data_dependence_relations, dump_dist_dir_vectors, dump_ddrs,
initialize_data_dependence_relation, finalize_ddr_dependent,
compute_all_dependences, find_data_references_in_loop,
compute_data_dependences_for_loop, analyze_all_data_dependences,
free_dependence_relation, free_dependence_relations,
free_data_refs): Idem.
* tree-data-ref.h (data_reference_p, subscript_p): New.
(data_dependence_relation, DDR_SUBSCRIPT, DDR_NUM_SUBSCRIPTS): Use
VEC instead of VARRAY.
(DDR_SUBSCRIPTS_VECTOR_INIT): Removed.
(find_data_references_in_loop, compute_data_dependences_for_loop,
dump_ddrs, dump_dist_dir_vectors, dump_data_references,
dump_data_dependence_relations, free_dependence_relations,
free_data_refs): Adjust declaration.
(lambda_transform_legal_p): Move declaration here...
* tree-vect-analyze.c (vect_analyze_data_ref_dependences,
vect_compute_data_refs_alignment, vect_verify_datarefs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_data_ref_accesses,
vect_analyze_data_refs): Use VEC instead of VARRAY.
* lambda.h (lambda_transform_legal_p): ...from here.
* lambda-code.c (lambda_transform_legal_p): Use VEC instead of VARRAY.
* tree-vect-transform.c (vect_update_inits_of_drs): Idem.
* Makefile.in (tree-loop-linear.o, lambda-mat.o, lambda-trans.o,
tree-chrec.o): Don't depend on VARRAY_H.
From-SVN: r112437
2006-03-28 06:19:26 +02:00
|
|
|
VEC (data_reference_p, heap) *datarefs;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
/* All data dependences in the loop. */
|
tree-loop-linear.c: Don't include varray.h.
* tree-loop-linear.c: Don't include varray.h.
(gather_interchange_stats, try_interchange_loops,
linear_transform_loops): Use VEC instead of VARRAY.
* lambda-mat.c: Don't include varray.h.
* tree-chrec.c: Same.
* lambda-trans.c: Same.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info): Use
VEC instead of VARRAY.
* tree-vectorizer.h: Idem.
* tree-data-ref.c (dump_data_references,
dump_data_dependence_relations, dump_dist_dir_vectors, dump_ddrs,
initialize_data_dependence_relation, finalize_ddr_dependent,
compute_all_dependences, find_data_references_in_loop,
compute_data_dependences_for_loop, analyze_all_data_dependences,
free_dependence_relation, free_dependence_relations,
free_data_refs): Idem.
* tree-data-ref.h (data_reference_p, subscript_p): New.
(data_dependence_relation, DDR_SUBSCRIPT, DDR_NUM_SUBSCRIPTS): Use
VEC instead of VARRAY.
(DDR_SUBSCRIPTS_VECTOR_INIT): Removed.
(find_data_references_in_loop, compute_data_dependences_for_loop,
dump_ddrs, dump_dist_dir_vectors, dump_data_references,
dump_data_dependence_relations, free_dependence_relations,
free_data_refs): Adjust declaration.
(lambda_transform_legal_p): Move declaration here...
* tree-vect-analyze.c (vect_analyze_data_ref_dependences,
vect_compute_data_refs_alignment, vect_verify_datarefs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_data_ref_accesses,
vect_analyze_data_refs): Use VEC instead of VARRAY.
* lambda.h (lambda_transform_legal_p): ...from here.
* lambda-code.c (lambda_transform_legal_p): Use VEC instead of VARRAY.
* tree-vect-transform.c (vect_update_inits_of_drs): Idem.
* Makefile.in (tree-loop-linear.o, lambda-mat.o, lambda-trans.o,
tree-chrec.o): Don't depend on VARRAY_H.
From-SVN: r112437
2006-03-28 06:19:26 +02:00
|
|
|
VEC (ddr_p, heap) *ddrs;
|
2005-02-03 17:13:17 +01:00
|
|
|
|
tree-vectorizer.c (new_loop_vec_info): Initialize new field.
gcc/ChangeLog
* tree-vectorizer.c (new_loop_vec_info): Initialize new
field.
(destroy_loop_vec_info): Add call to VEC_free.
* tree-vectorizer.h (may_alias_ddrs): Define.
(LOOP_VINFO_MAY_ALIAS_DDRS): Define.
* tree-vect-analyze.c (vect_analyze_data_ref_dependence):
Change reporting to dump.
(vect_is_duplicate_ddr): New.
(vect_mark_for_runtime_alias_test): New.
(vect_analyze_data_ref_dependences) Add call to
vect_mark_for_runtime_alias_test.
(vect_enhance_data_refs_alignment): Define local variable
vect_versioning_for_alias_required, don't perform
peeling for alignment if versioning for alias is
required.
(vect_enhance_data_refs_alignment): Use
PARAM_VECT_MAX_VERSION_FOR_ALIGNMENT_CHECKS instead of
PARAM_VECT_MAX_VERSION_CHECKS.
* tree-vect-transform.c
(vect_create_cond_for_alias_checks): New.
(vect_transform_loop): Add call to
vect_create_cond_for_alias_checks.
(vect_vfa_segment_size): New.
* params.def (PARAM_VECT_MAX_VERSION_FOR_ALIGNMENT_CHECKS):
Rename.
(PARAM_VECT_MAX_VERSION_FOR_ALIAS_CHECKS): Define.
* gcc/doc/invoke.texi
(vect-max-version-for-alignment-checks): Document.
(vect-max-version-for-alias-checks): Document.
(vect-max-version-checks): Remove.
gcc/testsuite/ChangeLog
* gcc.dg/vect/vect-vfa-01.c: New.
* gcc.dg/vect/vect-vfa-02.c: New.
* gcc.dg/vect/vect-vfa-03.c: New.
* gcc.dg/vect/vect-vfa-04.c: New.
* gcc.dg/vect/vect-102a.c, gcc.dg/vect/vect-51.c,
gcc.dg/vect/pr29145.c, gcc.dg/vect/vect-43.c,
gcc.dg/vect/vect-61.c, gcc.dg/vect/vect-53.c,
gcc.dg/vect/vect-45.c, gcc.dg/vect/vect-101.c,
gcc.dg/vect/vect-37.c, gcc.dg/vect/vect-79.c,
gcc.dg/vect/vect-102.c, gcc.dg/vect/vect-dv-2.c,
gcc.dg/vect/vect-57.c, gcc.dg/vect/vect-49.c,
gfortran.dg/vect/pr19049.f90: Rename to start with
prefix no-vfa-.
* gcc.dg/vect/vect.exp: Disable versioning for alias
when test starts with no-vfa-.
* gfortran.dg/vect/vect.exp: Likewise.
From-SVN: r127559
2007-08-16 16:20:39 +02:00
|
|
|
/* Data Dependence Relations defining address ranges that are candidates
|
|
|
|
for a run-time aliasing check. */
|
|
|
|
VEC (ddr_p, heap) *may_alias_ddrs;
|
|
|
|
|
2005-09-06 12:59:58 +02:00
|
|
|
/* Statements in the loop that have data references that are candidates for a
|
|
|
|
runtime (loop versioning) misalignment check. */
|
|
|
|
VEC(tree,heap) *may_misalign_stmts;
|
|
|
|
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
/* The loop location in the source. */
|
|
|
|
LOC loop_line_number;
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
|
|
|
|
/* All interleaving chains of stores in the loop, represented by the first
|
|
|
|
stmt in the chain. */
|
|
|
|
VEC(tree, heap) *strided_stores;
|
|
|
|
|
|
|
|
/* All SLP instances in the loop. This is a subset of the set of STRIDED_STORES
|
|
|
|
of the loop. */
|
|
|
|
VEC(slp_instance, heap) *slp_instances;
|
|
|
|
|
|
|
|
/* The unrolling factor needed to SLP the loop. In case of that pure SLP is
|
|
|
|
applied to the loop, i.e., no unrolling is needed, this is 1. */
|
|
|
|
unsigned slp_unrolling_factor;
|
2005-02-03 17:13:17 +01:00
|
|
|
} *loop_vec_info;
|
|
|
|
|
2005-09-06 12:59:58 +02:00
|
|
|
/* Access Functions. */
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
#define LOOP_VINFO_LOOP(L) (L)->loop
|
|
|
|
#define LOOP_VINFO_BBS(L) (L)->bbs
|
|
|
|
#define LOOP_VINFO_NITERS(L) (L)->num_iters
|
2007-12-06 17:18:55 +01:00
|
|
|
/* Since LOOP_VINFO_NITERS can change after prologue peeling
|
|
|
|
retain total unchanged scalar loop iterations for cost model. */
|
2007-12-10 23:49:43 +01:00
|
|
|
#define LOOP_VINFO_NITERS_UNCHANGED(L) (L)->num_iters_unchanged
|
2007-07-05 15:15:20 +02:00
|
|
|
#define LOOP_VINFO_COST_MODEL_MIN_ITERS(L) (L)->min_profitable_iters
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
#define LOOP_VINFO_VECTORIZABLE_P(L) (L)->vectorizable
|
|
|
|
#define LOOP_VINFO_VECT_FACTOR(L) (L)->vectorization_factor
|
2005-09-06 12:59:58 +02:00
|
|
|
#define LOOP_VINFO_PTR_MASK(L) (L)->ptr_mask
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
#define LOOP_VINFO_DATAREFS(L) (L)->datarefs
|
|
|
|
#define LOOP_VINFO_DDRS(L) (L)->ddrs
|
|
|
|
#define LOOP_VINFO_INT_NITERS(L) (TREE_INT_CST_LOW ((L)->num_iters))
|
2005-03-15 19:33:09 +01:00
|
|
|
#define LOOP_PEELING_FOR_ALIGNMENT(L) (L)->peeling_for_alignment
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
#define LOOP_VINFO_UNALIGNED_DR(L) (L)->unaligned_dr
|
2005-09-06 12:59:58 +02:00
|
|
|
#define LOOP_VINFO_MAY_MISALIGN_STMTS(L) (L)->may_misalign_stmts
|
expr.c (highest_pow2_factor): Make extern.
* expr.c (highest_pow2_factor): Make extern.
* tree-data-ref.c (ptr_decl_may_alias_p): New function.
(ptr_ptr_may_alias_p, may_alias_p, record_ptr_differ_p,
record_array_differ_p, array_ptr_differ_p): Likewise.
(base_object_differ_p): Rename (from array_base_name_differ_p). Support
additional cases. Call the above functions.
(base_addr_differ_p): Moved from tree-vect-analyze.c. Call
base_object_differ_p when there are two base objects. Otherwise, compare
base address and offset. Call may_alias_p.
(dump_data_reference): Use a correct field name.
(analyze_array): Make static. Initialize new data-ref fields.
(analyze_indirect_ref): New function.
(init_data_ref): Initialize new data-ref fields.
(strip_conversion): Moved from tree-vect-analyze.c.
(analyze_offset_expr, get_ptr_offset, address_analysis, object_analysis):
Likewise.
(analyze_offset): New function.
(create_data_ref): Likewise.
(initialize_data_dependence_relation): Call base_addr_differ_p. Compare
dimensions for ARRAY_REFs only.
(build_classic_dist_vector): Make static.
(access_functions_are_affine_or_constant_p): Call macro to get the
address of access functions.
(compute_all_dependences): Add new parameter
compute_self_and_read_read_dependences. Compute self and read-read
dependences if it is true.
(find_data_references_in_loop): Call create_data_ref. Initialize new
data-ref fields.
(compute_data_dependences_for_loop): Add new parameter
compute_self_and_read_read_dependences. Remove parameter nb_loops,
compute nb_loops. Call compute_all_dependences, build_classic_dist_vector
and build_classic_dir_vector with correct parameters.
(analyze_all_data_dependences): Call compute_data_dependences_for_loop with
correct parameters. Compare dimensions for ARRAY_REFs only.
(free_data_refs): Call macro to free access functions.
* tree-data-ref.h (struct first_location_in_loop): New structure. Move
fields from stmt_vinfo.
(struct base_object_info): New structure.
(struct data_reference): Move fields to base_object_info. Add fields
first_location and object_info for above structures. Move fields from
stmt_info: memtag, ptr_info, subvars, misalignment. Add new field aligned_to.
Add macros to access the new fields.
Update functions declarations.
* tree-flow.h (is_aliased_with): Declare.
* tree-loop-linear.c (linear_transform_loops): Call
compute_data_dependences_for_loop with correct parameters.
* tree-ssa-alias.c (is_aliased_with): New function.
* tree-vect-analyze.c (vect_get_ptr_offset): Remove.
(vect_analyze_offset_expr, vect_base_addr_differ_p): Likewise.
(vect_analyze_data_ref_dependence): Get ddr. Remove call to
vect_base_addr_differ_p, compute_subscript_distance and
build_classic_dist_vector. Add printings. Check absolute value of
distance.
(vect_analyze_data_ref_dependences): Go through ddrs instead of data-refs.
(vect_compute_data_ref_alignment): Get the fields of data-ref instead of
stmt. Check aligned_to. Check if the base is aligned. Remove conversion
to bytes. Add printing.
(vect_compute_data_refs_alignment): Go through loads and stores in one loop.
(vect_enhance_data_refs_alignment, vect_analyze_data_refs_alignment,
vect_analyze_data_ref_access): Likewise.
(vect_analyze_pointer_ref_access): Remove.
(vect_address_analysis, vect_object_analysis): Likewise.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop to find
and analyze data-refs in the loop.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref): Get the
fields of data-ref instead of stmt. Add init to the offset from the base.
(vect_create_data_ref_ptr): Get the fields of data-ref instead of stmt.
(vect_update_init_of_dr): Likewise.
(vect_update_inits_of_drs): Go through loads and stores in one loop.
* tree-vectorizer.c (new_stmt_vec_info): Remove initialization of removed
fields.
(new_loop_vec_info): Initialize new fields.
(destroy_loop_vec_info): Free new fields.
(vect_strip_conversion): Remove.
* tree-vectorizer.h (enum verbosity_levels): Add new verbosity level.
(struct _loop_vec_info): Unify data_ref_writes and data_ref_reads into
datarefs. Add new field ddrs.
Add macros for the new fields access.
(struct _stmt_vec_info): Remove: base_address, initial_offset, step,
base_aligned_p, misalignment, memtag, ptr_info and subvars.
Remove their macros.
* tree.h (highest_pow2_factor): Declare.
From-SVN: r102356
2005-07-25 14:05:07 +02:00
|
|
|
#define LOOP_VINFO_LOC(L) (L)->loop_line_number
|
tree-vectorizer.c (new_loop_vec_info): Initialize new field.
gcc/ChangeLog
* tree-vectorizer.c (new_loop_vec_info): Initialize new
field.
(destroy_loop_vec_info): Add call to VEC_free.
* tree-vectorizer.h (may_alias_ddrs): Define.
(LOOP_VINFO_MAY_ALIAS_DDRS): Define.
* tree-vect-analyze.c (vect_analyze_data_ref_dependence):
Change reporting to dump.
(vect_is_duplicate_ddr): New.
(vect_mark_for_runtime_alias_test): New.
(vect_analyze_data_ref_dependences) Add call to
vect_mark_for_runtime_alias_test.
(vect_enhance_data_refs_alignment): Define local variable
vect_versioning_for_alias_required, don't perform
peeling for alignment if versioning for alias is
required.
(vect_enhance_data_refs_alignment): Use
PARAM_VECT_MAX_VERSION_FOR_ALIGNMENT_CHECKS instead of
PARAM_VECT_MAX_VERSION_CHECKS.
* tree-vect-transform.c
(vect_create_cond_for_alias_checks): New.
(vect_transform_loop): Add call to
vect_create_cond_for_alias_checks.
(vect_vfa_segment_size): New.
* params.def (PARAM_VECT_MAX_VERSION_FOR_ALIGNMENT_CHECKS):
Rename.
(PARAM_VECT_MAX_VERSION_FOR_ALIAS_CHECKS): Define.
* gcc/doc/invoke.texi
(vect-max-version-for-alignment-checks): Document.
(vect-max-version-for-alias-checks): Document.
(vect-max-version-checks): Remove.
gcc/testsuite/ChangeLog
* gcc.dg/vect/vect-vfa-01.c: New.
* gcc.dg/vect/vect-vfa-02.c: New.
* gcc.dg/vect/vect-vfa-03.c: New.
* gcc.dg/vect/vect-vfa-04.c: New.
* gcc.dg/vect/vect-102a.c, gcc.dg/vect/vect-51.c,
gcc.dg/vect/pr29145.c, gcc.dg/vect/vect-43.c,
gcc.dg/vect/vect-61.c, gcc.dg/vect/vect-53.c,
gcc.dg/vect/vect-45.c, gcc.dg/vect/vect-101.c,
gcc.dg/vect/vect-37.c, gcc.dg/vect/vect-79.c,
gcc.dg/vect/vect-102.c, gcc.dg/vect/vect-dv-2.c,
gcc.dg/vect/vect-57.c, gcc.dg/vect/vect-49.c,
gfortran.dg/vect/pr19049.f90: Rename to start with
prefix no-vfa-.
* gcc.dg/vect/vect.exp: Disable versioning for alias
when test starts with no-vfa-.
* gfortran.dg/vect/vect.exp: Likewise.
From-SVN: r127559
2007-08-16 16:20:39 +02:00
|
|
|
#define LOOP_VINFO_MAY_ALIAS_DDRS(L) (L)->may_alias_ddrs
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
#define LOOP_VINFO_STRIDED_STORES(L) (L)->strided_stores
|
|
|
|
#define LOOP_VINFO_SLP_INSTANCES(L) (L)->slp_instances
|
|
|
|
#define LOOP_VINFO_SLP_UNROLLING_FACTOR(L) (L)->slp_unrolling_factor
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2007-04-18 19:43:27 +02:00
|
|
|
#define NITERS_KNOWN_P(n) \
|
|
|
|
(host_integerp ((n),0) \
|
|
|
|
&& TREE_INT_CST_LOW ((n)) > 0)
|
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
#define LOOP_VINFO_NITERS_KNOWN_P(L) \
|
2007-04-18 19:43:27 +02:00
|
|
|
NITERS_KNOWN_P((L)->num_iters)
|
2005-02-03 17:13:17 +01:00
|
|
|
|
2007-08-19 11:39:50 +02:00
|
|
|
static inline loop_vec_info
|
|
|
|
loop_vec_info_for_loop (struct loop *loop)
|
|
|
|
{
|
|
|
|
return (loop_vec_info) loop->aux;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline bool
|
|
|
|
nested_in_vect_loop_p (struct loop *loop, tree stmt)
|
|
|
|
{
|
|
|
|
return (loop->inner
|
|
|
|
&& (loop->inner == (bb_for_stmt (stmt))->loop_father));
|
|
|
|
}
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Info on vectorized defs. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
enum stmt_vec_info_type {
|
|
|
|
undef_vec_info_type = 0,
|
|
|
|
load_vec_info_type,
|
|
|
|
store_vec_info_type,
|
|
|
|
op_vec_info_type,
|
2006-11-27 13:41:51 +01:00
|
|
|
call_vec_info_type,
|
2005-04-12 03:35:54 +02:00
|
|
|
assignment_vec_info_type,
|
tree.def (REDUC_MAX_EXPR, [...]): New tree-codes.
* tree.def (REDUC_MAX_EXPR, REDUC_MIN_EXPR, REDUC_PLUS_EXPR): New
tree-codes.
* optabs.h (OTI_reduc_smax, OTI_reduc_umax, OTI_reduc_smin,
OTI_reduc_umin, OTI_reduc_plus): New optabs for reduction.
(reduc_smax_optab, reduc_umax_optab, reduc_smin_optab, reduc_umin_optab,
reduc_plus_optab): New optabs for reduction.
* expr.c (expand_expr_real_1): Handle new tree-codes.
* tree-inline.c (estimate_num_insns_1): Handle new tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio, op_symbol): Handle
new tree-codes.
* optabs.c (optab_for_tree_code): Handle new tree-codes.
(init_optabs): Initialize new optabs.
* genopinit.c (optabs): Define handlers for new optabs.
* tree-vect-analyze.c (vect_analyze_operations): Fail vectorization in
case of a phi that is marked as relevant. Call vectorizable_reduction.
(vect_mark_relevant): Phis may be marked as relevant.
(vect_mark_stmts_to_be_vectorized): The use corresponding to the
reduction variable in a reduction stmt does not mark its defining phi
as relevant. Update documentation accordingly.
(vect_can_advance_ivs_p): Skip reduction phis.
* tree-vect-transform.c (vect_get_vec_def_for_operand): Takes
additional argument. Handle reduction.
(vect_create_destination_var): Update call to vect_get_new_vect_var.
Handle non-vector argument.
(get_initial_def_for_reduction): New function.
(vect_create_epilog_for_reduction): New function.
(vectorizable_reduction): New function.
(vect_get_new_vect_var): Handle new vect_var_kind.
(vectorizable_assignment, vectorizable_operation, vectorizable_store,
vectorizable_condition): Update call to vect_get_new_vect_var.
(vect_transform_stmt): Call vectorizable_reduction.
(vect_update_ivs_after_vectorizer): Skip reduction phis.
(vect_transform_loop): Skip if stmt is both not relevant and not live.
* tree-vectorizer.c (reduction_code_for_scalar_code): New function.
(vect_is_simple_reduction): Was empty - added implementation.
* tree-vectorizer.h (vect_scalar_var): New enum vect_var_kind value.
(reduc_vec_info_type): New enum vect_def_type value.
* config/rs6000/altivec.md (reduc_smax_v4si, reduc_smax_v4sf,
reduc_umax_v4si, reduc_smin_v4si, reduc_umin_v4sf, reduc_smin_v4sf,
reduc_plus_v4si, reduc_plus_v4sf): New define_expands.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Remove
ENABLE_CHECKING around gcc_assert.
* tree-vect-transform.c (vect_do_peeling_for_loop_bound,
(vect_do_peeling_for_alignment, vect_transform_loop,
vect_get_vec_def_for_operand): Likewise.
From-SVN: r101155
2005-06-18 15:18:52 +02:00
|
|
|
condition_vec_info_type,
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
reduc_vec_info_type,
|
2007-04-17 09:31:45 +02:00
|
|
|
induc_vec_info_type,
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
type_promotion_vec_info_type,
|
2007-02-11 12:46:07 +01:00
|
|
|
type_demotion_vec_info_type,
|
2007-08-19 11:39:50 +02:00
|
|
|
type_conversion_vec_info_type,
|
|
|
|
loop_exit_ctrl_vec_info_type
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
};
|
|
|
|
|
|
|
|
/* Indicates whether/how a variable is used in the loop. */
|
|
|
|
enum vect_relevant {
|
|
|
|
vect_unused_in_loop = 0,
|
2007-08-19 11:39:50 +02:00
|
|
|
vect_used_in_outer_by_reduction,
|
|
|
|
vect_used_in_outer,
|
2007-02-22 09:16:18 +01:00
|
|
|
|
|
|
|
/* defs that feed computations that end up (only) in a reduction. These
|
|
|
|
defs may be used by non-reduction stmts, but eventually, any
|
|
|
|
computations/values that are affected by these defs are used to compute
|
|
|
|
a reduction (i.e. don't get stored to memory, for example). We use this
|
|
|
|
to identify computations that we can change the order in which they are
|
|
|
|
computed. */
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
vect_used_by_reduction,
|
2007-02-22 09:16:18 +01:00
|
|
|
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
vect_used_in_loop
|
2004-08-17 18:17:14 +02:00
|
|
|
};
|
|
|
|
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
/* The type of vectorization that can be applied to the stmt: regular loop-based
|
|
|
|
vectorization; pure SLP - the stmt is a part of SLP instances and does not
|
|
|
|
have uses outside SLP instances; or hybrid SLP and loop-based - the stmt is
|
|
|
|
a part of SLP instance and also must be loop-based vectorized, since it has
|
|
|
|
uses outside SLP sequences.
|
|
|
|
|
|
|
|
In the loop context the meanings of pure and hybrid SLP are slightly
|
|
|
|
different. By saying that pure SLP is applied to the loop, we mean that we
|
|
|
|
exploit only intra-iteration parallelism in the loop; i.e., the loop can be
|
|
|
|
vectorized without doing any conceptual unrolling, cause we don't pack
|
|
|
|
together stmts from different iterations, only within a single iteration.
|
|
|
|
Loop hybrid SLP means that we exploit both intra-iteration and
|
|
|
|
inter-iteration parallelism (e.g., number of elements in the vector is 4
|
|
|
|
and the slp-group-size is 2, in which case we don't have enough parallelism
|
|
|
|
within an iteration, so we obtain the rest of the parallelism from subsequent
|
|
|
|
iterations by unrolling the loop by 2). */
|
|
|
|
enum slp_vect_type {
|
|
|
|
loop_vect = 0,
|
|
|
|
pure_slp,
|
|
|
|
hybrid
|
|
|
|
};
|
|
|
|
|
|
|
|
|
2005-06-10 16:52:01 +02:00
|
|
|
typedef struct data_reference *dr_p;
|
|
|
|
DEF_VEC_P(dr_p);
|
|
|
|
DEF_VEC_ALLOC_P(dr_p,heap);
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
typedef struct _stmt_vec_info {
|
|
|
|
|
|
|
|
enum stmt_vec_info_type type;
|
|
|
|
|
|
|
|
/* The stmt to which this info struct refers to. */
|
|
|
|
tree stmt;
|
|
|
|
|
2005-02-03 17:13:17 +01:00
|
|
|
/* The loop_vec_info with respect to which STMT is vectorized. */
|
|
|
|
loop_vec_info loop_vinfo;
|
2004-08-17 18:17:14 +02:00
|
|
|
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
/* Not all stmts in the loop need to be vectorized. e.g, the increment
|
2004-08-17 18:17:14 +02:00
|
|
|
of the loop induction variable and computation of array indexes. relevant
|
|
|
|
indicates whether the stmt needs to be vectorized. */
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
enum vect_relevant relevant;
|
2004-08-17 18:17:14 +02:00
|
|
|
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
/* Indicates whether this stmts is part of a computation whose result is
|
|
|
|
used outside the loop. */
|
|
|
|
bool live;
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/* The vector type to be used. */
|
|
|
|
tree vectype;
|
|
|
|
|
|
|
|
/* The vectorized version of the stmt. */
|
|
|
|
tree vectorized_stmt;
|
|
|
|
|
|
|
|
|
|
|
|
/** The following is relevant only for stmts that contain a non-scalar
|
|
|
|
data-ref (array/pointer/struct access). A GIMPLE stmt is expected to have
|
|
|
|
at most one such data-ref. **/
|
|
|
|
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
/* Information about the data-ref (access function, etc),
|
|
|
|
relative to the inner-most containing loop. */
|
2004-08-17 18:17:14 +02:00
|
|
|
struct data_reference *data_ref_info;
|
|
|
|
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
/* Information about the data-ref relative to this loop
|
|
|
|
nest (the loop that is being considered for vectorization). */
|
|
|
|
tree dr_base_address;
|
|
|
|
tree dr_init;
|
|
|
|
tree dr_offset;
|
|
|
|
tree dr_step;
|
|
|
|
tree dr_aligned_to;
|
|
|
|
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
/* Stmt is part of some pattern (computation idiom) */
|
|
|
|
bool in_pattern_p;
|
|
|
|
|
builtins.c, [...]: Fix comment typos.
* builtins.c, c-pragma.h, c-typeck.c, cgraph.c, cgraphunit.c,
combine.c, common.opt, config/dfp-bit.c, config/i386/i386.c,
config/m68k/m68k.c, config/m68k/m68k.md, config/mt/mt.c,
config/mt/mt.h, config/s390/s390.md, df-core.c, df-problems.c,
df-scan.c, df.h, diagnostic.c, expr.c, function.h, gimplify.c,
loop-invariant.c, omp-low.c, opts.c, passes.c,
rtl-factoring.c, rtlanal.c, struct-equiv.c, tree-cfgcleanup.c,
tree-ssa-loop-niter.c, tree-ssa-loop-prefetch.c,
tree-ssa-structalias.c, tree-ssa-threadedge.c,
tree-ssa-threadupdate.c, tree-vect-patterns.c,
tree-vect-transform.c, tree-vectorizer.h, tree-vrp.c,
unwind-dw2.c: Fix comment typos. Follow spelling conventions.
From-SVN: r111721
2006-03-05 00:05:24 +01:00
|
|
|
/* Used for various bookkeeping purposes, generally holding a pointer to
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
some other stmt S that is in some way "related" to this stmt.
|
|
|
|
Current use of this field is:
|
|
|
|
If this stmt is part of a pattern (i.e. the field 'in_pattern_p' is
|
|
|
|
true): S is the "pattern stmt" that represents (and replaces) the
|
|
|
|
sequence of stmts that constitutes the pattern. Similarly, the
|
|
|
|
related_stmt of the "pattern stmt" points back to this stmt (which is
|
|
|
|
the last stmt in the original sequence of stmts that constitutes the
|
|
|
|
pattern). */
|
|
|
|
tree related_stmt;
|
|
|
|
|
2005-06-10 16:52:01 +02:00
|
|
|
/* List of datarefs that are known to have the same alignment as the dataref
|
|
|
|
of this stmt. */
|
|
|
|
VEC(dr_p,heap) *same_align_refs;
|
|
|
|
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
/* Classify the def of this stmt. */
|
|
|
|
enum vect_def_type def_type;
|
|
|
|
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
/* Interleaving info. */
|
|
|
|
/* First data-ref in the interleaving group. */
|
|
|
|
tree first_dr;
|
|
|
|
/* Pointer to the next data-ref in the group. */
|
|
|
|
tree next_dr;
|
|
|
|
/* The size of the interleaving group. */
|
|
|
|
unsigned int size;
|
|
|
|
/* For stores, number of stores from this group seen. We vectorize the last
|
|
|
|
one. */
|
|
|
|
unsigned int store_count;
|
|
|
|
/* For loads only, the gap from the previous load. For consecutive loads, GAP
|
|
|
|
is 1. */
|
|
|
|
unsigned int gap;
|
|
|
|
/* In case that two or more stmts share data-ref, this is the pointer to the
|
|
|
|
previously detected stmt with the same dr. */
|
|
|
|
tree same_dr_stmt;
|
2007-01-21 10:22:56 +01:00
|
|
|
/* For loads only, if there is a store with the same location, this field is
|
|
|
|
TRUE. */
|
|
|
|
bool read_write_dep;
|
2007-06-08 18:30:49 +02:00
|
|
|
|
|
|
|
/* Vectorization costs associated with statement. */
|
|
|
|
struct
|
|
|
|
{
|
|
|
|
int outside_of_loop; /* Statements generated outside loop. */
|
|
|
|
int inside_of_loop; /* Statements generated inside loop. */
|
|
|
|
} cost;
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
|
|
|
|
/* Whether the stmt is SLPed, loop-based vectorized, or both. */
|
|
|
|
enum slp_vect_type slp_type;
|
2004-08-17 18:17:14 +02:00
|
|
|
} *stmt_vec_info;
|
|
|
|
|
|
|
|
/* Access Functions. */
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
#define STMT_VINFO_TYPE(S) (S)->type
|
|
|
|
#define STMT_VINFO_STMT(S) (S)->stmt
|
|
|
|
#define STMT_VINFO_LOOP_VINFO(S) (S)->loop_vinfo
|
|
|
|
#define STMT_VINFO_RELEVANT(S) (S)->relevant
|
|
|
|
#define STMT_VINFO_LIVE_P(S) (S)->live
|
|
|
|
#define STMT_VINFO_VECTYPE(S) (S)->vectype
|
|
|
|
#define STMT_VINFO_VEC_STMT(S) (S)->vectorized_stmt
|
|
|
|
#define STMT_VINFO_DATA_REF(S) (S)->data_ref_info
|
tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.c (split_constant_offset): Expose.
* tree-data-refs.h (split_constant_offset): Add declaration.
* tree-vectorizer.h (dr_alignment_support): Renamed
dr_unaligned_software_pipeline to dr_explicit_realign_optimized.
Added a new value dr_explicit_realign.
(_stmt_vec_info): Added new fields: dr_base_address, dr_init,
dr_offset, dr_step, and dr_aligned_to, along with new access
functions for these fields: STMT_VINFO_DR_BASE_ADDRESS,
STMT_VINFO_DR_INIT, STMT_VINFO_DR_OFFSET, STMT_VINFO_DR_STEP, and
STMT_VINFO_DR_ALIGNED_TO.
* tree-vectorizer.c (vect_supportable_dr_alignment): Add
documentation.
In case of outer-loop vectorization with non-fixed misalignment - use
the dr_explicit_realign scheme instead of the optimized realignment
scheme.
(new_stmt_vec_info): Initialize new fields.
* tree-vect-analyze.c (vect_compute_data_ref_alignment): Handle the
'nested_in_vect_loop' case. Change verbosity level.
(vect_analyze_data_ref_access): Handle the 'nested_in_vect_loop' case.
Don't fail on zero step in the outer-loop for loads.
(vect_analyze_data_refs): Call split_constant_offset to calculate base,
offset and init relative to the outer-loop.
* tree-vect-transform.c (vect_create_data_ref_ptr): Replace the unused
BSI function argument with a new function argument - at_loop.
Simplify the condition that determines STEP. Takes additional argument
INV_P. Support outer-loop vectorization (handle the nested_in_vect_loop
case), including zero step in the outer-loop. Call
vect_create_addr_base_for_vector_ref with additional argument.
(vect_create_addr_base_for_vector_ref): Takes additional argument LOOP.
Updated function documentation. Handle the 'nested_in_vect_loop' case.
Fixed and simplified calculation of step.
(vectorizable_store): Call vect_create_data_ref_ptr with loop instead
of bsi, and with additional argument. Call bump_vector_ptr with
additional argument. Fix typos. Handle the 'nested_in_vect_loop' case.
(vect_setup_realignment): Takes additional arguments INIT_ADDR and
DR_ALIGNMENT_SUPPORT. Returns another value AT_LOOP. Handle the case
when the realignment setup needs to take place inside the loop. Support
the dr_explicit_realign scheme. Allow generating the optimized
realignment scheme for outer-loop vectorization. Added documentation.
(vectorizable_load): Support the dr_explicit_realign scheme. Handle the
'nested_in_vect_loop' case, including loads that are invariant in the
outer-loop and the realignment schemes. Handle the case when the
realignment setup needs to take place inside the loop. Call
vect_setup_realignment with additional arguments. Call
vect_create_data_ref_ptr with additional argument and with loop instead
of bsi. Fix 80-column overflow. Fix typos. Rename PHI_STMT to PHI.
(vect_gen_niters_for_prolog_loop): Call
vect_create_addr_base_for_vector_ref with additional arguments.
(vect_create_cond_for_align_checks): Likewise.
(bump_vector_ptr): Updated to support the new dr_explicit_realign
scheme: takes additional argument bump; argument ptr_incr is now
optional; updated documentation.
(vect_init_vector): Takes additional argument (bsi). Use it, if
available, to insert the vector initialization.
(get_initial_def_for_induction): Pass additional argument in call to
vect_init_vector.
(vect_get_vec_def_for_operand): Likewise.
(vect_setup_realignment): Likewise.
(vectorizable_load): Likewise.
From-SVN: r127624
2007-08-19 14:02:48 +02:00
|
|
|
|
|
|
|
#define STMT_VINFO_DR_BASE_ADDRESS(S) (S)->dr_base_address
|
|
|
|
#define STMT_VINFO_DR_INIT(S) (S)->dr_init
|
|
|
|
#define STMT_VINFO_DR_OFFSET(S) (S)->dr_offset
|
|
|
|
#define STMT_VINFO_DR_STEP(S) (S)->dr_step
|
|
|
|
#define STMT_VINFO_DR_ALIGNED_TO(S) (S)->dr_aligned_to
|
|
|
|
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
#define STMT_VINFO_IN_PATTERN_P(S) (S)->in_pattern_p
|
|
|
|
#define STMT_VINFO_RELATED_STMT(S) (S)->related_stmt
|
|
|
|
#define STMT_VINFO_SAME_ALIGN_REFS(S) (S)->same_align_refs
|
|
|
|
#define STMT_VINFO_DEF_TYPE(S) (S)->def_type
|
|
|
|
#define STMT_VINFO_DR_GROUP_FIRST_DR(S) (S)->first_dr
|
|
|
|
#define STMT_VINFO_DR_GROUP_NEXT_DR(S) (S)->next_dr
|
|
|
|
#define STMT_VINFO_DR_GROUP_SIZE(S) (S)->size
|
|
|
|
#define STMT_VINFO_DR_GROUP_STORE_COUNT(S) (S)->store_count
|
|
|
|
#define STMT_VINFO_DR_GROUP_GAP(S) (S)->gap
|
|
|
|
#define STMT_VINFO_DR_GROUP_SAME_DR_STMT(S)(S)->same_dr_stmt
|
2007-01-21 10:22:56 +01:00
|
|
|
#define STMT_VINFO_DR_GROUP_READ_WRITE_DEPENDENCE(S) (S)->read_write_dep
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
#define STMT_VINFO_STRIDED_ACCESS(S) ((S)->first_dr != NULL)
|
c-tree.texi: Document new tree codes.
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
From-SVN: r119088
2006-11-22 09:46:03 +01:00
|
|
|
|
|
|
|
#define DR_GROUP_FIRST_DR(S) (S)->first_dr
|
|
|
|
#define DR_GROUP_NEXT_DR(S) (S)->next_dr
|
|
|
|
#define DR_GROUP_SIZE(S) (S)->size
|
|
|
|
#define DR_GROUP_STORE_COUNT(S) (S)->store_count
|
|
|
|
#define DR_GROUP_GAP(S) (S)->gap
|
|
|
|
#define DR_GROUP_SAME_DR_STMT(S) (S)->same_dr_stmt
|
2007-01-21 10:22:56 +01:00
|
|
|
#define DR_GROUP_READ_WRITE_DEPENDENCE(S) (S)->read_write_dep
|
2004-08-17 18:17:14 +02:00
|
|
|
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
#define STMT_VINFO_RELEVANT_P(S) ((S)->relevant != vect_unused_in_loop)
|
2007-06-08 18:30:49 +02:00
|
|
|
#define STMT_VINFO_OUTSIDE_OF_LOOP_COST(S) (S)->cost.outside_of_loop
|
|
|
|
#define STMT_VINFO_INSIDE_OF_LOOP_COST(S) (S)->cost.inside_of_loop
|
|
|
|
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
#define HYBRID_SLP_STMT(S) ((S)->slp_type == hybrid)
|
|
|
|
#define PURE_SLP_STMT(S) ((S)->slp_type == pure_slp)
|
|
|
|
#define STMT_SLP_TYPE(S) (S)->slp_type
|
|
|
|
|
2007-06-08 18:30:49 +02:00
|
|
|
/* These are some defines for the initial implementation of the vectorizer's
|
|
|
|
cost model. These will later be target specific hooks. */
|
|
|
|
|
rsha Jagasia <harsha.jagasia@amd.com>
Jan Sjodin <jan.sjodin@amd.com>
* tree-vect-analyze.c (vect_analyze_operations): Change
comparison of loop iterations with threshold to less than
or equal to instead of less than. Reduce
min_scalar_loop_bound by one.
* tree-vect-transform.c (vect_estimate_min_profitable_iters):
Change prologue and epilogue iterations estimate to vf/2,
when unknown at compile-time. Change versioning guard
cost to taken_branch_cost. If peeling for alignment is
unknown at compile-time, change peel guard costs to one
taken branch and one not-taken branch per peeled loop.
If peeling for alignment is known but number of scalar loop
iterations is unknown at compile-time, change peel guard
costs to one taken branch per peeled loop. Change the cost
model equation to consider vector iterations as the loop
iterations less the prologue and epilogue iterations.
Change outside vector cost check to less than or equal to
zero instead of equal to zero.
(vect_do_peeling_for_loop_bound): Reduce
min_scalar_loop_bound by one.
* tree-vectorizer.h: Add TARG_COND_TAKEN_BRANCH_COST and
TARG_COND_NOT_TAKEN_BRANCH_COST.
* config/i386/i386.h (processor_costs): Add
scalar_stmt_cost, scalar_load_cost, scalar_store_cost,
vec_stmt_cost, vec_to_scalar_cost, scalar_to_vec_cost,
vec_align_load_cost, vect_unalign_load_cost,
vec_store_cost, cond_taken_branch_cost,
cond_not_taken_branch_cost.
Define macros for x86 costs.
* config/i386/i386.c:
(size_cost): Set scalar_stmt_cost, scalar_load_cost,
scalar_store_cost, vec_stmt_cost, vec_to_scalar_cost,
scalar_to_vec_cost, vec_align_load_cost,
vect_unalign_load_cost, vec_store_cost,
cond_taken_branch_cost, cond_not_taken_branch_cost to one.
(i386_cost, i486_cost, pentium_cost, pentiumpro_cost,
geode_cost, k6_cost, athlon_cost, pentium4_cost, nocona_cost,
core2_cost, generic64_cost, generic32_cost): Set to default
untuned costs.
(k8_cost, amdfam10_cost): Costs for vectorization tuned.
(x86_builtin_vectorization_cost): New.
2007-09-10 Harsha Jagasia <harsha.jagasia@amd.com>
* gcc.dg/vect/costmodel/i386/costmodel-vect-31.c:
Change dg-final to expect 1 non-profitable loop and
3 profitable loops.
* gcc.dg/vect/costmodel/x86-64/costmodel-vect-31.c:
Change dg-final to expect 1 non-profitable loop and
3 profitable loops.
* gcc.dg/vect/costmodel/x86-64/costmodel-fast-math-vect-pr29925.c:
Change dg-final to expect 1 profitable loop.
* gcc.dg/vect/costmodel/i386/costmodel-fast-math-vect-pr29925.c:
Change dg-final to expect 1 profitable loop.
Co-Authored-By: Jan Sjodin <jan.sjodin@amd.com>
From-SVN: r128353
2007-09-11 02:13:47 +02:00
|
|
|
/* Cost of conditional taken branch. */
|
|
|
|
#ifndef TARG_COND_TAKEN_BRANCH_COST
|
|
|
|
#define TARG_COND_TAKEN_BRANCH_COST 3
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* Cost of conditional not taken branch. */
|
|
|
|
#ifndef TARG_COND_NOT_TAKEN_BRANCH_COST
|
|
|
|
#define TARG_COND_NOT_TAKEN_BRANCH_COST 1
|
2007-06-08 18:30:49 +02:00
|
|
|
#endif
|
|
|
|
|
2007-07-12 14:17:03 +02:00
|
|
|
/* Cost of any scalar operation, excluding load and store. */
|
|
|
|
#ifndef TARG_SCALAR_STMT_COST
|
|
|
|
#define TARG_SCALAR_STMT_COST 1
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* Cost of scalar load. */
|
|
|
|
#ifndef TARG_SCALAR_LOAD_COST
|
|
|
|
#define TARG_SCALAR_LOAD_COST 1
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* Cost of scalar store. */
|
|
|
|
#ifndef TARG_SCALAR_STORE_COST
|
|
|
|
#define TARG_SCALAR_STORE_COST 1
|
|
|
|
#endif
|
|
|
|
|
2007-06-08 18:30:49 +02:00
|
|
|
/* Cost of any vector operation, excluding load, store or vector to scalar
|
|
|
|
operation. */
|
|
|
|
#ifndef TARG_VEC_STMT_COST
|
|
|
|
#define TARG_VEC_STMT_COST 1
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* Cost of vector to scalar operation. */
|
|
|
|
#ifndef TARG_VEC_TO_SCALAR_COST
|
|
|
|
#define TARG_VEC_TO_SCALAR_COST 1
|
|
|
|
#endif
|
|
|
|
|
2007-07-05 15:15:20 +02:00
|
|
|
/* Cost of scalar to vector operation. */
|
|
|
|
#ifndef TARG_SCALAR_TO_VEC_COST
|
|
|
|
#define TARG_SCALAR_TO_VEC_COST 1
|
|
|
|
#endif
|
|
|
|
|
2007-06-08 18:30:49 +02:00
|
|
|
/* Cost of aligned vector load. */
|
|
|
|
#ifndef TARG_VEC_LOAD_COST
|
|
|
|
#define TARG_VEC_LOAD_COST 1
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* Cost of misaligned vector load. */
|
|
|
|
#ifndef TARG_VEC_UNALIGNED_LOAD_COST
|
|
|
|
#define TARG_VEC_UNALIGNED_LOAD_COST 2
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* Cost of vector store. */
|
|
|
|
#ifndef TARG_VEC_STORE_COST
|
|
|
|
#define TARG_VEC_STORE_COST 1
|
|
|
|
#endif
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
|
2006-09-12 12:11:04 +02:00
|
|
|
static inline void set_stmt_info (stmt_ann_t ann, stmt_vec_info stmt_info);
|
2004-08-17 18:17:14 +02:00
|
|
|
static inline stmt_vec_info vinfo_for_stmt (tree stmt);
|
|
|
|
|
|
|
|
static inline void
|
2006-09-12 12:11:04 +02:00
|
|
|
set_stmt_info (stmt_ann_t ann, stmt_vec_info stmt_info)
|
2004-08-17 18:17:14 +02:00
|
|
|
{
|
|
|
|
if (ann)
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
ann->common.aux = (char *) stmt_info;
|
2004-08-17 18:17:14 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static inline stmt_vec_info
|
|
|
|
vinfo_for_stmt (tree stmt)
|
|
|
|
{
|
2006-09-12 12:11:04 +02:00
|
|
|
stmt_ann_t ann = stmt_ann (stmt);
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
return ann ? (stmt_vec_info) ann->common.aux : NULL;
|
2004-08-17 18:17:14 +02:00
|
|
|
}
|
|
|
|
|
2007-01-16 09:26:03 +01:00
|
|
|
static inline bool
|
|
|
|
is_pattern_stmt_p (stmt_vec_info stmt_info)
|
|
|
|
{
|
|
|
|
tree related_stmt;
|
|
|
|
stmt_vec_info related_stmt_info;
|
|
|
|
|
|
|
|
related_stmt = STMT_VINFO_RELATED_STMT (stmt_info);
|
|
|
|
if (related_stmt
|
|
|
|
&& (related_stmt_info = vinfo_for_stmt (related_stmt))
|
|
|
|
&& STMT_VINFO_IN_PATTERN_P (related_stmt_info))
|
|
|
|
return true;
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2007-08-19 11:39:50 +02:00
|
|
|
static inline bool
|
|
|
|
is_loop_header_bb_p (basic_block bb)
|
|
|
|
{
|
|
|
|
if (bb == (bb->loop_father)->header)
|
|
|
|
return true;
|
|
|
|
gcc_assert (EDGE_COUNT (bb->preds) == 1);
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2007-09-09 13:22:24 +02:00
|
|
|
static inline void
|
|
|
|
stmt_vinfo_set_inside_of_loop_cost (stmt_vec_info stmt_info, slp_tree slp_node,
|
|
|
|
int cost)
|
|
|
|
{
|
|
|
|
if (slp_node)
|
|
|
|
SLP_TREE_INSIDE_OF_LOOP_COST (slp_node) = cost;
|
|
|
|
else
|
|
|
|
STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info) = cost;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void
|
|
|
|
stmt_vinfo_set_outside_of_loop_cost (stmt_vec_info stmt_info, slp_tree slp_node,
|
|
|
|
int cost)
|
|
|
|
{
|
|
|
|
if (slp_node)
|
|
|
|
SLP_TREE_OUTSIDE_OF_LOOP_COST (slp_node) = cost;
|
|
|
|
else
|
|
|
|
STMT_VINFO_OUTSIDE_OF_LOOP_COST (stmt_info) = cost;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Info on data references alignment. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
|
2005-03-15 19:33:09 +01:00
|
|
|
/* Reflects actual alignment of first access in the vectorized loop,
|
|
|
|
taking into account peeling/versioning if applied. */
|
tree-vectorizer.h (DR_MISALIGNMENT): Cast aux to integer.
* tree-vectorizer.h (DR_MISALIGNMENT): Cast aux to integer.
(SET_DR_MISALIGNMENT): New.
* tree-vect-analyze.c (vect_compute_data_ref_alignment,
vect_update_misalignment_for_peel, vect_enhance_data_refs_alignment):
Use SET_DR_MISALIGNMENT.
* tree-predcom.c (split_data_refs_to_components): Cast dr->aux from
pointer.
* tree-data-ref.c (create_data_ref, compute_all_dependences,
find_loop_nest): Export.
* tree-data-ref.h (struct data_reference): Change aux field to pointer.
(create_data_ref, compute_all_dependences, find_loop_nest): Declare.
* tree-ssa-loop-prefetch.c: Include tree-data-ref.h.
(L1_CACHE_SIZE_BYTES, L2_CACHE_SIZE_BYTES, NONTEMPORAL_FRACTION):
New macros.
(struct mem_ref): Add field reuse_distance.
(find_or_create_group, record_ref): Use XNEW instead of xcalloc.
Initialize reuse_distance field.
(issue_prefetch_ref): Select temporality of prefetch according to
reuse_distance.
(volume_of_references, volume_of_dist_vector, add_subscript_strides,
self_reuse_distance, determine_loop_nest_reuse): New functions.
(loop_prefetch_arrays): Call determine_loop_nest_reuse.
(tree_ssa_prefetch_arrays): Dump L2 cache size.
* Makefile.in (tree-ssa-loop-prefetch.o): Add TREE_DATA_REF_H
dependency.
* gcc.dg/tree-ssa/prefetch-6.c: New test.
From-SVN: r125172
2007-05-29 23:55:47 +02:00
|
|
|
#define DR_MISALIGNMENT(DR) ((int) (size_t) (DR)->aux)
|
|
|
|
#define SET_DR_MISALIGNMENT(DR, VAL) ((DR)->aux = (void *) (size_t) (VAL))
|
2004-08-17 18:17:14 +02:00
|
|
|
|
|
|
|
static inline bool
|
|
|
|
aligned_access_p (struct data_reference *data_ref_info)
|
|
|
|
{
|
|
|
|
return (DR_MISALIGNMENT (data_ref_info) == 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline bool
|
2005-03-15 19:33:09 +01:00
|
|
|
known_alignment_for_access_p (struct data_reference *data_ref_info)
|
2004-08-17 18:17:14 +02:00
|
|
|
{
|
2005-03-15 19:33:09 +01:00
|
|
|
return (DR_MISALIGNMENT (data_ref_info) != -1);
|
2004-08-17 18:17:14 +02:00
|
|
|
}
|
|
|
|
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/* vect_dump will be set to stderr or dump_file if exist. */
|
|
|
|
extern FILE *vect_dump;
|
|
|
|
extern enum verbosity_levels vect_verbosity_level;
|
2004-08-17 18:17:14 +02:00
|
|
|
|
2005-08-18 18:07:54 +02:00
|
|
|
/* Bitmap of virtual variables to be renamed. */
|
[multiple changes]
2006-12-11 Diego Novillo <dnovillo@redhat.com>
* doc/tree-ssa.texi: Update documentation for virtual operands
and the use of push_stmt_changes/pop_stmt_changes.
* doc/invoke.texi: Remove documentation for params
global-var-threshold.
Update documentation on max-aliased-vops.
* tree-into-ssa.c: Cleanup comments, variables and
spacing in various functions.
(regs_to_rename): Declare.
(mem_syms_to_rename): Declare.
(dump_update_ssa): Declare.
(debug_update_ssa): Declare.
(dump_names_replaced_by): Declare.
(debug_names_replaced_by): Declare.
(dump_def_blocks): Declare.
(debug_def_blocks): Declare.
(dump_defs_stack): Declare.
(debug_defs_stack): Declare.
(dump_currdefs): Declare.
(debug_currdefs): Declare.
(mark_def_sites): Do not handle virtual operands.
(compute_idf): Rename from find_idf. Update users.
(register_new_def): Make local. Convert second argument
to 'tree'.
Use BLOCK_DEFS_STACK directly.
If pushing a non-register, also push the underlying
symbol.
(rewrite_stmt): Do not handle virtual operands.
(dump_tree_ssa): Call dump_def_blocks, dump_defs_stack,
dump_currdefs and dump_tree_ssa_stats.
(dump_tree_ssa_stats): Also dump REPL_TBL.
(replace_use): Remove. Update all users to call SET_USE
instead.
(rewrite_blocks): Move code to free memory to
fini_ssa_renamer.
(mark_def_site_blocks): Move initialization code to
init_ssa_renamer.
(init_ssa_renamer): New.
(fini_ssa_renamer): New.
(rewrite_into_ssa): Call them.
(prepare_block_for_update): Process SSA_OP_ALL_USES first
and SSA_OP_ALL_DEFS later. Do not process virtual
operands separately.
(dump_update_ssa): Call dump_decl_set.
(init_update_ssa): Initialize regs_to_rename and
mem_syms_to_rename.
Call init_ssa_renamer.
(delete_update_ssa): Call fini_ssa_renamer.
Free blocks_with_phis_to_rewrite.
(mark_sym_for_renaming): If the variable has
sub-variables, also mark them.
If the variable belongs to a partition, also mark it.
(mark_set_for_renaming): Call mark_sym_for_renaming on
every symbol in the set.
(switch_virtuals_to_full_rewrite): Call
mark_set_for_renaming.
(update_ssa): Separate syms_to_rename into regs_to_rename
and mem_syms_to_rename.
* tree-dump.c (dump_options): Add TDF_MEMSYMS.
* tree-pretty-print.c (debug_generic_expr): Add TDF_MEMSYMS.
(debug_generic_stmt): Likewise.
(debug_tree_chain): Likewise.
(dump_symbols): New.
(dump_generic_node): Check for TDF_MEMSYMS.
Handle MEMORY_PARTITION_TAG.
If the statement references memory and TDF_MEMSYMS is
given, call dump_symbols.
Indicate default names with (D).
(dump_vops): Update for new virtual operator format.
* tree.c (init_ttree): Add MEMORY_PARTITION_TAG to
tree_contains_struct.
(tree_code_size): Handle MEMORY_PARTITION_TAG.
(tree_node_structure): Likewise.
(needs_to_live_in_memory): Handle SSA names.
* tree.h (MTAG_P): Likewise.
(struct tree_memory_partition_tag): Declare.
(MPT_SYMBOLS): Define.
(union tree_node): Add field 'mpt'.
* treestruct.def (TS_MEMORY_PARTITION_TAG): Define.
* tree.def (MEMORY_PARTITION_TAG): Define.
* tree-pass.h (TDF_MEMSYMS): Define.
* params.h (GLOBAL_VAR_THRESHOLD): Remove.
* tree-ssa-alias.c: Include pointer-set.h
(struct alias_map_d): Remove fields total_alias_vops,
grouped_p and may_aliases. Update all users.
(struct mp_info_def): Declare.
(mp_info_t): New type.
(get_smt_for): Rename from get_tmt_for. Update all
users.
(add_may_alias): Add argument ALREADY_ADDED. If given,
use it to avoid adding duplicate entries to alias sets.
(replace_may_alias): Remove. Update all users.
(total_alias_vops_cmp): Remove. Update all users.
(group_aliases_into): Remove. Update all users.
(tree_pointer_compare): Remove. Update all users.
(compact_name_tags): Remove. Update all users.
(group_aliases): Remove. Update all users.
(mark_non_addressable): Move from tree-flow-inline.h.
Remove the symbol from the partition holding it, if
needed.
(dump_mp_info): New.
(debug_mp_info): New.
(sort_mp_info): New.
(create_partition_for): New.
(rewrite_alias_set_for): New.
(compute_memory_partitions): New.
(compute_may_aliases): Call it.
(init_alias_info): If computing aliases for the first
time, mark every memory symbol for renaming.
(have_common_aliases_p): New.
(compute_flow_insensitive_aliasing): Call it.
(setup_pointers_and_addressables): Do not cache
num_referenced_vars.
For register promoted symbols, mark their former
partition for renaming.
(maybe_create_global_var): Only create .GLOBAL_VAR if
there are no call-clobbered variables and a mix of pure
and non-pure functions were found.
(may_alias_p): Tidy comments.
(create_tag_raw): Remove unused variable new_type.
(dump_alias_info): call dump_memory_partitions.
(dump_points_to_info_for): Call dump_decl_set.
(may_be_aliased): Tidy comments and formatting.
* timevar.def (TV_MEMORY_PARTITIONING): Define.
* tree-vectorizer.c (vect_memsyms_to_rename): Rename from
vect_vnames_to_rename. Set DECL_UIDs instead of SSA name
versions in it.
(slpeel_update_phi_nodes_for_guard1): Ignore memory PHIs.
* tree-vect-transform.c (vect_transform_loop): Call
mark_set_for_renaming with vect_memsyms_to_rename.
* tree-flow-inline.h (zero_imm_uses_p): New.
(memory_partition): New.
(set_memory_partition): New.
(factoring_name_p): New.
(symbol_mem_tag): New. Update every function that used
to access the annotation directly.
(set_symbol_mem_tag): Likewise.
* tree-ssa-copy.c (may_propagate_copy): Allow copies
between a partition and a symbol as long as the symbol
belongs to the partition.
(merge_alias_info): Ignore merge requests when memory
partitions are involved.
* tree-ssa.c (verify_ssa_name): Check that default
definitions have empty defining statements.
(verify_use): Remove argument IS_VIRTUAL.
Don't call verify_ssa_name.
(verify_phi_args): Call verify_ssa_name.
(verify_flow_insensitive_alias_info): Handle MPTs.
(verify_flow_sensitive_alias_info): Likewise.
(verify_name_tags): Likewise.
(verify_call_clobbering): Likewise.
(verify_ssa): Check for VOPs only after aliasing
information is available.
Check virtuals and real operands separately.
Call verify_ssa_name on every operand.
(stmt_references_memory_p): Move to tree-ssa-operands.c.
(walk_use_def_chains_1): Guard against NULL PHI
arguments.
* tree-ssa-operands.c (stmt_references_memory_p): Move from
tree-ssa.c.
(get_mpt_for): New.
(dump_memory_partitions): New.
(debug_memory_partitions): New.
* tree-flow.h (struct var_ann_d): Add field mpt.
(struct stmt_ann_d): Add bitfield references_memory.
* Makefile.in (tree-ssa-structalias.o): Include
pointer-set.h
(tree-ssa-alias.o): Likewise.
* tree-ssa-structalias.c: (update_alias_info): Use
STORED_SYMS to determine which variables are being
written to by the store operation.
* tree-ssa-structalias.h (struct alias_info)
<total_alias_vops>: Remove. Update all users.
<written_vars>: Change to a pointer set. Update all
users.
<dereferenced_ptrs_store>: Likewise.
<dereferenced_ptrs_load>: Likewise.
(NUM_REFERENCES): Remove. Update all users.
(NUM_REFERENCES_CLEAR): Remove. Update all users.
(NUM_REFERENCES_INC): Remove. Update all users.
(NUM_REFERENCES_SET): Remove. Update all users.
* params.def (PARAM_GLOBAL_VAR_THRESHOLD): Remove.
Update all users.
(PARAM_MAX_ALIASED_VOPS): Set to 10.
* tree-ssanames.c (make_ssa_name): Initialize
SSA_NAME_IS_DEFAULT_DEF to 0.
2006-12-11 Aldy Hernandez <aldyh@redhat.com>
* tree-ssa-dse.c (aggregate_vardecl_d): New.
(dse_global_data): Add aggregate_vardecl field.
(dse_possible_dead_store_p): New.
Add prev_defvar variable.
Allow immediate uses and previous immediate uses to differ
if they are setting different parts of the whole.
(get_aggregate_vardecl): New.
(dse_record_partial_aggregate_store): New.
(dse_whole_aggregate_clobbered_p): New.
(dse_partial_kill_p): New.
(dse_optimize_stmt): Abstract code checking a possible dead store
into new function dse_possible_dead_store_p().
Call dse_maybe_record_aggregate_store().
When checking whether a STMT and its USE_STMT refer to the
same memory address, check also for partial kills that clobber
the whole.
Move some variable definitions to the block where they are used.
(aggregate_vardecl_hash): New.
(aggregate_vardecl_eq): New.
(aggregate_vardecl_free): New.
(aggregate_whole_store_p): New.
(tree_ssa_dse): Initialize and free aggregate_vardecl.
Mark which aggregate stores we care about.
2006-12-11 Andrew Macleod <amacleod@redhat.com>
* tree-ssa-operands.h (struct vuse_element_d): Declare.
(vuse_element_t): Declare.
(struct vuse_vec_d): Declare.
(vuse_vec_p): Declare.
(VUSE_VECT_NUM_ELEM): Define.
(VUSE_VECT_ELEMENT_NC): Define.
(VUSE_ELEMENT_PTR_NC): Define.
(VUSE_ELEMENT_VAR_NC): Define.
(VUSE_VECT_ELEMENT): Define.
(VUSE_ELEMENT_PTR): Define.
(VUSE_ELEMENT_VAR): Define.
(struct maydef_optype_d) <use_var>: Remove.
<use_ptr>: Remove.
<usev>: Add.
(struct vuse_optype_d) <kill_var>: Remove.
<use_ptr>: Remove.
<usev>: Add.
(struct mustdef_optype_d) <kill_var>: Remove.
<use_ptr>: Remove.
<usev>: Add.
(VUSE_OP_PTR): Add argument. Use VUSE_ELEMENT_PTR.
(VUSE_OP): Add argument. Use VUSE_ELEMENT_PTR.
(VUSE_NUM): Define.
(VUSE_VECT): Define.
(MAYDEF_OP_PTR): Add argument. Use VUSE_OP_PTR.
(MAYDEF_OP): Add argument. Use VUSE_OP.
(MAYDEF_NUM): Define.
(MAYDEF_VECT): Define.
(MUSTDEF_KILL_PTR): Use VUSE_OP_PTR.
(MUSTDEF_KILL): Use VUSE_OP.
(MUSTDEF_NUM): Define.
(MUSTDEF_VECT): Define.
(realloc_maydef): Declare.
(realloc_vuse): Declare.
(struct ssa_operand_iterator_d) <vuse_index>: Add.
<mayuse_index>: Add.
(LOADED_SYMS): Define.
(STORED_SYMS): Define.
(FOR_EACH_SSA_MUSTDEF_OPERAND): Call op_iter_next_mustdef.
* tree-into-ssa.c: Adapt for multi-operand V_MAY_DEF and VUSE
operators.
* tree-pretty-print.c: Likewise.
* tree-ssa-dse.c: Likewise.
* tree-flow-inline.h: Likewise.
(op_iter_next_mustdef): New.
* tree-ssa-operands.c: Likewise.
(ALLOC_OPTYPE): Remove.
Update all users.
(alloc_def): New.
(alloc_use): New.
(alloc_maydef): New.
(alloc_vuse): New.
(alloc_mustdef): New.
(realloc_maydef): New.
(realloc_vuse): New.
2006-12-11 Aldy Hernandez <aldyh@redhat.com>
* tree-ssa-operands.c: Remove build_v_must_defs.
(init_ssa_operands): Delete build_v_must_defs.
(finalize_ssa_v_must_def_ops): Remove.
(finalize_ssa_v_must_defs): Remove.
(finalize_ssa_stmt_operands): Do not call
finalize_ssa_v_must_defs.
(start_ssa_stmt_operands): Do not check build_v_must_defs.
(append_v_must_def): Delete.
(copy_virtual_operands): Do not copy V_MUST_DEFs.
(get_modify_expr_operands): Remove reference to V_MUST_DEF from
comment. Remove opf_kill_def.
(build_ssa_operands): Remove references to v_must_defs.
(copy_virtual_operands): Same.
(copy_virtual_operands): Same.
(fini_ssa_operands): Same.
(free_ssa_operands): Same.
(add_mustdef_op): Remove.
Remove mustdef_optype_p.
(alloc_mustdef): Remove.
Remove references to V_MUST_DEFs in comment at top of file.
(get_expr_operands): Remove opf_kill_def.
(opf_kill_def): Remove.
(add_virtual_operand): Remove opf_kill_def.
(get_indirect_ref_operands): Same.
(get_tmr_operands): Same.
* tree-vectorizer.c (rename_variables_in_bb): Remove
SSA_OP_ALL_KILLS.
* tree-ssa-loop-manip.c (find_uses_to_rename_stmt): Remove
SSA_OP_ALL_KILLS.
(check_loop_closed_ssa_stmt): Same.
* tree-ssa.c (verify_def): Remove V_MUST_DEF from comment.
(verify_use): Same.
(verify_ssa): Remove V_MUST_DEFs traces.
(verify_ssa): Remove SSA_OP_ALL_KILLS.
* tree-into-ssa.c (mark_def_sites): Change SSA_OP_VMUSTDEF to
SSA_OP_VMAYDEF.
(rewrite_update_stmt): Remove SSA_OP_VIRTUAL_KILLS.
(rewrite_stmt): Remove SSA_OP_ALL_KILLS.
* tree-ssa-operands.h (struct stmt_operands_d): Remove V_MUST_DEF
references.
(MUSTDEF_OPS): Remove.
(SSA_OP_VMUSTDEF): Remove.
(FOR_EACH_SSA_MUSTDEF_OPERAND): Remove.
(struct mustdef_optype_d): Remove.
Remove mustdef_optype_p.
(struct stmt_operands_d): Remove mustdef_ops.
(ssa_operand_iterator_d): Remove mustdefs and mustkills.
(SSA_OP_VIRTUAL_DEFS): Remove SSA_OP_VMUSTDEF.
(MUSTDEF_RESULT_PTR): Remove.
(MUSTDEF_RESULT): Remove.
(MUSTDEF_KILL_PTR): Remove.
(MUSTDEF_KILL): Remove.
(MUSTDEF_NUM): Remove.
(MUSTDEF_VECT): Remove.
(SSA_OP_VIRTUAL_KILLS): Remove.
(SSA_OP_ALL_VIRTUALS): Remove SSA_OP_VIRTUAL_KILLS.
(SSA_OP_VMUSTKILL): Remove.
(SSA_OP_ALL_KILLS): Remove.
(SSA_OP_ALL_OPERANDS): Remove SSA_OP_ALL_KILLS.
* tree-flow-inline.h (op_iter_init_def): Remove
SSA_OP_VIRTUAL_KILLS.
(delink_stmt_imm_use): Remove SSA_OP_ALL_KILLS.
* tree-ssa-pre.c (compute_rvuse_and_antic_safe): Remove
SSA_OP_VIRTUAL_KILLS.
* tree-ssa-loop-im.c (determine_max_movement): Remove
SSA_OP_VIRTUAL_KILLS.
(gather_mem_refs_stmt): Same.
(gather_mem_refs_stmt): Same.
* tree-ssa-dce.c (mark_really_necessary_kill_operand_phis): Delete.
(perform_tree_ssa_dce): Remove call to
mark_really_necessary_kill_operand_phis.
* tree-flow-inline.h (op_iter_init): Remove setting of mustdefs
and mustkills.
(op_iter_next_use): Do not check mustkills.
(op_iter_next_def): Do not check mustdefs.
(op_iter_next_tree): Do not check mustkills or mustdefs.
(clear_and_done_ssa_iter): Do not set mustdefs or mustkills.
(op_iter_next_maymustdef): Do not check mustkills.
(op_iter_init_must_and_may_def): Remove SSA_OP_VMUSTKILL.
(op_iter_init_mustdef): Remove.
* tree-ssa-live.c (create_ssa_var_map): Change SSA_OP_VMUSTDEF to
SSA_OP_VMAYDEF.
* tree-ssa-dse.c (dse_optimize_stmt): Remove SSA_OP_VMUSTDEF.
* tree-ssa-ccp.c: Remove V_MUST_DEF traces from comments.
(visit_assignment): Same.
* tree-ssa-copy.c (copy_prop_visit_assignment): Same.
* tree-sra.c (mark_all_v_defs_1): Remove V_MUST_DEF from comment.
* tree-outof-ssa.c (check_replaceable): Remove SSA_OP_VMUSTDEF.
* tree-pretty-print.c (dump_vops): Remove printing of V_MUST_DEF.
Remove kill_p variable.
* tree-dfa.c (struct dfa_stats_d): Remove num_v_must_defs.
(dump_dfa_stats): Remove code related to V_MUST_DEFs.
(collect_dfa_stats_r): Do not set num_v_must_defs.
(mark_new_vars_to_rename): Remove v_must_defs_{before,after}
code.
* tree-into-ssa.c (mark_def_sites): Change SSA_OP_VMUSTKILL to
SSA_OP_VMAYUSE.
* tree-ssa-pre.c (compute_rvuse_and_antic_safe): Remove
SSA_OP_VMUSTDEF and SSA_OP_VMUSTKILL.
* tree-ssa-propagate.c (stmt_makes_single_store): Remove
SSA_OP_VMUSTDEF.
From-SVN: r119760
2006-12-12 02:48:51 +01:00
|
|
|
extern bitmap vect_memsyms_to_rename;
|
2005-08-18 18:07:54 +02:00
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
/* Function prototypes. */
|
|
|
|
/*-----------------------------------------------------------------*/
|
|
|
|
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/*************************************************************************
|
|
|
|
Simple Loop Peeling Utilities - in tree-vectorizer.c
|
|
|
|
*************************************************************************/
|
|
|
|
/* Entry point for peeling of simple loops.
|
|
|
|
Peel the first/last iterations of a loop.
|
|
|
|
It can be used outside of the vectorizer for loops that are simple enough
|
|
|
|
(see function documentation). In the vectorizer it is used to peel the
|
|
|
|
last few iterations when the loop bound is unknown or does not evenly
|
|
|
|
divide by the vectorization factor, and to peel the first few iterations
|
|
|
|
to force the alignment of data references in the loop. */
|
|
|
|
extern struct loop *slpeel_tree_peel_loop_to_edge
|
2007-12-06 17:18:55 +01:00
|
|
|
(struct loop *, edge, tree, tree, bool, unsigned int, bool);
|
|
|
|
extern void set_prologue_iterations (basic_block, tree,
|
|
|
|
struct loop *, unsigned int);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
extern void slpeel_make_loop_iterate_ntimes (struct loop *, tree);
|
alias.c (component_uses_parent_alias_set): Constify.
* alias.c (component_uses_parent_alias_set): Constify.
* alias.h (component_uses_parent_alias_set): Likewise.
* cfgrtl.c (print_rtl_with_bb): Likewise.
* double-int.c (tree_to_double_int, double_int_fits_to_tree_p,
mpz_get_double_int): Likewise.
* double-int.h (double_int_fits_to_tree_p, tree_to_double_int,
mpz_get_double_int): Likewise.
* expr.c (is_aligning_offset, undefined_operand_subword_p,
mostly_zeros_p, all_zeros_p, safe_from_p, is_aligning_offset):
Likewise.
* expr.h (safe_from_p): Likewise.
* gimple-low.c (try_catch_may_fallthru, block_may_fallthru):
Likewise.
* gimplify.c (should_carry_locus_p, zero_sized_field_decl,
zero_sized_type, goa_lhs_expr_p): Likewise.
* omp-low.c (is_variable_sized, use_pointer_for_field): Likewise.
* rtl.h (print_rtl_with_bb): Likewise.
* sched-vis.c (print_exp, print_value, print_pattern): Likewise.
* tree-cfg.c (const_first_stmt, const_last_stmt): New.
* tree-flow-inline.h (bb_stmt_list): Constify.
(cbsi_start, cbsi_last, cbsi_end_p, cbsi_next, cbsi_prev,
cbsi_stmt): New.
* tree-flow.h (const_block_stmt_iterator, cbsi_start, cbsi_last,
const_first_stmt, const_last_stmt): New.
(block_may_fallthru, empty_block_p): Constify.
* tree-iterator.c (EXPR_FIRST_BODY, EXPR_LAST_BODY,
EXPR_ONLY_BODY): New.
(expr_first, expr_last, expr_only): Use macro for body.
(const_expr_first, const_expr_last, const_expr_only): New.
* tree-iterator.h (const_tree_stmt_iterator, ctsi_start,
ctsi_last, ctsi_end_p, ctsi_one_before_end_p, ctsi_next,
ctsi_prev, ctsi_stmt): New.
* tree-scalar-evolution.c (get_loop_exit_condition): Constify.
* tree-scalar-evolution.h (get_loop_exit_condition): Likewise.
* tree-ssa-loop-niter.c (loop_only_exit_p,
derive_constant_upper_bound): Likewise.
* tree-ssa-phiopt.c (empty_block_p): Likewise.
* tree-ssa-threadupdate.c (redirection_block_p): Likewise.
* tree-vectorizer.c (slpeel_can_duplicate_loop_p): Likewise.
* tree-vectorizer.h (slpeel_can_duplicate_loop_p): Likewise.
* tree-vrp.c (vrp_bitmap_equal_p): Likewise.
* tree.c (get_type_static_bounds): Likewise.
* tree.h (const_expr_first, const_expr_last, const_expr_only): New.
(get_type_static_bounds): Constify.
From-SVN: r127483
2007-08-14 17:18:11 +02:00
|
|
|
extern bool slpeel_can_duplicate_loop_p (const struct loop *, const_edge);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
#ifdef ENABLE_CHECKING
|
|
|
|
extern void slpeel_verify_cfg_after_peeling (struct loop *, struct loop *);
|
|
|
|
#endif
|
|
|
|
|
2004-08-17 18:17:14 +02:00
|
|
|
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/*************************************************************************
|
|
|
|
General Vectorization Utilities
|
|
|
|
*************************************************************************/
|
|
|
|
/** In tree-vectorizer.c **/
|
|
|
|
extern tree get_vectype_for_scalar_type (tree);
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
extern bool vect_is_simple_use (tree, loop_vec_info, tree *, tree *,
|
|
|
|
enum vect_def_type *);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
extern bool vect_is_simple_iv_evolution (unsigned, tree, tree *, tree *);
|
2007-08-19 11:39:50 +02:00
|
|
|
extern tree vect_is_simple_reduction (loop_vec_info, tree);
|
c-common.c (vector_types_convertible_p, [...]): Constify.
* c-common.c (vector_types_convertible_p,
decl_with_nonnull_addr_p, c_promoting_integer_type_p,
self_promoting_args_p): Constify.
* c-common.h (has_c_linkage, decl_with_nonnull_addr_p,
c_promoting_integer_type_p, self_promoting_args_p,
anon_aggr_type_p, vector_types_convertible_p): Likewise.
* c-decl.c (anon_aggr_type_p): Likewise.
* * c-dump.c (dump_stmt): Likewise.
* c-objc-common.c (has_c_linkage): Likewise.
* c-tree.h (same_translation_unit_p): Likewise.
* c-typeck.c (null_pointer_constant_p,
tagged_types_tu_compatible_p, function_types_compatible_p,
type_lists_compatible_p, lvalue_or_else, lvalue_p,
comptypes_internal, struct tagged_tu_seen_cache,
same_translation_unit_p, alloc_tagged_tu_seen_cache,
c_size_in_bytes): Likewise.
* ggc-common.c (compare_ptr_data, hash_descriptor, eq_descriptor,
hash_ptr, eq_ptr): Likewise.
* langhooks-def.h (lhd_decl_ok_for_sibcall,
LANG_HOOKS_OMP_PRIVATIZE_BY_REFERENCE): Likewise.
* langhooks.c (lhd_decl_ok_for_sibcall): Likewise.
* langhooks.h (ok_for_sibcall, omp_privatize_by_reference):
Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.h (optab_for_tree_code): Likewise.
* rtl.h (simplify_rtx): Likewise.
* simplify-rtx.c (simplify_rtx): Likewise.
* tree-dump.c (queue, queue_and_dump_index, queue_and_dump_type,
dump_flag, dump_node): Likewise.
* tree-dump.h (struct dump_info, dump_stmt, queue_and_dump_index,
queue_and_dump_type, dump_flag): Likewise.
* tree-flow.h (vect_can_force_dr_alignment_p): Likewise.
* tree-pass.h (dump_node): Likewise.
* tree-vectorizer.c (vect_can_force_dr_alignment_p,
supportable_narrowing_operation): Likewise.
* tree-vectorizer.h (vect_can_force_dr_alignment_p,
supportable_narrowing_operation): Likewise.
* tree-vrp.c (needs_overflow_infinity, supports_overflow_infinity,
is_negative_overflow_infinity, is_positive_overflow_infinity,
is_overflow_infinity, vrp_val_is_max, vrp_val_is_min,
nonnull_arg_p, get_value_range, vrp_operand_equal_p,
update_value_range, add_equivalence, ssa_name_nonnegative_p,
ssa_name_nonzero_p, fp_predicate): Likewise.
* tree.c (auto_var_in_fn_p, empty_body_p): Likewise.
* tree.h (empty_body_p, auto_var_in_fn_p, ssa_name_nonzero_p,
ssa_name_nonnegative_p): Likewise.
cp:
* call.c (sufficient_parms_p): Constify.
* class.c (same_signature_p): Likewise.
* cp-gimplify.c (is_invisiref_parm,
cxx_omp_privatize_by_reference): Likewise.
* cp-objcp-common.c (has_c_linkage): Likewise.
* cp-tree.h (NON_THUNK_FUNCTION_CHECK, THUNK_FUNCTION_CHECK,
sufficient_parms_p, same_signature_p, copy_fn_p, move_fn_p,
grok_ctor_properties, nothrow_libfn_p, skip_artificial_parms_for,
num_artificial_parms_for, comp_template_parms,
template_parameter_pack_p, any_dependent_template_arguments_p,
any_type_dependent_arguments_p, any_value_dependent_elements_p,
repo_export_class_p, cxx_omp_privatize_by_reference, pod_type_p,
zero_init_p, member_p, cp_lvalue_kind,
builtin_valid_in_constant_expr_p, decl_anon_ns_mem_p,
varargs_function_p, is_dummy_object, special_function_kind,
string_conv_p, type_unknown_p, comp_except_specs, compparms,
comp_cv_qualification, is_bitfield_expr_with_lowered_type,
unlowered_expr_type, ptr_reasonably_similar, cp_type_readonly,
cp_has_mutable_p, at_least_as_qualified_p,
invalid_nonstatic_memfn_p, lvalue_or_else, lvalue_p): Likewise.
* decl.c (copy_fn_p, move_fn_p, grok_ctor_properties): Likewise.
* except.c (nothrow_libfn_p): Likewise.
* method.c (skip_artificial_parms_for, num_artificial_parms_for):
Likewise.
* pt.c (comp_template_parms, template_parameter_pack_p,
any_type_dependent_arguments_p, any_value_dependent_elements_p,
any_dependent_template_arguments_p): Likewise.
* repo.c (repo_export_class_p): Likewise.
* semantics.c (anon_aggr_type_p): Likewise.
* tree.c (lvalue_p_1, real_lvalue_p, lvalue_p,
builtin_valid_in_constant_expr_p, decl_anon_ns_mem_p,
varargs_function_p, member_p, is_dummy_object, pod_type_p,
zero_init_p, special_function_p): Likewise.
* typeck.c (comp_array_types, type_unknown_p, comp_except_specs,
comp_array_types, at_least_as_qualified_p, comp_cv_qualification,
compparms, invalid_nonstatic_memfn_p,
is_bitfield_expr_with_lowered_type, unlowered_expr_type,
string_conv_p, ptr_reasonably_similar, cp_type_readonly,
cp_has_mutable_p, lvalue_or_else): Likewise.
fortran:
* trans-openmp.c (gfc_omp_privatize_by_reference): Constify.
* trans.h (gfc_omp_privatize_by_reference): Likewise.
java:
* lang.c (java_decl_ok_for_sibcall): Likewise.
From-SVN: r127799
2007-08-25 17:10:40 +02:00
|
|
|
extern bool vect_can_force_dr_alignment_p (const_tree, unsigned int);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
extern enum dr_alignment_support vect_supportable_dr_alignment
|
|
|
|
(struct data_reference *);
|
tree.def (REDUC_MAX_EXPR, [...]): New tree-codes.
* tree.def (REDUC_MAX_EXPR, REDUC_MIN_EXPR, REDUC_PLUS_EXPR): New
tree-codes.
* optabs.h (OTI_reduc_smax, OTI_reduc_umax, OTI_reduc_smin,
OTI_reduc_umin, OTI_reduc_plus): New optabs for reduction.
(reduc_smax_optab, reduc_umax_optab, reduc_smin_optab, reduc_umin_optab,
reduc_plus_optab): New optabs for reduction.
* expr.c (expand_expr_real_1): Handle new tree-codes.
* tree-inline.c (estimate_num_insns_1): Handle new tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio, op_symbol): Handle
new tree-codes.
* optabs.c (optab_for_tree_code): Handle new tree-codes.
(init_optabs): Initialize new optabs.
* genopinit.c (optabs): Define handlers for new optabs.
* tree-vect-analyze.c (vect_analyze_operations): Fail vectorization in
case of a phi that is marked as relevant. Call vectorizable_reduction.
(vect_mark_relevant): Phis may be marked as relevant.
(vect_mark_stmts_to_be_vectorized): The use corresponding to the
reduction variable in a reduction stmt does not mark its defining phi
as relevant. Update documentation accordingly.
(vect_can_advance_ivs_p): Skip reduction phis.
* tree-vect-transform.c (vect_get_vec_def_for_operand): Takes
additional argument. Handle reduction.
(vect_create_destination_var): Update call to vect_get_new_vect_var.
Handle non-vector argument.
(get_initial_def_for_reduction): New function.
(vect_create_epilog_for_reduction): New function.
(vectorizable_reduction): New function.
(vect_get_new_vect_var): Handle new vect_var_kind.
(vectorizable_assignment, vectorizable_operation, vectorizable_store,
vectorizable_condition): Update call to vect_get_new_vect_var.
(vect_transform_stmt): Call vectorizable_reduction.
(vect_update_ivs_after_vectorizer): Skip reduction phis.
(vect_transform_loop): Skip if stmt is both not relevant and not live.
* tree-vectorizer.c (reduction_code_for_scalar_code): New function.
(vect_is_simple_reduction): Was empty - added implementation.
* tree-vectorizer.h (vect_scalar_var): New enum vect_var_kind value.
(reduc_vec_info_type): New enum vect_def_type value.
* config/rs6000/altivec.md (reduc_smax_v4si, reduc_smax_v4sf,
reduc_umax_v4si, reduc_smin_v4si, reduc_umin_v4sf, reduc_smin_v4sf,
reduc_plus_v4si, reduc_plus_v4sf): New define_expands.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Remove
ENABLE_CHECKING around gcc_assert.
* tree-vect-transform.c (vect_do_peeling_for_loop_bound,
(vect_do_peeling_for_alignment, vect_transform_loop,
vect_get_vec_def_for_operand): Likewise.
From-SVN: r101155
2005-06-18 15:18:52 +02:00
|
|
|
extern bool reduction_code_for_scalar_code (enum tree_code, enum tree_code *);
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
extern bool supportable_widening_operation (enum tree_code, tree, tree,
|
|
|
|
tree *, tree *, enum tree_code *, enum tree_code *);
|
c-common.c (vector_types_convertible_p, [...]): Constify.
* c-common.c (vector_types_convertible_p,
decl_with_nonnull_addr_p, c_promoting_integer_type_p,
self_promoting_args_p): Constify.
* c-common.h (has_c_linkage, decl_with_nonnull_addr_p,
c_promoting_integer_type_p, self_promoting_args_p,
anon_aggr_type_p, vector_types_convertible_p): Likewise.
* c-decl.c (anon_aggr_type_p): Likewise.
* * c-dump.c (dump_stmt): Likewise.
* c-objc-common.c (has_c_linkage): Likewise.
* c-tree.h (same_translation_unit_p): Likewise.
* c-typeck.c (null_pointer_constant_p,
tagged_types_tu_compatible_p, function_types_compatible_p,
type_lists_compatible_p, lvalue_or_else, lvalue_p,
comptypes_internal, struct tagged_tu_seen_cache,
same_translation_unit_p, alloc_tagged_tu_seen_cache,
c_size_in_bytes): Likewise.
* ggc-common.c (compare_ptr_data, hash_descriptor, eq_descriptor,
hash_ptr, eq_ptr): Likewise.
* langhooks-def.h (lhd_decl_ok_for_sibcall,
LANG_HOOKS_OMP_PRIVATIZE_BY_REFERENCE): Likewise.
* langhooks.c (lhd_decl_ok_for_sibcall): Likewise.
* langhooks.h (ok_for_sibcall, omp_privatize_by_reference):
Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.h (optab_for_tree_code): Likewise.
* rtl.h (simplify_rtx): Likewise.
* simplify-rtx.c (simplify_rtx): Likewise.
* tree-dump.c (queue, queue_and_dump_index, queue_and_dump_type,
dump_flag, dump_node): Likewise.
* tree-dump.h (struct dump_info, dump_stmt, queue_and_dump_index,
queue_and_dump_type, dump_flag): Likewise.
* tree-flow.h (vect_can_force_dr_alignment_p): Likewise.
* tree-pass.h (dump_node): Likewise.
* tree-vectorizer.c (vect_can_force_dr_alignment_p,
supportable_narrowing_operation): Likewise.
* tree-vectorizer.h (vect_can_force_dr_alignment_p,
supportable_narrowing_operation): Likewise.
* tree-vrp.c (needs_overflow_infinity, supports_overflow_infinity,
is_negative_overflow_infinity, is_positive_overflow_infinity,
is_overflow_infinity, vrp_val_is_max, vrp_val_is_min,
nonnull_arg_p, get_value_range, vrp_operand_equal_p,
update_value_range, add_equivalence, ssa_name_nonnegative_p,
ssa_name_nonzero_p, fp_predicate): Likewise.
* tree.c (auto_var_in_fn_p, empty_body_p): Likewise.
* tree.h (empty_body_p, auto_var_in_fn_p, ssa_name_nonzero_p,
ssa_name_nonnegative_p): Likewise.
cp:
* call.c (sufficient_parms_p): Constify.
* class.c (same_signature_p): Likewise.
* cp-gimplify.c (is_invisiref_parm,
cxx_omp_privatize_by_reference): Likewise.
* cp-objcp-common.c (has_c_linkage): Likewise.
* cp-tree.h (NON_THUNK_FUNCTION_CHECK, THUNK_FUNCTION_CHECK,
sufficient_parms_p, same_signature_p, copy_fn_p, move_fn_p,
grok_ctor_properties, nothrow_libfn_p, skip_artificial_parms_for,
num_artificial_parms_for, comp_template_parms,
template_parameter_pack_p, any_dependent_template_arguments_p,
any_type_dependent_arguments_p, any_value_dependent_elements_p,
repo_export_class_p, cxx_omp_privatize_by_reference, pod_type_p,
zero_init_p, member_p, cp_lvalue_kind,
builtin_valid_in_constant_expr_p, decl_anon_ns_mem_p,
varargs_function_p, is_dummy_object, special_function_kind,
string_conv_p, type_unknown_p, comp_except_specs, compparms,
comp_cv_qualification, is_bitfield_expr_with_lowered_type,
unlowered_expr_type, ptr_reasonably_similar, cp_type_readonly,
cp_has_mutable_p, at_least_as_qualified_p,
invalid_nonstatic_memfn_p, lvalue_or_else, lvalue_p): Likewise.
* decl.c (copy_fn_p, move_fn_p, grok_ctor_properties): Likewise.
* except.c (nothrow_libfn_p): Likewise.
* method.c (skip_artificial_parms_for, num_artificial_parms_for):
Likewise.
* pt.c (comp_template_parms, template_parameter_pack_p,
any_type_dependent_arguments_p, any_value_dependent_elements_p,
any_dependent_template_arguments_p): Likewise.
* repo.c (repo_export_class_p): Likewise.
* semantics.c (anon_aggr_type_p): Likewise.
* tree.c (lvalue_p_1, real_lvalue_p, lvalue_p,
builtin_valid_in_constant_expr_p, decl_anon_ns_mem_p,
varargs_function_p, member_p, is_dummy_object, pod_type_p,
zero_init_p, special_function_p): Likewise.
* typeck.c (comp_array_types, type_unknown_p, comp_except_specs,
comp_array_types, at_least_as_qualified_p, comp_cv_qualification,
compparms, invalid_nonstatic_memfn_p,
is_bitfield_expr_with_lowered_type, unlowered_expr_type,
string_conv_p, ptr_reasonably_similar, cp_type_readonly,
cp_has_mutable_p, lvalue_or_else): Likewise.
fortran:
* trans-openmp.c (gfc_omp_privatize_by_reference): Constify.
* trans.h (gfc_omp_privatize_by_reference): Likewise.
java:
* lang.c (java_decl_ok_for_sibcall): Likewise.
From-SVN: r127799
2007-08-25 17:10:40 +02:00
|
|
|
extern bool supportable_narrowing_operation (enum tree_code, const_tree,
|
|
|
|
const_tree, enum tree_code *);
|
re PR tree-optimization/24659 (Conversions are not vectorized)
PR tree-optimization/24659
* optabs.h (enum optab_index): Add OTI_vec_unpacks_float_hi,
OTI_vec_unpacks_float_lo, OTI_vec_unpacku_float_hi,
OTI_vec_unpacku_float_lo, OTI_vec_pack_sfix_trunc and
OTI_vec_pack_ufix_trunc.
(vec_unpacks_float_hi_optab): Define new macro.
(vec_unpacks_float_lo_optab): Ditto.
(vec_unpacku_float_hi_optab): Ditto.
(vec_unpacku_float_lo_optab): Ditto.
(vec_pack_sfix_trunc_optab): Ditto.
(vec_pack_ufix_trunc_optab): Ditto.
* genopinit.c (optabs): Implement vec_unpack[s|u]_[hi|lo]_optab
and vec_pack_[s|u]fix_trunc_optab using
vec_unpack[s|u]_[hi\lo]_* and vec_pack_[u|s]fix_trunc_* patterns
* tree-vectorizer.c (supportable_widening_operation): Handle
FLOAT_EXPR and CONVERT_EXPR. Update comment.
(supportable_narrowing_operation): New function.
* tree-vectorizer.h (supportable_narrowing_operation): Prototype.
* tree-vect-transform.c (vectorizable_conversion): Handle
(nunits_in == nunits_out / 2) and (nunits_out == nunits_in / 2) cases.
(vect_gen_widened_results_half): Move before vectorizable_conversion.
(vectorizable_type_demotion): Call supportable_narrowing_operation()
to check for target support.
* optabs.c (optab_for_tree_code) Return vec_unpack[s|u]_float_hi_optab
for VEC_UNPACK_FLOAT_HI_EXPR, vec_unpack[s|u]_float_lo_optab
for VEC_UNPACK_FLOAT_LO_EXPR and vec_pack_[u|s]fix_trunc_optab
for VEC_PACK_FIX_TRUNC_EXPR.
(expand_binop): Special case mode of the result for
vec_pack_[u|s]fix_trunc_optab.
(init_optabs): Initialize vec_unpack[s|u]_[hi|lo]_optab and
vec_pack_[u|s]fix_trunc_optab.
* tree.def (VEC_UNPACK_FLOAT_HI_EXPR, VEC_UNPACK_FLOAT_LO_EXPR,
VEC_PACK_FIX_TRUNC_EXPR): New tree codes.
* tree-pretty-print.c (dump_generic_node): Handle
VEC_UNPACK_FLOAT_HI_EXPR, VEC_UNPACK_FLOAT_LO_EXPR and
VEC_PACK_FIX_TRUNC_EXPR.
(op_prio): Ditto.
* expr.c (expand_expr_real_1): Ditto.
* tree-inline.c (estimate_num_insns_1): Ditto.
* tree-vect-generic.c (expand_vector_operations_1): Ditto.
* config/i386/sse.md (vec_unpacks_float_hi_v8hi): New expander.
(vec_unpacks_float_lo_v8hi): Ditto.
(vec_unpacku_float_hi_v8hi): Ditto.
(vec_unpacku_float_lo_v8hi): Ditto.
(vec_unpacks_float_hi_v4si): Ditto.
(vec_unpacks_float_lo_v4si): Ditto.
(vec_pack_sfix_trunc_v2df): Ditto.
* doc/c-tree.texi (Expression trees) [VEC_UNPACK_FLOAT_HI_EXPR]:
Document.
[VEC_UNPACK_FLOAT_LO_EXPR]: Ditto.
[VEC_PACK_FIX_TRUNC_EXPR]: Ditto.
* doc/md.texi (Standard Names) [vec_pack_sfix_trunc]: Document.
[vec_pack_ufix_trunc]: Ditto.
[vec_unpacks_float_hi]: Ditto.
[vec_unpacks_float_lo]: Ditto.
[vec_unpacku_float_hi]: Ditto.
[vec_unpacku_float_lo]: Ditto.
testsuite/ChangeLog:
PR tree-optimization/24659
* gcc.dg/vect/vect-floatint-conversion-2.c: New test.
* gcc.dg/vect/vect-intfloat-conversion-1.c: Require vect_float,
not vect_int target.
* gcc.dg/vect/vect-intfloat-conversion-2.c: Require vect_float,
not vect_int target. Loop is vectorized for vect_intfloat_cvt
targets.
* gcc.dg/vect/vect-intfloat-conversion-3.c: New test.
* gcc.dg/vect/vect-intfloat-conversion-4a.c: New test.
* gcc.dg/vect/vect-intfloat-conversion-4b.c: New test.
From-SVN: r124784
2007-05-17 08:31:05 +02:00
|
|
|
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/* Creation and deletion of loop and stmt info structs. */
|
2004-08-17 18:17:14 +02:00
|
|
|
extern loop_vec_info new_loop_vec_info (struct loop *loop);
|
2007-08-19 11:39:50 +02:00
|
|
|
extern void destroy_loop_vec_info (loop_vec_info, bool);
|
2005-02-03 17:13:17 +01:00
|
|
|
extern stmt_vec_info new_stmt_vec_info (tree stmt, loop_vec_info);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/** In tree-vect-analyze.c **/
|
|
|
|
/* Driver for analysis stage. */
|
|
|
|
extern loop_vec_info vect_analyze_loop (struct loop *);
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
extern void vect_free_slp_tree (slp_tree);
|
2007-09-23 Razya Ladelsky
Zdenek Dvorak
OMP_ATOMIC Changes,
Reduction support for automatic parallelization.
* expr.c (expand_expr_real_1): Add cases for OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* Makefile.in: Add dependencies to expr.o, tree-parloops.o, omp-low.o
* tree-pretty-print.c (dump_generic_node): Add OMP_ATOMIC_LOAD
and OMP_ATOMIC_STORE.
* tree.h (OMP_DIRECTIVE_P): Add OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* gimple-low.c (lower_stmt): Same.
* gimplify.c (gimplify_expr): Same.
(gimplify_omp_atomic_fetch_op, gimplify_omp_atomic_pipeline,
gimplify_omp_atomic_mutex): Remove.
(gimplify_omp_atomic): Change it to simply gimplify the
statement instead of expanding it.
* omp-low.c: Add includes to optabs.h, cfgloop.h.
(expand_omp_atomic, expand_omp_atomic_pipeline,
goa_stabilize_expr, expand_omp_atomic_mutex,
expand_omp_atomic_fetch_op): New functions to implement
expansion of OMP_ATOMIC.
(expand_omp, build_omp_regions_1): Add support for
OMP_ATOMIC_LOAD/OMP_ATOMIC_STORE.
* tree-cfg.c (make_edges): add case for OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-gimple.c (is_gimple_stmt): Add OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-parloops.c: add include to tree-vectorizer.h.
(reduction_info): New structure for reduction.
(reduction_list): New list to represent list of reductions
per loop.
(struct data_arg): New helper structure for reduction.
(reduction_info_hash, reduction_info_eq, reduction_phi,
initialize_reductions,
create_call_for_reduction, create_phi_for_local_result,
create_call_for_reduction_1, create_loads_for_reductions,
create_final_loads_for_reduction): New functions.
(loop_parallel_p): Identify reductions, add reduction_list parameter.
(separate_decls_in_loop_name): Support reduction variables.
(separate_decls_in_loop): Add reduction_list and ld_st_data arguments,
call create_loads_for_reduction for each reduction.
(canonicalize_loop_ivs): Identify reductions, add reduction_list
parameter.
(transform_to_exit_first_loop): Add reduction support, add
reduction_list parameter.
(gen_parallel_loop): Add reduction_list parameter. Add call
separate_decls_in_loop with
the new argument. Traverse reductions and call
initialize_reductions, create_call_for_reduction.
(parallelize_loops): Create and delete the reduction list.
(add_field_for_name): Change use of data parameter. Add fields for
reductions.
* tree-vectorizer.h (vect_analyze_loop_form): Add declaration.
* tree-vect-analyze.c (vect_analyze_loop_form): export it.
* tree.def: Add definitions for OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-inline.c (estimate_num_insns_1): add cases for
OMP_ATOMIC_LOAD, OMP_ATOMIC_STORE.
* tree-cfg.c (make_edges): Add OMP_ATOMIC_LOAD,
OMP_ATOMIC_STORE.
* tree-ssa-operands.c (get_addr_dereference_operands):
New function. Subroutine of get_indirect_ref_operands.
(get_indirect_ref_operands): Call get_addr_dereference_operands.
(get_expr_operands): Support OMP_ATOMIC_LOAD, OMP_ATOMIC_STORE.
From-SVN: r129716
2007-10-29 12:05:04 +01:00
|
|
|
extern loop_vec_info vect_analyze_loop_form (struct loop *);
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
|
|
|
|
/** In tree-vect-patterns.c **/
|
|
|
|
/* Pattern recognition functions.
|
|
|
|
Additional pattern recognition functions can (and will) be added
|
|
|
|
in the future. */
|
|
|
|
typedef tree (* vect_recog_func_ptr) (tree, tree *, tree *);
|
2006-11-21 15:29:53 +01:00
|
|
|
#define NUM_PATTERNS 4
|
Makefile.in (tree-vect-patterns.o): Add rule for new file.
* Makefile.in (tree-vect-patterns.o): Add rule for new file.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Use
existing STMT_VINFO_VECTYPE if available.
(vect_mark_relevant): Add special handling for stmts that are
marked as STMT_VINFO_IN_PATTERN_P.
(vect_analyze_loop): Call vect_pattern_recog.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (in_pattern_p, related_stmt): New fields in
stmt_info.
(STMT_VINFO_IN_PATTERN_P, STMT_VINFO_RELATED_STMT): New macros.
(vect_recog_func_ptr): New function-pointer type.
* tree-vect-patterns.c: New file.
(vect_recog_widen_sum_pattern, vect_recog_widen_mult_pattern):
(vect_recog_dot_prod_pattern, vect_pattern_recog):
(vect_pattern_recog_1): New functions.
(vect_pattern_recog_funcs): New array of function pointers.
* tree-vectorizer.h (ternary_op): New enum value.
* tree-vect-transform.c (vect_create_epilog_for_reduction): Added
declaration. Revised documentation. Removed redundant dump prints.
Removed redundant argument. Added support for reduction patterns.
(vectorizable_reduction): Added support for reduction patterns.
(vect_transform_stmt): Added support for patterns.
* expr.c (expand_expr_real_1): Added case for DOT_PROD_EXPR.
* genopinit.c (udot_prod_optab, sdot_prod_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for DOT_PROD_EXPR.
(expand_widen_pattern_expr): New function.
(init_optabs): Initialize new optabs udot_prod_optab,
sdot_prod_optab.
* optabs.h (OTI_sdot_prod, OTI_udot_prod): New.
(sdot_prod_optab, udot_prod_optab): Define new optabs.
(expand_widen_pattern_expr): New function declaration.
* tree.def (DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR): New
tree-codes.
* tree-inline.c (estimate_num_insns_1): Added cases for new
tree-codes DOT_PROD_EXPR, WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-pretty-print.c (dump_generic_node): Likewise.
(op_prio): Likewise.
(op_symbol): Added cases for WIDEN_SUM_EXPR, WIDEN_MULT_EXPR.
* tree-ssa-operands.c (get_expr_operands): Added case for
DOT_PROD_EXPR.
* tree-vect-patterns.c (widened_name_p): New function.
(vect_recog_dot_prod_pattern): Added function implementation.
* tree-vect-transform.c (get_initial_def_for_reduction): Added
cases for DOT_PROD_EXPR, WIDEN_SUM_EXPR.
* config/rs6000/altivec.md (udot_prod<mode>, sdot_prodv8hi): New.
* config/i386/sse.md (sdot_prodv8hi, udot_prodv4si): New.
* expr.c (expand_expr_real_1): Added case for WIDEN_SUM_EXPR.
* genopinit.c (widen_ssum_optab, widen_usum_optab): Initialize.
* optabs.c (optab_for_tree_code): Added case for WIDEN_SUM_EXPR.
(init_optabs): Initialize new optabs widen_ssum_optab,
widen_usum_optab.
* optabs.h (OTI_widen_ssum, OTI_widen_usum): New.
(widen_ssum_optab, widen_usum_optab): Define new optabs.
* tree-vect-generic.c: (expand_vector_operations_1): Check type of
use instead of type of def.
* tree-vect-patterns.c (vect_recog_widen_sum_pattern): Added
function implementation.
* config/rs6000/altivec.md (widen_usum<mode>, widen_ssumv16qi,
widen_ssumv8hi): New.
* doc/tm.texi (ssum_widen, usum_widen, sdot_prod, udot_prod): New
patterns.
From-SVN: r109954
2006-01-19 11:24:00 +01:00
|
|
|
void vect_pattern_recog (loop_vec_info);
|
|
|
|
|
|
|
|
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/** In tree-vect-transform.c **/
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
extern bool vectorizable_load (tree, block_stmt_iterator *, tree *, slp_tree);
|
|
|
|
extern bool vectorizable_store (tree, block_stmt_iterator *, tree *, slp_tree);
|
|
|
|
extern bool vectorizable_operation (tree, block_stmt_iterator *, tree *,
|
|
|
|
slp_tree);
|
[multiple changes]
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p): Take
enum argument instead of bool.
(vect_analyze_operations): Call vectorizable_type_promotion.
* tree-vectorizer.h (type_promotion_vec_info_type): New enum
stmt_vec_info_type value.
(supportable_widening_operation, vectorizable_type_promotion): New
function declarations.
* tree-vect-transform.c (vect_gen_widened_results_half): New function.
(vectorizable_type_promotion): New function.
(vect_transform_stmt): Call vectorizable_type_promotion.
* tree-vect-analyze.c (supportable_widening_operation): New function.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
Add implementation.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR):
(VEC_UNPACK_HI_EXPR, VEC_UNPACK_LO_EXPR): New tree-codes.
* tree-inline.c (estimate_num_insns_1): Add cases for above new
tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* expr.c (expand_expr_real_1): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
(init_optabs): Initialize new optabs.
* genopinit.c (vec_widen_umult_hi_optab, vec_widen_smult_hi_optab,
vec_widen_smult_hi_optab, vec_widen_smult_lo_optab,
vec_unpacks_hi_optab, vec_unpacks_lo_optab, vec_unpacku_hi_optab,
vec_unpacku_lo_optab): Initialize new optabs.
* optabs.h (OTI_vec_widen_umult_hi, OTI_vec_widen_umult_lo):
(OTI_vec_widen_smult_h, OTI_vec_widen_smult_lo, OTI_vec_unpacks_hi,
OTI_vec_unpacks_lo, OTI_vec_unpacku_hi, OTI_vec_unpacku_lo): New
optab indices.
(vec_widen_umult_hi_optab, vec_widen_umult_lo_optab):
(vec_widen_smult_hi_optab, vec_widen_smult_lo_optab):
(vec_unpacks_hi_optab, vec_unpacku_hi_optab, vec_unpacks_lo_optab):
(vec_unpacku_lo_optab): New optabs.
* doc/md.texi (vec_unpacks_hi, vec_unpacks_lo, vec_unpacku_hi):
(vec_unpacku_lo, vec_widen_umult_hi, vec_widen_umult_lo):
(vec_widen_smult_hi, vec_widen_smult_lo): New.
* doc/c-tree.texi (VEC_LSHIFT_EXPR, VEC_RSHIFT_EXPR):
(VEC_WIDEN_MULT_HI_EXPR, VEC_WIDEN_MULT_LO_EXPR, VEC_UNPACK_HI_EXPR):
(VEC_UNPACK_LO_EXPR, VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New.
* config/rs6000/altivec.md (UNSPEC_VMULWHUB, UNSPEC_VMULWLUB):
(UNSPEC_VMULWHSB, UNSPEC_VMULWLSB, UNSPEC_VMULWHUH, UNSPEC_VMULWLUH):
(UNSPEC_VMULWHSH, UNSPEC_VMULWLSH): New.
(UNSPEC_VPERMSI, UNSPEC_VPERMHI): New.
(vec_vperm_v8hiv4si, vec_vperm_v16qiv8hi): New patterns used to
implement the unsigned unpacking patterns.
(vec_unpacks_hi_v16qi, vec_unpacks_hi_v8hi, vec_unpacks_lo_v16qi):
(vec_unpacks_lo_v8hi): New signed unpacking patterns.
(vec_unpacku_hi_v16qi, vec_unpacku_hi_v8hi, vec_unpacku_lo_v16qi):
(vec_unpacku_lo_v8hi): New unsigned unpacking patterns.
(vec_widen_umult_hi_v16qi, vec_widen_umult_lo_v16qi):
(vec_widen_smult_hi_v16qi, vec_widen_smult_lo_v16qi):
(vec_widen_umult_hi_v8hi, vec_widen_umult_lo_v8hi):
(vec_widen_smult_hi_v8hi, vec_widen_smult_lo_v8hi): New widening
multiplication patterns.
* target.h (builtin_mul_widen_even, builtin_mul_widen_odd): New.
* target-def.h (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN):
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
* config/rs6000/rs6000.c (rs6000_builtin_mul_widen_even): New.
(rs6000_builtin_mul_widen_odd): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): Defined.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): Defined.
* tree-vectorizer.h (enum vect_relevant): New enum type.
(_stmt_vec_info): Field relevant chaned from bool to enum
vect_relevant.
(STMT_VINFO_RELEVANT_P): Updated.
(STMT_VINFO_RELEVANT): New.
* tree-vectorizer.c (new_stmt_vec_info): Use STMT_VINFO_RELEVANT
instead of STMT_VINFO_RELEVANT_P.
* tree-vect-analyze.c (vect_mark_relevant, vect_stmt_relevant_p):
Replace calls to STMT_VINFO_RELEVANT_P with STMT_VINFO_RELEVANT,
and boolean variable with enum vect_relevant.
(vect_mark_stmts_to_be_vectorized): Likewise + update documentation.
* doc/tm.texi (TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN): New.
(TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_ODD): New.
2006-11-08 Richard Henderson <rth@redhat.com>
* config/i386/sse.md (vec_widen_umult_hi_v8hi,
vec_widen_umult_lo_v8hi): New.
(vec_widen_smult_hi_v4si, vec_widen_smult_lo_v4si,
vec_widen_umult_hi_v4si, vec_widen_umult_lo_v4si): New.
* config/i386/i386.c (ix86_expand_sse_unpack): New.
* config/i386/i386-protos.h (ix86_expand_sse_unpack): New.
* config/i386/sse.md (vec_unpacku_hi_v16qi, vec_unpacks_hi_v16qi,
vec_unpacku_lo_v16qi, vec_unpacks_lo_v16qi, vec_unpacku_hi_v8hi,
vec_unpacks_hi_v8hi, vec_unpacku_lo_v8hi, vec_unpacks_lo_v8hi,
vec_unpacku_hi_v4si, vec_unpacks_hi_v4si, vec_unpacku_lo_v4si,
vec_unpacks_lo_v4si): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_type_demotion): New function.
(vect_transform_stmt): Add case for type_demotion_vec_info_type.
(vect_analyze_operations): Call vectorizable_type_demotion.
* tree-vectorizer.h (type_demotion_vec_info_type): New enum
stmt_vec_info_type value.
(vectorizable_type_demotion): New function declaration.
* tree-vect-generic.c (expand_vector_operations_1): Consider correct
mode.
* tree.def (VEC_PACK_MOD_EXPR, VEC_PACK_SAT_EXPR): New tree-codes.
* expr.c (expand_expr_real_1): Add case for VEC_PACK_MOD_EXPR and
VEC_PACK_SAT_EXPR.
* tree-iniline.c (estimate_num_insns_1): Likewise.
* tree-pretty-print.c (dump_generic_node, op_prio): Likewise.
* optabs.c (optab_for_tree_code): Likewise.
* optabs.c (expand_binop): In case of vec_pack_*_optabs the mode
compared against the predicate of the result is not 'mode' (the input
to the function) but a mode with half the size of 'mode'.
(init_optab): Initialize new optabs.
* optabs.h (OTI_vec_pack_mod, OTI_vec_pack_ssat, OTI_vec_pack_usat):
New optab indices.
(vec_pack_mod_optab, vec_pack_ssat_optab, vec_pack_usat_optab): New
optabs.
* genopinit.c (vec_pack_mod_optab, vec_pack_ssat_optab):
(vec_pack_usat_optab): Initialize new optabs.
* doc/md.texi (vec_pack_mod, vec_pack_ssat, vec_pack_usat): New.
* config/rs6000/altivec.md (vec_pack_mod_v8hi, vec_pack_mod_v4si): New.
2006-11-08 Richard Henderson <rth@redehat.com>
* config/i386/sse.md (vec_pack_mod_v8hi, vec_pack_mod_v4si):
(vec_pack_mod_v2di, vec_interleave_highv16qi, vec_interleave_lowv16qi):
(vec_interleave_highv8hi, vec_interleave_lowv8hi):
(vec_interleave_highv4si, vec_interleave_lowv4si):
(vec_interleave_highv2di, vec_interleave_lowv2di): New.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_reduction): Support multiple
datatypes.
(vect_transform_stmt): Removed redundant code.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vectorizable_operation): Support multiple
datatypes.
2006-11-08 Dorit Nuzman <dorit@il.ibm.com>
* tree-vect-transform.c (vect_align_data_ref): Removed.
(vect_create_data_ref_ptr): Added additional argument - ptr_incr.
Updated function documentation. Return the increment stmt in ptr_incr.
(bump_vector_ptr): New function.
(vect_get_vec_def_for_stmt_copy): New function.
(vect_finish_stmt_generation): Create a stmt_info to newly created
vector stmts.
(vect_setup_realignment): Call vect_create_data_ref_ptr with additional
argument.
(vectorizable_reduction, vectorizable_assignment): Not supported yet if
VF is greater than the number of elements that can fit in one vector
word.
(vectorizable_operation, vectorizable_condition): Likewise.
(vectorizable_store, vectorizable_load): Support the case that the VF
is greater than the number of elements that can fit in one vector word.
(vect_transform_loop): Don't fail in case of multiple data-types.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Don't fail
in case of multiple data-types; the smallest type determines the VF.
(vect_analyze_data_ref_dependence): Don't record datarefs as same_align
if they are of different sizes.
(vect_update_misalignment_for_peel): Compare misalignments in terms of
number of elements rather than number of bytes.
(vect_enhance_data_refs_alignment): Fix/Add dump printouts.
(vect_can_advance_ivs_p): Fix a dump printout
From-SVN: r118577
2006-11-08 08:32:44 +01:00
|
|
|
extern bool vectorizable_type_promotion (tree, block_stmt_iterator *, tree *);
|
|
|
|
extern bool vectorizable_type_demotion (tree, block_stmt_iterator *, tree *);
|
2007-02-11 12:46:07 +01:00
|
|
|
extern bool vectorizable_conversion (tree, block_stmt_iterator *,
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
tree *, slp_tree);
|
|
|
|
extern bool vectorizable_assignment (tree, block_stmt_iterator *, tree *,
|
|
|
|
slp_tree);
|
2007-02-05 22:38:53 +01:00
|
|
|
extern tree vectorizable_function (tree, tree, tree);
|
2006-11-27 13:41:51 +01:00
|
|
|
extern bool vectorizable_call (tree, block_stmt_iterator *, tree *);
|
2005-04-12 03:35:54 +02:00
|
|
|
extern bool vectorizable_condition (tree, block_stmt_iterator *, tree *);
|
tree-flow.h (stmt_ann_d): Move aux to ...
* tree-flow.h (stmt_ann_d): Move aux to ...
(tree_ann_common_d): ... here.
* tree-ssa-loop-im.c (LIM_DATA, determine_invariantness_stmt,
move_computations_stmt, schedule_sm): Update references to
aux.
* tree-vectorizer.h (set_stmt_info, vinfo_for_stmt): Likewise.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_transform_loop): Likewise.
* tree-vectorizer.c (new_loop_vec_info, destroy_loop_vec_info):
Likewise.
* tree-vect-analyze.c (vect_analyze_scalar_cycles): Made void instead of
bool.
(vect_mark_relevant): Takes two additional arguments - live_p and
relevant_p. Set RELEVANT_P and LIVE_P according to these arguments.
(vect_stmt_relevant_p): Differentiate between a live stmt and a
relevant stmt. Return two values = live_p and relevant_p.
(vect_mark_stmts_to_be_vectorized): Call vect_mark_relevant and
vect_stmt_relevant_p with additional arguments. Phis are no longer
put into the worklist (analyzed seperately in analyze_scalar_cycles).
(vect_determine_vectorization_factor): Also check for LIVE_P, because a
stmt that is marked as irrelevant and live, cause it's only used out
side the loop, may need to be vectorized (e.g. reduction).
(vect_analyze_operations): Examine phis. Call
vectorizable_live_operation for for LIVE_P stmts. Check if
need_to_vectorize.
(vect_analyze_scalar_cycles): Update documentation. Don't fail
vectorization - just classify the scalar cycles created by the loop
phis. Call vect_is_simple_reduction.
(vect_analyze_loop): Call to analyze_scalar_cycles moved earlier.
* tree-vect-transform.c (vect_create_index_for_vector_ref): Update
call to set_stmt_info.
(vect_get_vec_def_for_operand): Code reorganized - the code that
classifies the type of use was factored out to vect_is_simple_use.
(vectorizable_store, vect_is_simple_cond): Call vect_is_simple_use with
additional arguments.
(vectorizable_assignment): Likewise. Also make sure the stmt is relevant
and computes a loop_vec_def.
(vectorizable_operation, vectorizable_load, vectorizable_condition):
Likewise.
(vectorizable_live_operation): New.
(vect_transform_stmt): Handle LIVE_P stmts.
* tree-vectorizer.c (new_stmt_vec_info): Initialize the new fields
STMT_VINFO_LIVE_P and STMT_VINFO_DEF_TYPE.
(new_loop_vec_info, destroy_loop_vec_info): Also handle phis.
(vect_is_simple_use): Determine the type of the def and return it
in a new function argument. Consider vect_reduction_def and
vect_induction_def, but for now these are not supported.
(vect_is_simple_reduction): New. Empty for now.
* tree-vectorizer.h (vect_def_type): New enum type.
(_stmt_vec_info): Added new fields - live and _stmt_vec_info.
(STMT_VINFO_LIVE_P, STMT_VINFO_DEF_TYPE): New accessor macros.
(vect_is_simple_use): New arguments added to function declaration.
(vect_is_simple_reduction): New function declaration.
(vectorizable_live_operation): New function declaration.
* tree-vect-analyze.c (vect_can_advance_ivs_p): Add debug printout.
(vect_can_advance_ivs_p): Likewise.
* tree-vect-transform.c (vect_update_ivs_after_vectorizer): Likewise.
From-SVN: r100617
2005-06-05 11:54:20 +02:00
|
|
|
extern bool vectorizable_live_operation (tree, block_stmt_iterator *, tree *);
|
tree.def (REDUC_MAX_EXPR, [...]): New tree-codes.
* tree.def (REDUC_MAX_EXPR, REDUC_MIN_EXPR, REDUC_PLUS_EXPR): New
tree-codes.
* optabs.h (OTI_reduc_smax, OTI_reduc_umax, OTI_reduc_smin,
OTI_reduc_umin, OTI_reduc_plus): New optabs for reduction.
(reduc_smax_optab, reduc_umax_optab, reduc_smin_optab, reduc_umin_optab,
reduc_plus_optab): New optabs for reduction.
* expr.c (expand_expr_real_1): Handle new tree-codes.
* tree-inline.c (estimate_num_insns_1): Handle new tree-codes.
* tree-pretty-print.c (dump_generic_node, op_prio, op_symbol): Handle
new tree-codes.
* optabs.c (optab_for_tree_code): Handle new tree-codes.
(init_optabs): Initialize new optabs.
* genopinit.c (optabs): Define handlers for new optabs.
* tree-vect-analyze.c (vect_analyze_operations): Fail vectorization in
case of a phi that is marked as relevant. Call vectorizable_reduction.
(vect_mark_relevant): Phis may be marked as relevant.
(vect_mark_stmts_to_be_vectorized): The use corresponding to the
reduction variable in a reduction stmt does not mark its defining phi
as relevant. Update documentation accordingly.
(vect_can_advance_ivs_p): Skip reduction phis.
* tree-vect-transform.c (vect_get_vec_def_for_operand): Takes
additional argument. Handle reduction.
(vect_create_destination_var): Update call to vect_get_new_vect_var.
Handle non-vector argument.
(get_initial_def_for_reduction): New function.
(vect_create_epilog_for_reduction): New function.
(vectorizable_reduction): New function.
(vect_get_new_vect_var): Handle new vect_var_kind.
(vectorizable_assignment, vectorizable_operation, vectorizable_store,
vectorizable_condition): Update call to vect_get_new_vect_var.
(vect_transform_stmt): Call vectorizable_reduction.
(vect_update_ivs_after_vectorizer): Skip reduction phis.
(vect_transform_loop): Skip if stmt is both not relevant and not live.
* tree-vectorizer.c (reduction_code_for_scalar_code): New function.
(vect_is_simple_reduction): Was empty - added implementation.
* tree-vectorizer.h (vect_scalar_var): New enum vect_var_kind value.
(reduc_vec_info_type): New enum vect_def_type value.
* config/rs6000/altivec.md (reduc_smax_v4si, reduc_smax_v4sf,
reduc_umax_v4si, reduc_smin_v4si, reduc_umin_v4sf, reduc_smin_v4sf,
reduc_plus_v4si, reduc_plus_v4sf): New define_expands.
* tree-vect-analyze.c (vect_determine_vectorization_factor): Remove
ENABLE_CHECKING around gcc_assert.
* tree-vect-transform.c (vect_do_peeling_for_loop_bound,
(vect_do_peeling_for_alignment, vect_transform_loop,
vect_get_vec_def_for_operand): Likewise.
From-SVN: r101155
2005-06-18 15:18:52 +02:00
|
|
|
extern bool vectorizable_reduction (tree, block_stmt_iterator *, tree *);
|
2007-04-17 09:31:45 +02:00
|
|
|
extern bool vectorizable_induction (tree, block_stmt_iterator *, tree *);
|
2007-06-08 18:30:49 +02:00
|
|
|
extern int vect_estimate_min_profitable_iters (loop_vec_info);
|
tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
* tree-vectorizer.h (enum vect_def_type): Start enumeration from 1.
(struct _slp_tree, struct _slp_instance): Define new data structures
along macros for their access.
(struct _loop_vec_info): Define new fields: strided_stores,
slp_instances, and slp_unrolling_factor along macros for their access.
(enum slp_vect_type): New.
(struct _stmt_vec_info): Define new field, slp_type, and macros for its
access.
(STMT_VINFO_STRIDED_ACCESS): New macro.
(vect_free_slp_tree): Declare.
(vectorizable_load): Add an argument of type slp_tree.
(vectorizable_store, vectorizable_operation, vectorizable_conversion,
vectorizable_assignment): Likewise.
(vect_model_simple_cost, vect_model_store_cost, vect_model_load_cost):
Declare (make extern).
* tree-vectorizer.c (new_stmt_vec_info): Initiliaze the new field.
(new_loop_vec_info): Likewise.
(destroy_loop_vec_info): Free memory allocated for SLP structures.
* tree-vect-analyze.c: Include recog.h.
(vect_update_slp_costs_according_to_vf): New.
(vect_analyze_operations): Add argument for calls to vectorizable_ ()
functions. For not pure SLP stmts with strided access check that the
group size is power of 2. Update the vectorization factor according to
SLP. Call vect_update_slp_costs_according_to_vf.
(vect_analyze_group_access): New.
(vect_analyze_data_ref_access): Call vect_analyze_group_access.
(vect_free_slp_tree): New functions.
(vect_get_and_check_slp_defs, vect_build_slp_tree, vect_print_slp_tree,
vect_mark_slp_stmts, vect_analyze_slp_instance, vect_analyze_slp,
vect_make_slp_decision, vect_detect_hybrid_slp_stmts,
vect_detect_hybrid_slp): Likewise.
(vect_analyze_loop): Call vect_analyze_slp, vect_make_slp_decision
and vect_detect_hybrid_slp.
* tree-vect-transform.c (vect_estimate_min_profitable_iters): Take
SLP costs into account.
(vect_get_cost_fields): New function.
(vect_model_simple_cost): Make extern, add SLP parameter and handle
SLP.
(vect_model_store_cost, vect_model_load_cost): Likewise.
(vect_get_constant_vectors): New function.
(vect_get_slp_vect_defs, vect_get_slp_defs,
vect_get_vec_defs_for_stmt_copy, vect_get_vec_defs_for_stmt_copy,
vect_get_vec_defs): Likewise.
(vectorizable_reduction): Don't handle SLP for now.
(vectorizable_call): Don't handle SLP for now. Add argument to
vect_model_simple_cost.
(vectorizable_conversion): Handle SLP (call vect_get_vec_defs to
get SLPed and vectorized defs). Fix indentation and spacing.
(vectorizable_assignment): Handle SLP.
(vectorizable_induction): Don't handle SLP for now.
(vectorizable_operation): Likewise.
(vectorizable_type_demotion): Add argument to
vect_model_simple_cost.
(vectorizable_type_promotion): Likewise.
(vectorizable_store, vectorizable_load): Handle SLP.
(vectorizable_condition): Don't handle SLP for now.
(vect_transform_stmt): Add a new argument for SLP. Check that there is
no SLP transformation required for unsupported cases. Add SLP
argument for supported cases.
(vect_remove_stores): New function.
(vect_schedule_slp_instance, vect_schedule_slp): Likewise.
(vect_transform_loop): Schedule SLP instances.
* Makefile.in: (tree-vect-analyze.o): Depend on recog.h.
From-SVN: r128289
2007-09-09 10:46:12 +02:00
|
|
|
extern void vect_model_simple_cost (stmt_vec_info, int, enum vect_def_type *,
|
|
|
|
slp_tree);
|
|
|
|
extern void vect_model_store_cost (stmt_vec_info, int, enum vect_def_type,
|
|
|
|
slp_tree);
|
|
|
|
extern void vect_model_load_cost (stmt_vec_info, int, slp_tree);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
/* Driver for transformation stage. */
|
tree-vrp.c (execute_vrp): Do not pass loops structure through arguments.
* tree-vrp.c (execute_vrp): Do not pass loops structure through
arguments.
* loop-unswitch.c (unswitch_loops, unswitch_single_loop,
unswitch_loop): Ditto.
* tree-loop-linear.c (linear_transform_loops): Ditto.
* tree-ssa-loop-im.c (determine_lsm, tree_ssa_lim_initialize,
tree_ssa_lim): Ditto.
* tree-ssa-loop-niter.c (estimate_numbers_of_iterations,
free_numbers_of_iterations_estimates): Ditto.
* tree-ssa-loop-unswitch.c (tree_ssa_unswitch_loops,
tree_unswitch_single_loop, tree_unswitch_loop): Ditto.
* cfgloopmanip.c (fix_bb_placement, fix_bb_placements,
remove_path, add_loop, loopify, unloop, fix_loop_placements,
place_new_loop, duplicate_loop, duplicate_subloops,
update_single_exit_for_duplicated_loops, copy_loops_to,
duplicate_loop_to_header_edge, create_preheaders,
force_single_succ_latches, loop_version, fix_loop_structure):
Ditto.
* tree-ssa-loop-manip.c (tree_duplicate_loop_to_header_edge,
tree_unroll_loop): Ditto.
* tree-ssa-loop-ch.c (copy_loop_headers): Ditto.
* tree-scalar-evolution.c (select_loops_exit_conditions,
scev_initialize, scev_analysis): Ditto.
* tree-scalar-evolution.h (scev_initialize): Ditto.
* cfghooks.c (cfg_hook_duplicate_loop_to_header_edge): Ditto.
* cfgloopanal.c (mark_irreducible_loops, mark_loop_exit_edges): Ditto.
* tree-ssa-loop-ivopts.c (tree_ssa_iv_optimize): Ditto.
* modulo-sched.c (sms_schedule): Ditto.
* tree-ssa-dom.c (tree_ssa_dominator_optimize): Ditto.
* loop-init.c (loop_optimizer_init, rtl_move_loop_invariants,
rtl_unswitch, rtl_unroll_and_peel_loops, rtl_doloop): Ditto.
* ifcvt.c (if_convert): Ditto.
* tree-ssa-loop-ivcanon.c (try_unroll_loop_completely,
canonicalize_loop_induction_variables,
canonicalize_induction_variables,
tree_unroll_loops_completely, remove_empty_loops): Ditto.
* tree-ssa-loop.c (tree_ssa_loop_init, tree_ssa_loop_im,
tree_ssa_loop_unswitch, tree_vectorize, tree_linear_transform,
tree_ssa_loop_ivcanon, tree_ssa_empty_loop, tree_ssa_loop_bounds,
tree_complete_unroll, tree_ssa_loop_prefetch, tree_ssa_loop_ivopts,
tree_ssa_loop_done): Ditto.
* predict.c (predict_loops, tree_estimate_probability, estimate_loops,
estimate_bb_frequencies): Ditto.
* tree-vectorizer.c (slpeel_tree_duplicate_loop_to_edge_cfg,
slpeel_tree_peel_loop_to_edge, vectorize_loops): Ditto.
* loop-unroll.c (unroll_and_peel_loops, peel_loops_completely,
decide_unrolling_and_peeling, peel_loop_completely,
unroll_loop_constant_iterations, unroll_loop_runtime_iterations,
peel_loop_simple, unroll_loop_stupid): Ditto.
* loop-doloop.c (doloop_optimize_loops): Ditto.
* tree-cfgcleanup.c (cleanup_tree_cfg_loop): Ditto.
* loop-invariant.c (move_loop_invariants): Ditto.
* tree-ssa-dce.c (tree_ssa_dce_loop): Ditto.
* tree-ssa-loop-prefetch.c (loop_prefetch_arrays,
tree_ssa_prefetch_arrays): Ditto.
* lambda-code.c (gcc_loopnest_to_lambda_loopnest, perfect_nestify):
Ditto.
* tree-vect-transform.c (vect_do_peeling_for_loop_bound,
vect_do_peeling_for_alignment, vect_transform_loop): Ditto.
* cfgloop.c (flow_loops_cfg_dump, flow_loops_dump,
mark_single_exit_loops, cancel_loop, cancel_loop_tree,
verify_loop_structure): Ditto.
* tree-flow.h (vectorize_loops, tree_ssa_lim, tree_ssa_unswitch_loops,
canonicalize_induction_variables, tree_unroll_loops_completely,
tree_ssa_prefetch_arrays, remove_empty_loops, tree_ssa_iv_optimize,
estimate_numbers_of_iterations, free_numbers_of_iterations_estimates,
tree_duplicate_loop_to_header_edge, tree_ssa_loop_version,
tree_unroll_loop, linear_transform_loops): Declaration changed.
* basic-block.h: Remove declaration of struct loops.
* cfghooks.h (struct cfg_hooks): Change type of
cfg_hook_duplicate_loop_to_header_edge.
(cfg_hook_duplicate_loop_to_header_edge): Declaration changed.
* tree-vectorizer.h (slpeel_tree_peel_loop_to_edge,
vect_transform_loop): Declaration changed.
* lambda.h (gcc_loopnest_to_lambda_loopnest): Declaration changed.
* cfgloop.h (flow_loops_dump, fix_loop_structure,
mark_irreducible_loops, mark_single_exit_loops, mark_loop_exit_edges,
cancel_loop_tree, create_preheaders, force_single_succ_latches,
verify_loop_structure, duplicate_loop, duplicate_loop_to_header_edge,
loopify, loop_version, remove_path, unswitch_loops,
unroll_and_peel_loops, doloop_optimize_loops, move_loop_invariants):
Declaration changed.
From-SVN: r119189
2006-11-25 11:34:13 +01:00
|
|
|
extern void vect_transform_loop (loop_vec_info);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
|
|
|
|
/*************************************************************************
|
|
|
|
Vectorization Debug Information - in tree-vectorizer.c
|
|
|
|
*************************************************************************/
|
tree-vectorizer.h (struct _loop_vec_info): Remove loop_line_number.
* tree-vectorizer.h (struct _loop_vec_info): Remove loop_line_number.
(LOOP_VINFO_LOC, LOOP_LOC): Remove.
* tree-vectorizer.c (vect_loop_location): New.
(vect_print_dump_info): Use vect_loop_location.
(new_loop_vec_info): Do not set LOOP_VINFO_LOC.
(vectorize_loops): Set vect_loop_location.
* tree-vect-analyze.c (vect_analyze_offset_expr,
vect_determin_vectorization_factor, vect_analyze_operations,
vect_analyze_scalar_cycles, vect_analyze_data_ref_dependence,
vect_analyze_data_ref_dependences, vect_compute_data_ref_alignment,
vect_analyze_data_refs_alignment, vect_analyze_data_ref_access,
vect_analyze_data_ref_accesses, vect_analyze_pointer_ref_access,
vect_object_analysis, vect_analyze_data_refs, vect_mark_relevant,
vect_stmt_relevant_p, vect_mark_stmts_to_be_vectorized,
vect_can_advance_ivs_p, vect_get_loop_niters, vect_analyze_loop_form,
vect_analyze_loop): Adjust vect_print_dump_info API.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref,
vect_create_data_ref_ptr, vect_init_vector, vect_get_vec_def_for_operand,
vect_finish_stmt_generation, vectorizable_assignment,
vectorizable_operation, vectorizable_store, vectorizable_load,
vectorizable_live_operation, vectorizable_condition, vect_transform_stmt,
vect_update_ivs_after_vectorizer, vect_do_peeling_for_loop_bound,
vect_gen_ninters_for_prolog_loop, vect_do_peeling_for_alignment,
vect_transform_loop): Same.
* tree-vectorizer.c (get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_reduction, vect_is_simple_iv_evolution, vectorize_loops):
Same.
From-SVN: r101649
2005-07-06 04:25:08 +02:00
|
|
|
extern bool vect_print_dump_info (enum verbosity_levels);
|
Makefile.in (tree-vect-analyze.o, [...]): New.
* Makefile.in (tree-vect-analyze.o, tree-vect-transform.o): New.
(tree-vectorizer.o): Added missing dependencies.
* tree-vectorizer.h (vect_dump, vect_verbosity_level): Added extern
decleration.
(slpeel_tree_peel_loop_to_edge): Function externalized (had a static
declaration in tree-vectorizer.c, now has an extern declaration in
tree-vectorizer.h).
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info, destroy_loop_vec_info,
new_stmt_vec_info, vect_analyze_loop, vectorizable_load,
vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop, vect_print_dump_info, vect_set_verbosity_level,
find_loop_location): Likewise.
* tree-vectorizer.c (langhooks.h): #include removed.
(slpeel_tree_peel_loop_to_edge): Function externalized. Declaration
moved to tree-vectorized.h.
(slpeel_make_loop_iterate_ntimes, slpeel_can_duplicate_loop_p,
slpeel_verify_cfg_after_peeling, vect_strip_conversion,
get_vectype_for_scalar_type, vect_is_simple_use,
vect_is_simple_iv_evolution, vect_can_force_dr_alignment_p,
vect_supportable_dr_alignment, new_loop_vec_info,
destroy_loop_vec_info, new_stmt_vec_info, vect_print_dump_info,
vect_set_verbosity_level, find_loop_location): Likewise.
(vect_analyze_loop): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-analyze.c.
(vect_analyze_loop_form): Moved to tree-vect-analyze.c.
(vect_mark_stmts_to_be_vectorized, vect_analyze_scalar_cycles,
vect_analyze_data_ref_accesses, vect_analyze_data_ref_dependences,
vect_analyze_data_refs_alignment, vect_compute_data_refs_alignment,
vect_enhance_data_refs_alignment, vect_analyze_operations,
exist_non_indexing_operands_for_use_p, vect_mark_relevant,
vect_stmt_relevant_p, vect_get_loop_niters,
vect_analyze_data_ref_dependence, vect_compute_data_ref_alignment,
vect_analyze_data_ref_access, vect_analyze_pointer_ref_access,
vect_can_advance_ivs_p, vect_get_ptr_offset, vect_analyze_offset_expr,
vect_base_addr_differ_p, vect_object_analysis, vect_address_analysis,
vect_get_memtag): Likewise.
(vectorizable_load): Function externalized. Declaration moved to
tree-vectorized.h. Function definition moved to tree-vect-transform.c.
(vectorizable_store, vectorizable_operation, vectorizable_assignment,
vect_transform_loop): Likewise.
(vect_transform_stmt): Moved to tree-vect-transform.c.
(vect_align_data_ref, vect_create_destination_var,
vect_create_data_ref_ptr, vect_create_index_for_vector_ref,
vect_create_addr_base_for_vector_ref, vect_get_new_vect_var,
vect_get_vec_def_for_operand, vect_init_vector,
vect_finish_stmt_generation, vect_generate_tmps_on_preheader,
vect_build_loop_niters, vect_update_ivs_after_vectorizer,
vect_gen_niters_for_prolog_loop, vect_update_inits_of_dr,
vect_update_inits_of_drs, vect_do_peeling_for_alignment,
vect_do_peeling_for_loop_bound): Likewise.
* tree-vect-analyze.c: New file.
* tree-vect-transform.c: New file.
From-SVN: r95153
2005-02-17 09:47:28 +01:00
|
|
|
extern void vect_set_verbosity_level (const char *);
|
|
|
|
extern LOC find_loop_location (struct loop *);
|
2004-08-17 18:17:14 +02:00
|
|
|
|
|
|
|
#endif /* GCC_TREE_VECTORIZER_H */
|