OpenE2K/gcc
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

invoke.texi (-fsched-pressure): Document it.

Browse Source 
2009-09-02  Vladimir Makarov  <vmakarov@redhat.com>

	* doc/invoke.texi (-fsched-pressure): Document it.
	(-fsched-reg-pressure-heuristic): Remove it.
	
	* reload.c (ira.h): Include.
	(find_reloads): Add choosing reload on number of small spilled
	classes.
	
	* haifa-sched.c (ira.h): Include.
	(sched_pressure_p, sched_regno_cover_class, curr_reg_pressure,
	saved_reg_pressure, curr_reg_live, saved_reg_live,
	region_ref_regs): New variables.
	(sched_init_region_reg_pressure_info, mark_regno_birth_or_death,
	initiate_reg_pressure_info, setup_ref_regs,
	initiate_bb_reg_pressure_info, save_reg_pressure,
	restore_reg_pressure, dying_use_p, print_curr_reg_pressure): New
	functions.
	(setup_insn_reg_pressure_info): New function.
	(rank_for_schedule): Add pressure checking and insn issue time.
	Remove comparison of insn reg weights.
	(ready_sort): Set insn reg pressure info.
	(update_register_pressure, setup_insn_max_reg_pressure,
	update_reg_and_insn_max_reg_pressure,
	sched_setup_bb_reg_pressure_info): New functions.
	(schedule_insn): Add code for printing and updating reg pressure
	info.
	(find_set_reg_weight, find_insn_reg_weight): Remove.
	(ok_for_early_queue_removal): Do nothing if pressure_only_p.
	(debug_ready_list): Print reg pressure info.
	(schedule_block): Ditto.  Check insn issue time.
	(sched_init): Set up sched_pressure_p.  Allocate and set up some
	reg pressure related info.
	(sched_finish): Free some reg pressure related info.
	(fix_tick_ready): Make insn always ready if pressure_p.
	(init_h_i_d): Don't call find_insn_reg_weight.
	(haifa_finish_h_i_d): Free insn reg pressure info.
	
	* ira-int.h (ira_hard_regno_cover_class, ira_reg_class_nregs,
	ira_memory_move_cost, ira_class_hard_regs,
	ira_class_hard_regs_num, ira_no_alloc_regs,
	ira_available_class_regs, ira_reg_class_cover_size,
	ira_reg_class_cover, ira_class_translate): Move to ira.h.

	* ira-lives.c (single_reg_class): Check mode to find how many
	registers are necessary for operand.
	(ira_implicitly_set_insn_hard_regs): New.

	* common.opt (fsched-pressure): New options.
	(fsched-reg-pressure-heuristic): Remove.

	* ira.c (setup_eliminable_regset): Rename to
	ira_setup_eliminable_regset.  Make it external.
	(expand_reg_info): Pass cover class to setup_reg_classes.
	(ira): Call resize_reg_info instead of allocate_reg_info.

	* sched-deps.c: Include ira.h.
	(implicit_reg_pending_clobbers, implicit_reg_pending_uses): New.
	(create_insn_reg_use, create_insn_reg_set, setup_insn_reg_uses,
	reg_pressure_info, insn_use_p, mark_insn_pseudo_birth,
	mark_insn_hard_regno_birth, mark_insn_reg_birth,
	mark_pseudo_death, mark_hard_regno_death, mark_reg_death,
	mark_insn_reg_store, mark_insn_reg_clobber,
	setup_insn_reg_pressure_info): New.
	(sched_analyze_1): Update implicit_reg_pending_uses.
	(sched_analyze_insn): Find implicit sets, uses, clobbers of regs.
	Use them to create dependencies.  Set insn reg uses and pressure
	info.  Process reg_pending_uses in one place.
	(free_deps): Free implicit sets.
	(remove_from_deps): Remove implicit sets if necessary.  Check
	implicit sets when clearing reg_last_in_use.
	(init_deps_global): Clear implicit_reg_pending_clobbers and
	implicit_reg_pending_uses.
	
	* ira.h (ira_hard_regno_cover_class, ira_reg_class_nregs,
	ira_memory_move_cost, ira_class_hard_regs,
	ira_class_hard_regs_num, ira_no_alloc_regs,
	ira_available_class_regs, ira_reg_class_cover_size,
	ira_reg_class_cover, ira_class_translate): Move from ira-int.h.
	(ira_setup_eliminable_regset, ira_set_pseudo_classes,
	ira_implicitly_set_insn_hard_regs): New prototypes.
	
	* ira-costs.c (pseudo_classes_defined_p, allocno_p,
	cost_elements_num): New variables.
	(allocno_costs, total_costs): Rename to costs and
	total_allocno_costs.
	(COSTS_OF_ALLOCNO): Rename to COSTS.
	(allocno_pref): Rename to pref.
	(allocno_pref_buffer): Rename to pref_buffer.
	(common_classes): Rename to regno_cover_class.
	(COST_INDEX): New.
	(record_reg_classes): Set allocno attributes only if allocno_p.
	(record_address_regs): Ditto.  Use COST_INDEX instead of
	ALLOCNO_NUM.
	(scan_one_insn): Use COST_INDEX and COSTS instead of ALLOCNO_NUM
	and COSTS_OF_ALLOCNO.
	(print_costs): Rename to print_allocno_costs.
	(print_pseudo_costs): New.
	(process_bb_node_for_costs): Split into 2 functions with new
	function process_bb_for_costs.  Pass BB to process_bb_for_costs.
	(find_allocno_class_costs): Rename to find_costs_and_classes.  Add
	new parameter dump_file.  Use cost_elements_num instead of
	ira_allocnos_num.  Make one iteration if preferred classes were
	already calculated for scheduler.  Make 2 versions of code
	depending on allocno_p.
	(setup_allocno_cover_class_and_costs): Check allocno_p.  Use
	regno_cover_class and COSTS instead of common_classes and
	COSTS_OF_ALLOCNO.
	(init_costs, finish_costs): New.
	(ira_costs): Set up allocno_p and cost_elements_num.  Call
	init_costs and finish_costs.
	(ira_set_pseudo_classes): New.

	* rtl.h (allocate_reg_info): Remove.
	(resize_reg_info): Change return type.
	(reg_cover_class): New.
	(setup_reg_classes): Add new parameter.
	
	* sched-int.h (struct deps_reg): New member implicit_sets.
	(sched_pressure_p, sched_regno_cover_class): New external
	definitions.
	(INCREASE_BITS): New macro.
	(struct reg_pressure_data, struct reg_use_data): New.
	(struct _haifa_insn_data): Remove reg_weight.  Add members
	reg_pressure, reg_use_list, reg_set_list, and
	reg_pressure_excess_cost_change.
	(struct deps): New member implicit_sets.
	(pressure_p): New variable.
	(COVER_CLASS_BITS, INCREASE_BITS): New macros.
	(struct reg_pressure_data, struct reg_use_data): New.
	(INSN_REG_WEIGHT): Remove.
	(INSN_REG_PRESSURE, INSN_MAX_REG_PRESSURE, INSN_REG_USE_LIST,
	INSN_REG_SET_LIST, INSN_REG_PRESSURE_EXCESS_COST_CHANGE): New
	macros.
	(sched_init_region_reg_pressure_info,
	sched_setup_bb_reg_pressure_info): New prototypes.
	
        * reginfo.c (struct reg_pref): New member coverclass.
	(reg_cover_class): New function.
	(reginfo_init, pass_reginfo_init): Move after free_reg_info.
	(reg_info_size): New variable.
	(allocate_reg_info): Make static.  Setup reg_info_size.
	(resize_reg_info): Use reg_info_size.  Return flag of resizing.
	(setup_reg_classes): Add a new parameter.  Setup cover class too.

	* Makefile.in (reload.o, haifa-sched.o, sched-deps.o): Add ira.h to the
	dependencies.

	* sched-rgn.c (deps_join): Set up implicit_sets.
	(schedule_region): Set up region and basic blocks pressure
	relative info.
	
	* passes.c (init_optimization_passes): Move
	pass_subregs_of_mode_init before pass_sched.

From-SVN: r151348
This commit is contained in:
Vladimir Makarov 2009-09-02 18:54:25 +00:00 committed by Vladimir Makarov
parent f8563a3ba7
commit ce18efcb54
17 changed files with 1825 additions and 531 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

155
gcc/ChangeLog
View File

@ -1,3 +1,158 @@
2009-09-02 Vladimir Makarov <vmakarov@redhat.com>
* doc/invoke.texi (-fsched-pressure): Document it.
(-fsched-reg-pressure-heuristic): Remove it.
* reload.c (ira.h): Include.
(find_reloads): Add choosing reload on number of small spilled
classes.
* haifa-sched.c (ira.h): Include.
(sched_pressure_p, sched_regno_cover_class, curr_reg_pressure,
saved_reg_pressure, curr_reg_live, saved_reg_live,
region_ref_regs): New variables.
(sched_init_region_reg_pressure_info, mark_regno_birth_or_death,
initiate_reg_pressure_info, setup_ref_regs,
initiate_bb_reg_pressure_info, save_reg_pressure,
restore_reg_pressure, dying_use_p, print_curr_reg_pressure): New
functions.
(setup_insn_reg_pressure_info): New function.
(rank_for_schedule): Add pressure checking and insn issue time.
Remove comparison of insn reg weights.
(ready_sort): Set insn reg pressure info.
(update_register_pressure, setup_insn_max_reg_pressure,
update_reg_and_insn_max_reg_pressure,
sched_setup_bb_reg_pressure_info): New functions.
(schedule_insn): Add code for printing and updating reg pressure
info.
(find_set_reg_weight, find_insn_reg_weight): Remove.
(ok_for_early_queue_removal): Do nothing if pressure_only_p.
(debug_ready_list): Print reg pressure info.
(schedule_block): Ditto. Check insn issue time.
(sched_init): Set up sched_pressure_p. Allocate and set up some
reg pressure related info.
(sched_finish): Free some reg pressure related info.
(fix_tick_ready): Make insn always ready if pressure_p.
(init_h_i_d): Don't call find_insn_reg_weight.
(haifa_finish_h_i_d): Free insn reg pressure info.
* ira-int.h (ira_hard_regno_cover_class, ira_reg_class_nregs,
ira_memory_move_cost, ira_class_hard_regs,
ira_class_hard_regs_num, ira_no_alloc_regs,
ira_available_class_regs, ira_reg_class_cover_size,
ira_reg_class_cover, ira_class_translate): Move to ira.h.
* ira-lives.c (single_reg_class): Check mode to find how many
registers are necessary for operand.
(ira_implicitly_set_insn_hard_regs): New.
* common.opt (fsched-pressure): New options.
(fsched-reg-pressure-heuristic): Remove.
* ira.c (setup_eliminable_regset): Rename to
ira_setup_eliminable_regset. Make it external.
(expand_reg_info): Pass cover class to setup_reg_classes.
(ira): Call resize_reg_info instead of allocate_reg_info.
* sched-deps.c: Include ira.h.
(implicit_reg_pending_clobbers, implicit_reg_pending_uses): New.
(create_insn_reg_use, create_insn_reg_set, setup_insn_reg_uses,
reg_pressure_info, insn_use_p, mark_insn_pseudo_birth,
mark_insn_hard_regno_birth, mark_insn_reg_birth,
mark_pseudo_death, mark_hard_regno_death, mark_reg_death,
mark_insn_reg_store, mark_insn_reg_clobber,
setup_insn_reg_pressure_info): New.
(sched_analyze_1): Update implicit_reg_pending_uses.
(sched_analyze_insn): Find implicit sets, uses, clobbers of regs.
Use them to create dependencies. Set insn reg uses and pressure
info. Process reg_pending_uses in one place.
(free_deps): Free implicit sets.
(remove_from_deps): Remove implicit sets if necessary. Check
implicit sets when clearing reg_last_in_use.
(init_deps_global): Clear implicit_reg_pending_clobbers and
implicit_reg_pending_uses.
* ira.h (ira_hard_regno_cover_class, ira_reg_class_nregs,
ira_memory_move_cost, ira_class_hard_regs,
ira_class_hard_regs_num, ira_no_alloc_regs,
ira_available_class_regs, ira_reg_class_cover_size,
ira_reg_class_cover, ira_class_translate): Move from ira-int.h.
(ira_setup_eliminable_regset, ira_set_pseudo_classes,
ira_implicitly_set_insn_hard_regs): New prototypes.
* ira-costs.c (pseudo_classes_defined_p, allocno_p,
cost_elements_num): New variables.
(allocno_costs, total_costs): Rename to costs and
total_allocno_costs.
(COSTS_OF_ALLOCNO): Rename to COSTS.
(allocno_pref): Rename to pref.
(allocno_pref_buffer): Rename to pref_buffer.
(common_classes): Rename to regno_cover_class.
(COST_INDEX): New.
(record_reg_classes): Set allocno attributes only if allocno_p.
(record_address_regs): Ditto. Use COST_INDEX instead of
ALLOCNO_NUM.
(scan_one_insn): Use COST_INDEX and COSTS instead of ALLOCNO_NUM
and COSTS_OF_ALLOCNO.
(print_costs): Rename to print_allocno_costs.
(print_pseudo_costs): New.
(process_bb_node_for_costs): Split into 2 functions with new
function process_bb_for_costs. Pass BB to process_bb_for_costs.
(find_allocno_class_costs): Rename to find_costs_and_classes. Add
new parameter dump_file. Use cost_elements_num instead of
ira_allocnos_num. Make one iteration if preferred classes were
already calculated for scheduler. Make 2 versions of code
depending on allocno_p.
(setup_allocno_cover_class_and_costs): Check allocno_p. Use
regno_cover_class and COSTS instead of common_classes and
COSTS_OF_ALLOCNO.
(init_costs, finish_costs): New.
(ira_costs): Set up allocno_p and cost_elements_num. Call
init_costs and finish_costs.
(ira_set_pseudo_classes): New.
* rtl.h (allocate_reg_info): Remove.
(resize_reg_info): Change return type.
(reg_cover_class): New.
(setup_reg_classes): Add new parameter.
* sched-int.h (struct deps_reg): New member implicit_sets.
(sched_pressure_p, sched_regno_cover_class): New external
definitions.
(INCREASE_BITS): New macro.
(struct reg_pressure_data, struct reg_use_data): New.
(struct _haifa_insn_data): Remove reg_weight. Add members
reg_pressure, reg_use_list, reg_set_list, and
reg_pressure_excess_cost_change.
(struct deps): New member implicit_sets.
(pressure_p): New variable.
(COVER_CLASS_BITS, INCREASE_BITS): New macros.
(struct reg_pressure_data, struct reg_use_data): New.
(INSN_REG_WEIGHT): Remove.
(INSN_REG_PRESSURE, INSN_MAX_REG_PRESSURE, INSN_REG_USE_LIST,
INSN_REG_SET_LIST, INSN_REG_PRESSURE_EXCESS_COST_CHANGE): New
macros.
(sched_init_region_reg_pressure_info,
sched_setup_bb_reg_pressure_info): New prototypes.
* reginfo.c (struct reg_pref): New member coverclass.
(reg_cover_class): New function.
(reginfo_init, pass_reginfo_init): Move after free_reg_info.
(reg_info_size): New variable.
(allocate_reg_info): Make static. Setup reg_info_size.
(resize_reg_info): Use reg_info_size. Return flag of resizing.
(setup_reg_classes): Add a new parameter. Setup cover class too.
* Makefile.in (reload.o, haifa-sched.o, sched-deps.o): Add ira.h to the
dependencies.
* sched-rgn.c (deps_join): Set up implicit_sets.
(schedule_region): Set up region and basic blocks pressure
relative info.
* passes.c (init_optimization_passes): Move
pass_subregs_of_mode_init before pass_sched.
2009-09-02 Martin Jambor <mjambor@suse.cz>
* tree-sra.c (struct access): New field grp_hint.

6
gcc/Makefile.in
View File

@ -3030,7 +3030,7 @@ vec.o : vec.c $(CONFIG_H) $(SYSTEM_H) coretypes.h vec.h $(GGC_H) \
reload.o : reload.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) \
$(FLAGS_H) output.h $(EXPR_H) $(OPTABS_H) reload.h $(RECOG_H) \
hard-reg-set.h insn-config.h $(REGS_H) $(FUNCTION_H) real.h $(TOPLEV_H) \
addresses.h $(TM_P_H) $(PARAMS_H) $(TARGET_H) $(REAL_H) $(DF_H)
addresses.h $(TM_P_H) $(PARAMS_H) $(TARGET_H) $(REAL_H) $(DF_H) ira.h
reload1.o : reload1.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) \
$(EXPR_H) $(OPTABS_H) reload.h $(REGS_H) hard-reg-set.h insn-config.h \
$(BASIC_BLOCK_H) $(RECOG_H) output.h $(FUNCTION_H) $(TOPLEV_H) $(TM_P_H) \
@ -3126,11 +3126,11 @@ modulo-sched.o : modulo-sched.c $(DDG_H) $(CONFIG_H) $(CONFIG_H) $(SYSTEM_H) \
haifa-sched.o : haifa-sched.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) \
$(SCHED_INT_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h $(FUNCTION_H) \
$(INSN_ATTR_H) $(TOPLEV_H) $(RECOG_H) $(EXCEPT_H) $(TM_P_H) $(TARGET_H) output.h \
$(PARAMS_H) $(DBGCNT_H)
$(PARAMS_H) $(DBGCNT_H) ira.h
sched-deps.o : sched-deps.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
$(RTL_H) $(SCHED_INT_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h \
$(FUNCTION_H) $(INSN_ATTR_H) $(TOPLEV_H) $(RECOG_H) $(EXCEPT_H) cselib.h \
$(PARAMS_H) $(TM_P_H)
ira.h $(PARAMS_H) $(TM_P_H) ira.h
sched-rgn.o : sched-rgn.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
$(RTL_H) $(SCHED_INT_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h \
$(FUNCTION_H) $(INSN_ATTR_H) $(TOPLEV_H) $(RECOG_H) $(EXCEPT_H) $(PARAMS_H) \

8
gcc/common.opt
View File

@ -982,6 +982,10 @@ fsched-interblock
Common Report Var(flag_schedule_interblock) Init(1) Optimization
Enable scheduling across basic blocks
fsched-pressure
Common Report Var(flag_sched_pressure) Init(0) Optimization
Enable register pressure sensitive insn scheduling
fsched-spec
Common Report Var(flag_schedule_speculative) Init(1) Optimization
Allow speculative motion of non-loads
@ -1071,10 +1075,6 @@ fsched-spec-insn-heuristic
Common Report Var(flag_sched_spec_insn_heuristic) Init(1) Optimization
Enable the speculative instruction heuristic in the scheduler
fsched-reg-pressure-heuristic
Common Report Var(flag_sched_reg_pressure_heuristic) Init(1) Optimization
Enable the register pressure heuristic in the scheduler
fsched-rank-heuristic
Common Report Var(flag_sched_rank_heuristic) Init(1) Optimization
Enable the rank heuristic in the scheduler

25
gcc/doc/invoke.texi
View File

@ -365,12 +365,12 @@ Objective-C and Objective-C++ Dialects}.
-freorder-blocks-and-partition -freorder-functions @gol
-frerun-cse-after-loop -freschedule-modulo-scheduled-loops @gol
-frounding-math -fsched2-use-superblocks @gol
-fsched2-use-traces -fsched-spec-load -fsched-spec-load-dangerous @gol
-fsched2-use-traces -fsched-pressure @gol
-fsched-spec-load -fsched-spec-load-dangerous @gol
-fsched-stalled-insns-dep[=@var{n}] -fsched-stalled-insns[=@var{n}] @gol
-fsched-group-heuristic -fsched-critical-path-heuristic @gol
-fsched-spec-insn-heuristic -fsched-reg-pressure-heuristic @gol
-fsched-rank-heuristic -fsched-last-insn-heuristic @gol
-fsched-dep-count-heuristic @gol
-fsched-spec-insn-heuristic -fsched-rank-heuristic @gol
-fsched-last-insn-heuristic -fsched-dep-count-heuristic @gol
-fschedule-insns -fschedule-insns2 -fsection-anchors @gol
-fselective-scheduling -fselective-scheduling2 @gol
-fsel-sched-pipelining -fsel-sched-pipelining-outer-loops @gol
@ -6226,6 +6226,16 @@ Don't allow speculative motion of non-load instructions. This is normally
enabled by default when scheduling before register allocation, i.e.@:
with @option{-fschedule-insns} or at @option{-O2} or higher.
@item -fsched-pressure
@opindex fsched-pressure
Enable register pressure sensitive insn scheduling before the register
allocation. This only makes sense when scheduling before register
allocation is enabled, i.e.@: with @option{-fschedule-insns} or at
@option{-O2} or higher. Usage of this option can improve the
generated code and decrease its size by preventing register pressure
increase above the number of available hard registers and as a
consequence register spills in the register allocation.
@item -fsched-spec-load
@opindex fsched-spec-load
Allow speculative motion of some load instructions. This only makes
@ -6294,13 +6304,6 @@ This is enabled by default when scheduling is enabled, i.e.@:
with @option{-fschedule-insns} or @option{-fschedule-insns2}
or at @option{-O2} or higher.
@item -fsched-reg-pressure-heuristic
@opindex fsched-reg-pressure-heuristic
Enable the register pressure heuristic in the scheduler. This heuristic
favors the instruction with smaller contribution to register pressure.
This only makes sense when scheduling before register allocation, i.e.@:
with @option{-fschedule-insns} or at @option{-O2} or higher.
@item -fsched-rank-heuristic
@opindex fsched-rank-heuristic
Enable the rank heuristic in the scheduler. This heuristic favors

537
gcc/haifa-sched.c
View File

@ -147,6 +147,7 @@ along with GCC; see the file COPYING3. If not see
#include "vecprim.h"
#include "dbgcnt.h"
#include "cfgloop.h"
#include "ira.h"
#ifdef INSN_SCHEDULING
@ -507,8 +508,6 @@ static int rank_for_schedule (const void *, const void *);
static void swap_sort (rtx *, int);
static void queue_insn (rtx, int);
static int schedule_insn (rtx);
static int find_set_reg_weight (const_rtx);
static void find_insn_reg_weight (const_rtx);
static void adjust_priority (rtx);
static void advance_one_cycle (void);
static void extend_h_i_d (void);
@ -588,6 +587,210 @@ schedule_insns (void)
}
#else
/* Do register pressure sensitive insn scheduling if the flag is set
up. */
bool sched_pressure_p;
/* Map regno -> its cover class. The map defined only when
SCHED_PRESSURE_P is true. */
enum reg_class *sched_regno_cover_class;
/* The current register pressure. Only elements corresponding cover
classes are defined. */
static int curr_reg_pressure[N_REG_CLASSES];
/* Saved value of the previous array. */
static int saved_reg_pressure[N_REG_CLASSES];
/* Register living at given scheduling point. */
static bitmap curr_reg_live;
/* Saved value of the previous array. */
static bitmap saved_reg_live;
/* Registers mentioned in the current region. */
static bitmap region_ref_regs;
/* Initiate register pressure relative info for scheduling the current
region. Currently it is only clearing register mentioned in the
current region. */
void
sched_init_region_reg_pressure_info (void)
{
bitmap_clear (region_ref_regs);
}
/* Update current register pressure related info after birth (if
BIRTH_P) or death of register REGNO. */
static void
mark_regno_birth_or_death (int regno, bool birth_p)
{
enum reg_class cover_class;
cover_class = sched_regno_cover_class[regno];
if (regno >= FIRST_PSEUDO_REGISTER)
{
if (cover_class != NO_REGS)
{
if (birth_p)
{
bitmap_set_bit (curr_reg_live, regno);
curr_reg_pressure[cover_class]
+= ira_reg_class_nregs[cover_class][PSEUDO_REGNO_MODE (regno)];
}
else
{
bitmap_clear_bit (curr_reg_live, regno);
curr_reg_pressure[cover_class]
-= ira_reg_class_nregs[cover_class][PSEUDO_REGNO_MODE (regno)];
}
}
}
else if (cover_class != NO_REGS
&& ! TEST_HARD_REG_BIT (ira_no_alloc_regs, regno))
{
if (birth_p)
{
bitmap_set_bit (curr_reg_live, regno);
curr_reg_pressure[cover_class]++;
}
else
{
bitmap_clear_bit (curr_reg_live, regno);
curr_reg_pressure[cover_class]--;
}
}
}
/* Initiate current register pressure related info from living
registers given by LIVE. */
static void
initiate_reg_pressure_info (bitmap live)
{
int i;
unsigned int j;
bitmap_iterator bi;
for (i = 0; i < ira_reg_class_cover_size; i++)
curr_reg_pressure[ira_reg_class_cover[i]] = 0;
bitmap_clear (curr_reg_live);
EXECUTE_IF_SET_IN_BITMAP (live, 0, j, bi)
if (current_nr_blocks == 1 || bitmap_bit_p (region_ref_regs, j))
mark_regno_birth_or_death (j, true);
}
/* Mark registers in X as mentioned in the current region. */
static void
setup_ref_regs (rtx x)
{
int i, j, regno;
const RTX_CODE code = GET_CODE (x);
const char *fmt;
if (REG_P (x))
{
regno = REGNO (x);
if (regno >= FIRST_PSEUDO_REGISTER)
bitmap_set_bit (region_ref_regs, REGNO (x));
else
for (i = hard_regno_nregs[regno][GET_MODE (x)] - 1; i >= 0; i--)
bitmap_set_bit (region_ref_regs, regno + i);
return;
}
fmt = GET_RTX_FORMAT (code);
for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
if (fmt[i] == 'e')
setup_ref_regs (XEXP (x, i));
else if (fmt[i] == 'E')
{
for (j = 0; j < XVECLEN (x, i); j++)
setup_ref_regs (XVECEXP (x, i, j));
}
}
/* Initiate current register pressure related info at the start of
basic block BB. */
static void
initiate_bb_reg_pressure_info (basic_block bb)
{
unsigned int i;
rtx insn;
if (current_nr_blocks > 1)
FOR_BB_INSNS (bb, insn)
if (INSN_P (insn))
setup_ref_regs (PATTERN (insn));
initiate_reg_pressure_info (df_get_live_in (bb));
#ifdef EH_RETURN_DATA_REGNO
if (bb_has_eh_pred (bb))
for (i = 0; ; ++i)
{
unsigned int regno = EH_RETURN_DATA_REGNO (i);
if (regno == INVALID_REGNUM)
break;
if (! bitmap_bit_p (df_get_live_in (bb), regno))
mark_regno_birth_or_death (regno, true);
}
#endif
}
/* Save current register pressure related info. */
static void
save_reg_pressure (void)
{
int i;
for (i = 0; i < ira_reg_class_cover_size; i++)
saved_reg_pressure[ira_reg_class_cover[i]]
= curr_reg_pressure[ira_reg_class_cover[i]];
bitmap_copy (saved_reg_live, curr_reg_live);
}
/* Restore saved register pressure related info. */
static void
restore_reg_pressure (void)
{
int i;
for (i = 0; i < ira_reg_class_cover_size; i++)
curr_reg_pressure[ira_reg_class_cover[i]]
= saved_reg_pressure[ira_reg_class_cover[i]];
bitmap_copy (curr_reg_live, saved_reg_live);
}
/* Return TRUE if the register is dying after its USE. */
static bool
dying_use_p (struct reg_use_data *use)
{
struct reg_use_data *next;
for (next = use->next_regno_use; next != use; next = next->next_regno_use)
if (QUEUE_INDEX (next->insn) != QUEUE_SCHEDULED)
return false;
return true;
}
/* Print info about the current register pressure and its excess for
each cover class. */
static void
print_curr_reg_pressure (void)
{
int i;
enum reg_class cl;
fprintf (sched_dump, ";;\t");
for (i = 0; i < ira_reg_class_cover_size; i++)
{
cl = ira_reg_class_cover[i];
gcc_assert (curr_reg_pressure[cl] >= 0);
fprintf (sched_dump, " %s:%d(%d)", reg_class_names[cl],
curr_reg_pressure[cl],
curr_reg_pressure[cl] - ira_available_class_regs[cl]);
}
fprintf (sched_dump, "\n");
}
/* Pointer to the last instruction scheduled. Used by rank_for_schedule,
so that insns independent of the last scheduled insn will be preferred
over dependent instructions. */
@ -657,7 +860,8 @@ dep_cost_1 (dep_t link, dw_t dw)
/* A USE insn should never require the value used to be computed.
This allows the computation of a function's result and parameter
values to overlap the return and call. */
values to overlap the return and call. We don't care about the
the dependence cost when only decreasing register pressure. */
if (recog_memoized (used) < 0)
{
cost = 0;
@ -686,10 +890,8 @@ dep_cost_1 (dep_t link, dw_t dw)
if (targetm.sched.adjust_cost_2)
{
cost = targetm.sched.adjust_cost_2 (used, (int) dep_type, insn, cost,
dw);
}
cost = targetm.sched.adjust_cost_2 (used, (int) dep_type, insn, cost,
dw);
else if (targetm.sched.adjust_cost != NULL)
{
/* This variable is used for backward compatibility with the
@ -906,6 +1108,53 @@ do { if ((N_READY) == 2) \
qsort (READY, N_READY, sizeof (rtx), rank_for_schedule); } \
while (0)
/* Setup info about the current register pressure impact of scheduling
INSN at the current scheduling point. */
static void
setup_insn_reg_pressure_info (rtx insn)
{
int i, change, before, after, hard_regno;
int excess_cost_change;
enum machine_mode mode;
enum reg_class cl;
struct reg_pressure_data *pressure_info;
int *max_reg_pressure;
struct reg_use_data *use;
static int death[N_REG_CLASSES];
excess_cost_change = 0;
for (i = 0; i < ira_reg_class_cover_size; i++)
death[ira_reg_class_cover[i]] = 0;
for (use = INSN_REG_USE_LIST (insn); use != NULL; use = use->next_insn_use)
if (dying_use_p (use))
{
cl = sched_regno_cover_class[use->regno];
if (use->regno < FIRST_PSEUDO_REGISTER)
death[cl]++;
else
death[cl] += ira_reg_class_nregs[cl][PSEUDO_REGNO_MODE (use->regno)];
}
pressure_info = INSN_REG_PRESSURE (insn);
max_reg_pressure = INSN_MAX_REG_PRESSURE (insn);
gcc_assert (pressure_info != NULL && max_reg_pressure != NULL);
for (i = 0; i < ira_reg_class_cover_size; i++)
{
cl = ira_reg_class_cover[i];
gcc_assert (curr_reg_pressure[cl] >= 0);
change = (int) pressure_info[i].set_increase - death[cl];
before = MAX (0, max_reg_pressure[i] - ira_available_class_regs[cl]);
after = MAX (0, max_reg_pressure[i] + change
- ira_available_class_regs[cl]);
hard_regno = ira_class_hard_regs[cl][0];
gcc_assert (hard_regno >= 0);
mode = reg_raw_mode[hard_regno];
excess_cost_change += ((after - before)
* (ira_memory_move_cost[mode][cl][0]
+ ira_memory_move_cost[mode][cl][1]));
}
INSN_REG_PRESSURE_EXCESS_COST_CHANGE (insn) = excess_cost_change;
}
/* Returns a positive value if x is preferred; returns a negative value if
y is preferred. Should never return 0, since that will make the sort
unstable. */
@ -917,7 +1166,7 @@ rank_for_schedule (const void *x, const void *y)
rtx tmp2 = *(const rtx *) x;
rtx last;
int tmp_class, tmp2_class;
int val, priority_val, weight_val, info_val;
int val, priority_val, info_val;
if (MAY_HAVE_DEBUG_INSNS)
{
@ -936,12 +1185,38 @@ rank_for_schedule (const void *x, const void *y)
/* Make sure that priority of TMP and TMP2 are initialized. */
gcc_assert (INSN_PRIORITY_KNOWN (tmp) && INSN_PRIORITY_KNOWN (tmp2));
if (sched_pressure_p)
{
int diff;
/* Prefer insn whose scheduling results in the smallest register
pressure excess. */
if ((diff = (INSN_REG_PRESSURE_EXCESS_COST_CHANGE (tmp)
+ (INSN_TICK (tmp) > clock_var
? INSN_TICK (tmp) - clock_var : 0)
- INSN_REG_PRESSURE_EXCESS_COST_CHANGE (tmp2)
- (INSN_TICK (tmp2) > clock_var
? INSN_TICK (tmp2) - clock_var : 0))) != 0)
return diff;
}
if (sched_pressure_p
&& (INSN_TICK (tmp2) > clock_var || INSN_TICK (tmp) > clock_var))
{
if (INSN_TICK (tmp) <= clock_var)
return -1;
else if (INSN_TICK (tmp2) <= clock_var)
return 1;
else
return INSN_TICK (tmp) - INSN_TICK (tmp2);
}
/* Prefer insn with higher priority. */
priority_val = INSN_PRIORITY (tmp2) - INSN_PRIORITY (tmp);
if (flag_sched_critical_path_heuristic && priority_val)
return priority_val;
/* Prefer speculative insn with greater dependencies weakness. */
if (flag_sched_spec_insn_heuristic && spec_info)
{
@ -966,11 +1241,6 @@ rank_for_schedule (const void *x, const void *y)
return dw;
}
/* Prefer an insn with smaller contribution to registers-pressure. */
if (flag_sched_reg_pressure_heuristic && !reload_completed &&
(weight_val = INSN_REG_WEIGHT (tmp) - INSN_REG_WEIGHT (tmp2)))
return weight_val;
info_val = (*current_sched_info->rank) (tmp, tmp2);
if(flag_sched_rank_heuristic && info_val)
return info_val;
@ -1222,7 +1492,14 @@ ready_remove_insn (rtx insn)
void
ready_sort (struct ready_list *ready)
{
int i;
rtx *first = ready_lastpos (ready);
if (sched_pressure_p)
{
for (i = 0; i < ready->n_ready; i++)
setup_insn_reg_pressure_info (first[i]);
}
SCHED_SORT (first, ready->n_ready);
}
@ -1278,6 +1555,93 @@ advance_one_cycle (void)
/* Clock at which the previous instruction was issued. */
static int last_clock_var;
/* Update register pressure after scheduling INSN. */
static void
update_register_pressure (rtx insn)
{
struct reg_use_data *use;
struct reg_set_data *set;
for (use = INSN_REG_USE_LIST (insn); use != NULL; use = use->next_insn_use)
if (dying_use_p (use) && bitmap_bit_p (curr_reg_live, use->regno))
mark_regno_birth_or_death (use->regno, false);
for (set = INSN_REG_SET_LIST (insn); set != NULL; set = set->next_insn_set)
mark_regno_birth_or_death (set->regno, true);
}
/* Set up or update (if UPDATE_P) max register pressure (see its
meaning in sched-int.h::_haifa_insn_data) for all current BB insns
after insn AFTER. */
static void
setup_insn_max_reg_pressure (rtx after, bool update_p)
{
int i, p;
bool eq_p;
rtx insn;
static int max_reg_pressure[N_REG_CLASSES];
save_reg_pressure ();
for (i = 0; i < ira_reg_class_cover_size; i++)
max_reg_pressure[ira_reg_class_cover[i]]
= curr_reg_pressure[ira_reg_class_cover[i]];
for (insn = NEXT_INSN (after);
insn != NULL_RTX && BLOCK_FOR_INSN (insn) == BLOCK_FOR_INSN (after);
insn = NEXT_INSN (insn))
if (NONDEBUG_INSN_P (insn))
{
eq_p = true;
for (i = 0; i < ira_reg_class_cover_size; i++)
{
p = max_reg_pressure[ira_reg_class_cover[i]];
if (INSN_MAX_REG_PRESSURE (insn)[i] != p)
{
eq_p = false;
INSN_MAX_REG_PRESSURE (insn)[i]
= max_reg_pressure[ira_reg_class_cover[i]];
}
}
if (update_p && eq_p)
break;
update_register_pressure (insn);
for (i = 0; i < ira_reg_class_cover_size; i++)
if (max_reg_pressure[ira_reg_class_cover[i]]
< curr_reg_pressure[ira_reg_class_cover[i]])
max_reg_pressure[ira_reg_class_cover[i]]
= curr_reg_pressure[ira_reg_class_cover[i]];
}
restore_reg_pressure ();
}
/* Update the current register pressure after scheduling INSN. Update
also max register pressure for unscheduled insns of the current
BB. */
static void
update_reg_and_insn_max_reg_pressure (rtx insn)
{
int i;
int before[N_REG_CLASSES];
for (i = 0; i < ira_reg_class_cover_size; i++)
before[i] = curr_reg_pressure[ira_reg_class_cover[i]];
update_register_pressure (insn);
for (i = 0; i < ira_reg_class_cover_size; i++)
if (curr_reg_pressure[ira_reg_class_cover[i]] != before[i])
break;
if (i < ira_reg_class_cover_size)
setup_insn_max_reg_pressure (insn, true);
}
/* Set up register pressure at the beginning of basic block BB whose
insns starting after insn AFTER. Set up also max register pressure
for all insns of the basic block. */
void
sched_setup_bb_reg_pressure_info (basic_block bb, rtx after)
{
gcc_assert (sched_pressure_p);
initiate_bb_reg_pressure_info (bb);
setup_insn_max_reg_pressure (after, false);
}
/* INSN is the "currently executing insn". Launch each insn which was
waiting on INSN. READY is the ready list which contains the insns
that are ready to fire. CLOCK is the current cycle. The function
@ -1289,10 +1653,12 @@ schedule_insn (rtx insn)
{
sd_iterator_def sd_it;
dep_t dep;
int i;
int advance = 0;
if (sched_verbose >= 1)
{
struct reg_pressure_data *pressure_info;
char buf[2048];
print_insn (buf, insn, 0);
@ -1303,9 +1669,21 @@ schedule_insn (rtx insn)
fprintf (sched_dump, "nothing");
else
print_reservation (sched_dump, insn);
pressure_info = INSN_REG_PRESSURE (insn);
if (pressure_info != NULL)
{
fputc (':', sched_dump);
for (i = 0; i < ira_reg_class_cover_size; i++)
fprintf (sched_dump, "%s%+d(%d)",
reg_class_names[ira_reg_class_cover[i]],
pressure_info[i].set_increase, pressure_info[i].change);
}
fputc ('\n', sched_dump);
}
if (sched_pressure_p)
update_reg_and_insn_max_reg_pressure (insn);
/* Scheduling instruction should have all its dependencies resolved and
should have been removed from the ready list. */
gcc_assert (sd_lists_empty_p (insn, SD_LIST_BACK));
@ -1614,66 +1992,6 @@ restore_other_notes (rtx head, basic_block head_bb)
return head;
}
/* Functions for computation of registers live/usage info. */
/* This function looks for a new register being defined.
If the destination register is already used by the source,
a new register is not needed. */
static int
find_set_reg_weight (const_rtx x)
{
if (GET_CODE (x) == CLOBBER
&& register_operand (SET_DEST (x), VOIDmode))
return 1;
if (GET_CODE (x) == SET
&& register_operand (SET_DEST (x), VOIDmode))
{
if (REG_P (SET_DEST (x)))
{
if (!reg_mentioned_p (SET_DEST (x), SET_SRC (x)))
return 1;
else
return 0;
}
return 1;
}
return 0;
}
/* Calculate INSN_REG_WEIGHT for INSN. */
static void
find_insn_reg_weight (const_rtx insn)
{
int reg_weight = 0;
rtx x;
/* Handle register life information. */
if (! INSN_P (insn))
return;
/* Increment weight for each register born here. */
x = PATTERN (insn);
reg_weight += find_set_reg_weight (x);
if (GET_CODE (x) == PARALLEL)
{
int j;
for (j = XVECLEN (x, 0) - 1; j >= 0; j--)
{
x = XVECEXP (PATTERN (insn), 0, j);
reg_weight += find_set_reg_weight (x);
}
}
/* Decrement weight for each register that dies here. */
for (x = REG_NOTES (insn); x; x = XEXP (x, 1))
{
if (REG_NOTE_KIND (x) == REG_DEAD
|| REG_NOTE_KIND (x) == REG_UNUSED)
reg_weight--;
}
INSN_REG_WEIGHT (insn) = reg_weight;
}
/* Move insns that became ready to fire from queue to ready list. */
static void
@ -1943,7 +2261,18 @@ debug_ready_list (struct ready_list *ready)
p = ready_lastpos (ready);
for (i = 0; i < ready->n_ready; i++)
fprintf (sched_dump, " %s", (*current_sched_info->print_insn) (p[i], 0));
{
fprintf (sched_dump, " %s:%d",
(*current_sched_info->print_insn) (p[i], 0),
INSN_LUID (p[i]));
if (sched_pressure_p)
fprintf (sched_dump, "(cost=%d",
INSN_REG_PRESSURE_EXCESS_COST_CHANGE (p[i]));
if (INSN_TICK (p[i]) > clock_var)
fprintf (sched_dump, ":delay=%d", INSN_TICK (p[i]) - clock_var);
if (sched_pressure_p)
fprintf (sched_dump, ")");
}
fprintf (sched_dump, "\n");
}
@ -2666,6 +2995,8 @@ schedule_block (basic_block *target_bb)
fprintf (sched_dump, ";;\tReady list (t = %3d): ",
clock_var);
debug_ready_list (&ready);
if (sched_pressure_p)
print_curr_reg_pressure ();
}
if (ready.n_ready == 0
@ -2708,6 +3039,13 @@ schedule_block (basic_block *target_bb)
else
insn = ready_remove_first (&ready);
if (sched_pressure_p && INSN_TICK (insn) > clock_var)
{
ready_add (&ready, insn, true);
advance = 1;
break;
}
if (targetm.sched.dfa_new_cycle
&& targetm.sched.dfa_new_cycle (sched_dump, sched_verbose,
insn, last_clock_var,
@ -2745,6 +3083,8 @@ schedule_block (basic_block *target_bb)
fatal error for unrecognizable insns. */
cost = 0;
}
else if (sched_pressure_p)
cost = 0;
else
{
cost = state_transition (temp_state, insn);
@ -2826,7 +3166,6 @@ schedule_block (basic_block *target_bb)
else if (GET_CODE (PATTERN (insn)) != USE
&& GET_CODE (PATTERN (insn)) != CLOBBER)
can_issue_more--;
advance = schedule_insn (insn);
/* After issuing an asm insn we should start a new cycle. */
@ -3033,6 +3372,11 @@ sched_init (void)
flag_schedule_speculative_load = 0;
#endif
sched_pressure_p = (flag_sched_pressure && ! reload_completed
&& common_sched_info->sched_pass_id == SCHED_RGN_PASS);
if (sched_pressure_p)
ira_setup_eliminable_regset ();
/* Initialize SPEC_INFO. */
if (targetm.sched.set_sched_flags)
{
@ -3108,6 +3452,23 @@ sched_init (void)
targetm.sched.md_init_global (sched_dump, sched_verbose,
get_max_uid () + 1);
if (sched_pressure_p)
{
int i, max_regno = max_reg_num ();
ira_set_pseudo_classes (sched_verbose ? sched_dump : NULL);
sched_regno_cover_class
= (enum reg_class *) xmalloc (max_regno * sizeof (enum reg_class));
for (i = 0; i < max_regno; i++)
sched_regno_cover_class[i]
= (i < FIRST_PSEUDO_REGISTER
? ira_class_translate[REGNO_REG_CLASS (i)]
: reg_cover_class (i));
curr_reg_live = BITMAP_ALLOC (NULL);
saved_reg_live = BITMAP_ALLOC (NULL);
region_ref_regs = BITMAP_ALLOC (NULL);
}
curr_state = xmalloc (dfa_state_size);
}
@ -3205,6 +3566,13 @@ void
sched_finish (void)
{
haifa_finish_h_i_d ();
if (sched_pressure_p)
{
free (sched_regno_cover_class);
BITMAP_FREE (region_ref_regs);
BITMAP_FREE (saved_reg_live);
BITMAP_FREE (curr_reg_live);
}
free (curr_state);
if (targetm.sched.md_finish_global)
@ -3514,7 +3882,7 @@ fix_tick_ready (rtx next)
INSN_TICK (next) = tick;
delay = tick - clock_var;
if (delay <= 0)
if (delay <= 0 || sched_pressure_p)
delay = QUEUE_READY;
change_queue_index (next, delay);
@ -5091,7 +5459,6 @@ init_h_i_d (rtx insn)
if (INSN_LUID (insn) > 0)
{
INSN_COST (insn) = -1;
find_insn_reg_weight (insn);
QUEUE_INDEX (insn) = QUEUE_NOWHERE;
INSN_TICK (insn) = INVALID_TICK;
INTER_TICK (insn) = INVALID_TICK;
@ -5118,6 +5485,20 @@ haifa_init_h_i_d (bb_vec_t bbs, basic_block bb, insn_vec_t insns, rtx insn)
void
haifa_finish_h_i_d (void)
{
int i;
haifa_insn_data_t data;
struct reg_use_data *use, *next;
for (i = 0; VEC_iterate (haifa_insn_data_def, h_i_d, i, data); i++)
{
if (data->reg_pressure != NULL)
free (data->reg_pressure);
for (use = data->reg_use_list; use != NULL; use = next)
{
next = use->next_insn_use;
free (use);
}
}
VEC_free (haifa_insn_data_def, heap, h_i_d);
}

488
gcc/ira-costs.c
View File

@ -1,4 +1,4 @@
/* IRA hard register and memory cost calculation for allocnos.
/* IRA hard register and memory cost calculation for allocnos or pseudos.
Copyright (C) 2006, 2007, 2008, 2009
Free Software Foundation, Inc.
Contributed by Vladimir Makarov <vmakarov@redhat.com>.
@ -38,18 +38,25 @@ along with GCC; see the file COPYING3. If not see
#include "params.h"
#include "ira-int.h"
/* The file contains code is similar to one in regclass but the code
works on the allocno basis. */
/* The flags is set up every time when we calculate pseudo register
classes through function ira_set_pseudo_classes. */
static bool pseudo_classes_defined_p = false;
/* TRUE if we work with allocnos. Otherwise we work with pseudos. */
static bool allocno_p;
/* Number of elements in arrays `in_inc_dec' and `costs'. */
static int cost_elements_num;
#ifdef FORBIDDEN_INC_DEC_CLASSES
/* Indexed by n, is TRUE if allocno with number N is used in an
auto-inc or auto-dec context. */
/* Indexed by n, is TRUE if allocno or pseudo with number N is used in
an auto-inc or auto-dec context. */
static bool *in_inc_dec;
#endif
/* The `costs' struct records the cost of using hard registers of each
class considered for the calculation and of using memory for each
allocno. */
allocno or pseudo. */
struct costs
{
int mem_cost;
@ -74,8 +81,11 @@ static struct costs *temp_costs;
static struct costs *op_costs[MAX_RECOG_OPERANDS];
static struct costs *this_op_costs[MAX_RECOG_OPERANDS];
/* Original and accumulated costs of each class for each allocno. */
static struct costs *allocno_costs, *total_costs;
/* Costs of each class for each allocno or pseudo. */
static struct costs *costs;
/* Accumulated costs of each class for each allocno. */
static struct costs *total_allocno_costs;
/* Classes used for cost calculation. They may be different on
different iterations of the cost calculations or in different
@ -92,21 +102,26 @@ static int cost_class_nums[N_REG_CLASSES];
/* It is the current size of struct costs. */
static int struct_costs_size;
/* Return pointer to structure containing costs of allocno with given
NUM in array ARR. */
#define COSTS_OF_ALLOCNO(arr, num) \
/* Return pointer to structure containing costs of allocno or pseudo
with given NUM in array ARR. */
#define COSTS(arr, num) \
((struct costs *) ((char *) (arr) + (num) * struct_costs_size))
/* Record register class preferences of each allocno. Null value
means no preferences. It happens on the 1st iteration of the cost
calculation. */
static enum reg_class *allocno_pref;
/* Return index in COSTS when processing reg with REGNO. */
#define COST_INDEX(regno) (allocno_p \
? ALLOCNO_NUM (ira_curr_regno_allocno_map[regno]) \
: (int) regno)
/* Allocated buffers for allocno_pref. */
static enum reg_class *allocno_pref_buffer;
/* Record register class preferences of each allocno or pseudo. Null
value means no preferences. It happens on the 1st iteration of the
cost calculation. */
static enum reg_class *pref;
/* Record register class of each allocno with the same regno. */
static enum reg_class *common_classes;
/* Allocated buffers for pref. */
static enum reg_class *pref_buffer;
/* Record cover register class of each allocno with the same regno. */
static enum reg_class *regno_cover_class;
/* Execution frequency of the current insn. */
static int frequency;
@ -189,7 +204,7 @@ static void
record_reg_classes (int n_alts, int n_ops, rtx *ops,
enum machine_mode *modes, const char **constraints,
rtx insn, struct costs **op_costs,
enum reg_class *allocno_pref)
enum reg_class *pref)
{
int alt;
int i, j, k;
@ -320,12 +335,9 @@ record_reg_classes (int n_alts, int n_ops, rtx *ops,
were not in the appropriate class. We could use
cover class here but it is less accurate
approximation. */
if (allocno_pref)
if (pref)
{
enum reg_class pref_class
= allocno_pref[ALLOCNO_NUM
(ira_curr_regno_allocno_map
[REGNO (op)])];
enum reg_class pref_class = pref[COST_INDEX (REGNO (op))];
if (pref_class == NO_REGS)
alt_cost
@ -564,12 +576,9 @@ record_reg_classes (int n_alts, int n_ops, rtx *ops,
were not in the appropriate class. We could use
cover class here but it is less accurate
approximation. */
if (allocno_pref)
if (pref)
{
enum reg_class pref_class
= allocno_pref[ALLOCNO_NUM
(ira_curr_regno_allocno_map
[REGNO (op)])];
enum reg_class pref_class = pref[COST_INDEX (REGNO (op))];
if (pref_class == NO_REGS)
alt_cost
@ -637,17 +646,18 @@ record_reg_classes (int n_alts, int n_ops, rtx *ops,
}
}
for (i = 0; i < n_ops; i++)
{
ira_allocno_t a;
rtx op = ops[i];
if (! REG_P (op) || REGNO (op) < FIRST_PSEUDO_REGISTER)
continue;
a = ira_curr_regno_allocno_map [REGNO (op)];
if (! ALLOCNO_BAD_SPILL_P (a) && insn_allows_mem[i] == 0)
ALLOCNO_BAD_SPILL_P (a) = true;
}
if (allocno_p)
for (i = 0; i < n_ops; i++)
{
ira_allocno_t a;
rtx op = ops[i];
if (! REG_P (op) || REGNO (op) < FIRST_PSEUDO_REGISTER)
continue;
a = ira_curr_regno_allocno_map [REGNO (op)];
if (! ALLOCNO_BAD_SPILL_P (a) && insn_allows_mem[i] == 0)
ALLOCNO_BAD_SPILL_P (a) = true;
}
/* If this insn is a single set copying operand 1 to operand 0 and
one operand is an allocno with the other a hard reg or an allocno
@ -877,8 +887,7 @@ record_address_regs (enum machine_mode mode, rtx x, int context,
#ifdef FORBIDDEN_INC_DEC_CLASSES
if (REG_P (XEXP (x, 0))
&& REGNO (XEXP (x, 0)) >= FIRST_PSEUDO_REGISTER)
in_inc_dec[ALLOCNO_NUM (ira_curr_regno_allocno_map
[REGNO (XEXP (x, 0))])] = true;
in_inc_dec[COST_INDEX (REGNO (XEXP (x, 0)))] = true;
#endif
record_address_regs (mode, XEXP (x, 0), 0, code, SCRATCH, 2 * scale);
break;
@ -892,10 +901,9 @@ record_address_regs (enum machine_mode mode, rtx x, int context,
if (REGNO (x) < FIRST_PSEUDO_REGISTER)
break;
ALLOCNO_BAD_SPILL_P (ira_curr_regno_allocno_map[REGNO (x)]) = true;
pp = COSTS_OF_ALLOCNO (allocno_costs,
ALLOCNO_NUM (ira_curr_regno_allocno_map
[REGNO (x)]));
if (allocno_p)
ALLOCNO_BAD_SPILL_P (ira_curr_regno_allocno_map[REGNO (x)]) = true;
pp = COSTS (costs, COST_INDEX (REGNO (x)));
pp->mem_cost += (ira_memory_move_cost[Pmode][rclass][1] * scale) / 2;
for (k = 0; k < cost_classes_num; k++)
{
@ -922,8 +930,7 @@ record_address_regs (enum machine_mode mode, rtx x, int context,
/* Calculate the costs of insn operands. */
static void
record_operand_costs (rtx insn, struct costs **op_costs,
enum reg_class *allocno_pref)
record_operand_costs (rtx insn, struct costs **op_costs, enum reg_class *pref)
{
const char *constraints[MAX_RECOG_OPERANDS];
enum machine_mode modes[MAX_RECOG_OPERANDS];
@ -976,11 +983,11 @@ record_operand_costs (rtx insn, struct costs **op_costs,
xconstraints[i+1] = constraints[i];
record_reg_classes (recog_data.n_alternatives, recog_data.n_operands,
recog_data.operand, modes,
xconstraints, insn, op_costs, allocno_pref);
xconstraints, insn, op_costs, pref);
}
record_reg_classes (recog_data.n_alternatives, recog_data.n_operands,
recog_data.operand, modes,
constraints, insn, op_costs, allocno_pref);
constraints, insn, op_costs, pref);
}
@ -1015,17 +1022,17 @@ scan_one_insn (rtx insn)
{
enum reg_class cl = GENERAL_REGS;
rtx reg = SET_DEST (set);
int num = ALLOCNO_NUM (ira_curr_regno_allocno_map[REGNO (reg)]);
int num = COST_INDEX (REGNO (reg));
if (allocno_pref)
cl = allocno_pref[num];
COSTS_OF_ALLOCNO (allocno_costs, num)->mem_cost
if (pref)
cl = pref[num];
COSTS (costs, num)->mem_cost
-= ira_memory_move_cost[GET_MODE (reg)][cl][1] * frequency;
record_address_regs (GET_MODE (SET_SRC (set)), XEXP (SET_SRC (set), 0),
0, MEM, SCRATCH, frequency * 2);
}
record_operand_costs (insn, op_costs, allocno_pref);
record_operand_costs (insn, op_costs, pref);
/* Now add the cost for each operand to the total costs for its
allocno. */
@ -1034,9 +1041,7 @@ scan_one_insn (rtx insn)
&& REGNO (recog_data.operand[i]) >= FIRST_PSEUDO_REGISTER)
{
int regno = REGNO (recog_data.operand[i]);
struct costs *p
= COSTS_OF_ALLOCNO (allocno_costs,
ALLOCNO_NUM (ira_curr_regno_allocno_map[regno]));
struct costs *p = COSTS (costs, COST_INDEX (regno));
struct costs *q = op_costs[i];
p->mem_cost += q->mem_cost;
@ -1051,12 +1056,13 @@ scan_one_insn (rtx insn)
/* Print allocnos costs to file F. */
static void
print_costs (FILE *f)
print_allocno_costs (FILE *f)
{
int k;
ira_allocno_t a;
ira_allocno_iterator ai;
ira_assert (allocno_p);
fprintf (f, "\n");
FOR_EACH_ALLOCNO (a, ai)
{
@ -1085,57 +1091,114 @@ print_costs (FILE *f)
)
{
fprintf (f, " %s:%d", reg_class_names[rclass],
COSTS_OF_ALLOCNO (allocno_costs, i)->cost[k]);
COSTS (costs, i)->cost[k]);
if (flag_ira_region == IRA_REGION_ALL
|| flag_ira_region == IRA_REGION_MIXED)
fprintf (f, ",%d", COSTS_OF_ALLOCNO (total_costs, i)->cost[k]);
fprintf (f, ",%d", COSTS (total_allocno_costs, i)->cost[k]);
}
}
fprintf (f, " MEM:%i\n", COSTS_OF_ALLOCNO (allocno_costs, i)->mem_cost);
fprintf (f, " MEM:%i\n", COSTS (costs, i)->mem_cost);
}
}
/* Print pseudo costs to file F. */
static void
print_pseudo_costs (FILE *f)
{
int regno, k;
int rclass;
ira_assert (! allocno_p);
fprintf (f, "\n");
for (regno = max_reg_num () - 1; regno >= FIRST_PSEUDO_REGISTER; regno--)
{
if (regno_reg_rtx[regno] == NULL_RTX)
continue;
fprintf (f, " r%d costs:", regno);
for (k = 0; k < cost_classes_num; k++)
{
rclass = cost_classes[k];
if (contains_reg_of_mode[rclass][PSEUDO_REGNO_MODE (regno)]
#ifdef FORBIDDEN_INC_DEC_CLASSES
&& (! in_inc_dec[regno] || ! forbidden_inc_dec_class[rclass])
#endif
#ifdef CANNOT_CHANGE_MODE_CLASS
&& ! invalid_mode_change_p (regno, (enum reg_class) rclass,
PSEUDO_REGNO_MODE (regno))
#endif
)
fprintf (f, " %s:%d", reg_class_names[rclass],
COSTS (costs, regno)->cost[k]);
}
fprintf (f, " MEM:%i\n", COSTS (costs, regno)->mem_cost);
}
}
/* Traverse the BB represented by LOOP_TREE_NODE to update the allocno
costs. */
static void
process_bb_for_costs (basic_block bb)
{
rtx insn;
frequency = REG_FREQ_FROM_BB (bb);
if (frequency == 0)
frequency = 1;
FOR_BB_INSNS (bb, insn)
insn = scan_one_insn (insn);
}
/* Traverse the BB represented by LOOP_TREE_NODE to update the allocno
costs. */
static void
process_bb_node_for_costs (ira_loop_tree_node_t loop_tree_node)
{
basic_block bb;
rtx insn;
bb = loop_tree_node->bb;
if (bb == NULL)
return;
frequency = REG_FREQ_FROM_BB (bb);
if (frequency == 0)
frequency = 1;
FOR_BB_INSNS (bb, insn)
insn = scan_one_insn (insn);
if (bb != NULL)
process_bb_for_costs (bb);
}
/* Find costs of register classes and memory for allocnos and their
best costs. */
/* Find costs of register classes and memory for allocnos or pseudos
and their best costs. Set up preferred, alternative and cover
classes for pseudos. */
static void
find_allocno_class_costs (void)
find_costs_and_classes (FILE *dump_file)
{
int i, k;
int i, k, start;
int pass;
basic_block bb;
init_recog ();
#ifdef FORBIDDEN_INC_DEC_CLASSES
in_inc_dec = ira_allocate (sizeof (bool) * ira_allocnos_num);
in_inc_dec = ira_allocate (sizeof (bool) * cost_elements_num);
#endif /* FORBIDDEN_INC_DEC_CLASSES */
allocno_pref = NULL;
pref = NULL;
start = 0;
if (!resize_reg_info () && allocno_p && pseudo_classes_defined_p)
{
ira_allocno_t a;
ira_allocno_iterator ai;
pref = pref_buffer;
FOR_EACH_ALLOCNO (a, ai)
pref[ALLOCNO_NUM (a)] = reg_preferred_class (ALLOCNO_REGNO (a));
if (flag_expensive_optimizations)
start = 1;
}
if (allocno_p)
/* Clear the flag for the next compiled function. */
pseudo_classes_defined_p = false;
/* Normally we scan the insns once and determine the best class to
use for each allocno. However, if -fexpensive-optimizations are
on, we do so twice, the second time using the tentative best
classes to guide the selection. */
for (pass = 0; pass <= flag_expensive_optimizations; pass++)
for (pass = start; pass <= flag_expensive_optimizations; pass++)
{
if (internal_flag_ira_verbose > 0 && ira_dump_file)
fprintf (ira_dump_file, "\nPass %i for finding allocno costs\n\n",
pass);
if ((!allocno_p || internal_flag_ira_verbose > 0) && dump_file)
fprintf (dump_file,
"\nPass %i for finding pseudo/allocno costs\n\n", pass);
/* We could use only cover classes. Unfortunately it does not
work well for some targets where some subclass of cover class
is costly and wrong cover class is chosen. */
@ -1154,20 +1217,31 @@ find_allocno_class_costs (void)
= sizeof (struct costs) + sizeof (int) * (cost_classes_num - 1);
/* Zero out our accumulation of the cost of each class for each
allocno. */
memset (allocno_costs, 0, ira_allocnos_num * struct_costs_size);
memset (costs, 0, cost_elements_num * struct_costs_size);
#ifdef FORBIDDEN_INC_DEC_CLASSES
memset (in_inc_dec, 0, ira_allocnos_num * sizeof (bool));
memset (in_inc_dec, 0, cost_elements_num * sizeof (bool));
#endif
/* Scan the instructions and record each time it would save code
to put a certain allocno in a certain class. */
ira_traverse_loop_tree (true, ira_loop_tree_root,
process_bb_node_for_costs, NULL);
if (allocno_p)
{
/* Scan the instructions and record each time it would save code
to put a certain allocno in a certain class. */
ira_traverse_loop_tree (true, ira_loop_tree_root,
process_bb_node_for_costs, NULL);
memcpy (total_allocno_costs, costs,
max_struct_costs_size * ira_allocnos_num);
}
else
{
basic_block bb;
FOR_EACH_BB (bb)
process_bb_for_costs (bb);
}
memcpy (total_costs, allocno_costs,
max_struct_costs_size * ira_allocnos_num);
if (pass == 0)
allocno_pref = allocno_pref_buffer;
pref = pref_buffer;
/* Now for each allocno look at how desirable each class is and
find which class is preferred. */
@ -1182,41 +1256,52 @@ find_allocno_class_costs (void)
int inc_dec_p = false;
#endif
if (ira_regno_allocno_map[i] == NULL)
continue;
memset (temp_costs, 0, struct_costs_size);
/* Find cost of all allocnos with the same regno. */
for (a = ira_regno_allocno_map[i];
a != NULL;
a = ALLOCNO_NEXT_REGNO_ALLOCNO (a))
if (! allocno_p)
{
a_num = ALLOCNO_NUM (a);
if ((flag_ira_region == IRA_REGION_ALL
|| flag_ira_region == IRA_REGION_MIXED)
&& (parent = ALLOCNO_LOOP_TREE_NODE (a)->parent) != NULL
&& (parent_a = parent->regno_allocno_map[i]) != NULL
/* There are no caps yet. */
&& bitmap_bit_p (ALLOCNO_LOOP_TREE_NODE (a)->border_allocnos,
ALLOCNO_NUM (a)))
{
/* Propagate costs to upper levels in the region
tree. */
parent_a_num = ALLOCNO_NUM (parent_a);
for (k = 0; k < cost_classes_num; k++)
COSTS_OF_ALLOCNO (total_costs, parent_a_num)->cost[k]
+= COSTS_OF_ALLOCNO (total_costs, a_num)->cost[k];
COSTS_OF_ALLOCNO (total_costs, parent_a_num)->mem_cost
+= COSTS_OF_ALLOCNO (total_costs, a_num)->mem_cost;
}
for (k = 0; k < cost_classes_num; k++)
temp_costs->cost[k]
+= COSTS_OF_ALLOCNO (allocno_costs, a_num)->cost[k];
temp_costs->mem_cost
+= COSTS_OF_ALLOCNO (allocno_costs, a_num)->mem_cost;
if (regno_reg_rtx[i] == NULL_RTX)
continue;
#ifdef FORBIDDEN_INC_DEC_CLASSES
if (in_inc_dec[a_num])
inc_dec_p = true;
inc_dec_p = in_inc_dec[i];
#endif
memcpy (temp_costs, COSTS (costs, i), struct_costs_size);
}
else
{
if (ira_regno_allocno_map[i] == NULL)
continue;
memset (temp_costs, 0, struct_costs_size);
/* Find cost of all allocnos with the same regno. */
for (a = ira_regno_allocno_map[i];
a != NULL;
a = ALLOCNO_NEXT_REGNO_ALLOCNO (a))
{
a_num = ALLOCNO_NUM (a);
if ((flag_ira_region == IRA_REGION_ALL
|| flag_ira_region == IRA_REGION_MIXED)
&& (parent = ALLOCNO_LOOP_TREE_NODE (a)->parent) != NULL
&& (parent_a = parent->regno_allocno_map[i]) != NULL
/* There are no caps yet. */
&& bitmap_bit_p (ALLOCNO_LOOP_TREE_NODE
(a)->border_allocnos,
ALLOCNO_NUM (a)))
{
/* Propagate costs to upper levels in the region
tree. */
parent_a_num = ALLOCNO_NUM (parent_a);
for (k = 0; k < cost_classes_num; k++)
COSTS (total_allocno_costs, parent_a_num)->cost[k]
+= COSTS (total_allocno_costs, a_num)->cost[k];
COSTS (total_allocno_costs, parent_a_num)->mem_cost
+= COSTS (total_allocno_costs, a_num)->mem_cost;
}
for (k = 0; k < cost_classes_num; k++)
temp_costs->cost[k] += COSTS (costs, a_num)->cost[k];
temp_costs->mem_cost += COSTS (costs, a_num)->mem_cost;
#ifdef FORBIDDEN_INC_DEC_CLASSES
if (in_inc_dec[a_num])
inc_dec_p = true;
#endif
}
}
best_cost = (1 << (HOST_BITS_PER_INT - 2)) - 1;
best = ALL_REGS;
@ -1252,35 +1337,42 @@ find_allocno_class_costs (void)
alt_class = reg_class_subunion[alt_class][rclass];
}
alt_class = ira_class_translate[alt_class];
if (best_cost > temp_costs->mem_cost)
regno_cover_class[i] = NO_REGS;
else if (flag_ira_algorithm == IRA_ALGORITHM_PRIORITY)
/* Make the common class the biggest class of best and
alt_class. */
regno_cover_class[i] = alt_class == NO_REGS ? best : alt_class;
else
/* Make the common class a cover class. Remember all
allocnos with the same regno should have the same cover
class. */
regno_cover_class[i] = ira_class_translate[best];
if (pass == flag_expensive_optimizations)
{
if (best_cost > temp_costs->mem_cost)
best = alt_class = NO_REGS;
else if (best == alt_class)
alt_class = NO_REGS;
setup_reg_classes (i, best, alt_class);
if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL)
fprintf (ira_dump_file,
" r%d: preferred %s, alternative %s\n",
i, reg_class_names[best], reg_class_names[alt_class]);
setup_reg_classes (i, best, alt_class, regno_cover_class[i]);
if ((!allocno_p || internal_flag_ira_verbose > 2)
&& dump_file != NULL)
fprintf (dump_file,
" r%d: preferred %s, alternative %s, cover %s\n",
i, reg_class_names[best], reg_class_names[alt_class],
reg_class_names[regno_cover_class[i]]);
}
if (! allocno_p)
{
pref[i] = best_cost > temp_costs->mem_cost ? NO_REGS : best;
continue;
}
if (best_cost > temp_costs->mem_cost)
common_classes[i] = NO_REGS;
else if (flag_ira_algorithm == IRA_ALGORITHM_PRIORITY)
/* Make the common class the biggest class of best and
alt_class. */
common_classes[i] = alt_class == NO_REGS ? best : alt_class;
else
/* Make the common class a cover class. Remember all
allocnos with the same regno should have the same cover
class. */
common_classes[i] = ira_class_translate[best];
for (a = ira_regno_allocno_map[i];
a != NULL;
a = ALLOCNO_NEXT_REGNO_ALLOCNO (a))
{
a_num = ALLOCNO_NUM (a);
if (common_classes[i] == NO_REGS)
if (regno_cover_class[i] == NO_REGS)
best = NO_REGS;
else
{
@ -1292,7 +1384,7 @@ find_allocno_class_costs (void)
for (k = 0; k < cost_classes_num; k++)
{
rclass = cost_classes[k];
if (! ira_class_subset_p[rclass][common_classes[i]])
if (! ira_class_subset_p[rclass][regno_cover_class[i]])
continue;
/* Ignore classes that are too small for this
operand or invalid for an operand that was
@ -1307,50 +1399,50 @@ find_allocno_class_costs (void)
#endif
)
;
else if (COSTS_OF_ALLOCNO (total_costs, a_num)->cost[k]
else if (COSTS (total_allocno_costs, a_num)->cost[k]
< best_cost)
{
best_cost
= COSTS_OF_ALLOCNO (total_costs, a_num)->cost[k];
allocno_cost
= COSTS_OF_ALLOCNO (allocno_costs, a_num)->cost[k];
= COSTS (total_allocno_costs, a_num)->cost[k];
allocno_cost = COSTS (costs, a_num)->cost[k];
best = (enum reg_class) rclass;
}
else if (COSTS_OF_ALLOCNO (total_costs, a_num)->cost[k]
else if (COSTS (total_allocno_costs, a_num)->cost[k]
== best_cost)
{
best = ira_reg_class_union[best][rclass];
allocno_cost
= MAX (allocno_cost,
COSTS_OF_ALLOCNO (allocno_costs,
a_num)->cost[k]);
= MAX (allocno_cost, COSTS (costs, a_num)->cost[k]);
}
}
ALLOCNO_COVER_CLASS_COST (a) = allocno_cost;
}
ira_assert (flag_ira_algorithm == IRA_ALGORITHM_PRIORITY
|| ira_class_translate[best] == common_classes[i]);
if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL
&& (pass == 0 || allocno_pref[a_num] != best))
|| ira_class_translate[best] == regno_cover_class[i]);
if (internal_flag_ira_verbose > 2 && dump_file != NULL
&& (pass == 0 || pref[a_num] != best))
{
fprintf (ira_dump_file, " a%d (r%d,", a_num, i);
fprintf (dump_file, " a%d (r%d,", a_num, i);
if ((bb = ALLOCNO_LOOP_TREE_NODE (a)->bb) != NULL)
fprintf (ira_dump_file, "b%d", bb->index);
fprintf (dump_file, "b%d", bb->index);
else
fprintf (ira_dump_file, "l%d",
fprintf (dump_file, "l%d",
ALLOCNO_LOOP_TREE_NODE (a)->loop->num);
fprintf (ira_dump_file, ") best %s, cover %s\n",
fprintf (dump_file, ") best %s, cover %s\n",
reg_class_names[best],
reg_class_names[common_classes[i]]);
reg_class_names[regno_cover_class[i]]);
}
allocno_pref[a_num] = best;
pref[a_num] = best;
}
}
if (internal_flag_ira_verbose > 4 && ira_dump_file)
if (internal_flag_ira_verbose > 4 && dump_file)
{
print_costs (ira_dump_file);
fprintf (ira_dump_file,"\n");
if (allocno_p)
print_allocno_costs (dump_file);
else
print_pseudo_costs (dump_file);
fprintf (dump_file,"\n");
}
}
#ifdef FORBIDDEN_INC_DEC_CLASSES
@ -1443,23 +1535,22 @@ setup_allocno_cover_class_and_costs (void)
int *reg_costs;
enum reg_class cover_class, rclass;
enum machine_mode mode;
HARD_REG_SET *pref;
ira_allocno_t a;
ira_allocno_iterator ai;
ira_assert (allocno_p);
FOR_EACH_ALLOCNO (a, ai)
{
i = ALLOCNO_NUM (a);
mode = ALLOCNO_MODE (a);
cover_class = common_classes[ALLOCNO_REGNO (a)];
ira_assert (allocno_pref[i] == NO_REGS || cover_class != NO_REGS);
ALLOCNO_MEMORY_COST (a) = COSTS_OF_ALLOCNO (allocno_costs, i)->mem_cost;
cover_class = regno_cover_class[ALLOCNO_REGNO (a)];
ira_assert (pref[i] == NO_REGS || cover_class != NO_REGS);
ALLOCNO_MEMORY_COST (a) = COSTS (costs, i)->mem_cost;
ira_set_allocno_cover_class (a, cover_class);
if (cover_class == NO_REGS)
continue;
ALLOCNO_AVAILABLE_REGS_NUM (a) = ira_available_class_regs[cover_class];
pref = &reg_class_contents[allocno_pref[i]];
if (optimize && ALLOCNO_COVER_CLASS (a) != allocno_pref[i])
if (optimize && ALLOCNO_COVER_CLASS (a) != pref[i])
{
n = ira_class_hard_regs_num[cover_class];
ALLOCNO_HARD_REG_COSTS (a)
@ -1467,7 +1558,7 @@ setup_allocno_cover_class_and_costs (void)
for (j = n - 1; j >= 0; j--)
{
regno = ira_class_hard_regs[cover_class][j];
if (TEST_HARD_REG_BIT (*pref, regno))
if (TEST_HARD_REG_BIT (reg_class_contents[pref[i]], regno))
reg_costs[j] = ALLOCNO_COVER_CLASS_COST (a);
else
{
@ -1482,7 +1573,7 @@ setup_allocno_cover_class_and_costs (void)
== cover_class);
num = cost_class_nums[cover_class];
}
reg_costs[j] = COSTS_OF_ALLOCNO (allocno_costs, i)->cost[num];
reg_costs[j] = COSTS (costs, i)->cost[num];
}
}
}
@ -1569,27 +1660,58 @@ ira_finish_costs_once (void)
/* Common initialization function for ira_costs and
ira_set_pseudo_classes. */
static void
init_costs (void)
{
costs = (struct costs *) ira_allocate (max_struct_costs_size
* cost_elements_num);
pref_buffer
= (enum reg_class *) ira_allocate (sizeof (enum reg_class)
* cost_elements_num);
regno_cover_class
= (enum reg_class *) ira_allocate (sizeof (enum reg_class)
* max_reg_num ());
}
/* Common finalization function for ira_costs and
ira_set_pseudo_classes. */
static void
finish_costs (void)
{
ira_free (regno_cover_class);
ira_free (pref_buffer);
ira_free (costs);
}
/* Entry function which defines cover class, memory and hard register
costs for each allocno. */
void
ira_costs (void)
{
allocno_costs = (struct costs *) ira_allocate (max_struct_costs_size
* ira_allocnos_num);
total_costs = (struct costs *) ira_allocate (max_struct_costs_size
* ira_allocnos_num);
allocno_pref_buffer
= (enum reg_class *) ira_allocate (sizeof (enum reg_class)
* ira_allocnos_num);
common_classes
= (enum reg_class *) ira_allocate (sizeof (enum reg_class)
* max_reg_num ());
find_allocno_class_costs ();
allocno_p = true;
cost_elements_num = ira_allocnos_num;
init_costs ();
total_allocno_costs = (struct costs *) ira_allocate (max_struct_costs_size
* ira_allocnos_num);
find_costs_and_classes (ira_dump_file);
setup_allocno_cover_class_and_costs ();
ira_free (common_classes);
ira_free (allocno_pref_buffer);
ira_free (total_costs);
ira_free (allocno_costs);
finish_costs ();
ira_free (total_allocno_costs);
}
/* Entry function which defines classes for pseudos. */
void
ira_set_pseudo_classes (FILE *dump_file)
{
allocno_p = false;
internal_flag_ira_verbose = flag_ira_verbose;
cost_elements_num = max_reg_num ();
init_costs ();
find_costs_and_classes (dump_file);
pseudo_classes_defined_p = true;
finish_costs ();
}

50
gcc/ira-int.h
View File

@ -565,18 +565,7 @@ extern int ira_reg_cost, ira_mem_cost;
extern int ira_load_cost, ira_store_cost, ira_shuffle_cost;
extern int ira_move_loops_num, ira_additional_jumps_num;
/* Map: hard register number -> cover class it belongs to. If the
corresponding class is NO_REGS, the hard register is not available
for allocation. */
extern enum reg_class ira_hard_regno_cover_class[FIRST_PSEUDO_REGISTER];
/* Map: register class x machine mode -> number of hard registers of
given class needed to store value of given mode. If the number for
some hard-registers of the register class is different, the size
will be negative. */
extern int ira_reg_class_nregs[N_REG_CLASSES][MAX_MACHINE_MODE];
/* Maximal value of the previous array elements. */
/* Maximal value of element of array ira_reg_class_nregs. */
extern int ira_max_nregs;
/* The number of bits in each element of array used to implement a bit
@ -730,10 +719,9 @@ ira_allocno_set_iter_next (ira_allocno_set_iterator *i)
extern HARD_REG_SET ira_reg_mode_hard_regset
[FIRST_PSEUDO_REGISTER][NUM_MACHINE_MODES];
/* Arrays analogous to macros MEMORY_MOVE_COST and REGISTER_MOVE_COST.
Don't use ira_register_move_cost directly. Use function of
/* Array analogous to macro REGISTER_MOVE_COST. Don't use
ira_register_move_cost directly. Use function of
ira_get_may_move_cost instead. */
extern short ira_memory_move_cost[MAX_MACHINE_MODE][N_REG_CLASSES][2];
extern move_table *ira_register_move_cost[MAX_MACHINE_MODE];
/* Similar to may_move_in_cost but it is calculated in IRA instead of
@ -755,29 +743,12 @@ extern move_table *ira_may_move_out_cost[MAX_MACHINE_MODE];
allocation. */
extern int ira_class_subset_p[N_REG_CLASSES][N_REG_CLASSES];
/* Array of number of hard registers of given class which are
available for the allocation. The order is defined by the
allocation order. */
extern short ira_class_hard_regs[N_REG_CLASSES][FIRST_PSEUDO_REGISTER];
/* The number of elements of the above array for given register
class. */
extern int ira_class_hard_regs_num[N_REG_CLASSES];
/* Index (in ira_class_hard_regs) for given register class and hard
register (in general case a hard register can belong to several
register classes). The index is negative for hard registers
unavailable for the allocation. */
extern short ira_class_hard_reg_index[N_REG_CLASSES][FIRST_PSEUDO_REGISTER];
/* Function specific hard registers can not be used for the register
allocation. */
extern HARD_REG_SET ira_no_alloc_regs;
/* Number of given class hard registers available for the register
allocation for given classes. */
extern int ira_available_class_regs[N_REG_CLASSES];
/* Array whose values are hard regset of hard registers available for
the allocation of given register class whose HARD_REGNO_MODE_OK
values for given mode are zero. */
@ -789,16 +760,6 @@ extern HARD_REG_SET prohibited_class_mode_regs
prohibited. */
extern HARD_REG_SET ira_prohibited_mode_move_regs[NUM_MACHINE_MODES];
/* Number of cover classes. Cover classes is non-intersected register
classes containing all hard-registers available for the
allocation. */
extern int ira_reg_class_cover_size;
/* The array containing cover classes (see also comments for macro
IRA_COVER_CLASSES). Only first IRA_REG_CLASS_COVER_SIZE elements are
used for this. */
extern enum reg_class ira_reg_class_cover[N_REG_CLASSES];
/* The value is number of elements in the subsequent array. */
extern int ira_important_classes_num;
@ -812,11 +773,6 @@ extern enum reg_class ira_important_classes[N_REG_CLASSES];
classes. */
extern int ira_important_class_nums[N_REG_CLASSES];
/* Map of all register classes to corresponding cover class containing
the given class. If given class is not a subset of a cover class,
we translate it into the cheapest cover class. */
extern enum reg_class ira_class_translate[N_REG_CLASSES];
/* The biggest important class inside of intersection of the two
classes (that is calculated taking only hard registers available
for allocation into account). If the both classes contain no hard

65
gcc/ira-lives.c
View File

@ -702,7 +702,8 @@ single_reg_class (const char *constraints, rtx op, rtx equiv_const)
? GENERAL_REGS
: REG_CLASS_FROM_CONSTRAINT (c, constraints));
if ((cl != NO_REGS && next_cl != cl)
|| ira_available_class_regs[next_cl] > 1)
|| (ira_available_class_regs[next_cl]
> ira_reg_class_nregs[next_cl][GET_MODE (op)]))
return NO_REGS;
cl = next_cl;
break;
@ -712,8 +713,10 @@ single_reg_class (const char *constraints, rtx op, rtx equiv_const)
next_cl
= single_reg_class (recog_data.constraints[c - '0'],
recog_data.operand[c - '0'], NULL_RTX);
if ((cl != NO_REGS && next_cl != cl) || next_cl == NO_REGS
|| ira_available_class_regs[next_cl] > 1)
if ((cl != NO_REGS && next_cl != cl)
|| next_cl == NO_REGS
|| (ira_available_class_regs[next_cl]
> ira_reg_class_nregs[next_cl][GET_MODE (op)]))
return NO_REGS;
cl = next_cl;
break;
@ -736,6 +739,62 @@ single_reg_operand_class (int op_num)
recog_data.operand[op_num], NULL_RTX);
}
/* The function sets up hard register set *SET to hard registers which
might be used by insn reloads because the constraints are too
strict. */
void
ira_implicitly_set_insn_hard_regs (HARD_REG_SET *set)
{
int i, c, regno;
bool ignore_p;
enum reg_class cl;
rtx op;
enum machine_mode mode;
CLEAR_HARD_REG_SET (*set);
for (i = 0; i < recog_data.n_operands; i++)
{
op = recog_data.operand[i];
if (GET_CODE (op) == SUBREG)
op = SUBREG_REG (op);
if (GET_CODE (op) == SCRATCH
|| (REG_P (op) && (regno = REGNO (op)) >= FIRST_PSEUDO_REGISTER))
{
const char *p = recog_data.constraints[i];
mode = (GET_CODE (op) == SCRATCH
? GET_MODE (op) : PSEUDO_REGNO_MODE (regno));
cl = NO_REGS;
for (ignore_p = false; (c = *p); p += CONSTRAINT_LEN (c, p))
if (c == '#')
ignore_p = true;
else if (c == ',')
ignore_p = false;
else if (! ignore_p)
switch (c)
{
case 'r':
case 'a': case 'b': case 'c': case 'd': case 'e': case 'f':
case 'h': case 'j': case 'k': case 'l':
case 'q': case 't': case 'u':
case 'v': case 'w': case 'x': case 'y': case 'z':
case 'A': case 'B': case 'C': case 'D':
case 'Q': case 'R': case 'S': case 'T': case 'U':
case 'W': case 'Y': case 'Z':
cl = (c == 'r'
? GENERAL_REGS
: REG_CLASS_FROM_CONSTRAINT (c, p));
if (cl != NO_REGS
&& (ira_available_class_regs[cl]
<= ira_reg_class_nregs[cl][mode]))
IOR_HARD_REG_SET (*set, reg_class_contents[cl]);
break;
}
}
}
}
/* Processes input operands, if IN_P, or output operands otherwise of
the current insn with FREQ to find allocno which can use only one
hard register and makes other currently living allocnos conflicting

16
gcc/ira.c
View File

@ -1422,8 +1422,8 @@ compute_regs_asm_clobbered (char *regs_asm_clobbered)
/* Set up ELIMINABLE_REGSET, IRA_NO_ALLOC_REGS, and REGS_EVER_LIVE. */
static void
setup_eliminable_regset (void)
void
ira_setup_eliminable_regset (void)
{
/* Like regs_ever_live, but 1 if a reg is set or clobbered from an
asm. Unlike regs_ever_live, elements of this array corresponding
@ -1827,7 +1827,8 @@ setup_preferred_alternate_classes_for_new_pseudos (int start)
old_regno = ORIGINAL_REGNO (regno_reg_rtx[i]);
ira_assert (i != old_regno);
setup_reg_classes (i, reg_preferred_class (old_regno),
reg_alternate_class (old_regno));
reg_alternate_class (old_regno),
reg_cover_class (old_regno));
if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL)
fprintf (ira_dump_file,
" New r%d: setting preferred %s, alternative %s\n",
@ -1848,10 +1849,7 @@ expand_reg_info (int old_size)
resize_reg_info ();
for (i = old_size; i < size; i++)
{
reg_renumber[i] = -1;
setup_reg_classes (i, GENERAL_REGS, ALL_REGS);
}
setup_reg_classes (i, GENERAL_REGS, ALL_REGS, GENERAL_REGS);
}
/* Return TRUE if there is too high register pressure in the function.
@ -3160,8 +3158,8 @@ ira (FILE *f)
}
max_regno_before_ira = allocated_reg_info_size = max_reg_num ();
allocate_reg_info ();
setup_eliminable_regset ();
resize_reg_info ();
ira_setup_eliminable_regset ();
ira_overall_cost = ira_reg_cost = ira_mem_cost = 0;
ira_load_cost = ira_store_cost = ira_shuffle_cost = 0;

49
gcc/ira.h
View File

@ -20,14 +20,63 @@ You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
/* Number of given class hard registers available for the register
allocation for given classes. */
extern int ira_available_class_regs[N_REG_CLASSES];
/* Map: hard register number -> cover class it belongs to. If the
corresponding class is NO_REGS, the hard register is not available
for allocation. */
extern enum reg_class ira_hard_regno_cover_class[FIRST_PSEUDO_REGISTER];
/* Number of cover classes. Cover classes is non-intersected register
classes containing all hard-registers available for the
allocation. */
extern int ira_reg_class_cover_size;
/* The array containing cover classes (see also comments for macro
IRA_COVER_CLASSES). Only first IRA_REG_CLASS_COVER_SIZE elements are
used for this. */
extern enum reg_class ira_reg_class_cover[N_REG_CLASSES];
/* Map of all register classes to corresponding cover class containing
the given class. If given class is not a subset of a cover class,
we translate it into the cheapest cover class. */
extern enum reg_class ira_class_translate[N_REG_CLASSES];
/* Map: register class x machine mode -> number of hard registers of
given class needed to store value of given mode. If the number for
some hard-registers of the register class is different, the size
will be negative. */
extern int ira_reg_class_nregs[N_REG_CLASSES][MAX_MACHINE_MODE];
/* Function specific hard registers can not be used for the register
allocation. */
extern HARD_REG_SET ira_no_alloc_regs;
/* True if we have allocno conflicts. It is false for non-optimized
mode or when the conflict table is too big. */
extern bool ira_conflicts_p;
/* Array analogous to macro MEMORY_MOVE_COST. */
extern short ira_memory_move_cost[MAX_MACHINE_MODE][N_REG_CLASSES][2];
/* Array of number of hard registers of given class which are
available for the allocation. The order is defined by the
allocation order. */
extern short ira_class_hard_regs[N_REG_CLASSES][FIRST_PSEUDO_REGISTER];
/* The number of elements of the above array for given register
class. */
extern int ira_class_hard_regs_num[N_REG_CLASSES];
extern void ira_init_once (void);
extern void ira_init (void);
extern void ira_finish_once (void);
extern void ira_setup_eliminable_regset (void);
extern rtx ira_eliminate_regs (rtx, enum machine_mode);
extern void ira_set_pseudo_classes (FILE *);
extern void ira_implicitly_set_insn_hard_regs (HARD_REG_SET *);
extern void ira_sort_regnos_for_alter_reg (int *, int, unsigned int *);
extern void ira_mark_allocation_change (int);

2
gcc/passes.c
View File

@ -776,8 +776,8 @@ init_optimization_passes (void)
NEXT_PASS (pass_mode_switching);
NEXT_PASS (pass_match_asm_constraints);
NEXT_PASS (pass_sms);
NEXT_PASS (pass_sched);
NEXT_PASS (pass_subregs_of_mode_init);
NEXT_PASS (pass_sched);
NEXT_PASS (pass_ira);
NEXT_PASS (pass_subregs_of_mode_finish);
NEXT_PASS (pass_postreload);

114
gcc/reginfo.c
View File

@ -898,6 +898,10 @@ struct reg_pref
but since it is recommended that there be a class corresponding to the
union of most major pair of classes, that generality is not required. */
char altclass;
/* coverclass is a register class that IRA uses for allocating
the pseudo. */
char coverclass;
};
/* Record preferences of each pseudo. This is available after RA is
@ -925,65 +929,51 @@ reg_alternate_class (int regno)
return (enum reg_class) reg_pref[regno].altclass;
}
/* Initialize some global data for this pass. */
static unsigned int
reginfo_init (void)
/* Return the reg_class which is used by IRA for its allocation. */
enum reg_class
reg_cover_class (int regno)
{
if (df)
df_compute_regs_ever_live (true);
if (reg_pref == 0)
return NO_REGS;
/* This prevents dump_flow_info from losing if called
before reginfo is run. */
reg_pref = NULL;
/* No more global register variables may be declared. */
no_global_reg_vars = 1;
return 1;
return (enum reg_class) reg_pref[regno].coverclass;
}
struct rtl_opt_pass pass_reginfo_init =
{
{
RTL_PASS,
"reginfo", /* name */
NULL, /* gate */
reginfo_init, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_NONE, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
0, /* todo_flags_start */
0 /* todo_flags_finish */
}
};
/* Current size of reg_info. */
static int reg_info_size;
/* Allocate space for reg info. */
void
static void
allocate_reg_info (void)
{
int size = max_reg_num ();
reg_info_size = max_reg_num ();
gcc_assert (! reg_pref && ! reg_renumber);
reg_renumber = XNEWVEC (short, size);
reg_pref = XCNEWVEC (struct reg_pref, size);
memset (reg_renumber, -1, size * sizeof (short));
reg_renumber = XNEWVEC (short, reg_info_size);
reg_pref = XCNEWVEC (struct reg_pref, reg_info_size);
memset (reg_renumber, -1, reg_info_size * sizeof (short));
}
/* Resize reg info. The new elements will be uninitialized. */
void
bool
resize_reg_info (void)
{
int size = max_reg_num ();
int old;
gcc_assert (reg_pref != NULL);
if (reg_info_size == max_reg_num ())
return false;
old = reg_info_size;
reg_info_size = max_reg_num ();
gcc_assert (reg_pref && reg_renumber);
reg_renumber = XRESIZEVEC (short, reg_renumber, size);
reg_pref = XRESIZEVEC (struct reg_pref, reg_pref, size);
reg_renumber = XRESIZEVEC (short, reg_renumber, reg_info_size);
reg_pref = XRESIZEVEC (struct reg_pref, reg_pref, reg_info_size);
memset (reg_pref + old, -1,
(reg_info_size - old) * sizeof (struct reg_pref));
memset (reg_renumber + old, -1, (reg_info_size - old) * sizeof (short));
return true;
}
@ -1004,19 +994,55 @@ free_reg_info (void)
}
}
/* Initialize some global data for this pass. */
static unsigned int
reginfo_init (void)
{
if (df)
df_compute_regs_ever_live (true);
/* This prevents dump_flow_info from losing if called
before reginfo is run. */
reg_pref = NULL;
allocate_reg_info ();
/* No more global register variables may be declared. */
no_global_reg_vars = 1;
return 1;
}
struct rtl_opt_pass pass_reginfo_init =
{
{
RTL_PASS,
"reginfo", /* name */
NULL, /* gate */
reginfo_init, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_NONE, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
0, /* todo_flags_start */
0 /* todo_flags_finish */
}
};
/* Set up preferred and alternate classes for REGNO as PREFCLASS and
ALTCLASS. */
/* Set up preferred, alternate, and cover classes for REGNO as
PREFCLASS, ALTCLASS, and COVERCLASS. */
void
setup_reg_classes (int regno,
enum reg_class prefclass, enum reg_class altclass)
enum reg_class prefclass, enum reg_class altclass,
enum reg_class coverclass)
{
if (reg_pref == NULL)
return;
reg_pref[regno].prefclass = prefclass;
reg_pref[regno].altclass = altclass;
reg_pref[regno].coverclass = coverclass;
}

55
gcc/reload.c
View File

@ -112,6 +112,7 @@ a register with any other reload. */
#include "params.h"
#include "target.h"
#include "df.h"
#include "ira.h"
/* True if X is a constant that can be forced into the constant pool. */
#define CONST_POOL_OK_P(X) \
@ -2589,6 +2590,7 @@ find_reloads (rtx insn, int replace, int ind_levels, int live_known,
char goal_alternative_earlyclobber[MAX_RECOG_OPERANDS];
int goal_alternative_swapped;
int best;
int best_small_class_operands_num;
int commutative;
char operands_match[MAX_RECOG_OPERANDS][MAX_RECOG_OPERANDS];
rtx substed_operand[MAX_RECOG_OPERANDS];
@ -2914,6 +2916,7 @@ find_reloads (rtx insn, int replace, int ind_levels, int live_known,
all the operands together against the register constraints. */
best = MAX_RECOG_OPERANDS * 2 + 600;
best_small_class_operands_num = 0;
swapped = 0;
goal_alternative_swapped = 0;
@ -3697,22 +3700,48 @@ find_reloads (rtx insn, int replace, int ind_levels, int live_known,
/* If this alternative can be made to work by reloading,
and it needs less reloading than the others checked so far,
record it as the chosen goal for reloading. */
if (! bad && best > losers)
if (! bad)
{
for (i = 0; i < noperands; i++)
bool change_p = false;
int small_class_operands_num = 0;
if (best >= losers)
{
goal_alternative[i] = this_alternative[i];
goal_alternative_win[i] = this_alternative_win[i];
goal_alternative_match_win[i] = this_alternative_match_win[i];
goal_alternative_offmemok[i] = this_alternative_offmemok[i];
goal_alternative_matches[i] = this_alternative_matches[i];
goal_alternative_earlyclobber[i]
= this_alternative_earlyclobber[i];
for (i = 0; i < noperands; i++)
small_class_operands_num
+= SMALL_REGISTER_CLASS_P (this_alternative[i]) ? 1 : 0;
if (best > losers
|| (best == losers
/* If the cost of the reloads is the same,
prefer alternative which requires minimal
number of small register classes for the
operands. This improves chances of reloads
for insn requiring small register
classes. */
&& (small_class_operands_num
< best_small_class_operands_num)))
change_p = true;
}
if (change_p)
{
for (i = 0; i < noperands; i++)
{
goal_alternative[i] = this_alternative[i];
goal_alternative_win[i] = this_alternative_win[i];
goal_alternative_match_win[i]
= this_alternative_match_win[i];
goal_alternative_offmemok[i]
= this_alternative_offmemok[i];
goal_alternative_matches[i] = this_alternative_matches[i];
goal_alternative_earlyclobber[i]
= this_alternative_earlyclobber[i];
}
goal_alternative_swapped = swapped;
best = losers;
best_small_class_operands_num = small_class_operands_num;
goal_alternative_number = this_alternative_number;
goal_earlyclobber = this_earlyclobber;
}
goal_alternative_swapped = swapped;
best = losers;
goal_alternative_number = this_alternative_number;
goal_earlyclobber = this_earlyclobber;
}
}

8
gcc/rtl.h
View File

@ -1898,10 +1898,8 @@ extern rtx remove_free_EXPR_LIST_node (rtx *);
/* Initialize may_move_cost and friends for mode M. */
extern void init_move_cost (enum machine_mode);
/* Allocate register info memory. */
extern void allocate_reg_info (void);
/* Resize reg info. */
extern void resize_reg_info (void);
extern bool resize_reg_info (void);
/* Free up register info memory. */
extern void free_reg_info (void);
@ -1912,7 +1910,9 @@ extern const char *decode_asm_operands (rtx, rtx *, rtx **, const char **,
extern enum reg_class reg_preferred_class (int);
extern enum reg_class reg_alternate_class (int);
extern void setup_reg_classes (int, enum reg_class, enum reg_class);
extern enum reg_class reg_cover_class (int);
extern void setup_reg_classes (int, enum reg_class, enum reg_class,
enum reg_class);
extern void split_all_insns (void);
extern unsigned int split_all_insns_noflow (void);

659
gcc/sched-deps.c
View File

@ -41,6 +41,7 @@ along with GCC; see the file COPYING3. If not see
#include "sched-int.h"
#include "params.h"
#include "cselib.h"
#include "ira.h"
#ifdef INSN_SCHEDULING
@ -396,6 +397,15 @@ static regset reg_pending_clobbers;
static regset reg_pending_uses;
static enum reg_pending_barrier_mode reg_pending_barrier;
/* Hard registers implicitly clobbered or used (or may be implicitly
clobbered or used) by the currently analyzed insn. For example,
insn in its constraint has one register class. Even if there is
currently no hard register in the insn, the particular hard
register will be in the insn after reload pass because the
constraint requires it. */
static HARD_REG_SET implicit_reg_pending_clobbers;
static HARD_REG_SET implicit_reg_pending_uses;
/* To speed up the test for duplicate dependency links we keep a
record of dependencies created by add_dependence when the average
number of instructions in a basic block is very large.
@ -417,8 +427,8 @@ static int cache_size;
static int deps_may_trap_p (const_rtx);
static void add_dependence_list (rtx, rtx, int, enum reg_note);
static void add_dependence_list_and_free (struct deps *, rtx,
rtx *, int, enum reg_note);
static void add_dependence_list_and_free (struct deps *, rtx,
rtx *, int, enum reg_note);
static void delete_all_dependences (rtx);
static void fixup_sched_groups (rtx);
@ -1367,7 +1377,7 @@ add_dependence_list (rtx insn, rtx list, int uncond, enum reg_note dep_type)
is not readonly. */
static void
add_dependence_list_and_free (struct deps *deps, rtx insn, rtx *listp,
add_dependence_list_and_free (struct deps *deps, rtx insn, rtx *listp,
int uncond, enum reg_note dep_type)
{
rtx list, next;
@ -1625,7 +1635,7 @@ haifa_note_mem_dep (rtx mem, rtx pending_mem, rtx pending_insn, ds_t ds)
{
dep_def _dep, *dep = &_dep;
init_dep_1 (dep, pending_insn, cur_insn, ds_to_dt (ds),
init_dep_1 (dep, pending_insn, cur_insn, ds_to_dt (ds),
current_sched_info->flags & USE_DEPS_LIST ? ds : -1);
maybe_add_or_update_dep_1 (dep, false, pending_mem, mem);
}
@ -1691,6 +1701,327 @@ ds_to_dt (ds_t ds)
return REG_DEP_ANTI;
}
}
/* Functions for computation of info needed for register pressure
sensitive insn scheduling. */
/* Allocate and return reg_use_data structure for REGNO and INSN. */
static struct reg_use_data *
create_insn_reg_use (int regno, rtx insn)
{
struct reg_use_data *use;
use = (struct reg_use_data *) xmalloc (sizeof (struct reg_use_data));
use->regno = regno;
use->insn = insn;
use->next_insn_use = INSN_REG_USE_LIST (insn);
INSN_REG_USE_LIST (insn) = use;
return use;
}
/* Allocate and return reg_set_data structure for REGNO and INSN. */
static struct reg_set_data *
create_insn_reg_set (int regno, rtx insn)
{
struct reg_set_data *set;
set = (struct reg_set_data *) xmalloc (sizeof (struct reg_set_data));
set->regno = regno;
set->insn = insn;
set->next_insn_set = INSN_REG_SET_LIST (insn);
INSN_REG_SET_LIST (insn) = set;
return set;
}
/* Set up insn register uses for INSN and dependency context DEPS. */
static void
setup_insn_reg_uses (struct deps *deps, rtx insn)
{
unsigned i;
reg_set_iterator rsi;
rtx list;
struct reg_use_data *use, *use2, *next;
struct deps_reg *reg_last;
EXECUTE_IF_SET_IN_REG_SET (reg_pending_uses, 0, i, rsi)
{
if (i < FIRST_PSEUDO_REGISTER
&& TEST_HARD_REG_BIT (ira_no_alloc_regs, i))
continue;
if (find_regno_note (insn, REG_DEAD, i) == NULL_RTX
&& ! REGNO_REG_SET_P (reg_pending_sets, i)
&& ! REGNO_REG_SET_P (reg_pending_clobbers, i))
/* Ignore use which is not dying. */
continue;
use = create_insn_reg_use (i, insn);
use->next_regno_use = use;
reg_last = &deps->reg_last[i];
/* Create the cycle list of uses. */
for (list = reg_last->uses; list; list = XEXP (list, 1))
{
use2 = create_insn_reg_use (i, XEXP (list, 0));
next = use->next_regno_use;
use->next_regno_use = use2;
use2->next_regno_use = next;
}
}
}
/* Register pressure info for the currently processed insn. */
static struct reg_pressure_data reg_pressure_info[N_REG_CLASSES];
/* Return TRUE if INSN has the use structure for REGNO. */
static bool
insn_use_p (rtx insn, int regno)
{
struct reg_use_data *use;
for (use = INSN_REG_USE_LIST (insn); use != NULL; use = use->next_insn_use)
if (use->regno == regno)
return true;
return false;
}
/* Update the register pressure info after birth of pseudo register REGNO
in INSN. Arguments CLOBBER_P and UNUSED_P say correspondingly that
the register is in clobber or unused after the insn. */
static void
mark_insn_pseudo_birth (rtx insn, int regno, bool clobber_p, bool unused_p)
{
int incr, new_incr;
enum reg_class cl;
gcc_assert (regno >= FIRST_PSEUDO_REGISTER);
cl = sched_regno_cover_class[regno];
if (cl != NO_REGS)
{
incr = ira_reg_class_nregs[cl][PSEUDO_REGNO_MODE (regno)];
if (clobber_p)
{
new_incr = reg_pressure_info[cl].clobber_increase + incr;
reg_pressure_info[cl].clobber_increase = new_incr;
}
else if (unused_p)
{
new_incr = reg_pressure_info[cl].unused_set_increase + incr;
reg_pressure_info[cl].unused_set_increase = new_incr;
}
else
{
new_incr = reg_pressure_info[cl].set_increase + incr;
reg_pressure_info[cl].set_increase = new_incr;
if (! insn_use_p (insn, regno))
reg_pressure_info[cl].change += incr;
create_insn_reg_set (regno, insn);
}
gcc_assert (new_incr < (1 << INCREASE_BITS));
}
}
/* Like mark_insn_pseudo_regno_birth except that NREGS saying how many
hard registers involved in the birth. */
static void
mark_insn_hard_regno_birth (rtx insn, int regno, int nregs,
bool clobber_p, bool unused_p)
{
enum reg_class cl;
int new_incr, last = regno + nregs;
while (regno < last)
{
gcc_assert (regno < FIRST_PSEUDO_REGISTER);
if (! TEST_HARD_REG_BIT (ira_no_alloc_regs, regno))
{
cl = sched_regno_cover_class[regno];
if (cl != NO_REGS)
{
if (clobber_p)
{
new_incr = reg_pressure_info[cl].clobber_increase + 1;
reg_pressure_info[cl].clobber_increase = new_incr;
}
else if (unused_p)
{
new_incr = reg_pressure_info[cl].unused_set_increase + 1;
reg_pressure_info[cl].unused_set_increase = new_incr;
}
else
{
new_incr = reg_pressure_info[cl].set_increase + 1;
reg_pressure_info[cl].set_increase = new_incr;
if (! insn_use_p (insn, regno))
reg_pressure_info[cl].change += 1;
create_insn_reg_set (regno, insn);
}
gcc_assert (new_incr < (1 << INCREASE_BITS));
}
}
regno++;
}
}
/* Update the register pressure info after birth of pseudo or hard
register REG in INSN. Arguments CLOBBER_P and UNUSED_P say
correspondingly that the register is in clobber or unused after the
insn. */
static void
mark_insn_reg_birth (rtx insn, rtx reg, bool clobber_p, bool unused_p)
{
int regno;
if (GET_CODE (reg) == SUBREG)
reg = SUBREG_REG (reg);
if (! REG_P (reg))
return;
regno = REGNO (reg);
if (regno < FIRST_PSEUDO_REGISTER)
mark_insn_hard_regno_birth (insn, regno,
hard_regno_nregs[regno][GET_MODE (reg)],
clobber_p, unused_p);
else
mark_insn_pseudo_birth (insn, regno, clobber_p, unused_p);
}
/* Update the register pressure info after death of pseudo register
REGNO. */
static void
mark_pseudo_death (int regno)
{
int incr;
enum reg_class cl;
gcc_assert (regno >= FIRST_PSEUDO_REGISTER);
cl = sched_regno_cover_class[regno];
if (cl != NO_REGS)
{
incr = ira_reg_class_nregs[cl][PSEUDO_REGNO_MODE (regno)];
reg_pressure_info[cl].change -= incr;
}
}
/* Like mark_pseudo_death except that NREGS saying how many hard
registers involved in the death. */
static void
mark_hard_regno_death (int regno, int nregs)
{
enum reg_class cl;
int last = regno + nregs;
while (regno < last)
{
gcc_assert (regno < FIRST_PSEUDO_REGISTER);
if (! TEST_HARD_REG_BIT (ira_no_alloc_regs, regno))
{
cl = sched_regno_cover_class[regno];
if (cl != NO_REGS)
reg_pressure_info[cl].change -= 1;
}
regno++;
}
}
/* Update the register pressure info after death of pseudo or hard
register REG. */
static void
mark_reg_death (rtx reg)
{
int regno;
if (GET_CODE (reg) == SUBREG)
reg = SUBREG_REG (reg);
if (! REG_P (reg))
return;
regno = REGNO (reg);
if (regno < FIRST_PSEUDO_REGISTER)
mark_hard_regno_death (regno, hard_regno_nregs[regno][GET_MODE (reg)]);
else
mark_pseudo_death (regno);
}
/* Process SETTER of REG. DATA is an insn containing the setter. */
static void
mark_insn_reg_store (rtx reg, const_rtx setter, void *data)
{
if (setter != NULL_RTX && GET_CODE (setter) != SET)
return;
mark_insn_reg_birth
((rtx) data, reg, false,
find_reg_note ((const_rtx) data, REG_UNUSED, reg) != NULL_RTX);
}
/* Like mark_insn_reg_store except notice just CLOBBERs; ignore SETs. */
static void
mark_insn_reg_clobber (rtx reg, const_rtx setter, void *data)
{
if (GET_CODE (setter) == CLOBBER)
mark_insn_reg_birth ((rtx) data, reg, true, false);
}
/* Set up reg pressure info related to INSN. */
static void
setup_insn_reg_pressure_info (rtx insn)
{
int i, len;
enum reg_class cl;
static struct reg_pressure_data *pressure_info;
rtx link;
gcc_assert (sched_pressure_p);
if (! INSN_P (insn))
return;
for (i = 0; i < ira_reg_class_cover_size; i++)
{
cl = ira_reg_class_cover[i];
reg_pressure_info[cl].clobber_increase = 0;
reg_pressure_info[cl].set_increase = 0;
reg_pressure_info[cl].unused_set_increase = 0;
reg_pressure_info[cl].change = 0;
}
note_stores (PATTERN (insn), mark_insn_reg_clobber, insn);
note_stores (PATTERN (insn), mark_insn_reg_store, insn);
#ifdef AUTO_INC_DEC
for (link = REG_NOTES (insn); link; link = XEXP (link, 1))
if (REG_NOTE_KIND (link) == REG_INC)
mark_insn_reg_store (XEXP (link, 0), NULL_RTX, insn);
#endif
for (link = REG_NOTES (insn); link; link = XEXP (link, 1))
if (REG_NOTE_KIND (link) == REG_DEAD)
mark_reg_death (XEXP (link, 0));
len = sizeof (struct reg_pressure_data) * ira_reg_class_cover_size;
pressure_info
= INSN_REG_PRESSURE (insn) = (struct reg_pressure_data *) xmalloc (len);
INSN_MAX_REG_PRESSURE (insn) = (int *) xmalloc (ira_reg_class_cover_size
* sizeof (int));
for (i = 0; i < ira_reg_class_cover_size; i++)
{
cl = ira_reg_class_cover[i];
pressure_info[i].clobber_increase
= reg_pressure_info[cl].clobber_increase;
pressure_info[i].set_increase = reg_pressure_info[cl].set_increase;
pressure_info[i].unused_set_increase
= reg_pressure_info[cl].unused_set_increase;
pressure_info[i].change = reg_pressure_info[cl].change;
}
}
/* Internal variable for sched_analyze_[12] () functions.
@ -1905,10 +2236,16 @@ sched_analyze_1 (struct deps *deps, rtx x, rtx insn)
/* Treat all writes to a stack register as modifying the TOS. */
if (regno >= FIRST_STACK_REG && regno <= LAST_STACK_REG)
{
int nregs;
/* Avoid analyzing the same register twice. */
if (regno != FIRST_STACK_REG)
sched_analyze_reg (deps, FIRST_STACK_REG, mode, code, insn);
sched_analyze_reg (deps, FIRST_STACK_REG, mode, USE, insn);
nregs = hard_regno_nregs[FIRST_STACK_REG][mode];
while (--nregs >= 0)
SET_HARD_REG_BIT (implicit_reg_pending_uses,
FIRST_STACK_REG + nregs);
}
#endif
}
@ -2243,6 +2580,16 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
unsigned i;
reg_set_iterator rsi;
if (! reload_completed)
{
HARD_REG_SET temp;
extract_insn (insn);
preprocess_constraints ();
ira_implicitly_set_insn_hard_regs (&temp);
IOR_HARD_REG_SET (implicit_reg_pending_clobbers, temp);
}
can_start_lhs_rhs_p = (NONJUMP_INSN_P (insn)
&& code == SET);
@ -2263,7 +2610,8 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
and others know that a value is dead. Depend on the last call
instruction so that reg-stack won't get confused. */
if (code == CLOBBER)
add_dependence_list (insn, deps->last_function_call, 1, REG_DEP_OUTPUT);
add_dependence_list (insn, deps->last_function_call, 1,
REG_DEP_OUTPUT);
}
else if (code == PARALLEL)
{
@ -2326,6 +2674,8 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_ANTI);
add_dependence_list (insn, reg_last->implicit_sets,
0, REG_DEP_ANTI);
add_dependence_list (insn, reg_last->clobbers, 0,
REG_DEP_ANTI);
@ -2381,6 +2731,12 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
|| (NONJUMP_INSN_P (insn) && control_flow_insn_p (insn)))
reg_pending_barrier = MOVE_BARRIER;
if (sched_pressure_p)
{
setup_insn_reg_uses (deps, insn);
setup_insn_reg_pressure_info (insn);
}
/* Add register dependencies for insn. */
if (DEBUG_INSN_P (insn))
{
@ -2421,119 +2777,160 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
if (prev && NONDEBUG_INSN_P (prev))
add_dependence (insn, prev, REG_DEP_ANTI);
}
/* If the current insn is conditional, we can't free any
of the lists. */
else if (sched_has_condition_p (insn))
{
EXECUTE_IF_SET_IN_REG_SET (reg_pending_uses, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
if (!deps->readonly)
{
reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
reg_last->uses_length++;
}
}
EXECUTE_IF_SET_IN_REG_SET (reg_pending_clobbers, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
if (!deps->readonly)
{
reg_last->clobbers = alloc_INSN_LIST (insn, reg_last->clobbers);
reg_last->clobbers_length++;
}
}
EXECUTE_IF_SET_IN_REG_SET (reg_pending_sets, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
if (!deps->readonly)
{
reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
SET_REGNO_REG_SET (&deps->reg_conditional_sets, i);
}
}
}
else
{
EXECUTE_IF_SET_IN_REG_SET (reg_pending_uses, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
add_dependence_list (insn, reg_last->implicit_sets, 0, REG_DEP_ANTI);
add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
if (!deps->readonly)
{
reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
reg_last->uses_length++;
}
}
for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
if (TEST_HARD_REG_BIT (implicit_reg_pending_uses, i))
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
add_dependence_list (insn, reg_last->implicit_sets, 0,
REG_DEP_ANTI);
add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
if (!deps->readonly)
{
reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
reg_last->uses_length++;
}
}
if (!deps->readonly)
{
reg_last->uses_length++;
reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
}
}
EXECUTE_IF_SET_IN_REG_SET (reg_pending_clobbers, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
if (reg_last->uses_length > MAX_PENDING_LIST_LENGTH
|| reg_last->clobbers_length > MAX_PENDING_LIST_LENGTH)
{
add_dependence_list_and_free (deps, insn, &reg_last->sets, 0,
REG_DEP_OUTPUT);
add_dependence_list_and_free (deps, insn, &reg_last->uses, 0,
REG_DEP_ANTI);
add_dependence_list_and_free (deps, insn, &reg_last->clobbers, 0,
REG_DEP_OUTPUT);
if (!deps->readonly)
{
reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
reg_last->clobbers_length = 0;
reg_last->uses_length = 0;
}
}
else
{
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
}
if (!deps->readonly)
{
reg_last->clobbers_length++;
reg_last->clobbers = alloc_INSN_LIST (insn, reg_last->clobbers);
}
}
EXECUTE_IF_SET_IN_REG_SET (reg_pending_sets, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list_and_free (deps, insn, &reg_last->sets, 0,
REG_DEP_OUTPUT);
add_dependence_list_and_free (deps, insn, &reg_last->clobbers, 0,
REG_DEP_OUTPUT);
add_dependence_list_and_free (deps, insn, &reg_last->uses, 0,
REG_DEP_ANTI);
if (!deps->readonly)
{
reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
reg_last->uses_length = 0;
reg_last->clobbers_length = 0;
CLEAR_REGNO_REG_SET (&deps->reg_conditional_sets, i);
}
}
/* If the current insn is conditional, we can't free any
of the lists. */
if (sched_has_condition_p (insn))
{
EXECUTE_IF_SET_IN_REG_SET (reg_pending_clobbers, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->implicit_sets, 0,
REG_DEP_ANTI);
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
if (!deps->readonly)
{
reg_last->clobbers
= alloc_INSN_LIST (insn, reg_last->clobbers);
reg_last->clobbers_length++;
}
}
EXECUTE_IF_SET_IN_REG_SET (reg_pending_sets, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->implicit_sets, 0,
REG_DEP_ANTI);
add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
if (!deps->readonly)
{
reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
SET_REGNO_REG_SET (&deps->reg_conditional_sets, i);
}
}
}
else
{
EXECUTE_IF_SET_IN_REG_SET (reg_pending_clobbers, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
if (reg_last->uses_length > MAX_PENDING_LIST_LENGTH
|| reg_last->clobbers_length > MAX_PENDING_LIST_LENGTH)
{
add_dependence_list_and_free (deps, insn, &reg_last->sets, 0,
REG_DEP_OUTPUT);
add_dependence_list_and_free (deps, insn,
&reg_last->implicit_sets, 0,
REG_DEP_ANTI);
add_dependence_list_and_free (deps, insn, &reg_last->uses, 0,
REG_DEP_ANTI);
add_dependence_list_and_free
(deps, insn, &reg_last->clobbers, 0, REG_DEP_OUTPUT);
if (!deps->readonly)
{
reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
reg_last->clobbers_length = 0;
reg_last->uses_length = 0;
}
}
else
{
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
add_dependence_list (insn, reg_last->implicit_sets, 0,
REG_DEP_ANTI);
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
}
if (!deps->readonly)
{
reg_last->clobbers_length++;
reg_last->clobbers
= alloc_INSN_LIST (insn, reg_last->clobbers);
}
}
EXECUTE_IF_SET_IN_REG_SET (reg_pending_sets, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list_and_free (deps, insn, &reg_last->sets, 0,
REG_DEP_OUTPUT);
add_dependence_list_and_free (deps, insn,
&reg_last->implicit_sets,
0, REG_DEP_ANTI);
add_dependence_list_and_free (deps, insn, &reg_last->clobbers, 0,
REG_DEP_OUTPUT);
add_dependence_list_and_free (deps, insn, &reg_last->uses, 0,
REG_DEP_ANTI);
if (!deps->readonly)
{
reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
reg_last->uses_length = 0;
reg_last->clobbers_length = 0;
CLEAR_REGNO_REG_SET (&deps->reg_conditional_sets, i);
}
}
}
}
for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
if (TEST_HARD_REG_BIT (implicit_reg_pending_clobbers, i))
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->sets, 0, REG_DEP_ANTI);
add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_ANTI);
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
if (!deps->readonly)
reg_last->implicit_sets
= alloc_INSN_LIST (insn, reg_last->implicit_sets);
}
if (!deps->readonly)
{
IOR_REG_SET (&deps->reg_last_in_use, reg_pending_uses);
IOR_REG_SET (&deps->reg_last_in_use, reg_pending_clobbers);
IOR_REG_SET (&deps->reg_last_in_use, reg_pending_sets);
for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
if (TEST_HARD_REG_BIT (implicit_reg_pending_uses, i)
|| TEST_HARD_REG_BIT (implicit_reg_pending_clobbers, i))
SET_REGNO_REG_SET (&deps->reg_last_in_use, i);
/* Set up the pending barrier found. */
deps->last_reg_pending_barrier = reg_pending_barrier;
@ -2542,6 +2939,8 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
CLEAR_REG_SET (reg_pending_uses);
CLEAR_REG_SET (reg_pending_clobbers);
CLEAR_REG_SET (reg_pending_sets);
CLEAR_HARD_REG_SET (implicit_reg_pending_clobbers);
CLEAR_HARD_REG_SET (implicit_reg_pending_uses);
/* Add dependencies if a scheduling barrier was found. */
if (reg_pending_barrier)
@ -2554,12 +2953,14 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
{
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
add_dependence_list
(insn, reg_last->sets, 0,
reg_pending_barrier == TRUE_BARRIER ? REG_DEP_TRUE : REG_DEP_ANTI);
add_dependence_list
(insn, reg_last->clobbers, 0,
reg_pending_barrier == TRUE_BARRIER ? REG_DEP_TRUE : REG_DEP_ANTI);
add_dependence_list (insn, reg_last->sets, 0,
reg_pending_barrier == TRUE_BARRIER
? REG_DEP_TRUE : REG_DEP_ANTI);
add_dependence_list (insn, reg_last->implicit_sets, 0,
REG_DEP_ANTI);
add_dependence_list (insn, reg_last->clobbers, 0,
reg_pending_barrier == TRUE_BARRIER
? REG_DEP_TRUE : REG_DEP_ANTI);
}
}
else
@ -2569,12 +2970,15 @@ sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
struct deps_reg *reg_last = &deps->reg_last[i];
add_dependence_list_and_free (deps, insn, &reg_last->uses, 0,
REG_DEP_ANTI);
add_dependence_list_and_free
(deps, insn, &reg_last->sets, 0,
reg_pending_barrier == TRUE_BARRIER ? REG_DEP_TRUE : REG_DEP_ANTI);
add_dependence_list_and_free
(deps, insn, &reg_last->clobbers, 0,
reg_pending_barrier == TRUE_BARRIER ? REG_DEP_TRUE : REG_DEP_ANTI);
add_dependence_list_and_free (deps, insn, &reg_last->sets, 0,
reg_pending_barrier == TRUE_BARRIER
? REG_DEP_TRUE : REG_DEP_ANTI);
add_dependence_list_and_free (deps, insn,
&reg_last->implicit_sets, 0,
REG_DEP_ANTI);
add_dependence_list_and_free (deps, insn, &reg_last->clobbers, 0,
reg_pending_barrier == TRUE_BARRIER
? REG_DEP_TRUE : REG_DEP_ANTI);
if (!deps->readonly)
{
@ -2750,7 +3154,7 @@ deps_analyze_insn (struct deps *deps, rtx insn)
if (global_regs[i])
{
SET_REGNO_REG_SET (reg_pending_sets, i);
SET_REGNO_REG_SET (reg_pending_uses, i);
SET_HARD_REG_BIT (implicit_reg_pending_uses, i);
}
/* Other call-clobbered hard regs may be clobbered.
Since we only have a choice between 'might be clobbered'
@ -2763,7 +3167,7 @@ deps_analyze_insn (struct deps *deps, rtx insn)
by the function, but it is certain that the stack pointer
is among them, but be conservative. */
else if (fixed_regs[i])
SET_REGNO_REG_SET (reg_pending_uses, i);
SET_HARD_REG_BIT (implicit_reg_pending_uses, i);
/* The frame pointer is normally not used by the function
itself, but by the debugger. */
/* ??? MIPS o32 is an exception. It uses the frame pointer
@ -2772,7 +3176,7 @@ deps_analyze_insn (struct deps *deps, rtx insn)
else if (i == FRAME_POINTER_REGNUM
|| (i == HARD_FRAME_POINTER_REGNUM
&& (! reload_completed || frame_pointer_needed)))
SET_REGNO_REG_SET (reg_pending_uses, i);
SET_HARD_REG_BIT (implicit_reg_pending_uses, i);
}
/* For each insn which shouldn't cross a call, add a dependence
@ -2988,6 +3392,8 @@ free_deps (struct deps *deps)
free_INSN_LIST_list (&reg_last->uses);
if (reg_last->sets)
free_INSN_LIST_list (&reg_last->sets);
if (reg_last->implicit_sets)
free_INSN_LIST_list (&reg_last->implicit_sets);
if (reg_last->clobbers)
free_INSN_LIST_list (&reg_last->clobbers);
}
@ -3025,9 +3431,12 @@ remove_from_deps (struct deps *deps, rtx insn)
remove_from_dependence_list (insn, &reg_last->uses);
if (reg_last->sets)
remove_from_dependence_list (insn, &reg_last->sets);
if (reg_last->implicit_sets)
remove_from_dependence_list (insn, &reg_last->implicit_sets);
if (reg_last->clobbers)
remove_from_dependence_list (insn, &reg_last->clobbers);
if (!reg_last->uses && !reg_last->sets && !reg_last->clobbers)
if (!reg_last->uses && !reg_last->sets && !reg_last->implicit_sets
&& !reg_last->clobbers)
CLEAR_REGNO_REG_SET (&deps->reg_last_in_use, i);
}
@ -3167,6 +3576,8 @@ sched_deps_finish (void)
void
init_deps_global (void)
{
CLEAR_HARD_REG_SET (implicit_reg_pending_clobbers);
CLEAR_HARD_REG_SET (implicit_reg_pending_uses);
reg_pending_sets = ALLOC_REG_SET (&reg_obstack);
reg_pending_clobbers = ALLOC_REG_SET (&reg_obstack);
reg_pending_uses = ALLOC_REG_SET (&reg_obstack);

89
gcc/sched-int.h
View File

@ -441,6 +441,7 @@ struct deps_reg
{
rtx uses;
rtx sets;
rtx implicit_sets;
rtx clobbers;
int uses_length;
int clobbers_length;
@ -642,6 +643,14 @@ extern spec_info_t spec_info;
extern struct haifa_sched_info *current_sched_info;
/* Do register pressure sensitive insn scheduling if the flag is set
up. */
extern bool sched_pressure_p;
/* Map regno -> its cover class. The map defined only when
SCHED_PRESSURE_P is true. */
extern enum reg_class *sched_regno_cover_class;
/* Indexed by INSN_UID, the collection of all data associated with
a single instruction. */
@ -687,6 +696,52 @@ struct _haifa_deps_insn_data
unsigned int cant_move : 1;
};
/* Bits used for storing values of the fields in the following
structure. */
#define INCREASE_BITS 8
/* The structure describes how the corresponding insn increases the
register pressure for each cover class. */
struct reg_pressure_data
{
/* Pressure increase for given class because of clobber. */
unsigned int clobber_increase : INCREASE_BITS;
/* Increase in register pressure for given class because of register
sets. */
unsigned int set_increase : INCREASE_BITS;
/* Pressure increase for given class because of unused register
set. */
unsigned int unused_set_increase : INCREASE_BITS;
/* Pressure change: #sets - #deaths. */
int change : INCREASE_BITS;
};
/* The following structure describes usage of registers by insns. */
struct reg_use_data
{
/* Regno used in the insn. */
int regno;
/* Insn using the regno. */
rtx insn;
/* Cyclic list of elements with the same regno. */
struct reg_use_data *next_regno_use;
/* List of elements with the same insn. */
struct reg_use_data *next_insn_use;
};
/* The following structure describes used sets of registers by insns.
Registers are pseudos whose cover class is not NO_REGS or hard
registers available for allocations. */
struct reg_set_data
{
/* Regno used in the insn. */
int regno;
/* Insn setting the regno. */
rtx insn;
/* List of elements with the same insn. */
struct reg_set_data *next_insn_set;
};
struct _haifa_insn_data
{
/* We can't place 'struct _deps_list' into h_i_d instead of deps_list_t
@ -712,10 +767,6 @@ struct _haifa_insn_data
short cost;
/* This weight is an estimation of the insn's contribution to
register pressure. */
short reg_weight;
/* Set if there's DEF-USE dependence between some speculatively
moved load insn and this one. */
unsigned int fed_by_spec_load : 1;
@ -740,6 +791,26 @@ struct _haifa_insn_data
/* Original pattern of the instruction. */
rtx orig_pat;
/* The following array contains info how the insn increases register
pressure. There is an element for each cover class of pseudos
referenced in insns. */
struct reg_pressure_data *reg_pressure;
/* The following array contains maximal reg pressure between last
scheduled insn and given insn. There is an element for each
cover class of pseudos referenced in insns. This info updated
after scheduling each insn for each insn between the two
mentioned insns. */
int *max_reg_pressure;
/* The following list contains info about used pseudos and hard
registers available for allocation. */
struct reg_use_data *reg_use_list;
/* The following list contains info about set pseudos and hard
registers available for allocation. */
struct reg_set_data *reg_set_list;
/* Info about how scheduling the insn changes cost of register
pressure excess (between source and target). */
int reg_pressure_excess_cost_change;
};
typedef struct _haifa_insn_data haifa_insn_data_def;
@ -755,7 +826,12 @@ extern VEC(haifa_insn_data_def, heap) *h_i_d;
/* Accessor macros for h_i_d. There are more in haifa-sched.c and
sched-rgn.c. */
#define INSN_PRIORITY(INSN) (HID (INSN)->priority)
#define INSN_REG_WEIGHT(INSN) (HID (INSN)->reg_weight)
#define INSN_REG_PRESSURE(INSN) (HID (INSN)->reg_pressure)
#define INSN_MAX_REG_PRESSURE(INSN) (HID (INSN)->max_reg_pressure)
#define INSN_REG_USE_LIST(INSN) (HID (INSN)->reg_use_list)
#define INSN_REG_SET_LIST(INSN) (HID (INSN)->reg_set_list)
#define INSN_REG_PRESSURE_EXCESS_COST_CHANGE(INSN) \
(HID (INSN)->reg_pressure_excess_cost_change)
#define INSN_PRIORITY_STATUS(INSN) (HID (INSN)->priority_status)
typedef struct _haifa_deps_insn_data haifa_deps_insn_data_def;
@ -1153,7 +1229,9 @@ extern void extend_dependency_caches (int, bool);
extern void debug_ds (ds_t);
/* Functions in haifa-sched.c. */
extern void sched_init_region_reg_pressure_info (void);
extern int haifa_classify_insn (const_rtx);
extern void get_ebb_head_tail (basic_block, basic_block, rtx *, rtx *);
extern int no_real_insns_p (const_rtx, const_rtx);
@ -1163,6 +1241,7 @@ extern int dep_cost_1 (dep_t, dw_t);
extern int dep_cost (dep_t);
extern int set_priorities (rtx, rtx);
extern void sched_setup_bb_reg_pressure_info (basic_block, rtx);
extern void schedule_block (basic_block *);
extern int cycle_issued_insns;

30
gcc/sched-rgn.c
View File

@ -2613,6 +2613,8 @@ deps_join (struct deps *succ_deps, struct deps *pred_deps)
succ_rl->uses = concat_INSN_LIST (pred_rl->uses, succ_rl->uses);
succ_rl->sets = concat_INSN_LIST (pred_rl->sets, succ_rl->sets);
succ_rl->implicit_sets
= concat_INSN_LIST (pred_rl->implicit_sets, succ_rl->implicit_sets);
succ_rl->clobbers = concat_INSN_LIST (pred_rl->clobbers,
succ_rl->clobbers);
succ_rl->uses_length += pred_rl->uses_length;
@ -2690,12 +2692,14 @@ propagate_deps (int bb, struct deps *pred_deps)
bb's successors.
Specifically for reg-reg data dependences, the block insns are
scanned by sched_analyze () top-to-bottom. Two lists are
scanned by sched_analyze () top-to-bottom. Three lists are
maintained by sched_analyze (): reg_last[].sets for register DEFs,
and reg_last[].uses for register USEs.
reg_last[].implicit_sets for implicit hard register DEFs, and
reg_last[].uses for register USEs.
When analysis is completed for bb, we update for its successors:
; - DEFS[succ] = Union (DEFS [succ], DEFS [bb])
; - IMPLICIT_DEFS[succ] = Union (IMPLICIT_DEFS [succ], IMPLICIT_DEFS [bb])
; - USES[succ] = Union (USES [succ], DEFS [bb])
The mechanism for computing mem-mem data dependence is very
@ -2934,6 +2938,28 @@ schedule_region (int rgn)
sched_extend_ready_list (rgn_n_insns);
if (sched_pressure_p)
{
sched_init_region_reg_pressure_info ();
for (bb = 0; bb < current_nr_blocks; bb++)
{
basic_block first_bb, last_bb;
rtx head, tail;
first_bb = EBB_FIRST_BB (bb);
last_bb = EBB_LAST_BB (bb);
get_ebb_head_tail (first_bb, last_bb, &head, &tail);
if (no_real_insns_p (head, tail))
{
gcc_assert (first_bb == last_bb);
continue;
}
sched_setup_bb_reg_pressure_info (first_bb, PREV_INSN (head));
}
}
/* Now we can schedule all blocks. */
for (bb = 0; bb < current_nr_blocks; bb++)
{