OpenE2K/gcc
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

basic-block.h (struct edge_def): Add new field, crossing_edge.

Browse Source 
2004-04-09  Caroline Tice  <ctice@apple.com>

        * basic-block.h (struct edge_def):  Add new field, crossing_edge.
        (struct basic_block_def):  Add new field, partition.
        (UNPARTITIONED, HOT_PARTITION, COLD_PARTITION):  New constant macro
        definitions.
        (partition_hot_cold_basic_blocks): Add extern function
        declaration.
        * bb-reorder.c (function.h, obstack.h, expr.h, regs.h):  Add four new
        include statements.
        (N_ROUNDS): Increase the maximum number of rounds by 1.
        (branch_threshold): Add array value for new round.
        (exec_threshold): Add array value for new round.
        (push_to_next_round_p): New function.
        (add_unlikely_executed_notes): New function.
        (find_rarely_executed_basic_blocks_and_crossing_edges): New function.
        (mark_bb_for_unlikely_executed_section): New function.
        (add_labels_and_missing_jumps): New function.
        (add_reg_crossing_jump_notes): New function.
        (fix_up_fall_thru_edges): New function.
        (find_jump_block): New function.
        (fix_crossing_conditional_branches): New function.
        (fix_crossing_unconditional_branches): New function.
        (fix_edges_for_rarely_executed_code): New function.
        (partition_hot_cold_basic_blocks): New function.
        (find_traces):  Add an extra round for partitioning hot/cold
        basic blocks.
        (find_traces_1_round): Add a parameter.  Modify to push all cold blocks,
        and only cold blocks, into the last (extra) round of collecting traces.
        (better_edge_p): Add a parameter.  Modify to favor non-crossing edges
        over crossing edges.
        (bb_to_key):  Add code to correctly identify cold blocks when
        doing partitioning.
        (connect_traces): Modify to connect all the non-cold traces first, then
        go back and connect up all the cold traces.
        (reorder_basic_blocks):  Add call to add_unlikely_executed_notes.
        * cfg.c (entry_exit_blocks): Add initialization for partition field in
        entry and exit blocks.
        * cfgbuild.c (make_edges): Update current_function_has_computed_jump
        if we are doing hot/cold partitioning.
        * cfgcleanup.c (cfglayout.h): Add new include statement.
        (try_simplify_condjump): Modify to not attempt on blocks with jumps
        that cross section boundaries.
        (try_forward_edges): Likewise.
        (merge_blocks_move_predecessor_nojumps): Likewise.
        (merge_blocks_move_successor_nojumps): Likewise.
        (merge_blocks_move): Likewise.
        (try_crossjump_to_edge): Modify to not attempt after we have done
        the block partitioning.
        (try_crossjump_bb): Modify to not attempt on blocks with jumps that
        cross section boundaries.
        (try_optimize_cfg): Likewise.
        * cfghooks.c (tidy_fallthru_edges): Modify to not remove indirect
        jumps that cross section boundaries.
        * cfglayout.c (flags.h): Add new include statement.
        (update_unlikely_executed_notes):  New function.
        (fixup_reorder_chain):  Add code so when a new jumping basic block is
        added, it's UNLIKELY_EXECUTED_CODE and REG_CROSSING_JUMP notes are
        updated appropriately.
        (duplicate_insn_chain):  Add code to duplicate the new NOTE insn
        introduced by this optimization.
        * cfglayout.h (scan_ahead_for_unlikely_executed_note):  Add new
        extern function declaration.
        * cfgrtl.c (can_delete_note_p):  Add NOTE_INSN_UNLIKELY_EXECUTED_CODE to
        list of notes that can be deleted.
        (create_basic_block_structure):  Add initialization for partition field.
        (rtl_can_merge_blocks):  Modify to test blocks for jumps that cross
        section boundaries.
        (try_redirect_by_replacing_jump):  Modify to not attempt on jumps that
        cross section boundaries.
        (commit_one_edge_insertion): Add code so newly created basic block
        ends up in correct (hot or cold) section.  Modify to disallow
        insertions before NOTE_INSN_UNLIKELY_EXECUTED_CODE notes.
        (rtl_verify_flow_info_1):  Add code to verify that no fall_thru edge
        crosses section boundaries.
        (cfg_layout_can_merge_blocks_p): Modify to test blocks for jumps that
        cross section boundaries.
        (force_nonfallthru_and_redirect): Modify to make sure new basic block
        ends up in correct section, with correct notes attached.
        * common.opt (freorder-blocks-and-partition):  Add new flag for this
        optimization.
        * dbxout.c (dbx_function_end):  Add code to make sure scope labels at
        the end of functions are written into the correct (hot or cold)
        section.
        (dbx_source_file): Add code so writing debug file information
        doesn't incorrectly change sections.
        * defaults.h (NORMAL_TEXT_SECTION_NAME): New constant macro, for use
        in partitioning hot/cold basic blocks into separate sections.
        (SECTION_FORMAT_STRING): New constant macro, for linux/i386 hot/cold
        section partitioning.
        (HAS_LONG_COND_BRANCH): New constant macro, indicating whether or not
        conditional branches can span all of memory.
        (HAS_LONG_UNCOND_BRANCH): New constant macro, indicationg whether or not
        unconditional branches can span all of memory.
        * final.c (scan_ahead_for_unlikely_executed_note):  New function.
        (final_scan_insn):  Add code to check for NOTE instruction indicating
        whether basic block belongs in hot or cold section, and to make sure
        the current basic block is being written to the appropriate section.
        Also added code to ensure that jump table basic blocks end up in the
        correct section.
        * flags.h (flag_reorder_blocks_and_partition):  New flag.
        * ifcvt.c (find_if_case_1):  Modify to not attempt if conversion if
        one of the branches has a jump that crosses between sections.
        (find_if_case_2): Likewise.
        (ifcvt): Modify to not attempt to mark loop exit edges after
        hot/cold partitioning has occurred.
        * opts.c (decode_options): Code to handle new flag,
        flag_reorder_blocks_and_partition; also to turn it off if
        flag_exceptions is on.
        (common_handle_option): Code to handle new flag,
        flag_reorder_blocks_and_partition.
        * output.h (unlikely_text_section): New extern function declaration.
        (in_unlikely_text_section): New extern function declaration.
        * passes.c (rest_of_handle_stack_regs):  Add
        flag_reorder_blocks_and_partition as an 'or' condition for calling
        reorder_basic_blocks.
        (rest_of_handle_reorder_blocks):  Add flag_reorder_blocks_and_partition
        as an 'or' condition for calling reorder_basic_blocks.
        (rest_of_compilation):  Add call to partition_hot_cold_basic_blocks.
        * print-rtl.c (print_rtx): Add code for handling new note,
        NOTE_INSN_UNLIKELY_EXECUTED_CODE
        * rtl.c  (NOTE_INSN_UNLIKELY_EXECUTED_CODE): New note insn (see below).
        (REG_CROSSING_JUMP): New kind of reg_note, to mark jumps that
        cross between section boundaries.
        * rtl.h (NOTE_INSN_UNLIKELY_EXECUTED_CODE):  New note instruction,
        indicating the basic block containing it belongs in the cold section.
        (REG_CROSSING_JUMP): New type of reg_note, to mark jumps that cross
        between hot and cold sections.
        * toplev.c (flag_reorder_blocks_and_partition):  Add code to
        initialize this flag, and to tie it to the command-line option
        freorder-blocks-and-partition.
        * varasm.c (cfglayout.h):  Add new include statement.
        (unlikely_section_label_printed):  New global variable, used for
        determining when to output section name labels for cold sections.
        (in_section):  Add in_unlikely_executed_text to enum data structure.
        (text_section):  Modify code to use SECTION_FORMAT_STRING and
        NORMAL_TEXT_SECTION_NAME macros.
        (unlikely_text_section):  New function.
        (in_unlikely_text_section):  New function.
        (function_section):  Add code to make sure beginning of function is
        written into correct section (hot or cold).
        (assemble_start_function):  Add code to make sure stuff is written to
        the correct section.
        (assemble_zeros):  Add in_unlikely_text_section as an 'or' condition
        to an if statement that was checking 'in_text_section'.
        (assemble_variable):  Add 'in_unlikely_text_section' as an 'or'
        condition to an if statement that was checking 'in_text_section'.
        (default_section_type_flags_1):  Add check: if in cold section
        flags = SECTION_CODE.
        * config/darwin.c (darwin_asm_named_section):  Modify to use
        SECTION_FORMAT_STRING if we are partitioning hot/cold blocks.
        * config/i386/i386.h (HAS_LONG_COND_BRANCH): Defined this macro
        specifically for the i386.
        (HAS_LONG_UNCOND_BRANCH):  Defined this macro specifically for the i386.
        * config/rs6000/darwin.h (UNLIKELY_EXECUTED_TEXT_SECTION_NAME): Change
        text string to something more informative.
        (NORMAL_TEXT_SECTION_NAME): Add new definition.
        (SECTION_FORMAT_STRING):  Add new definition.
        * config/rs6000/rs6000.c (rs6000_assemble_integer):  Add
        '!in_unlikely_text_section' as an 'and' condition to an if statement
        that was already checking '!in_text_section'.
        * config/rs6000/sysv4.h (HOT_TEXT_SECTION_NAME,NORMAL_TEXT_SECTION_NAME,
        UNLIKELY_EXECUTED_TEXT_SECTION_NAME,SECTION_FORMAT_STRING): Make
        sure these are properly defined for linux on ppc.
        * doc/invoke.texi  (freorder-blocks-and-partition): Add documentation
        for this new flag.
        * doc/rtl.texi (REG_CROSSING_JUMP):  Add documentation for new
        reg_note.
        * doc/tm.texi (NORMAL_TEXT_SECTION_NAME, SECTION_FORMAT_STRING,
        HAS_LONG_COND_BRANCH, HAS_LONG_UNCOND_BRANCH): Add documentation for
        these new macros.

From-SVN: r80564
This commit is contained in:
Caroline Tice 2004-04-09 19:57:47 +00:00 committed by Caroline Tice
parent d770b996d2
commit 750054a2ee
32 changed files with 1659 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

172
gcc/ChangeLog
View File

@ -1,3 +1,175 @@
2004-04-09 Caroline Tice <ctice@apple.com>
* basic-block.h (struct edge_def): Add new field, crossing_edge.
(struct basic_block_def): Add new field, partition.
(UNPARTITIONED, HOT_PARTITION, COLD_PARTITION): New constant macro
definitions.
(partition_hot_cold_basic_blocks): Add extern function
declaration.
* bb-reorder.c (function.h, obstack.h, expr.h, regs.h): Add four new
include statements.
(N_ROUNDS): Increase the maximum number of rounds by 1.
(branch_threshold): Add array value for new round.
(exec_threshold): Add array value for new round.
(push_to_next_round_p): New function.
(add_unlikely_executed_notes): New function.
(find_rarely_executed_basic_blocks_and_crossing_edges): New function.
(mark_bb_for_unlikely_executed_section): New function.
(add_labels_and_missing_jumps): New function.
(add_reg_crossing_jump_notes): New function.
(fix_up_fall_thru_edges): New function.
(find_jump_block): New function.
(fix_crossing_conditional_branches): New function.
(fix_crossing_unconditional_branches): New function.
(fix_edges_for_rarely_executed_code): New function.
(partition_hot_cold_basic_blocks): New function.
(find_traces): Add an extra round for partitioning hot/cold
basic blocks.
(find_traces_1_round): Add a parameter. Modify to push all cold blocks,
and only cold blocks, into the last (extra) round of collecting traces.
(better_edge_p): Add a parameter. Modify to favor non-crossing edges
over crossing edges.
(bb_to_key): Add code to correctly identify cold blocks when
doing partitioning.
(connect_traces): Modify to connect all the non-cold traces first, then
go back and connect up all the cold traces.
(reorder_basic_blocks): Add call to add_unlikely_executed_notes.
* cfg.c (entry_exit_blocks): Add initialization for partition field in
entry and exit blocks.
* cfgbuild.c (make_edges): Update current_function_has_computed_jump
if we are doing hot/cold partitioning.
* cfgcleanup.c (cfglayout.h): Add new include statement.
(try_simplify_condjump): Modify to not attempt on blocks with jumps
that cross section boundaries.
(try_forward_edges): Likewise.
(merge_blocks_move_predecessor_nojumps): Likewise.
(merge_blocks_move_successor_nojumps): Likewise.
(merge_blocks_move): Likewise.
(try_crossjump_to_edge): Modify to not attempt after we have done
the block partitioning.
(try_crossjump_bb): Modify to not attempt on blocks with jumps that
cross section boundaries.
(try_optimize_cfg): Likewise.
* cfghooks.c (tidy_fallthru_edges): Modify to not remove indirect
jumps that cross section boundaries.
* cfglayout.c (flags.h): Add new include statement.
(update_unlikely_executed_notes): New function.
(fixup_reorder_chain): Add code so when a new jumping basic block is
added, it's UNLIKELY_EXECUTED_CODE and REG_CROSSING_JUMP notes are
updated appropriately.
(duplicate_insn_chain): Add code to duplicate the new NOTE insn
introduced by this optimization.
* cfglayout.h (scan_ahead_for_unlikely_executed_note): Add new
extern function declaration.
* cfgrtl.c (can_delete_note_p): Add NOTE_INSN_UNLIKELY_EXECUTED_CODE to
list of notes that can be deleted.
(create_basic_block_structure): Add initialization for partition field.
(rtl_can_merge_blocks): Modify to test blocks for jumps that cross
section boundaries.
(try_redirect_by_replacing_jump): Modify to not attempt on jumps that
cross section boundaries.
(commit_one_edge_insertion): Add code so newly created basic block
ends up in correct (hot or cold) section. Modify to disallow
insertions before NOTE_INSN_UNLIKELY_EXECUTED_CODE notes.
(rtl_verify_flow_info_1): Add code to verify that no fall_thru edge
crosses section boundaries.
(cfg_layout_can_merge_blocks_p): Modify to test blocks for jumps that
cross section boundaries.
(force_nonfallthru_and_redirect): Modify to make sure new basic block
ends up in correct section, with correct notes attached.
* common.opt (freorder-blocks-and-partition): Add new flag for this
optimization.
* dbxout.c (dbx_function_end): Add code to make sure scope labels at
the end of functions are written into the correct (hot or cold)
section.
(dbx_source_file): Add code so writing debug file information
doesn't incorrectly change sections.
* defaults.h (NORMAL_TEXT_SECTION_NAME): New constant macro, for use
in partitioning hot/cold basic blocks into separate sections.
(SECTION_FORMAT_STRING): New constant macro, for linux/i386 hot/cold
section partitioning.
(HAS_LONG_COND_BRANCH): New constant macro, indicating whether or not
conditional branches can span all of memory.
(HAS_LONG_UNCOND_BRANCH): New constant macro, indicationg whether or not
unconditional branches can span all of memory.
* final.c (scan_ahead_for_unlikely_executed_note): New function.
(final_scan_insn): Add code to check for NOTE instruction indicating
whether basic block belongs in hot or cold section, and to make sure
the current basic block is being written to the appropriate section.
Also added code to ensure that jump table basic blocks end up in the
correct section.
* flags.h (flag_reorder_blocks_and_partition): New flag.
* ifcvt.c (find_if_case_1): Modify to not attempt if conversion if
one of the branches has a jump that crosses between sections.
(find_if_case_2): Likewise.
(ifcvt): Modify to not attempt to mark loop exit edges after
hot/cold partitioning has occurred.
* opts.c (decode_options): Code to handle new flag,
flag_reorder_blocks_and_partition; also to turn it off if
flag_exceptions is on.
(common_handle_option): Code to handle new flag,
flag_reorder_blocks_and_partition.
* output.h (unlikely_text_section): New extern function declaration.
(in_unlikely_text_section): New extern function declaration.
* passes.c (rest_of_handle_stack_regs): Add
flag_reorder_blocks_and_partition as an 'or' condition for calling
reorder_basic_blocks.
(rest_of_handle_reorder_blocks): Add flag_reorder_blocks_and_partition
as an 'or' condition for calling reorder_basic_blocks.
(rest_of_compilation): Add call to partition_hot_cold_basic_blocks.
* print-rtl.c (print_rtx): Add code for handling new note,
NOTE_INSN_UNLIKELY_EXECUTED_CODE
* rtl.c (NOTE_INSN_UNLIKELY_EXECUTED_CODE): New note insn (see below).
(REG_CROSSING_JUMP): New kind of reg_note, to mark jumps that
cross between section boundaries.
* rtl.h (NOTE_INSN_UNLIKELY_EXECUTED_CODE): New note instruction,
indicating the basic block containing it belongs in the cold section.
(REG_CROSSING_JUMP): New type of reg_note, to mark jumps that cross
between hot and cold sections.
* toplev.c (flag_reorder_blocks_and_partition): Add code to
initialize this flag, and to tie it to the command-line option
freorder-blocks-and-partition.
* varasm.c (cfglayout.h): Add new include statement.
(unlikely_section_label_printed): New global variable, used for
determining when to output section name labels for cold sections.
(in_section): Add in_unlikely_executed_text to enum data structure.
(text_section): Modify code to use SECTION_FORMAT_STRING and
NORMAL_TEXT_SECTION_NAME macros.
(unlikely_text_section): New function.
(in_unlikely_text_section): New function.
(function_section): Add code to make sure beginning of function is
written into correct section (hot or cold).
(assemble_start_function): Add code to make sure stuff is written to
the correct section.
(assemble_zeros): Add in_unlikely_text_section as an 'or' condition
to an if statement that was checking 'in_text_section'.
(assemble_variable): Add 'in_unlikely_text_section' as an 'or'
condition to an if statement that was checking 'in_text_section'.
(default_section_type_flags_1): Add check: if in cold section
flags = SECTION_CODE.
* config/darwin.c (darwin_asm_named_section): Modify to use
SECTION_FORMAT_STRING if we are partitioning hot/cold blocks.
* config/i386/i386.h (HAS_LONG_COND_BRANCH): Defined this macro
specifically for the i386.
(HAS_LONG_UNCOND_BRANCH): Defined this macro specifically for the i386.
* config/rs6000/darwin.h (UNLIKELY_EXECUTED_TEXT_SECTION_NAME): Change
text string to something more informative.
(NORMAL_TEXT_SECTION_NAME): Add new definition.
(SECTION_FORMAT_STRING): Add new definition.
* config/rs6000/rs6000.c (rs6000_assemble_integer): Add
'!in_unlikely_text_section' as an 'and' condition to an if statement
that was already checking '!in_text_section'.
* config/rs6000/sysv4.h (HOT_TEXT_SECTION_NAME,NORMAL_TEXT_SECTION_NAME,
UNLIKELY_EXECUTED_TEXT_SECTION_NAME,SECTION_FORMAT_STRING): Make
sure these are properly defined for linux on ppc.
* doc/invoke.texi (freorder-blocks-and-partition): Add documentation
for this new flag.
* doc/rtl.texi (REG_CROSSING_JUMP): Add documentation for new
reg_note.
* doc/tm.texi (NORMAL_TEXT_SECTION_NAME, SECTION_FORMAT_STRING,
HAS_LONG_COND_BRANCH, HAS_LONG_UNCOND_BRANCH): Add documentation for
these new macros.
2004-04-08 Roger Sayle <roger@eyesopen.com>
* function.c (gen_mem_addressof): When changing the RTX from a REG

12
gcc/basic-block.h
View File

@ -138,6 +138,8 @@ typedef struct edge_def {
int probability; /* biased by REG_BR_PROB_BASE */
gcov_type count; /* Expected number of executions calculated
in profile.c */
bool crossing_edge; /* Crosses between hot and cold sections, when
we do partitioning. */
} *edge;
#define EDGE_FALLTHRU 1 /* 'Straight line' flow */
@ -246,6 +248,9 @@ typedef struct basic_block_def {
/* Various flags. See BB_* below. */
int flags;
/* Which section block belongs in, when partitioning basic blocks. */
int partition;
/* Additional data maintained by cfg_layout routines. */
struct reorder_block_def *rbi;
} *basic_block;
@ -260,6 +265,12 @@ typedef struct basic_block_def {
#define BB_IRREDUCIBLE_LOOP 16
#define BB_SUPERBLOCK 32
/* Partitions, to be used when partitioning hot and cold basic blocks into
separate sections. */
#define UNPARTITIONED 0
#define HOT_PARTITION 1
#define COLD_PARTITION 2
/* Number of basic blocks in the current function. */
extern int n_basic_blocks;
@ -611,6 +622,7 @@ extern bool control_flow_insn_p (rtx);
/* In bb-reorder.c */
extern void reorder_basic_blocks (void);
extern void partition_hot_cold_basic_blocks (void);
/* In dominance.c */

926
gcc/bb-reorder.c
View File

File diff suppressed because it is too large Load Diff

2
gcc/cfg.c
View File

@ -115,6 +115,7 @@ struct basic_block_def entry_exit_blocks[2]
0, /* count */
0, /* frequency */
0, /* flags */
0, /* partition */
NULL /* rbi */
},
{
@ -138,6 +139,7 @@ struct basic_block_def entry_exit_blocks[2]
0, /* count */
0, /* frequency */
0, /* flags */
0, /* partition */
NULL /* rbi */
}
};

6
gcc/cfgbuild.c
View File

@ -271,6 +271,12 @@ make_edges (rtx label_value_list, basic_block min, basic_block max, int update_p
/* Assume no computed jump; revise as we create edges. */
current_function_has_computed_jump = 0;
/* If we are partitioning hot and cold basic blocks into separate
sections, we cannot assume there is no computed jump. */
if (flag_reorder_blocks_and_partition)
current_function_has_computed_jump = 1;
/* Heavy use of computed goto in machine-generated code can lead to
nearly fully-connected CFGs. In that case we spend a significant
amount of time searching the edge lists for duplicates. */

64
gcc/cfgcleanup.c
View File

@ -49,6 +49,7 @@ Software Foundation, 59 Temple Place - Suite 330, Boston, MA
#include "tm_p.h"
#include "target.h"
#include "regs.h"
#include "cfglayout.h"
#include "expr.h"
/* cleanup_cfg maintains following flags for each basic block. */
@ -150,6 +151,15 @@ try_simplify_condjump (basic_block cbranch_block)
return false;
jump_dest_block = jump_block->succ->dest;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& (jump_block->partition != jump_dest_block->partition
|| cbranch_jump_edge->crossing_edge))
return false;
/* The conditional branch must target the block after the
unconditional branch. */
cbranch_dest_block = cbranch_jump_edge->dest;
@ -428,6 +438,14 @@ try_forward_edges (int mode, basic_block b)
bool changed = false;
edge e, next, *threaded_edges = NULL;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& find_reg_note (BB_END (b), REG_CROSSING_JUMP, NULL_RTX))
return false;
for (e = b->succ; e; e = next)
{
basic_block target, first;
@ -675,6 +693,15 @@ merge_blocks_move_predecessor_nojumps (basic_block a, basic_block b)
{
rtx barrier;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& (a->partition != b->partition
|| find_reg_note (BB_END (a), REG_CROSSING_JUMP, NULL_RTX)))
return;
barrier = next_nonnote_insn (BB_END (a));
if (GET_CODE (barrier) != BARRIER)
abort ();
@ -718,6 +745,15 @@ merge_blocks_move_successor_nojumps (basic_block a, basic_block b)
rtx barrier, real_b_end;
rtx label, table;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& (find_reg_note (BB_END (a), REG_CROSSING_JUMP, NULL_RTX)
|| a->partition != b->partition))
return;
real_b_end = BB_END (b);
/* If there is a jump table following block B temporarily add the jump table
@ -782,6 +818,18 @@ merge_blocks_move (edge e, basic_block b, basic_block c, int mode)
&& tail_recursion_label_p (BB_HEAD (c)))
return NULL;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& (find_reg_note (BB_END (b), REG_CROSSING_JUMP, NULL_RTX)
|| find_reg_note (BB_END (c), REG_CROSSING_JUMP, NULL_RTX)
|| b->partition != c->partition))
return NULL;
/* If B has a fallthru edge to C, no need to move anything. */
if (e->flags & EDGE_FALLTHRU)
{
@ -1453,6 +1501,12 @@ try_crossjump_to_edge (int mode, edge e1, edge e2)
rtx newpos1, newpos2;
edge s;
/* If we have partitioned hot/cold basic blocks, it is a bad idea
to try this optimization. */
if (flag_reorder_blocks_and_partition && no_new_pseudos)
return false;
/* Search backward through forwarder blocks. We don't need to worry
about multiple entry or chained forwarders, as they will be optimized
away. We do this to look past the unconditional jump following a
@ -1639,6 +1693,15 @@ try_crossjump_bb (int mode, basic_block bb)
if (!bb->pred || !bb->pred->pred_next)
return false;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& (bb->pred->src->partition != bb->pred->pred_next->src->partition
|| bb->pred->crossing_edge))
return false;
/* It is always cheapest to redirect a block that ends in a branch to
a block that falls through into BB, as that adds no branches to the
program. We'll try that combination first. */
@ -1895,6 +1958,7 @@ try_optimize_cfg (int mode)
&& ! b->succ->succ_next
&& b->succ->dest != EXIT_BLOCK_PTR
&& onlyjump_p (BB_END (b))
&& !find_reg_note (BB_END (b), REG_CROSSING_JUMP, NULL_RTX)
&& try_redirect_by_replacing_jump (b->succ, b->succ->dest,
(mode & CLEANUP_CFGLAYOUT) != 0))
{

3
gcc/cfghooks.c
View File

@ -600,7 +600,8 @@ tidy_fallthru_edges (void)
if ((s = b->succ) != NULL
&& ! (s->flags & EDGE_COMPLEX)
&& s->succ_next == NULL
&& s->dest == c)
&& s->dest == c
&& !find_reg_note (BB_END (b), REG_CROSSING_JUMP, NULL_RTX))
tidy_fallthru_edge (s);
}
}

53
gcc/cfglayout.c
View File

@ -35,6 +35,7 @@ Software Foundation, 59 Temple Place - Suite 330, Boston, MA
#include "target.h"
#include "ggc.h"
#include "alloc-pool.h"
#include "flags.h"
/* The contents of the current function definition are allocated
in this obstack, and all are freed at the end of the function. */
@ -57,6 +58,7 @@ void verify_insn_chain (void);
static void fixup_fallthru_exit_predecessor (void);
static rtx duplicate_insn_chain (rtx, rtx);
static tree insn_scope (rtx);
static void update_unlikely_executed_notes (basic_block);
rtx
unlink_insn_chain (rtx first, rtx last)
@ -635,6 +637,7 @@ fixup_reorder_chain (void)
edge e_fall, e_taken, e;
rtx bb_end_insn;
basic_block nb;
basic_block old_bb;
if (bb->succ == NULL)
continue;
@ -711,6 +714,11 @@ fixup_reorder_chain (void)
}
}
/* If the "jumping" edge is a crossing edge, and the fall
through edge is non-crossing, leave things as they are. */
else if (e_taken->crossing_edge && !e_fall->crossing_edge)
continue;
/* Otherwise we can try to invert the jump. This will
basically never fail, however, keep up the pretense. */
else if (invert_jump (bb_end_insn,
@ -768,7 +776,34 @@ fixup_reorder_chain (void)
nb->rbi->next = bb->rbi->next;
bb->rbi->next = nb;
/* Don't process this new block. */
old_bb = bb;
bb = nb;
/* Make sure new bb is tagged for correct section (same as
fall-thru source). */
e_fall->src->partition = bb->pred->src->partition;
if (flag_reorder_blocks_and_partition)
{
if (bb->pred->src->partition == COLD_PARTITION)
{
rtx new_note;
rtx note = BB_HEAD (e_fall->src);
while (!INSN_P (note)
&& note != BB_END (e_fall->src))
note = NEXT_INSN (note);
new_note = emit_note_before
(NOTE_INSN_UNLIKELY_EXECUTED_CODE,
note);
NOTE_BASIC_BLOCK (new_note) = bb;
}
if (GET_CODE (BB_END (bb)) == JUMP_INSN
&& !any_condjump_p (BB_END (bb))
&& bb->succ->crossing_edge )
REG_NOTES (BB_END (bb)) = gen_rtx_EXPR_LIST
(REG_CROSSING_JUMP, NULL_RTX, REG_NOTES (BB_END (bb)));
}
}
}
@ -803,6 +838,8 @@ fixup_reorder_chain (void)
bb->index = index;
BASIC_BLOCK (index) = bb;
update_unlikely_executed_notes (bb);
bb->prev_bb = prev_bb;
prev_bb->next_bb = bb;
}
@ -820,6 +857,21 @@ fixup_reorder_chain (void)
}
}
/* Update the basic block number information in any
NOTE_INSN_UNLIKELY_EXECUTED_CODE notes within the basic block. */
static void
update_unlikely_executed_notes (basic_block bb)
{
rtx cur_insn;
for (cur_insn = BB_HEAD (bb); cur_insn != BB_END (bb);
cur_insn = NEXT_INSN (cur_insn))
if (GET_CODE (cur_insn) == NOTE
&& NOTE_LINE_NUMBER (cur_insn) == NOTE_INSN_UNLIKELY_EXECUTED_CODE)
NOTE_BASIC_BLOCK (cur_insn) = bb;
}
/* Perform sanity checks on the insn chain.
1. Check that next/prev pointers are consistent in both the forward and
reverse direction.
@ -990,6 +1042,7 @@ duplicate_insn_chain (rtx from, rtx to)
abort ();
break;
case NOTE_INSN_REPEATED_LINE_NUMBER:
case NOTE_INSN_UNLIKELY_EXECUTED_CODE:
emit_note_copy (insn);
break;

1
gcc/cfglayout.h
View File

@ -45,3 +45,4 @@ extern bool can_copy_bbs_p (basic_block *, unsigned);
extern void copy_bbs (basic_block *, unsigned, basic_block *,
edge *, unsigned, edge *, struct loop *);
extern void cfg_layout_initialize_rbi (basic_block);
extern bool scan_ahead_for_unlikely_executed_note (rtx);

111
gcc/cfgrtl.c
View File

@ -99,6 +99,7 @@ can_delete_note_p (rtx note)
{
return (NOTE_LINE_NUMBER (note) == NOTE_INSN_DELETED
|| NOTE_LINE_NUMBER (note) == NOTE_INSN_BASIC_BLOCK
|| NOTE_LINE_NUMBER (note) == NOTE_INSN_UNLIKELY_EXECUTED_CODE
|| NOTE_LINE_NUMBER (note) == NOTE_INSN_PREDICTION);
}
@ -319,6 +320,7 @@ create_basic_block_structure (rtx head, rtx end, rtx bb_note, basic_block after)
link_block (bb, after);
BASIC_BLOCK (bb->index) = bb;
update_bb_for_insn (bb);
bb->partition = UNPARTITIONED;
/* Tag the block so that we know it has been used when considering
other basic block notes. */
@ -620,11 +622,24 @@ rtl_merge_blocks (basic_block a, basic_block b)
static bool
rtl_can_merge_blocks (basic_block a,basic_block b)
{
bool partitions_ok = true;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& (find_reg_note (BB_END (a), REG_CROSSING_JUMP, NULL_RTX)
|| find_reg_note (BB_END (b), REG_CROSSING_JUMP, NULL_RTX)
|| a->partition != b->partition))
partitions_ok = false;
/* There must be exactly one edge in between the blocks. */
return (a->succ && !a->succ->succ_next && a->succ->dest == b
&& !b->pred->pred_next && a != b
/* Must be simple edge. */
&& !(a->succ->flags & EDGE_COMPLEX)
&& partitions_ok
&& a->next_bb == b
&& a != ENTRY_BLOCK_PTR && b != EXIT_BLOCK_PTR
/* If the jump insn has side effects,
@ -665,6 +680,15 @@ try_redirect_by_replacing_jump (edge e, basic_block target, bool in_cfglayout)
rtx set;
int fallthru = 0;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& find_reg_note (insn, REG_CROSSING_JUMP, NULL_RTX))
return false;
/* Verify that all targets will be TARGET. */
for (tmp = src->succ; tmp; tmp = tmp->succ_next)
if (tmp->dest != target && tmp != e)
@ -1066,6 +1090,33 @@ force_nonfallthru_and_redirect (edge e, basic_block target)
target->global_live_at_start);
}
/* Make sure new block ends up in correct hot/cold section. */
jump_block->partition = e->src->partition;
if (flag_reorder_blocks_and_partition)
{
if (e->src->partition == COLD_PARTITION)
{
rtx bb_note, new_note;
for (bb_note = BB_HEAD (jump_block);
bb_note && bb_note != NEXT_INSN (BB_END (jump_block));
bb_note = NEXT_INSN (bb_note))
if (GET_CODE (bb_note) == NOTE
&& NOTE_LINE_NUMBER (bb_note) == NOTE_INSN_BASIC_BLOCK)
break;
new_note = emit_note_after (NOTE_INSN_UNLIKELY_EXECUTED_CODE,
bb_note);
NOTE_BASIC_BLOCK (new_note) = jump_block;
jump_block->partition = COLD_PARTITION;
}
if (GET_CODE (BB_END (jump_block)) == JUMP_INSN
&& !any_condjump_p (BB_END (jump_block))
&& jump_block->succ->crossing_edge )
REG_NOTES (BB_END (jump_block)) = gen_rtx_EXPR_LIST
(REG_CROSSING_JUMP, NULL_RTX,
REG_NOTES (BB_END (jump_block)));
}
/* Wire edge in. */
new_edge = make_edge (e->src, jump_block, EDGE_FALLTHRU);
new_edge->probability = e->probability;
@ -1480,6 +1531,10 @@ commit_one_edge_insertion (edge e, int watch_calls)
tmp = NEXT_INSN (tmp);
if (NOTE_INSN_BASIC_BLOCK_P (tmp))
tmp = NEXT_INSN (tmp);
if (tmp
&& GET_CODE (tmp) == NOTE
&& NOTE_LINE_NUMBER (tmp) == NOTE_INSN_UNLIKELY_EXECUTED_CODE)
tmp = NEXT_INSN (tmp);
if (tmp == BB_HEAD (bb))
before = tmp;
else if (tmp)
@ -1522,6 +1577,38 @@ commit_one_edge_insertion (edge e, int watch_calls)
{
bb = split_edge (e);
after = BB_END (bb);
/* If we are partitioning hot/cold basic blocks, we must make sure
that the new basic block ends up in the correct section. */
bb->partition = e->src->partition;
if (flag_reorder_blocks_and_partition
&& e->src != ENTRY_BLOCK_PTR
&& e->src->partition == COLD_PARTITION)
{
rtx bb_note, new_note, cur_insn;
bb_note = NULL_RTX;
for (cur_insn = BB_HEAD (bb); cur_insn != NEXT_INSN (BB_END (bb));
cur_insn = NEXT_INSN (cur_insn))
if (GET_CODE (cur_insn) == NOTE
&& NOTE_LINE_NUMBER (cur_insn) == NOTE_INSN_BASIC_BLOCK)
{
bb_note = cur_insn;
break;
}
new_note = emit_note_after (NOTE_INSN_UNLIKELY_EXECUTED_CODE,
bb_note);
NOTE_BASIC_BLOCK (new_note) = bb;
if (GET_CODE (BB_END (bb)) == JUMP_INSN
&& !any_condjump_p (BB_END (bb))
&& bb->succ->crossing_edge )
REG_NOTES (BB_END (bb)) = gen_rtx_EXPR_LIST
(REG_CROSSING_JUMP, NULL_RTX, REG_NOTES (BB_END (bb)));
if (after == bb_note)
after = new_note;
}
}
}
@ -1791,6 +1878,7 @@ update_br_prob_note (basic_block bb)
- tails of basic blocks (ensure that boundary is necessary)
- scans body of the basic block for JUMP_INSN, CODE_LABEL
and NOTE_INSN_BASIC_BLOCK
- verify that no fall_thru edge crosses hot/cold partition boundaries
In future it can be extended check a lot of other stuff as well
(reachability of basic blocks, life information, etc. etc.). */
@ -1878,7 +1966,15 @@ rtl_verify_flow_info_1 (void)
for (e = bb->succ; e; e = e->succ_next)
{
if (e->flags & EDGE_FALLTHRU)
n_fallthru++, fallthru = e;
{
n_fallthru++, fallthru = e;
if (e->crossing_edge)
{
error ("Fallthru edge crosses section boundary (bb %i)",
e->src->index);
err = 1;
}
}
if ((e->flags & ~(EDGE_DFS_BACK
| EDGE_CAN_FALLTHRU
@ -2553,11 +2649,24 @@ cfg_layout_delete_block (basic_block bb)
static bool
cfg_layout_can_merge_blocks_p (basic_block a, basic_block b)
{
bool partitions_ok = true;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& (find_reg_note (BB_END (a), REG_CROSSING_JUMP, NULL_RTX)
|| find_reg_note (BB_END (b), REG_CROSSING_JUMP, NULL_RTX)
|| a->partition != b->partition))
partitions_ok = false;
/* There must be exactly one edge in between the blocks. */
return (a->succ && !a->succ->succ_next && a->succ->dest == b
&& !b->pred->pred_next && a != b
/* Must be simple edge. */
&& !(a->succ->flags & EDGE_COMPLEX)
&& partitions_ok
&& a != ENTRY_BLOCK_PTR && b != EXIT_BLOCK_PTR
/* If the jump insn has side effects,
we can't kill the edge. */

4
gcc/common.opt
View File

@ -569,6 +569,10 @@ freorder-blocks
Common
Reorder basic blocks to improve code placement
freorder-blocks-and-partition
Common
Reorder basic blocks and partition into hot and cold sections
freorder-functions
Common
Reorder functions to improve code placement

5
gcc/config/darwin.c
View File

@ -1304,7 +1304,10 @@ darwin_globalize_label (FILE *stream, const char *name)
void
darwin_asm_named_section (const char *name, unsigned int flags ATTRIBUTE_UNUSED)
{
fprintf (asm_out_file, ".section %s\n", name);
if (flag_reorder_blocks_and_partition)
fprintf (asm_out_file, SECTION_FORMAT_STRING, name);
else
fprintf (asm_out_file, ".section %s\n", name);
}
unsigned int

3
gcc/config/i386/i386.h
View File

@ -207,6 +207,9 @@ extern int target_flags;
#endif
#endif
#define HAS_LONG_COND_BRANCH 1
#define HAS_LONG_UNCOND_BRANCH 1
/* Avoid adding %gs:0 in TLS references; use %gs:address directly. */
#define TARGET_TLS_DIRECT_SEG_REFS (target_flags & MASK_TLS_DIRECT_SEG_REFS)

4
gcc/config/rs6000/darwin.h
View File

@ -145,8 +145,10 @@ do { \
/* These are used by -fbranch-probabilities */
#define HOT_TEXT_SECTION_NAME "__TEXT,__text,regular,pure_instructions"
#define NORMAL_TEXT_SECTION_NAME "__TEXT,__text,regular,pure_instructions"
#define UNLIKELY_EXECUTED_TEXT_SECTION_NAME \
"__TEXT,__text2,regular,pure_instructions"
"__TEXT,__unlikely,regular,pure_instructions"
#define SECTION_FORMAT_STRING ".section %s\n\t.align 2\n"
/* Define cutoff for using external functions to save floating point.
Currently on Darwin, always use inline stores. */

1
gcc/config/rs6000/rs6000.c
View File

@ -9702,6 +9702,7 @@ rs6000_assemble_integer (rtx x, unsigned int size, int aligned_p)
if (TARGET_RELOCATABLE
&& !in_toc_section ()
&& !in_text_section ()
&& !in_text_unlikely_section ()
&& !recurse
&& GET_CODE (x) != CONST_INT
&& GET_CODE (x) != CONST_DOUBLE

5
gcc/config/rs6000/sysv4.h
View File

@ -434,6 +434,11 @@ do { \
#define BSS_SECTION_ASM_OP "\t.section\t\".bss\""
#define HOT_TEXT_SECTION_NAME ".text"
#define NORMAL_TEXT_SECTION_NAME ".text"
#define UNLIKELY_EXECUTED_TEXT_SECTION_NAME ".text.unlikely"
#define SECTION_FORMAT_STRING ".section\t\"%s\"\n\t.align 2\n"
/* Override elfos.h definition. */
#undef INIT_SECTION_ASM_OP
#define INIT_SECTION_ASM_OP "\t.section\t\".init\",\"ax\""

8
gcc/dbxout.c
View File

@ -448,6 +448,9 @@ static void
dbxout_function_end (void)
{
char lscope_label_name[100];
function_section (current_function_decl);
/* Convert Ltext into the appropriate format for local labels in case
the system doesn't insert underscores in front of user generated
labels. */
@ -728,7 +731,10 @@ dbxout_source_file (FILE *file, const char *filename)
&& DECL_SECTION_NAME (current_function_decl) != NULL_TREE)
; /* Don't change section amid function. */
else
text_section ();
{
if (!in_text_section () && !in_unlikely_text_section ())
text_section ();
}
targetm.asm_out.internal_label (file, "Ltext", source_label_number);
source_label_number++;
lastfile = filename;

16
gcc/defaults.h
View File

@ -623,10 +623,26 @@ You Lose! You must define PREFERRED_DEBUGGING_TYPE!
#define HOT_TEXT_SECTION_NAME "text.hot"
#endif
#ifndef NORMAL_TEXT_SECTION_NAME
#define NORMAL_TEXT_SECTION_NAME ".text"
#endif
#ifndef UNLIKELY_EXECUTED_TEXT_SECTION_NAME
#define UNLIKELY_EXECUTED_TEXT_SECTION_NAME "text.unlikely"
#endif
#ifndef SECTION_FORMAT_STRING
#define SECTION_FORMAT_STRING "\t.section\t%s\n\t.align 2\n"
#endif
#ifndef HAS_LONG_COND_BRANCH
#define HAS_LONG_COND_BRANCH 0
#endif
#ifndef HAS_LONG_UNCOND_BRANCH
#define HAS_LONG_UNCOND_BRANCH 0
#endif
#ifndef VECTOR_MODE_SUPPORTED_P
#define VECTOR_MODE_SUPPORTED_P(MODE) 0
#endif

11
gcc/doc/invoke.texi
View File

@ -284,7 +284,7 @@ in the following sections.
-foptimize-sibling-calls -fprefetch-loop-arrays @gol
-fprofile-generate -fprofile-use @gol
-freduce-all-givs -fregmove -frename-registers @gol
-freorder-blocks -freorder-functions @gol
-freorder-blocks -freorder-blocks-and-partition -freorder-functions @gol
-frerun-cse-after-loop -frerun-loop-opt @gol
-frounding-math -fschedule-insns -fschedule-insns2 @gol
-fno-sched-interblock -fno-sched-spec -fsched-spec-load @gol
@ -3680,7 +3680,7 @@ optimizations designed to reduce code size.
@option{-Os} disables the following optimization flags:
@gccoptlist{-falign-functions -falign-jumps -falign-loops @gol
-falign-labels -freorder-blocks -fprefetch-loop-arrays}
-falign-labels -freorder-blocks -freorder-blocks-and-partition -fprefetch-loop-arrays}
If you use multiple @option{-O} options, with or without level numbers,
the last such option is the one that is effective.
@ -4206,6 +4206,13 @@ taken branches and improve code locality.
Enabled at levels @option{-O2}, @option{-O3}.
@item -freorder-blocks-and-partition
@opindex freorder-blocks-and-partition
In addition to reordering basic blocks in the compiled function, in order
to reduce number of taken branches, partitions hot and cold basic blocks
into separate sections of the assembly and .o files, to improve
paging and cache locality performance.
@item -freorder-functions
@opindex freorder-functions
Reorder basic blocks in the compiled function in order to reduce number of

10
gcc/doc/rtl.texi
View File

@ -3288,6 +3288,16 @@ This insn uses @var{op}, a @code{code_label} or a @code{note} of type
be held in a register. The presence of this note allows jump
optimization to be aware that @var{op} is, in fact, being used, and flow
optimization to build an accurate flow graph.
@findex REG_CROSSING_JUMP
@item REG_CROSSING_JUMP
This insn is an branching instruction (either an unconditional jump or
an indirect jump) which crosses between hot and cold sections, which
could potentially be very far apart in the executable. The presence
of this note indicates to other optimizations that this this branching
instruction should not be ``collapsed'' into a simpler branching
construct. It is used when the optimization to partition basic blocks
into hot and cold sections is turned on.
@end table
The following notes describe attributes of outputs of an insn:

31
gcc/doc/tm.texi
View File

@ -5802,6 +5802,13 @@ frequently executed functions of the program. If not defined, GCC will provide
a default definition if the target supports named sections.
@end defmac
@defmac NORMAL_TEXT_SECTION_NAME
If defined, a C string constant for the name of the section containing code
that does not go into the ``unlikely executed text'' partition. This is used
as part of the optimization that partitions hot and cold basic blocks into
separate sections in the .o and executable files.
@end defmac
@defmac UNLIKELY_EXECUTED_TEXT_SECTION_NAME
If defined, a C string constant for the name of the section containing unlikely
executed functions in the program.
@ -5853,6 +5860,12 @@ finalization code. If not defined, GCC will assume such a section does
not exist.
@end defmac
@defmac SECTION_FORMAT_STRING
If defined, format string used by fprintf to write out the
text section headers for the hot and cold sections of the
assembly file, when hot and cold partitioning is being performed.
@end defmac
@defmac CRT_CALL_STATIC_FUNCTION (@var{section_op}, @var{function})
If defined, an ASM statement that switches to a different section
via @var{section_op}, calls @var{function}, and switches back to
@ -8454,6 +8467,24 @@ For each predicate function named in @code{PREDICATE_CODES}, a
declaration will be generated in @file{insn-codes.h}.
@end defmac
@defmac HAS_LONG_COND_BRANCH
Define this boolean macro to indicate whether or not your architecture
has conditional branches that can span all of memory. It is used in
conjunction with an optimization that partitions hot and cold basic
blocks into separate sections of the executable. If this macro is
set to false, gcc will convert any conditional branches that attempt
to cross between sections into unconditional branches or indirect jumps.
@end defmac
@defmac HAS_LONG_UNCOND_BRANCH
Define this boolean macro to indicate whether or not your architecture
has unconditional branches that can span all of memory. It is used in
conjunction with an optimization that partitions hot and cold basic
blocks into separate sections of the executable. If this macro is
set to false, gcc will convert any unconditional branches that attempt
to cross between sections into indirect jumps.
@end defmac
@defmac SPECIAL_MODE_PREDICATES
Define this if you have special predicates that know special things
about modes. Genrecog will warn about certain forms of

74
gcc/final.c
View File

@ -1623,6 +1623,35 @@ output_alternate_entry_point (FILE *file, rtx insn)
}
}
/* Return boolean indicating if there is a NOTE_INSN_UNLIKELY_EXECUTED_CODE
note in the instruction chain (going forward) between the current
instruction, and the next 'executable' instruction. */
bool
scan_ahead_for_unlikely_executed_note (rtx insn)
{
rtx temp;
int bb_note_count = 0;
for (temp = insn; temp; temp = NEXT_INSN (temp))
{
if (GET_CODE (temp) == NOTE
&& NOTE_LINE_NUMBER (temp) == NOTE_INSN_UNLIKELY_EXECUTED_CODE)
return true;
if (GET_CODE (temp) == NOTE
&& NOTE_LINE_NUMBER (temp) == NOTE_INSN_BASIC_BLOCK)
{
bb_note_count++;
if (bb_note_count > 1)
return false;
}
if (INSN_P (temp))
return false;
}
return false;
}
/* The final scan for one insn, INSN.
Args are same as in `final', except that INSN
is the insn being scanned.
@ -1672,7 +1701,31 @@ final_scan_insn (rtx insn, FILE *file, int optimize ATTRIBUTE_UNUSED,
case NOTE_INSN_EXPECTED_VALUE:
break;
case NOTE_INSN_UNLIKELY_EXECUTED_CODE:
/* The presence of this note indicates that this basic block
belongs in the "cold" section of the .o file. If we are
not already writing to the cold section we need to change
to it. */
unlikely_text_section ();
break;
case NOTE_INSN_BASIC_BLOCK:
/* If we are performing the optimization that paritions
basic blocks into hot & cold sections of the .o file,
then at the start of each new basic block, before
beginning to write code for the basic block, we need to
check to see whether the basic block belongs in the hot
or cold section of the .o file, and change the section we
are writing to appropriately. */
if (flag_reorder_blocks_and_partition
&& in_unlikely_text_section()
&& !scan_ahead_for_unlikely_executed_note (insn))
text_section ();
#ifdef IA64_UNWIND_INFO
IA64_UNWIND_EMIT (asm_out_file, insn);
#endif
@ -1859,6 +1912,27 @@ final_scan_insn (rtx insn, FILE *file, int optimize ATTRIBUTE_UNUSED,
if (LABEL_NAME (insn))
(*debug_hooks->label) (insn);
/* If we are doing the optimization that partitions hot & cold
basic blocks into separate sections of the .o file, we need
to ensure the jump table ends up in the correct section... */
if (flag_reorder_blocks_and_partition)
{
rtx tmp_table, tmp_label;
if (GET_CODE (insn) == CODE_LABEL
&& tablejump_p (NEXT_INSN (insn), &tmp_label, &tmp_table))
{
/* Do nothing; Do NOT change the current section. */
}
else if (scan_ahead_for_unlikely_executed_note (insn))
unlikely_text_section ();
else
{
if (in_unlikely_text_section ())
text_section ();
}
}
if (app_on)
{
fputs (ASM_APP_OFF, file);

5
gcc/flags.h
View File

@ -210,6 +210,11 @@ extern int flag_branch_probabilities;
extern int flag_reorder_blocks;
/* Nonzero if basic blocks should be partitioned into hot and cold
sections of the .o file, in addition to being reordered. */
extern int flag_reorder_blocks_and_partition;
/* Nonzero if functions should be reordered. */
extern int flag_reorder_functions;

27
gcc/ifcvt.c
View File

@ -2851,6 +2851,18 @@ find_if_case_1 (basic_block test_bb, edge then_edge, edge else_edge)
edge then_succ = then_bb->succ;
int then_bb_index;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& ((BB_END (then_bb)
&& find_reg_note (BB_END (then_bb), REG_CROSSING_JUMP, NULL_RTX))
|| (BB_END (else_bb)
&& find_reg_note (BB_END (else_bb), REG_CROSSING_JUMP,
NULL_RTX))))
return FALSE;
/* THEN has one successor. */
if (!then_succ || then_succ->succ_next != NULL)
return FALSE;
@ -2919,6 +2931,18 @@ find_if_case_2 (basic_block test_bb, edge then_edge, edge else_edge)
edge else_succ = else_bb->succ;
rtx note;
/* If we are partitioning hot/cold basic blocks, we don't want to
mess up unconditional or indirect jumps that cross between hot
and cold sections. */
if (flag_reorder_blocks_and_partition
&& ((BB_END (then_bb)
&& find_reg_note (BB_END (then_bb), REG_CROSSING_JUMP, NULL_RTX))
|| (BB_END (else_bb)
&& find_reg_note (BB_END (else_bb), REG_CROSSING_JUMP,
NULL_RTX))))
return FALSE;
/* ELSE has one successor. */
if (!else_succ || else_succ->succ_next != NULL)
return FALSE;
@ -3263,7 +3287,8 @@ if_convert (int x_life_data_ok)
num_true_changes = 0;
life_data_ok = (x_life_data_ok != 0);
if (! targetm.cannot_modify_jumps_p ())
if ((! targetm.cannot_modify_jumps_p ())
&& (!flag_reorder_blocks_and_partition || !no_new_pseudos))
mark_loop_exit_edges ();
/* Free up basic_block_for_insn so that we don't have to keep it

18
gcc/opts.c
View File

@ -592,6 +592,7 @@ decode_options (unsigned int argc, const char **argv)
or less automatically remove extra jumps, but would also try to
use more short jumps instead of long jumps. */
flag_reorder_blocks = 0;
flag_reorder_blocks_and_partition = 0;
}
if (optimize_size)
@ -657,6 +658,19 @@ decode_options (unsigned int argc, const char **argv)
if (flag_really_no_inline == 2)
flag_really_no_inline = flag_no_inline;
/* The optimization to partition hot and cold basic blocks into separate
sections of the .o and executable files does not work (currently)
with exception handling. If flag_exceptions is turned on we need to
turn off the partitioning optimization. */
if (flag_exceptions && flag_reorder_blocks_and_partition)
{
warning
("-freorder-blocks-and-partition does not work with exceptions");
flag_reorder_blocks_and_partition = 0;
flag_reorder_blocks = 1;
}
}
/* Handle target- and language-independent options. Return zero to
@ -1268,6 +1282,10 @@ common_handle_option (size_t scode, const char *arg,
flag_reorder_blocks = value;
break;
case OPT_freorder_blocks_and_partition:
flag_reorder_blocks_and_partition = value;
break;
case OPT_freorder_functions:
flag_reorder_functions = value;
break;

6
gcc/output.h
View File

@ -162,6 +162,9 @@ extern void check_function_return_warnings (void);
/* Tell assembler to switch to text section. */
extern void text_section (void);
/* Tell assembler to switch to unlikely-to-be-executed text section. */
extern void unlikely_text_section (void);
/* Tell assembler to switch to data section. */
extern void data_section (void);
@ -172,6 +175,9 @@ extern void readonly_data_section (void);
/* Determine if we're in the text section. */
extern int in_text_section (void);
/* Determine if we're in the unlikely-to-be-executed text section. */
extern int in_unlikely_text_section (void);
#ifdef CTORS_SECTION_ASM_OP
extern void ctors_section (void);
#endif

20
gcc/passes.c
View File

@ -522,7 +522,7 @@ rest_of_handle_stack_regs (tree decl, rtx insns)
{
if (cleanup_cfg (CLEANUP_EXPENSIVE | CLEANUP_POST_REGSTACK
| (flag_crossjumping ? CLEANUP_CROSSJUMP : 0))
&& flag_reorder_blocks)
&& (flag_reorder_blocks || flag_reorder_blocks_and_partition))
{
reorder_basic_blocks ();
cleanup_cfg (CLEANUP_EXPENSIVE | CLEANUP_POST_REGSTACK);
@ -718,9 +718,9 @@ rest_of_handle_reorder_blocks (tree decl, rtx insns)
if (flag_sched2_use_traces && flag_schedule_insns_after_reload)
tracer ();
if (flag_reorder_blocks)
if (flag_reorder_blocks || flag_reorder_blocks_and_partition)
reorder_basic_blocks ();
if (flag_reorder_blocks
if (flag_reorder_blocks || flag_reorder_blocks_and_partition
|| (flag_sched2_use_traces && flag_schedule_insns_after_reload))
changed |= cleanup_cfg (CLEANUP_EXPENSIVE
| (!HAVE_conditional_execution
@ -1806,6 +1806,20 @@ rest_of_compilation (tree decl)
if (flag_if_conversion)
rest_of_handle_if_after_combine (decl, insns);
/* The optimization to partition hot/cold basic blocks into separate
sections of the .o file does not work well with exception handling.
Don't call it if there are exceptions. */
if (flag_reorder_blocks_and_partition && !flag_exceptions)
{
no_new_pseudos = 0;
partition_hot_cold_basic_blocks ();
allocate_reg_life_data ();
update_life_info (NULL, UPDATE_LIFE_GLOBAL_RM_NOTES,
PROP_LOG_LINKS | PROP_REG_INFO | PROP_DEATH_NOTES);
no_new_pseudos = 1;
}
if (optimize > 0 && (flag_regmove || flag_expensive_optimizations))
rest_of_handle_regmove (decl, insns);

8
gcc/print-rtl.c
View File

@ -291,6 +291,14 @@ print_rtx (rtx in_rtx)
fprintf (outfile, " [ ERROR ]");
break;
case NOTE_INSN_UNLIKELY_EXECUTED_CODE:
{
basic_block bb = NOTE_BASIC_BLOCK (in_rtx);
if (bb != 0)
fprintf (outfile, " [bb %d]", bb->index);
break;
}
case NOTE_INSN_VAR_LOCATION:
fprintf (outfile, " (");
print_mem_expr (outfile, NOTE_VAR_LOCATION_DECL (in_rtx));

6
gcc/rtl.c
View File

@ -122,7 +122,9 @@ const char * const note_insn_name[NOTE_INSN_MAX - NOTE_INSN_BIAS] =
"NOTE_INSN_EH_REGION_BEG", "NOTE_INSN_EH_REGION_END",
"NOTE_INSN_REPEATED_LINE_NUMBER",
"NOTE_INSN_BASIC_BLOCK", "NOTE_INSN_EXPECTED_VALUE",
"NOTE_INSN_PREDICTION", "NOTE_INSN_VAR_LOCATION"
"NOTE_INSN_PREDICTION",
"NOTE_INSN_UNLIKELY_EXECUTED_CODE",
"NOTE_INSN_VAR_LOCATION"
};
const char * const reg_note_name[] =
@ -134,7 +136,7 @@ const char * const reg_note_name[] =
"REG_VALUE_PROFILE", "REG_NOALIAS", "REG_SAVE_AREA", "REG_BR_PRED",
"REG_FRAME_RELATED_EXPR", "REG_EH_CONTEXT", "REG_EH_REGION",
"REG_SAVE_NOTE", "REG_MAYBE_DEAD", "REG_NORETURN",
"REG_NON_LOCAL_GOTO", "REG_SETJMP", "REG_ALWAYS_RETURN",
"REG_NON_LOCAL_GOTO", "REG_CROSSING_JUMP", "REG_SETJMP", "REG_ALWAYS_RETURN",
"REG_VTABLE_REF"
};

9
gcc/rtl.h
View File

@ -857,6 +857,11 @@ enum reg_note
computed goto. */
REG_NON_LOCAL_GOTO,
/* Indicates that a jump crosses between hot and cold sections
in a (partitioned) assembly or .o file, and therefore should not be
reduced to a simpler jump by optimizations. */
REG_CROSSING_JUMP,
/* This kind of note is generated at each to `setjmp',
and similar functions that can return twice. */
REG_SETJMP,
@ -1018,6 +1023,10 @@ enum insn_note
/* Record a prediction. Uses NOTE_PREDICTION. */
NOTE_INSN_PREDICTION,
/* Record that the current basic block is unlikely to be executed and
should be moved to the UNLIKELY_EXECUTED_TEXT_SECTION. */
NOTE_INSN_UNLIKELY_EXECUTED_CODE,
/* The location of a variable. */
NOTE_INSN_VAR_LOCATION,

6
gcc/toplev.c
View File

@ -249,6 +249,11 @@ int flag_branch_probabilities = 0;
int flag_reorder_blocks = 0;
/* Nonzero if blocks should be partitioned into hot and cold sections in
addition to being reordered. */
int flag_reorder_blocks_and_partition = 0;
/* Nonzero if functions should be reordered. */
int flag_reorder_functions = 0;
@ -955,6 +960,7 @@ static const lang_independent_options f_options[] =
{"branch-probabilities", &flag_branch_probabilities, 1 },
{"profile", &profile_flag, 1 },
{"reorder-blocks", &flag_reorder_blocks, 1 },
{"reorder-blocks-and-partition", &flag_reorder_blocks_and_partition, 1},
{"reorder-functions", &flag_reorder_functions, 1 },
{"rename-registers", &flag_rename_registers, 1 },
{"cprop-registers", &flag_cprop_registers, 1 },

71
gcc/varasm.c
View File

@ -49,6 +49,7 @@ Software Foundation, 59 Temple Place - Suite 330, Boston, MA
#include "debug.h"
#include "target.h"
#include "cgraph.h"
#include "cfglayout.h"
#ifdef XCOFF_DEBUGGING_INFO
#include "xcoffout.h" /* Needed for external data
@ -100,6 +101,14 @@ int size_directive_output;
tree last_assemble_variable_decl;
/* The following global variable indicates if the section label for the
"cold" section of code has been output yet to the assembler. The
label is useful when running gdb. This is part of the optimization that
partitions hot and cold basic blocks into separate sections of the .o
file. */
bool unlikely_section_label_printed = false;
/* RTX_UNCHANGING_P in a MEM can mean it is stored into, for initialization.
So giving constant the alias set for the type will allow such
initializations to appear to conflict with the load of the constant. We
@ -145,7 +154,8 @@ static bool asm_emit_uninitialised (tree, const char*,
unsigned HOST_WIDE_INT);
static void mark_weak (tree);
enum in_section { no_section, in_text, in_data, in_named
enum in_section { no_section, in_text, in_unlikely_executed_text, in_data,
in_named
#ifdef BSS_SECTION_ASM_OP
, in_bss
#endif
@ -198,7 +208,34 @@ text_section (void)
if (in_section != in_text)
{
in_section = in_text;
fprintf (asm_out_file, "%s\n", TEXT_SECTION_ASM_OP);
fprintf (asm_out_file, SECTION_FORMAT_STRING, NORMAL_TEXT_SECTION_NAME);
}
}
/* Tell assembler to switch to unlikely-to-be-executed text section. */
void
unlikely_text_section (void)
{
if ((in_section != in_unlikely_executed_text)
&& (in_section != in_named
|| strcmp (in_named_name, UNLIKELY_EXECUTED_TEXT_SECTION_NAME) != 0))
{
#ifdef TARGET_ASM_NAMED_SECTION
named_section (NULL_TREE, UNLIKELY_EXECUTED_TEXT_SECTION_NAME, 0);
#else
in_section = in_unlikely_executed_text;
fprintf (asm_out_file, SECTION_FORMAT_STRING,
UNLIKELY_EXECUTED_TEXT_SECTION_NAME);
#endif /* ifdef TARGET_ASM_NAMED_SECTION */
if (!unlikely_section_label_printed)
{
fprintf (asm_out_file, "__%s_unlikely_section:\n",
current_function_name ());
unlikely_section_label_printed = true;
}
}
}
@ -244,6 +281,14 @@ in_text_section (void)
return in_section == in_text;
}
/* Determine if we're in the unlikely-to-be-executed text section. */
int
in_unlikely_text_section (void)
{
return in_section == in_unlikely_executed_text;
}
/* Determine if we're in the data section. */
int
@ -483,11 +528,16 @@ asm_output_aligned_bss (FILE *file, tree decl ATTRIBUTE_UNUSED,
void
function_section (tree decl)
{
if (decl != NULL_TREE
&& DECL_SECTION_NAME (decl) != NULL_TREE)
named_section (decl, (char *) 0, 0);
if (scan_ahead_for_unlikely_executed_note (get_insns()))
unlikely_text_section ();
else
text_section ();
{
if (decl != NULL_TREE
&& DECL_SECTION_NAME (decl) != NULL_TREE)
named_section (decl, (char *) 0, 0);
else
text_section ();
}
}
/* Switch to section for variable DECL. RELOC is the same as the
@ -1030,6 +1080,8 @@ assemble_start_function (tree decl, const char *fnname)
{
int align;
unlikely_section_label_printed = false;
/* The following code does not need preprocessing in the assembler. */
app_disable ();
@ -1117,7 +1169,8 @@ assemble_zeros (unsigned HOST_WIDE_INT size)
#ifdef ASM_NO_SKIP_IN_TEXT
/* The `space' pseudo in the text section outputs nop insns rather than 0s,
so we must output 0s explicitly in the text section. */
if (ASM_NO_SKIP_IN_TEXT && in_text_section ())
if ((ASM_NO_SKIP_IN_TEXT && in_text_section ())
|| (ASM_NO_SKIP_IN_TEXT && in_unlikely_text_section ()))
{
unsigned HOST_WIDE_INT i;
for (i = 0; i < size; i++)
@ -1479,7 +1532,7 @@ assemble_variable (tree decl, int top_level ATTRIBUTE_UNUSED,
variable_section (decl, reloc);
/* dbxout.c needs to know this. */
if (in_text_section ())
if (in_text_section () || in_unlikely_text_section ())
DECL_IN_TEXT_SECTION (decl) = 1;
/* Output the alignment of this data. */
@ -4328,6 +4381,8 @@ default_section_type_flags_1 (tree decl, const char *name, int reloc,
flags = SECTION_CODE;
else if (decl && decl_readonly_section_1 (decl, reloc, shlib))
flags = 0;
else if (strcmp (name, UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0)
flags = SECTION_CODE;
else
flags = SECTION_WRITE;