OpenE2K/gcc
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Correct fixes for various hot/cold partitioning concerns.

Browse Source 
2005-04-27  Caroline Tice  <ctice@apple.com>

        * bb-reorder.c (find_rarely_executed_basic_blocks_and_crossing_edges):
        Remove targetm.have_named_sections test.
        (fix_edges_for_rarely_executed_code): Likewise.
        (insert_section_boundary_note): Likewise.
        (reorder_basic_blocks): Check partitioning flag before calling
        verify_hot_cold_block_grouping.
        * dbxout.c (dbxout_function_end): Get hot/cold section labels from
        the function struct rather than global variables.
        * dwarf2out.c (COLD_TEXT_SECTION_LABEL): New macro.
        (COLD_END_LABEL): Likewise
        (cold_text_section_label): New static global variable.
        (cold_end_label): Likewise.
        (dwarf2out_switch_text_section): Get hot/cold section labels from
        the function struct rather than global variables; test to make sure
        cfun is defined.
        (output_aranges): Use cold_text_section_label and cold_end_label;
        check partitioning flag before putting out delta.
        (output_ranges): Remove incorrect code attempting to use
        hot/cold labels.
        (output_line_info): Get cold section label from function struct; test
        to make sure cfun is defined.
        (add_location_or_const_value_attribute): Likewise.
        (dwarf2out_var_location): Likewise.
        (dwarf2out_init): Generate cold_text_section_label and cold_end_label;
        write out cold_text_section_label if partition flag is set.
        (dwarf2out_finish): Write out cold_end_label if partition flag is set;
        * function.h (struct function): Add new fields to point to hot/cold
        section labels: hot_section_label, cold_section_label,
        hot_section_end_label and cold_section_end_label; also add new field
        for cold text section name, unlikely_text_section_name.
        * opts.c (decode_options): Turn off partitioning flag if
        !targetm.have_named_sections.
        * output.h (hot_section_label): Remove.
        (hot_section_end_label): Remove.
        (cold_section_end_label): Remove.
        (unlikely_section_label): Remove.
        (unlikely_text_section_name): Remove.
        * passes.c (rest_of_handle_final):  Remove extra blank line.
        * varasm.c (unlikely_section_label): Remove.
        (hot_section_label): Remove.
        (hot_section_end_label): Remove.
        (cold_section_end_label): Remove.
        (unlikely_text_section_name):  Remove.
        (initialize_cold_section_name): Modify to call
        targetm.strip_name_encoding; to store cold section name in current
        function struct, if it exists; and to only use the decl_section_name
        if flag_named_sections is true.
        (unlikely_text_section): Modify to get section name out of current
        function struct, if there is one; otherwise build it from
        UNLIKELY_EXECUTED_TEXT_SECTION_NAME.
        (in_unlikely_text_section): Likewise.
        (named_section): Modify to get/put cold section name in current function
        struct, if there is one.
        (function_section):  Change 'bool unlikely' to 'int reloc'; check
        targetm.have_named_sections before calling named_section.
        (current_function_section): Likewise.
        (assemble_start_function): Modify to get/put unlikely_text_section_name
        in current function struct; modify to get hot/cold section labels
        from function struct; initialize labels using
        ASM_GENERATE_INTERNAL_LABEL;
        test partitioning flag before writing out hot section label.
        (assemble_end_function): Test partitioning flag before writing out
        hot/cold section labels.
        (default_section_type_flags_1):  Get cold text section name from
        function struct if there is one; Set flags correctly for
        cold text section if there is not a current function struct.

From-SVN: r98885
This commit is contained in:
Caroline Tice 2005-04-27 21:35:20 +00:00 committed by Caroline Tice
parent 32cf6a3bfe
commit c7466deed4
9 changed files with 304 additions and 196 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

69
gcc/ChangeLog
View File

@ -1,3 +1,72 @@
2005-04-27 Caroline Tice <ctice@apple.com>
* bb-reorder.c (find_rarely_executed_basic_blocks_and_crossing_edges):
Remove targetm.have_named_sections test.
(fix_edges_for_rarely_executed_code): Likewise.
(insert_section_boundary_note): Likewise.
(reorder_basic_blocks): Check partitioning flag before calling
verify_hot_cold_block_grouping.
* dbxout.c (dbxout_function_end): Get hot/cold section labels from
the function struct rather than global variables.
* dwarf2out.c (COLD_TEXT_SECTION_LABEL): New macro.
(COLD_END_LABEL): Likewise
(cold_text_section_label): New static global variable.
(cold_end_label): Likewise.
(dwarf2out_switch_text_section): Get hot/cold section labels from
the function struct rather than global variables; test to make sure
cfun is defined.
(output_aranges): Use cold_text_section_label and cold_end_label;
check partitioning flag before putting out delta.
(output_ranges): Remove incorrect code attempting to use
hot/cold labels.
(output_line_info): Get cold section label from function struct; test
to make sure cfun is defined.
(add_location_or_const_value_attribute): Likewise.
(dwarf2out_var_location): Likewise.
(dwarf2out_init): Generate cold_text_section_label and cold_end_label;
write out cold_text_section_label if partition flag is set.
(dwarf2out_finish): Write out cold_end_label if partition flag is set;
* function.h (struct function): Add new fields to point to hot/cold
section labels: hot_section_label, cold_section_label,
hot_section_end_label and cold_section_end_label; also add new field
for cold text section name, unlikely_text_section_name.
* opts.c (decode_options): Turn off partitioning flag if
!targetm.have_named_sections.
* output.h (hot_section_label): Remove.
(hot_section_end_label): Remove.
(cold_section_end_label): Remove.
(unlikely_section_label): Remove.
(unlikely_text_section_name): Remove.
* passes.c (rest_of_handle_final): Remove extra blank line.
* varasm.c (unlikely_section_label): Remove.
(hot_section_label): Remove.
(hot_section_end_label): Remove.
(cold_section_end_label): Remove.
(unlikely_text_section_name): Remove.
(initialize_cold_section_name): Modify to call
targetm.strip_name_encoding; to store cold section name in current
function struct, if it exists; and to only use the decl_section_name
if flag_named_sections is true.
(unlikely_text_section): Modify to get section name out of current
function struct, if there is one; otherwise build it from
UNLIKELY_EXECUTED_TEXT_SECTION_NAME.
(in_unlikely_text_section): Likewise.
(named_section): Modify to get/put cold section name in current function
struct, if there is one.
(function_section): Change 'bool unlikely' to 'int reloc'; check
targetm.have_named_sections before calling named_section.
(current_function_section): Likewise.
(assemble_start_function): Modify to get/put unlikely_text_section_name
in current function struct; modify to get hot/cold section labels
from function struct; initialize labels using
ASM_GENERATE_INTERNAL_LABEL;
test partitioning flag before writing out hot section label.
(assemble_end_function): Test partitioning flag before writing out
hot/cold section labels.
(default_section_type_flags_1): Get cold text section name from
function struct if there is one; Set flags correctly for
cold text section if there is not a current function struct.
2005-04-27 Richard Guenther <rguenth@gcc.gnu.org> 2005-04-27 Richard Guenther <rguenth@gcc.gnu.org>
* tree-ssa-propagate.c (set_rhs): Revert last change. * tree-ssa-propagate.c (set_rhs): Revert last change.

17
gcc/bb-reorder.c
View File

@ -1237,8 +1237,6 @@ find_rarely_executed_basic_blocks_and_crossing_edges (edge *crossing_edges,
/* Mark every edge that crosses between sections. */ /* Mark every edge that crosses between sections. */
i = 0; i = 0;
if (targetm.have_named_sections)
{
FOR_EACH_BB (bb) FOR_EACH_BB (bb)
FOR_EACH_EDGE (e, ei, bb->succs) FOR_EACH_EDGE (e, ei, bb->succs)
{ {
@ -1258,7 +1256,6 @@ find_rarely_executed_basic_blocks_and_crossing_edges (edge *crossing_edges,
else else
e->flags &= ~EDGE_CROSSING; e->flags &= ~EDGE_CROSSING;
} }
}
*n_crossing_edges = i; *n_crossing_edges = i;
} }
@ -1821,15 +1818,6 @@ fix_edges_for_rarely_executed_code (edge *crossing_edges,
fix_up_fall_thru_edges (); fix_up_fall_thru_edges ();
/* Only do the parts necessary for writing separate sections if
the target architecture has the ability to write separate sections
(i.e. it has named sections). Otherwise, the hot/cold partitioning
information will be used when reordering blocks to try to put all
the hot blocks together, then all the cold blocks, but no actual
section partitioning will be done. */
if (targetm.have_named_sections)
{
/* If the architecture does not have conditional branches that can /* If the architecture does not have conditional branches that can
span all of memory, convert crossing conditional branches into span all of memory, convert crossing conditional branches into
crossing unconditional branches. */ crossing unconditional branches. */
@ -1850,7 +1838,6 @@ fix_edges_for_rarely_executed_code (edge *crossing_edges,
} }
add_reg_crossing_jump_notes (); add_reg_crossing_jump_notes ();
}
} }
/* Verify, in the basic block chain, that there is at most one switch /* Verify, in the basic block chain, that there is at most one switch
@ -1942,6 +1929,7 @@ reorder_basic_blocks (unsigned int flags)
dump_flow_info (dump_file); dump_flow_info (dump_file);
cfg_layout_finalize (); cfg_layout_finalize ();
if (flag_reorder_blocks_and_partition)
verify_hot_cold_block_grouping (); verify_hot_cold_block_grouping ();
timevar_pop (TV_REORDER_BLOCKS); timevar_pop (TV_REORDER_BLOCKS);
@ -1962,8 +1950,7 @@ insert_section_boundary_note (void)
rtx new_note; rtx new_note;
int first_partition = 0; int first_partition = 0;
if (flag_reorder_blocks_and_partition if (flag_reorder_blocks_and_partition)
&& targetm.have_named_sections)
FOR_EACH_BB (bb) FOR_EACH_BB (bb)
{ {
if (!first_partition) if (!first_partition)

7
gcc/dbxout.c
View File

@ -939,10 +939,11 @@ dbxout_function_end (tree decl)
if (flag_reorder_blocks_and_partition) if (flag_reorder_blocks_and_partition)
{ {
dbxout_begin_empty_stabs (N_FUN); dbxout_begin_empty_stabs (N_FUN);
dbxout_stab_value_label_diff (hot_section_end_label, hot_section_label); dbxout_stab_value_label_diff (cfun->hot_section_end_label,
cfun->hot_section_label);
dbxout_begin_empty_stabs (N_FUN); dbxout_begin_empty_stabs (N_FUN);
dbxout_stab_value_label_diff (cold_section_end_label, dbxout_stab_value_label_diff (cfun->cold_section_end_label,
unlikely_section_label); cfun->cold_section_label);
} }
else else
{ {

81
gcc/dwarf2out.c
View File

@ -4122,6 +4122,9 @@ static int maybe_emit_file (int);
#ifndef TEXT_SECTION_LABEL #ifndef TEXT_SECTION_LABEL
#define TEXT_SECTION_LABEL "Ltext" #define TEXT_SECTION_LABEL "Ltext"
#endif #endif
#ifndef COLD_TEXT_SECTION_LABEL
#define COLD_TEXT_SECTION_LABEL "Ltext_cold"
#endif
#ifndef DEBUG_LINE_SECTION_LABEL #ifndef DEBUG_LINE_SECTION_LABEL
#define DEBUG_LINE_SECTION_LABEL "Ldebug_line" #define DEBUG_LINE_SECTION_LABEL "Ldebug_line"
#endif #endif
@ -4149,6 +4152,8 @@ static int maybe_emit_file (int);
static char text_end_label[MAX_ARTIFICIAL_LABEL_BYTES]; static char text_end_label[MAX_ARTIFICIAL_LABEL_BYTES];
static char text_section_label[MAX_ARTIFICIAL_LABEL_BYTES]; static char text_section_label[MAX_ARTIFICIAL_LABEL_BYTES];
static char cold_text_section_label[MAX_ARTIFICIAL_LABEL_BYTES];
static char cold_end_label[MAX_ARTIFICIAL_LABEL_BYTES];
static char abbrev_section_label[MAX_ARTIFICIAL_LABEL_BYTES]; static char abbrev_section_label[MAX_ARTIFICIAL_LABEL_BYTES];
static char debug_info_section_label[MAX_ARTIFICIAL_LABEL_BYTES]; static char debug_info_section_label[MAX_ARTIFICIAL_LABEL_BYTES];
static char debug_line_section_label[MAX_ARTIFICIAL_LABEL_BYTES]; static char debug_line_section_label[MAX_ARTIFICIAL_LABEL_BYTES];
@ -4159,6 +4164,9 @@ static char ranges_section_label[2 * MAX_ARTIFICIAL_LABEL_BYTES];
#ifndef TEXT_END_LABEL #ifndef TEXT_END_LABEL
#define TEXT_END_LABEL "Letext" #define TEXT_END_LABEL "Letext"
#endif #endif
#ifndef COLD_END_LABEL
#define COLD_END_LABEL "Letext_cold"
#endif
#ifndef BLOCK_BEGIN_LABEL #ifndef BLOCK_BEGIN_LABEL
#define BLOCK_BEGIN_LABEL "LBB" #define BLOCK_BEGIN_LABEL "LBB"
#endif #endif
@ -6800,12 +6808,15 @@ dwarf2out_switch_text_section (void)
{ {
dw_fde_ref fde; dw_fde_ref fde;
if (!cfun)
internal_error ("Attempt to switch text sections without any code.");
fde = &fde_table[fde_table_in_use - 1]; fde = &fde_table[fde_table_in_use - 1];
fde->dw_fde_switched_sections = true; fde->dw_fde_switched_sections = true;
fde->dw_fde_hot_section_label = xstrdup (hot_section_label); fde->dw_fde_hot_section_label = cfun->hot_section_label;
fde->dw_fde_hot_section_end_label = xstrdup (hot_section_end_label); fde->dw_fde_hot_section_end_label = cfun->hot_section_end_label;
fde->dw_fde_unlikely_section_label = xstrdup (unlikely_section_label); fde->dw_fde_unlikely_section_label = cfun->cold_section_label;
fde->dw_fde_unlikely_section_end_label = xstrdup (cold_section_end_label); fde->dw_fde_unlikely_section_end_label = cfun->cold_section_end_label;
separate_line_info_table_in_use++; separate_line_info_table_in_use++;
} }
@ -7235,14 +7246,15 @@ output_aranges (void)
} }
dw2_asm_output_addr (DWARF2_ADDR_SIZE, text_section_label, "Address"); dw2_asm_output_addr (DWARF2_ADDR_SIZE, text_section_label, "Address");
if (last_text_section == in_unlikely_executed_text
|| (last_text_section == in_named
&& last_text_section_name == unlikely_text_section_name))
dw2_asm_output_delta (DWARF2_ADDR_SIZE, text_end_label,
unlikely_section_label, "Length");
else
dw2_asm_output_delta (DWARF2_ADDR_SIZE, text_end_label, dw2_asm_output_delta (DWARF2_ADDR_SIZE, text_end_label,
text_section_label, "Length"); text_section_label, "Length");
if (flag_reorder_blocks_and_partition)
{
dw2_asm_output_addr (DWARF2_ADDR_SIZE, cold_text_section_label,
"Address");
dw2_asm_output_delta (DWARF2_ADDR_SIZE, cold_end_label,
cold_text_section_label, "Length");
}
for (i = 0; i < arange_table_in_use; i++) for (i = 0; i < arange_table_in_use; i++)
{ {
@ -7331,18 +7343,6 @@ output_ranges (void)
unit base address defaults to DW_AT_low_pc, which is the unit base address defaults to DW_AT_low_pc, which is the
base of the text section. */ base of the text section. */
if (separate_line_info_table_in_use == 0) if (separate_line_info_table_in_use == 0)
{
if (last_text_section == in_unlikely_executed_text
|| (last_text_section == in_named
&& last_text_section_name == unlikely_text_section_name))
{
dw2_asm_output_delta (DWARF2_ADDR_SIZE, blabel,
unlikely_section_label,
fmt, i * 2 * DWARF2_ADDR_SIZE);
dw2_asm_output_delta (DWARF2_ADDR_SIZE, elabel,
unlikely_section_label, NULL);
}
else
{ {
dw2_asm_output_delta (DWARF2_ADDR_SIZE, blabel, dw2_asm_output_delta (DWARF2_ADDR_SIZE, blabel,
text_section_label, text_section_label,
@ -7350,7 +7350,6 @@ output_ranges (void)
dw2_asm_output_delta (DWARF2_ADDR_SIZE, elabel, dw2_asm_output_delta (DWARF2_ADDR_SIZE, elabel,
text_section_label, NULL); text_section_label, NULL);
} }
}
/* Otherwise, we add a DW_AT_entry_pc attribute to force the /* Otherwise, we add a DW_AT_entry_pc attribute to force the
compilation unit base address to zero, which allows us to compilation unit base address to zero, which allows us to
@ -7735,10 +7734,11 @@ output_line_info (void)
current_file = 1; current_file = 1;
current_line = 1; current_line = 1;
if (last_text_section == in_unlikely_executed_text if (cfun
&& (last_text_section == in_unlikely_executed_text
|| (last_text_section == in_named || (last_text_section == in_named
&& last_text_section_name == unlikely_text_section_name)) && last_text_section_name == cfun->unlikely_text_section_name)))
strcpy (prev_line_label, unlikely_section_label); strcpy (prev_line_label, cfun->cold_section_label);
else else
strcpy (prev_line_label, text_section_label); strcpy (prev_line_label, text_section_label);
for (lt_index = 1; lt_index < line_info_table_in_use; ++lt_index) for (lt_index = 1; lt_index < line_info_table_in_use; ++lt_index)
@ -10112,7 +10112,6 @@ add_location_or_const_value_attribute (dw_die_ref die, tree decl,
dw_loc_list_ref list; dw_loc_list_ref list;
rtx varloc; rtx varloc;
/* We need to figure out what section we should use as the base /* We need to figure out what section we should use as the base
for the address ranges where a given location is valid. for the address ranges where a given location is valid.
1. If this particular DECL has a section associated with it, 1. If this particular DECL has a section associated with it,
@ -10134,10 +10133,12 @@ add_location_or_const_value_attribute (dw_die_ref die, tree decl,
tree sectree = DECL_SECTION_NAME (current_function_decl); tree sectree = DECL_SECTION_NAME (current_function_decl);
secname = TREE_STRING_POINTER (sectree); secname = TREE_STRING_POINTER (sectree);
} }
else if (last_text_section == in_unlikely_executed_text else if (cfun
&& (last_text_section == in_unlikely_executed_text
|| (last_text_section == in_named || (last_text_section == in_named
&& last_text_section_name == unlikely_text_section_name)) && last_text_section_name ==
secname = unlikely_section_label; cfun->unlikely_text_section_name)))
secname = cfun->cold_section_label;
else else
secname = text_section_label; secname = text_section_label;
@ -13260,10 +13261,11 @@ dwarf2out_var_location (rtx loc_note)
newloc->var_loc_note = loc_note; newloc->var_loc_note = loc_note;
newloc->next = NULL; newloc->next = NULL;
if (last_text_section == in_unlikely_executed_text if (cfun
&& (last_text_section == in_unlikely_executed_text
|| (last_text_section == in_named || (last_text_section == in_named
&& last_text_section_name == unlikely_text_section_name)) && last_text_section_name == cfun->unlikely_text_section_name)))
newloc->section_label = unlikely_section_label; newloc->section_label = cfun->cold_section_label;
else else
newloc->section_label = text_section_label; newloc->section_label = text_section_label;
@ -13501,6 +13503,9 @@ dwarf2out_init (const char *filename ATTRIBUTE_UNUSED)
ASM_GENERATE_INTERNAL_LABEL (abbrev_section_label, ASM_GENERATE_INTERNAL_LABEL (abbrev_section_label,
DEBUG_ABBREV_SECTION_LABEL, 0); DEBUG_ABBREV_SECTION_LABEL, 0);
ASM_GENERATE_INTERNAL_LABEL (text_section_label, TEXT_SECTION_LABEL, 0); ASM_GENERATE_INTERNAL_LABEL (text_section_label, TEXT_SECTION_LABEL, 0);
ASM_GENERATE_INTERNAL_LABEL (cold_text_section_label,
COLD_TEXT_SECTION_LABEL, 0);
ASM_GENERATE_INTERNAL_LABEL (cold_end_label, COLD_END_LABEL, 0);
ASM_GENERATE_INTERNAL_LABEL (debug_info_section_label, ASM_GENERATE_INTERNAL_LABEL (debug_info_section_label,
DEBUG_INFO_SECTION_LABEL, 0); DEBUG_INFO_SECTION_LABEL, 0);
@ -13525,6 +13530,11 @@ dwarf2out_init (const char *filename ATTRIBUTE_UNUSED)
text_section (); text_section ();
ASM_OUTPUT_LABEL (asm_out_file, text_section_label); ASM_OUTPUT_LABEL (asm_out_file, text_section_label);
if (flag_reorder_blocks_and_partition)
{
unlikely_text_section ();
ASM_OUTPUT_LABEL (asm_out_file, cold_text_section_label);
}
} }
/* A helper function for dwarf2out_finish called through /* A helper function for dwarf2out_finish called through
@ -13856,6 +13866,11 @@ dwarf2out_finish (const char *filename)
/* Output a terminator label for the .text section. */ /* Output a terminator label for the .text section. */
text_section (); text_section ();
targetm.asm_out.internal_label (asm_out_file, TEXT_END_LABEL, 0); targetm.asm_out.internal_label (asm_out_file, TEXT_END_LABEL, 0);
if (flag_reorder_blocks_and_partition)
{
unlikely_text_section ();
targetm.asm_out.internal_label (asm_out_file, COLD_END_LABEL, 0);
}
/* Output the source line correspondence table. We must do this /* Output the source line correspondence table. We must do this
even if there is no line information. Otherwise, on an empty even if there is no line information. Otherwise, on an empty

14
gcc/function.h
View File

@ -352,6 +352,20 @@ struct function GTY(())
/* The variables unexpanded so far. */ /* The variables unexpanded so far. */
tree unexpanded_var_list; tree unexpanded_var_list;
/* Assembly labels for the hot and cold text sections, to
be used by debugger functions for determining the size of text
sections. */
const char * hot_section_label;
const char * cold_section_label;
const char * hot_section_end_label;
const char * cold_section_end_label;
/* String to be used for name of cold text sections, via
targetm.asm_out.named_section. */
const char *unlikely_text_section_name;
/* Collected bit flags. */ /* Collected bit flags. */
/* Nonzero if function being compiled needs to be given an address /* Nonzero if function being compiled needs to be given an address

9
gcc/opts.c
View File

@ -678,6 +678,15 @@ decode_options (unsigned int argc, const char **argv)
flag_reorder_blocks_and_partition = 0; flag_reorder_blocks_and_partition = 0;
flag_reorder_blocks = 1; flag_reorder_blocks = 1;
} }
if (flag_reorder_blocks_and_partition
&& !targetm.have_named_sections)
{
inform
("-freorder-blocks-and-partition does not work on this architecture.");
flag_reorder_blocks_and_partition = 0;
flag_reorder_blocks = 1;
}
} }
/* Handle target- and language-independent options. Return zero to /* Handle target- and language-independent options. Return zero to

5
gcc/output.h
View File

@ -453,11 +453,6 @@ enum in_section { no_section, in_text, in_unlikely_executed_text, in_data,
#endif #endif
}; };
extern char *unlikely_section_label;
extern char *hot_section_label;
extern char *hot_section_end_label;
extern char *cold_section_end_label;
extern char *unlikely_text_section_name;
extern const char *last_text_section_name; extern const char *last_text_section_name;
extern enum in_section last_text_section; extern enum in_section last_text_section;
extern bool first_function_block_is_cold; extern bool first_function_block_is_cold;

1
gcc/passes.c
View File

@ -328,7 +328,6 @@ rest_of_handle_final (void)
timevar_push (TV_SYMOUT); timevar_push (TV_SYMOUT);
(*debug_hooks->function_decl) (current_function_decl); (*debug_hooks->function_decl) (current_function_decl);
timevar_pop (TV_SYMOUT); timevar_pop (TV_SYMOUT);
ggc_collect (); ggc_collect ();

185
gcc/varasm.c
View File

@ -101,40 +101,6 @@ tree last_assemble_variable_decl;
bool first_function_block_is_cold; bool first_function_block_is_cold;
/* The following global variable indicates the label name to be put at
the start of the first cold section within each function, when
partitioning basic blocks into hot and cold sections. Used for
debug info. */
char *unlikely_section_label;
/* The following global variable indicates the label name to be put at
the start of the first hot section within each function, when
partitioning basic blocks into hot and cold sections. Used for
debug info. */
char *hot_section_label;
/* The following global variable indicates the label name to be put at
the end of the last hot section within each function, when
partitioning basic blocks into hot and cold sections. Used for
debug info. */
char *hot_section_end_label;
/* The following global variable indicates the label name to be put at
the end of the last cold section within each function, when
partitioning basic blocks into hot and cold sections. Used for
debug info.*/
char *cold_section_end_label;
/* The following global variable indicates the section name to be used
for the current cold section, when partitioning hot and cold basic
blocks into separate sections. */
char *unlikely_text_section_name;
/* We give all constants their own alias set. Perhaps redundant with /* We give all constants their own alias set. Perhaps redundant with
MEM_READONLY_P, but pre-dates it. */ MEM_READONLY_P, but pre-dates it. */
@ -211,28 +177,36 @@ static void
initialize_cold_section_name (void) initialize_cold_section_name (void)
{ {
const char *name; const char *name;
const char *stripped_name;
char *buffer;
int len; int len;
if (! unlikely_text_section_name) if (cfun
&& current_function_decl)
{ {
if (DECL_SECTION_NAME (current_function_decl) if (!cfun->unlikely_text_section_name)
&& (strcmp (TREE_STRING_POINTER (DECL_SECTION_NAME
(current_function_decl)),
HOT_TEXT_SECTION_NAME) != 0)
&& (strcmp (TREE_STRING_POINTER (DECL_SECTION_NAME
(current_function_decl)),
UNLIKELY_EXECUTED_TEXT_SECTION_NAME) != 0))
{ {
name = TREE_STRING_POINTER (DECL_SECTION_NAME if (flag_function_sections
(current_function_decl)); && DECL_SECTION_NAME (current_function_decl))
len = strlen (name); {
unlikely_text_section_name = xmalloc (len + 10); name = xstrdup (TREE_STRING_POINTER (DECL_SECTION_NAME
sprintf (unlikely_text_section_name, "%s%s", name, "_unlikely"); (current_function_decl)));
stripped_name = targetm.strip_name_encoding (name);
len = strlen (stripped_name);
buffer = (char *) xmalloc (len + 10);
sprintf (buffer, "%s%s", stripped_name, "_unlikely");
cfun->unlikely_text_section_name = ggc_strdup (buffer);
free (buffer);
free ((char *) name);
} }
else else
unlikely_text_section_name = cfun->unlikely_text_section_name =
xstrdup (UNLIKELY_EXECUTED_TEXT_SECTION_NAME); UNLIKELY_EXECUTED_TEXT_SECTION_NAME;
} }
}
else
internal_error
("initialize_cold_section_name called without valid current_function_decl.");
} }
/* Tell assembler to switch to text section. */ /* Tell assembler to switch to text section. */
@ -253,14 +227,23 @@ text_section (void)
void void
unlikely_text_section (void) unlikely_text_section (void)
{ {
if (! unlikely_text_section_name) if (cfun)
{
if (!cfun->unlikely_text_section_name)
initialize_cold_section_name (); initialize_cold_section_name ();
if ((in_section != in_unlikely_executed_text) if ((in_section != in_unlikely_executed_text)
&& (in_section != in_named && (in_section != in_named
|| strcmp (in_named_name, unlikely_text_section_name) != 0)) || strcmp (in_named_name, cfun->unlikely_text_section_name) != 0))
{ {
named_section (NULL_TREE, unlikely_text_section_name, 0); named_section (NULL_TREE, cfun->unlikely_text_section_name, 0);
in_section = in_unlikely_executed_text;
last_text_section = in_unlikely_executed_text;
}
}
else
{
named_section (NULL_TREE, UNLIKELY_EXECUTED_TEXT_SECTION_NAME, 0);
in_section = in_unlikely_executed_text; in_section = in_unlikely_executed_text;
last_text_section = in_unlikely_executed_text; last_text_section = in_unlikely_executed_text;
} }
@ -315,10 +298,21 @@ in_unlikely_text_section (void)
{ {
bool ret_val; bool ret_val;
if (cfun)
{
ret_val = ((in_section == in_unlikely_executed_text) ret_val = ((in_section == in_unlikely_executed_text)
|| (in_section == in_named || (in_section == in_named
&& unlikely_text_section_name && cfun->unlikely_text_section_name
&& strcmp (in_named_name, unlikely_text_section_name) == 0)); && strcmp (in_named_name,
cfun->unlikely_text_section_name) == 0));
}
else
{
ret_val = ((in_section == in_unlikely_executed_text)
|| (in_section == in_named
&& strcmp (in_named_name,
UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0));
}
return ret_val; return ret_val;
} }
@ -463,9 +457,9 @@ named_section (tree decl, const char *name, int reloc)
name = TREE_STRING_POINTER (DECL_SECTION_NAME (decl)); name = TREE_STRING_POINTER (DECL_SECTION_NAME (decl));
if (strcmp (name, UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0 if (strcmp (name, UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0
&& !unlikely_text_section_name) && cfun
unlikely_text_section_name = && ! cfun->unlikely_text_section_name)
xstrdup (UNLIKELY_EXECUTED_TEXT_SECTION_NAME); cfun->unlikely_text_section_name = UNLIKELY_EXECUTED_TEXT_SECTION_NAME;
flags = targetm.section_type_flags (decl, name, reloc); flags = targetm.section_type_flags (decl, name, reloc);
@ -574,16 +568,17 @@ asm_output_aligned_bss (FILE *file, tree decl ATTRIBUTE_UNUSED,
void void
function_section (tree decl) function_section (tree decl)
{ {
bool unlikely = false; int reloc = 0;
if (first_function_block_is_cold) if (first_function_block_is_cold)
unlikely = true; reloc = 1;
#ifdef USE_SELECT_SECTION_FOR_FUNCTIONS #ifdef USE_SELECT_SECTION_FOR_FUNCTIONS
targetm.asm_out.select_section (decl, unlikely, DECL_ALIGN (decl)); targetm.asm_out.select_section (decl, reloc, DECL_ALIGN (decl));
#else #else
if (decl != NULL_TREE if (decl != NULL_TREE
&& DECL_SECTION_NAME (decl) != NULL_TREE) && DECL_SECTION_NAME (decl) != NULL_TREE
&& targetm.have_named_sections)
named_section (decl, (char *) 0, 0); named_section (decl, (char *) 0, 0);
else else
text_section (); text_section ();
@ -594,16 +589,20 @@ void
current_function_section (tree decl) current_function_section (tree decl)
{ {
#ifdef USE_SELECT_SECTION_FOR_FUNCTIONS #ifdef USE_SELECT_SECTION_FOR_FUNCTIONS
bool unlikely = (in_unlikely_text_section () int reloc = 0;
|| (last_text_section == in_unlikely_executed_text));
targetm.asm_out.select_section (decl, unlikely, DECL_ALIGN (decl)); if (in_unlikely_text_section ()
|| last_text_section == in_unlikely_executed_text)
reloc = 1;
targetm.asm_out.select_section (decl, reloc, DECL_ALIGN (decl));
#else #else
if (last_text_section == in_unlikely_executed_text) if (last_text_section == in_unlikely_executed_text)
unlikely_text_section (); unlikely_text_section ();
else if (last_text_section == in_text) else if (last_text_section == in_text)
text_section (); text_section ();
else if (last_text_section == in_named) else if (last_text_section == in_named
&& targetm.have_named_sections)
named_section (NULL_TREE, last_text_section_name, 0); named_section (NULL_TREE, last_text_section_name, 0);
else else
function_section (decl); function_section (decl);
@ -1224,18 +1223,31 @@ void
assemble_start_function (tree decl, const char *fnname) assemble_start_function (tree decl, const char *fnname)
{ {
int align; int align;
char tmp_label[100];
bool hot_label_written = false; bool hot_label_written = false;
unlikely_text_section_name = NULL; cfun->unlikely_text_section_name = NULL;
first_function_block_is_cold = false; first_function_block_is_cold = false;
hot_section_label = reconcat (hot_section_label, fnname, ".hot_section", NULL); if (flag_reorder_blocks_and_partition)
unlikely_section_label = reconcat (unlikely_section_label, {
fnname, ".unlikely_section", NULL); ASM_GENERATE_INTERNAL_LABEL (tmp_label, "HOTB", const_labelno);
hot_section_end_label = reconcat (hot_section_end_label, cfun->hot_section_label = ggc_strdup (tmp_label);
fnname, ".end", NULL); ASM_GENERATE_INTERNAL_LABEL (tmp_label, "COLDB", const_labelno);
cold_section_end_label = reconcat (cold_section_end_label, cfun->cold_section_label = ggc_strdup (tmp_label);
fnname, ".end.cold", NULL); ASM_GENERATE_INTERNAL_LABEL (tmp_label, "HOTE", const_labelno);
cfun->hot_section_end_label = ggc_strdup (tmp_label);
ASM_GENERATE_INTERNAL_LABEL (tmp_label, "COLDE", const_labelno);
cfun->cold_section_end_label = ggc_strdup (tmp_label);
const_labelno++;
}
else
{
cfun->hot_section_label = NULL;
cfun->cold_section_label = NULL;
cfun->hot_section_end_label = NULL;
cfun->cold_section_end_label = NULL;
}
/* The following code does not need preprocessing in the assembler. */ /* The following code does not need preprocessing in the assembler. */
@ -1253,7 +1265,7 @@ assemble_start_function (tree decl, const char *fnname)
{ {
unlikely_text_section (); unlikely_text_section ();
assemble_align (FUNCTION_BOUNDARY); assemble_align (FUNCTION_BOUNDARY);
ASM_OUTPUT_LABEL (asm_out_file, unlikely_section_label); ASM_OUTPUT_LABEL (asm_out_file, cfun->cold_section_label);
if (BB_PARTITION (ENTRY_BLOCK_PTR->next_bb) == BB_COLD_PARTITION) if (BB_PARTITION (ENTRY_BLOCK_PTR->next_bb) == BB_COLD_PARTITION)
{ {
/* Since the function starts with a cold section, we need to /* Since the function starts with a cold section, we need to
@ -1261,7 +1273,7 @@ assemble_start_function (tree decl, const char *fnname)
section label. */ section label. */
text_section (); text_section ();
assemble_align (FUNCTION_BOUNDARY); assemble_align (FUNCTION_BOUNDARY);
ASM_OUTPUT_LABEL (asm_out_file, hot_section_label); ASM_OUTPUT_LABEL (asm_out_file, cfun->hot_section_label);
hot_label_written = true; hot_label_written = true;
first_function_block_is_cold = true; first_function_block_is_cold = true;
} }
@ -1291,8 +1303,8 @@ assemble_start_function (tree decl, const char *fnname)
s[i] = (TREE_STRING_POINTER (DECL_SECTION_NAME (decl)))[i]; s[i] = (TREE_STRING_POINTER (DECL_SECTION_NAME (decl)))[i];
s[len] = '\0'; s[len] = '\0';
if (unlikely_text_section_name if (cfun->unlikely_text_section_name
&& (strcmp (s, unlikely_text_section_name) == 0)) && (strcmp (s, cfun->unlikely_text_section_name) == 0))
first_function_block_is_cold = true; first_function_block_is_cold = true;
} }
@ -1304,7 +1316,7 @@ assemble_start_function (tree decl, const char *fnname)
function_section (decl); function_section (decl);
if (flag_reorder_blocks_and_partition if (flag_reorder_blocks_and_partition
&& !hot_label_written) && !hot_label_written)
ASM_OUTPUT_LABEL (asm_out_file, hot_section_label); ASM_OUTPUT_LABEL (asm_out_file, cfun->hot_section_label);
/* Tell assembler to move to target machine's alignment for functions. */ /* Tell assembler to move to target machine's alignment for functions. */
align = floor_log2 (FUNCTION_BOUNDARY / BITS_PER_UNIT); align = floor_log2 (FUNCTION_BOUNDARY / BITS_PER_UNIT);
@ -1366,7 +1378,6 @@ assemble_start_function (tree decl, const char *fnname)
void void
assemble_end_function (tree decl, const char *fnname) assemble_end_function (tree decl, const char *fnname)
{ {
enum in_section save_text_section;
#ifdef ASM_DECLARE_FUNCTION_SIZE #ifdef ASM_DECLARE_FUNCTION_SIZE
ASM_DECLARE_FUNCTION_SIZE (asm_out_file, fnname, decl); ASM_DECLARE_FUNCTION_SIZE (asm_out_file, fnname, decl);
#endif #endif
@ -1379,11 +1390,13 @@ assemble_end_function (tree decl, const char *fnname)
debug info.) */ debug info.) */
if (flag_reorder_blocks_and_partition) if (flag_reorder_blocks_and_partition)
{ {
enum in_section save_text_section;
save_text_section = in_section; save_text_section = in_section;
unlikely_text_section (); unlikely_text_section ();
ASM_OUTPUT_LABEL (asm_out_file, cold_section_end_label); ASM_OUTPUT_LABEL (asm_out_file, cfun->cold_section_end_label);
text_section (); text_section ();
ASM_OUTPUT_LABEL (asm_out_file, hot_section_end_label); ASM_OUTPUT_LABEL (asm_out_file, cfun->hot_section_end_label);
if (save_text_section == in_unlikely_executed_text) if (save_text_section == in_unlikely_executed_text)
unlikely_text_section (); unlikely_text_section ();
} }
@ -4780,8 +4793,14 @@ default_section_type_flags_1 (tree decl, const char *name, int reloc,
flags = SECTION_CODE; flags = SECTION_CODE;
else if (decl && decl_readonly_section_1 (decl, reloc, shlib)) else if (decl && decl_readonly_section_1 (decl, reloc, shlib))
flags = 0; flags = 0;
else if (unlikely_text_section_name else if (current_function_decl
&& strcmp (name, unlikely_text_section_name) == 0) && cfun
&& cfun->unlikely_text_section_name
&& strcmp (name, cfun->unlikely_text_section_name) == 0)
flags = SECTION_CODE;
else if (!decl
&& (!current_function_decl || !cfun)
&& strcmp (name, UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0)
flags = SECTION_CODE; flags = SECTION_CODE;
else else
flags = SECTION_WRITE; flags = SECTION_WRITE;