gcc/ada/
* exp_aggr.adb
(Convert_To_Positional): Add Dims local variable
and pass it in calls to Is_Flat and Flatten.
(Check_Static_Components): Pass Dims in call to
Is_Static_Element.
(Nonflattenable_Next_Aggr): New predicate.
(Flatten): Add Dims parameter and Expr local variable. Call
Nonflattenable_Next_Aggr in a couple of places. In the case
when an Others choice is present, check that the element is
either static or a nested aggregate that can be flattened,
before disregarding the replication limit for elaboration
purposes. Check that a nested array is flattenable in the case
of a multidimensional array in any position. Remove redundant
check in the Others case and pass Dims in call to
Is_Static_Element. Use Expr variable.
(Is_Flat): Change type of Dims parameter from Int to Nat.
(Is_Static_Element): Add Dims parameter. Replace tests on
literals with call to Compile_Time_Known_Value. If everything
else failed and the dimension is 1, preanalyze the expression
before calling again Compile_Time_Known_Value on it. Return
true for null.
(Late_Expansion): Do not expand further if the assignment to the
target can be done directly by the back end.
gcc/ada/
* libgnat/g-socket.adb (Wait_On_Socket): Fix memory leaks and
file descriptor leaks. A memory leak was created each time the
routine was called without a selector (Selector = Null). Also,
in case of exception in the routine a memory leak and descriptor
leak was created as the created file selector was not closed.
gcc/ada/
* sem_util.adb
(Immediate_Context_Implies_Is_Potentially_Unevaluated): New
subprogram.
(Is_Potentially_Unevaluated): Do not stop climbing the tree on
the first candidate subexpression; required to handle nested
expressions.
gcc/ada/
* sem_ch3.adb (Check_Completion): Refactor chained
if-then-elsif-... statement to be more like a case
statement (note: we can't simply use case statement because of
Is_Intrinsic_Subprogram in the first condition).
gcc/ada/
* exp_aggr.adb (Max_Aggregate_Size): Use small limit for
aggregate inside subprograms.
* sprint.adb (Sprint_Node_Actual [N_Object_Declaration]): Do not
print the initialization expression if the No_Initialization
flag is set.
* sem_util.ads, sem_util.adb (Predicate_Enabled): New.
* exp_ch4.adb (Expand_N_Type_Conversion): Code cleanup and apply
predicate check consistently.
* exp_ch6.adb (Expand_Actuals.By_Ref_Predicate_Check): Ditto.
* sem_ch3.adb (Analyze_Object_Declaration): Ditto.
* exp_ch3.adb (Build_Assignment): Revert handling of predicate
check for allocators with qualified expressions, now handled in
Freeze_Expression directly.
* sem_aggr.adb: Fix typos.
* checks.adb: Code refactoring: use Predicate_Enabled.
(Apply_Predicate_Check): Code cleanup.
* freeze.adb (Freeze_Expression): Freeze the subtype mark before
a qualified expression on an allocator.
* exp_util.ads, exp_util.adb (Within_Internal_Subprogram):
Renamed Predicate_Check_In_Scope to clarify usage, refine
handling of predicates within init procs which should be enabled
when the node comes from source.
* sem_ch13.adb (Freeze_Entity_Checks): Update call to
Predicate_Check_In_Scope.
gcc/ada/
* exp_ch4.adb (Expand_Array_Comparison): Reformat.
(Expand_Concatenate): Use standard size values directly and use
Standard_Long_Long_Unsigned instead of RE_Long_Long_Unsigned.
(Expand_Modular_Op): Use Standard_Long_Long_Integer in case the
modulus is larger than Integer.
(Expand_N_Op_Expon): Use standard size value directly.
(Narrow_Large_Operation): Use Uint instead of Nat for sizes and
use a local variable for the size of the type.
(Get_Size_For_Range): Return Uint instead of Nat.
(Is_OK_For_Range): Take Uint instead of Nat.
gcc/ada/
* exp_ch6.adb (Make_Build_In_Place_Iface_Call_In_Allocator):
Build the internal anonymous access type using as a reference
the designated type imposed by the context (instead of using the
return type of the called function).
gcc/ada/
* sem_ch3.adb (Process_Discriminants): Revert recent change to
location of Set_Ekind; detect effectively volatile discriminants
by their type only.
gcc/ada/
* aspects.ads: Add Aspect_Aggregate.
* exp_aggr.adb (Expand_Container_Aggregate): Expand positional
container aggregates into separate initialization and insertion
operations.
* sem_aggr.ads (Resolve_Container_Aggregate): New subprogram.
* sem_aggr.adb (Resolve_Container_Aggregate): Parse aspect
aggregate, establish element types and key types if present, and
resolve aggregate components.
* sem_ch13.ads (Parse_Aspect_Aggregate): Public subprogram used
in validation, resolution and expansion of container aggregates
* sem_ch13.adb
(Parse_Aspect_Aggregate): Retrieve names of primitives specified
in aspect specification.
(Validate_Aspect_Aggregate): Check legality of specified
operations given in aspect specification, before nane
resolution.
(Resolve_Aspect_Aggregate): At freeze point resolve operations
and verify that given operations have the required profile.
* sem_res.adb (Resolve): Call Resolve_Aspect_Aggregate if aspect
is present for type.
* snames.ads-tmpl: Add names used in aspect Aggregate: Empty,
Add_Named, Add_Unnamed, New_Indexed, Assign_Indexed.
gcc/ada/
* Makefile.rtl (GNATRTL_NONTASKING_OBJS): Add s-shabig.o.
* libgnat/s-shabig.ads: New file to share definitions.
* libgnat/s-genbig.ads, libgnat/s-genbig.adb: Reorganized to
make it more generic and flexible in terms of memory allocation
and data structure returned.
(To_String): Moved to System.Generic_Bignums to allow sharing
this code.
(Big_And, Big_Or, Big_Shift_Left, Big_Shift_Right): New.
* libgnat/s-bignum.adb, libgnat/s-bignum.ads: Adapt to new
System.Generic_Bignums spec.
* libgnat/a-nbnbin.adb: Likewise.
(To_String): Moved to System.Generic_Bignums to allow sharing
this code.
* libgnat/a-nbnbre.adb (Normalize): Fix handling of Num = 0
leading to an exception.
gcc/ada/
* einfo.adb (Write_Field24_Name): Handle E_Loop_Parameter.
* freeze.adb (Freeze_Expr_Types): Freeze the iterator type used as
Default_Iterator of the name of an N_Iterator_Specification node.
gcc/ada/
* checks.adb (Determine_Range): Deal with Min and Max attributes.
* exp_ch6.adb (Expand_Call_Helper): When generating code to pass
the accessibility level to the caller in the case of an actual
which is an if-expression, also remove the nodes created after
the declaration of the dummy temporary.
* sem_ch6.adb (Analyze_Subprogram_Body_Helper): Use Natural as
the type of the minimum accessibility level object.
gcc/ada/
* sem_ch3.adb (Process_Discriminants): Set Ekind of the
processed discriminant entity before passing to
Is_Effectively_Volatile, which was crashing on a failed
assertion.
* sem_prag.adb (Analyze_External_Property_In_Decl_Part): Prevent
call to No_Caching_Enabled with entities other than variables,
which was crashing on a failed assertion.
(Analyze_Pragma): Style cleanups.
* sem_util.adb (Is_Effectively_Volatile): Enforce comment with
an assertion; prevent call to No_Caching_Enabled with entities
other than variables.
(Is_Effectively_Volatile_Object): Only call
Is_Effectively_Volatile on objects, not on types.
(No_Caching_Enabled): Enforce comment with an assertion.
gcc/ada/
* exp_spark.adb (Expand_SPARK_Delta_Or_Update): Refactored from
Expand_SPARK_N_Attribute_Reference; rewrite into N_Aggregate or
N_Delta_Aggregate depending on what is being rewritten.
(Expand_SPARK_N_Delta_Aggregate): New routine to expand
delta_aggregate.
(Expand_SPARK_N_Attribute_Reference): Call the refactored
routine.
This fixes the case where we try to fold a read from an
array initalizer and happen to cross the boundary of
multiple CTORs which isn't really supported. For the
interesting cases like the testcase we actually handle
the folding by encoding the whole initializer.
2020-07-10 Richard Biener <rguenther@suse.de>
PR tree-optimization/96133
* gimple-fold.c (fold_array_ctor_reference): Do not
recurse to folding a CTOR that does not fully cover the
asked for object.
* gcc.dg/torture/pr96133.c: New testcase.
When using -fprofile-report, -fdump-profile-report can be used to
print the report to a foo.c.000i.profile-report file instead
of stderr. I see it handy for comparison purpose.
gcc/ChangeLog:
* dumpfile.c [profile-report]: Add new profile dump.
* dumpfile.h (enum tree_dump_index): Ad TDI_profile_report.
* passes.c (pass_manager::dump_profile_report): Change stderr
to dump_file.
This patch is derived from the review of vector with length patch
series. I relaxed the guard on LOOP_VINFO_PEELING_FOR_ALIGNMENT for
vector with length as Richard S.'s suggestion, then encountered one
failure from case gcc.dg/vect/vect-ifcvt-11.c with param
vect-partial-vector-usage=2 enablement run. The root cause is that
we still use the original niters for the loop body vectorization,
it leads the access to go out of bound, instead we should use
LOOP_VINFO_NITERS which has been adjusted in vect_do_peeling by
considering the peeling number for prologue.
Bootstrapped/regtested on aarch64-linux-gnu and powerpc64le-linux-gnu.
gcc/ChangeLog:
* tree-vect-loop.c (vect_transform_loop): Use LOOP_VINFO_NITERS which
is adjusted by considering peeled prologue for non
vect_use_loop_mask_for_alignment_p cases.
This is a fix for the pointer (or array) size inadvertently being used
for the bias with attach and detach mapping kinds, for both C and C++.
2020-07-09 Julian Brown <julian@codesourcery.com>
Thomas Schwinge <thomas@codesourcery.com>
gcc/c/
PR middle-end/95270
* c-typeck.c (c_finish_omp_clauses): Set OMP_CLAUSE_SIZE (bias) to zero
for standalone attach/detach clauses.
gcc/cp/
PR middle-end/95270
* semantics.c (finish_omp_clauses): Likewise.
include/
PR middle-end/95270
* gomp-constants.h (gomp_map_kind): Expand comment for attach/detach
mapping kinds.
gcc/testsuite/
PR middle-end/95270
* c-c++-common/goacc/mdc-1.c: Update expected dump output for zero
bias.
libgomp/
PR middle-end/95270
* testsuite/libgomp.oacc-c-c++-common/pr95270-1.c: New test.
* testsuite/libgomp.oacc-c-c++-common/pr95270-2.c: New test.
Arrange for GOMP_MAP_ATTACH to be grouped together with a preceding
GOMP_MAP_TO_PSET or other "to" data movement clause, except in cases
where an explicit "attach" clause is used.
2020-07-09 Julian Brown <julian@codesourcery.com>
include/
* gomp-constants.h (gomp_map_kind): Update comment for GOMP_MAP_TO_PSET.
libgomp/
* oacc-mem.c (find_group_last): Group data-movement clauses
(GOMP_MAP_TO_PSET, GOMP_MAP_TO, etc.) together with a subsequent
GOMP_MAP_ATTACH. Allow standalone GOMP_MAP_ATTACH also.
Fix a bug with mapping Fortran components which themselves have derived
types in the OpenACC 2.5+ manual deep-copy support.
2020-07-09 Julian Brown <julian@codesourcery.com>
gcc/fortran/
* trans-openmp.c (gfc_trans_omp_clauses): Use 'inner' not 'decl' for
derived type members which themselves have derived types.
gcc/testsuite/
* gfortran.dg/goacc/mapping-tests-3.f90: New test.
* gfortran.dg/goacc/mapping-tests-4.f90: New test.
Built-in initialization occurs only once and fairly early, when the
command line options are in force. If the -mcpu=<CPU> is pre-power10,
then we fail to initialize the MMA built-ins, so they are not
available to call in a #pragma target/attribute target function.
The fix is to basically always (on server type cpus) initialize the MMA
built-ins so we can use them in #pragma target/attribute target functions.
2020-07-09 Peter Bergner <bergner@linux.ibm.com>
gcc/
PR target/96125
* config/rs6000/rs6000-call.c (rs6000_init_builtins): Define the MMA
specific types __vector_quad and __vector_pair, and initialize the
MMA built-ins if TARGET_EXTRA_BUILTINS is set.
(mma_init_builtins): Don't test for mask set in rs6000_builtin_mask.
Remove now unneeded mask variable.
* config/rs6000/rs6000.c (rs6000_option_override_internal): Add the
OPTION_MASK_MMA flag for power10 if not already set.
gcc/testsuite/
PR target/96125
* gcc.target/powerpc/pr96125.c: New test.
This fixes a thinko where we end up combining a BIT_FIELD_REF
and a memory access, fixed by checking all stmts are a load or
none.
2020-07-09 Richard Biener <rguenther@suse.de>
PR tree-optimization/96133
* tree-vect-slp.c (vect_build_slp_tree_1): Compare load_p
status between stmts.
I was mistaken to assume that a dependent type is necessarily
incomplete, and indeed there are multiple places in the frontend where
we check a type for both dependency and completeness. So this patch
partially reverts the fix for PR95497, restoring the dependent_type_p
check that guarded the call to is_really_empty_class below.
gcc/cp/ChangeLog:
PR c++/96132
* constexpr.c (potential_constant_expression_1) <case PARM_DECL>:
Restore dependent_type_p check that guarded the call to
is_really_empty_class.
gcc/testsuite/ChangeLog:
PR c++/96132
* g++.dg/template/incomplete12.C: New test.
Enable FMA in rsqrt<mode>2 expander and fold rsqrtv16sf2 expander into
rsqrt<mode>2 expander which expands to UNSPEC_RSQRT28 for TARGET_AVX512ER.
Although it doesn't show performance change in our workloads, FMA can
improve other workloads.
gcc/
PR target/88713
* config/i386/i386-expand.c (ix86_emit_swsqrtsf): Enable FMA.
* config/i386/sse.md (VF_AVX512VL_VF1_128_256): New.
(rsqrt<mode>2): Replace VF1_128_256 with VF_AVX512VL_VF1_128_256.
(rsqrtv16sf2): Removed.
gcc/testsuite/
PR target/88713
* gcc.target/i386/pr88713-1.c: New test.
* gcc.target/i386/pr88713-2.c: Likewise.
This followup removes vect_verify_datarefs_alignment and its
premature cancellation of vectorization leaving the actual
decision whether alignment is supported to the functions
deciding whether we can vectorize a load or store.
2020-07-08 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (vect_verify_datarefs_alignment): Remove.
(vect_slp_analyze_and_verify_instance_alignment): Rename to ...
(vect_slp_analyze_instance_alignment): ... this.
* tree-vect-data-refs.c (verify_data_ref_alignment): Remove.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Do not call
vect_verify_datarefs_alignment.
(vect_slp_analyze_node_alignment): Rename from
vect_slp_analyze_and_verify_node_alignment and do not
call verify_data_ref_alignment.
(vect_slp_analyze_instance_alignment): Rename from
vect_slp_analyze_and_verify_instance_alignment.
* tree-vect-stmts.c (vectorizable_store): Dump when
we vectorize an unaligned access.
(vectorizable_load): Likewise.
* tree-vect-loop.c (vect_analyze_loop_2): Do not call
vect_verify_datarefs_alignment.
* tree-vect-slp.c (vect_slp_analyze_bb_1): Adjust.
* gcc.dg/vect/bb-slp-10.c: Adjust.
* gcc.dg/vect/slp-45.c: Likewise.
* gcc.dg/vect/vect-109.c: Likewise.
If reduction partition's SCC is broken by runtime alias checks, force
a negative post order to it so that it will be scheduled in the last.
2020-07-09 Bin Cheng <bin.cheng@linux.alibaba.com>
gcc/
PR tree-optimization/95804
* tree-loop-distribution.c (break_alias_scc_partitions): Force
negative post order to reduction partition.
gcc/testsuite/
PR tree-optimization/95804
* gcc.dg/tree-ssa/pr95804.c: New test.
This patch implements the optimized logical to actual iterators
computation for triangular loops.
I have a rough implementation using integers, but this one uses floating
point. There is a small problem that -fopenmp programs aren't linked with
-lm, so it does it only if the hw has sqrt optab (and uses ifn rather than
__builtin_sqrt because it obviously doesn't need errno handling etc.).
Do you think it is ok this way, or should I use the integral computation
using inlined isqrt (we have inequation of the form
start >= x * t10 + t11 * (((x - 1) * x) / 2)
where t10 and t11 are signed long long values and start unsigned long long,
and the division by 2 actually is a problem for accuracy in some cases, so
if we do it in integral, we need to do actually
long long t12 = 2 * t10 - t11;
unsigned long long t13 = t12 * t12 + start * 8 * t11;
unsigned long long isqrt_ = isqrtull (t13);
long long x = (((long long) isqrt_ - t12) / t11) >> 1;
with careful overflow checking on all the computations before isqrtull
(and on overflows use the fallback implementation).
2020-07-09 Jakub Jelinek <jakub@redhat.com>
* omp-general.h (struct omp_for_data): Add min_inner_iterations
and factor members.
* omp-general.c (omp_extract_for_data): Initialize them and remember
them in OMP_CLAUSE_COLLAPSE_COUNT if needed and restore from there.
* omp-expand.c (expand_omp_for_init_counts): Fix up computation of
counts[fd->last_nonrect] if fd->loop.n2 is INTEGER_CST.
(expand_omp_for_init_vars): For
fd->first_nonrect + 1 == fd->last_nonrect loops with for now
INTEGER_CST fd->loop.n2 find quadratic equation roots instead of
using fallback method when possible.
* testsuite/libgomp.c/loop-19.c: New test.
* testsuite/libgomp.c/loop-20.c: New test.
While this is an OpenMP 5.1 change, it is undesirable to let people use different
values and then deal with ABI backwards compatibility in a year or two.
2020-07-09 Jakub Jelinek <jakub@redhat.com>
* omp.h.in (omp_alloctrait_value_t): Change omp_atv_default from
2 to -1. Add omp_atv_serialized and define omp_atv_sequential using
it. Remove __omp_alloctrait_value_max__.
* allocator.c (omp_init_allocator): Handle omp_atv_default for
omp_atk_alignment and omp_atk_pool_size.
The variables first_moveable_pseudo and last_moveable_pseudo aren't
reset after compiling a function, which means they leak into the first
scheduler pass of the following function. In some cases, this can cause
an extra spill during register allocation of the second function.
gcc/ChangeLog:
* ira.c (move_unallocated_pseudos): Zero first_moveable_pseudo and
last_moveable_pseudo before returning.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/nospill.c: New test.
sjlj.S did not have the GNU property note markup and the BTI c
instructions that are necessary when it is built with branch
protection.
The notes are only added when libitm is built with branch
protection, because old linkers mishandle the note (merge
them incorrectly or emit warnings), the BTI instructions
are added unconditionally.
2020-07-09 Szabolcs Nagy <szabolcs.nagy@arm.com>
libitm/ChangeLog:
* config/aarch64/sjlj.S: Add BTI marking and related definitions,
and add BTI c to function entries.
lse.S did not have the GNU property note markup and the BTI c
instructions that are necessary when it is built with branch
protection.
The notes are only added when libgcc is built with branch
protection, because old linkers mishandle the note (merge
them incorrectly or emit warnings), the BTI instructions
are added unconditionally.
Note: BTI c is only necessary at function entry if the function
may be called indirectly, currently lse functions are not called
indirectly, but BTI is added for ABI reasons e.g. to allow
linkers later to emit stub code with indirect jump.
2020-07-09 Szabolcs Nagy <szabolcs.nagy@arm.com>
libgcc/ChangeLog:
PR target/96001
* config/aarch64/lse.S: Add BTI marking and related definitions,
and add BTI c to function entries.
lse.S did not have GNU stack note, this may cause missing
PT_GNU_STACK in binaries on Linux and FreeBSD.
2020-07-09 Szabolcs Nagy <szabolcs.nagy@arm.com>
libgcc/ChangeLog:
* config/aarch64/lse.S: Add stack note.
sjlj.S only had the note on Linux, but it is supposed
to have it on FreeBSD too.
2020-07-09 Szabolcs Nagy <szabolcs.nagy@arm.com>
libitm/ChangeLog:
* config/aarch64/sjlj.S: Add stack note if __FreeBSD__ is defined.
Define the __ARM_FEATURE_BTI_DEFAULT feature test
macro when BTI branch protection is enabled.
2020-07-09 Szabolcs Nagy <szabolcs.nagy@arm.com>
gcc/ChangeLog:
* config/aarch64/aarch64-c.c (aarch64_update_cpp_builtins): Add
__ARM_FEATURE_BTI_DEFAULT support.
This patch introduces the mitigation for Straight Line Speculation past
the BLR instruction.
This mitigation replaces BLR instructions with a BL to a stub which uses
a BR to jump to the original value. These function stubs are then
appended with a speculation barrier to ensure no straight line
speculation happens after these jumps.
When optimising for speed we use a set of stubs for each function since
this should help the branch predictor make more accurate predictions
about where a stub should branch.
When optimising for size we use one set of stubs for all functions.
This set of stubs can have human readable names, and we are using
`__call_indirect_x<N>` for register x<N>.
When BTI branch protection is enabled the BLR instruction can jump to a
`BTI c` instruction using any register, while the BR instruction can
only jump to a `BTI c` instruction using the x16 or x17 registers.
Hence, in order to ensure this transformation is safe we mov the value
of the original register into x16 and use x16 for the BR.
As an example when optimising for size:
a
BLR x0
instruction would get transformed to something like
BL __call_indirect_x0
where __call_indirect_x0 labels a thunk that contains
__call_indirect_x0:
MOV X16, X0
BR X16
<speculation barrier>
The first version of this patch used local symbols specific to a
compilation unit to try and avoid relocations.
This was mistaken since functions coming from the same compilation unit
can still be in different sections, and the assembler will insert
relocations at jumps between sections.
On any relocation the linker is permitted to emit a veneer to handle
jumps between symbols that are very far apart. The registers x16 and
x17 may be clobbered by these veneers.
Hence the function stubs cannot rely on the values of x16 and x17 being
the same as just before the function stub is called.
Similar can be said for the hot/cold partitioning of single functions,
so function-local stubs have the same restriction.
This updated version of the patch never emits function stubs for x16 and
x17, and instead forces other registers to be used.
Given the above, there is now no benefit to local symbols (since they
are not enough to avoid dealing with linker intricacies). This patch
now uses global symbols with hidden visibility each stored in their own
COMDAT section. This means stubs can be shared between compilation
units while still avoiding the PLT indirection.
This patch also removes the `__call_indirect_x30` stub (and
function-local equivalent) which would simply jump back to the original
location.
The function-local stubs are emitted to the assembly output file in one
chunk, which means we need not add the speculation barrier directly
after each one.
This is because we know for certain that the instructions directly after
the BR in all but the last function stub will be from another one of
these stubs and hence will not contain a speculation gadget.
Instead we add a speculation barrier at the end of the sequence of
stubs.
The global stubs are emitted in COMDAT/.linkonce sections by
themselves so that the linker can remove duplicates from multiple object
files. This means they are not emitted in one chunk, and each one must
include the speculation barrier.
Another difference is that since the global stubs are shared across
compilation units we do not know that all functions will be targeting an
architecture supporting the SB instruction.
Rather than provide multiple stubs for each architecture, we provide a
stub that will work for all architectures -- using the DSB+ISB barrier.
This mitigation does not apply for BLR instructions in the following
places:
- Some accesses to thread-local variables use a code sequence with a BLR
instruction. This code sequence is part of the binary interface between
compiler and linker. If this BLR instruction needs to be mitigated, it'd
probably be best to do so in the linker. It seems that the code sequence
for thread-local variable access is unlikely to lead to a Spectre Revalation
Gadget.
- PLT stubs are produced by the linker and each contain a BLR instruction.
It seems that at most only after the last PLT stub a Spectre Revalation
Gadget might appear.
Testing:
Bootstrap and regtest on AArch64
(with BOOT_CFLAGS="-mharden-sls=retbr,blr")
Used a temporary hack(1) in gcc-dg.exp to use these options on every
test in the testsuite, a slight modification to emit the speculation
barrier after every function stub, and a script to check that the
output never emitted a BLR, or unmitigated BR or RET instruction.
Similar on an aarch64-none-elf cross-compiler.
1) Temporary hack emitted a speculation barrier at the end of every stub
function, and used a script to ensure that:
a) Every RET or BR is immediately followed by a speculation barrier.
b) No BLR instruction is emitted by compiler.
gcc/ChangeLog:
* config/aarch64/aarch64-protos.h (aarch64_indirect_call_asm):
New declaration.
* config/aarch64/aarch64.c (aarch64_regno_regclass): Handle new
stub registers class.
(aarch64_class_max_nregs): Likewise.
(aarch64_register_move_cost): Likewise.
(aarch64_sls_shared_thunks): Global array to store stub labels.
(aarch64_sls_emit_function_stub): New.
(aarch64_create_blr_label): New.
(aarch64_sls_emit_blr_function_thunks): New.
(aarch64_sls_emit_shared_blr_thunks): New.
(aarch64_asm_file_end): New.
(aarch64_indirect_call_asm): New.
(TARGET_ASM_FILE_END): Use aarch64_asm_file_end.
(TARGET_ASM_FUNCTION_EPILOGUE): Use
aarch64_sls_emit_blr_function_thunks.
* config/aarch64/aarch64.h (STB_REGNUM_P): New.
(enum reg_class): Add STUB_REGS class.
(machine_function): Introduce `call_via` array for
function-local stub labels.
* config/aarch64/aarch64.md (*call_insn, *call_value_insn): Use
aarch64_indirect_call_asm to emit code when hardening BLR
instructions.
* config/aarch64/constraints.md (Ucr): New constraint
representing registers for indirect calls. Is GENERAL_REGS
usually, and STUB_REGS when hardening BLR instruction against
SLS.
* config/aarch64/predicates.md (aarch64_general_reg): STUB_REGS class
is also a general register.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/sls-mitigation/sls-miti-blr-bti.c: New test.
* gcc.target/aarch64/sls-mitigation/sls-miti-blr.c: New test.
Instructions following RET or BR are not necessarily executed. In order
to avoid speculation past RET and BR we can simply append a speculation
barrier.
Since these speculation barriers will not be architecturally executed,
they are not expected to add a high performance penalty.
The speculation barrier is to be SB when targeting architectures which
have this enabled, and DSB SY + ISB otherwise.
We add tests for each of the cases where such an instruction was seen.
This is implemented by modifying each machine description pattern that
emits either a RET or a BR instruction. We choose not to use something
like `TARGET_ASM_FUNCTION_EPILOGUE` since it does not affect the
`indirect_jump`, `jump`, `sibcall_insn` and `sibcall_value_insn`
patterns and we find it preferable to implement the functionality in the
same way for every pattern.
There is one particular case which is slightly tricky. The
implementation of TARGET_ASM_TRAMPOLINE_TEMPLATE uses a BR which needs
to be mitigated against. The trampoline template is used *once* per
compilation unit, and the TRAMPOLINE_SIZE is exposed to the user via the
builtin macro __LIBGCC_TRAMPOLINE_SIZE__.
In the future we may implement function specific attributes to turn on
and off hardening on a per-function basis.
The fixed nature of the trampoline described above implies it will be
safer to ensure this speculation barrier is always used.
Testing:
Bootstrap and regtest done on aarch64-none-linux
Used a temporary hack(1) to use these options on every test in the
testsuite and a script to check that the output never emitted an
unmitigated RET or BR.
1) Temporary hack was a change to the testsuite to always use
`-save-temps` and run a script on the assembly output of those
compilations which produced one to ensure every RET or BR is immediately
followed by a speculation barrier.
gcc/ChangeLog:
* config/aarch64/aarch64-protos.h (aarch64_sls_barrier): New.
* config/aarch64/aarch64.c (aarch64_output_casesi): Emit
speculation barrier after BR instruction if needs be.
(aarch64_trampoline_init): Handle ptr_mode value & adjust size
of code copied.
(aarch64_sls_barrier): New.
(aarch64_asm_trampoline_template): Add needed barriers.
* config/aarch64/aarch64.h (AARCH64_ISA_SB): New.
(TARGET_SB): New.
(TRAMPOLINE_SIZE): Account for barrier.
* config/aarch64/aarch64.md (indirect_jump, *casesi_dispatch,
simple_return, *do_return, *sibcall_insn, *sibcall_value_insn):
Emit barrier if needs be, also account for possible barrier using
"sls_length" attribute.
(sls_length): New attribute.
(length): Determine default using any non-default sls_length
value.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/sls-mitigation/sls-miti-retbr.c: New test.
* gcc.target/aarch64/sls-mitigation/sls-miti-retbr-pacret.c:
New test.
* gcc.target/aarch64/sls-mitigation/sls-mitigation.exp: New file.
* lib/target-supports.exp (check_effective_target_aarch64_asm_sb_ok):
New proc.
Here we introduce the flags that will be used for straight line speculation.
The new flag introduced is `-mharden-sls=`.
This flag can take arguments of `none`, `all`, or a comma seperated list of one
or more of `retbr` or `blr`.
`none` indicates no special mitigation of the straight line speculation
vulnerability.
`all` requests all mitigations currently implemented.
`retbr` requests that the RET and BR instructions have a speculation barrier
inserted after them.
`blr` requests that BLR instructions are replaced by a BL to a function stub
using a BR with a speculation barrier after it.
Setting this on a per-function basis using attributes or the like is not
enabled, but may be in the future.
gcc/ChangeLog:
2020-06-02 Matthew Malcomson <matthew.malcomson@arm.com>
* config/aarch64/aarch64-protos.h (aarch64_harden_sls_retbr_p):
New.
(aarch64_harden_sls_blr_p): New.
* config/aarch64/aarch64.c (enum aarch64_sls_hardening_type):
New.
(aarch64_harden_sls_retbr_p): New.
(aarch64_harden_sls_blr_p): New.
(aarch64_validate_sls_mitigation): New.
(aarch64_override_options): Parse options for SLS mitigation.
* config/aarch64/aarch64.opt (-mharden-sls): New option.
* doc/invoke.texi: Document new option.
