When the preprocessor lexes preprocessing numbers in lex_number, it
accepts digit separators in more cases than actually permitted in
pp-numbers by the standard syntax.
One thing this accepts is adjacent digit separators; there is some
code to reject those later, but as noted in bug 83873 it fails to
cover the case of adjacent digit separators within a floating-point
exponent. Accepting adjacent digit separators only results in a
missing diagnostic, not in valid code being rejected or being accepted
with incorrect semantics, because the correct lexing in such a case
would have '' start the following preprocessing tokens, and no valid
preprocessing token starts '' while ' isn't valid on its own as a
preprocessing token either. So this patch fixes that case by moving
the error for adjacent digit separators to lex_number (allowing a more
specific diagnostic than if '' were excluded from the pp-number
completely).
Other cases inappropriately accepted involve digit separators before
'.', 'e+', 'e-', 'p+' or 'p-' (or corresponding uppercase variants).
In those cases, as shown by the test digit-sep-pp-number.C added, this
can result in valid code being wrongly rejected as a result of too
many characters being included in the pp-number. So this case is
fixed by terminating the pp-number at the correct character according
to the standard. That test also covers the case where a digit
separator was followed by an identifier-nondigit that is not a
nondigit (e.g. a UCN); that case was already handled correctly.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
libcpp/
PR c++/83873
PR preprocessor/97604
* lex.c (lex_number): Reject adjacent digit separators here. Do
not allow digit separators before '.' or an exponent with sign.
* expr.c (cpp_classify_number): Do not check for adjacent digit
separators here.
gcc/testsuite/
PR c++/83873
PR preprocessor/97604
* g++.dg/cpp1y/digit-sep-neg-2.C,
g++.dg/cpp1y/digit-sep-pp-number.C: New tests.
* g++.dg/cpp1y/digit-sep-line-neg.C, g++.dg/cpp1y/digit-sep-neg.C:
Adjust expected messages.
As reported in bug 82359, the preprocessor does not allow C++ digit
separators in the line number in a #line directive, despite the
standard syntax for that directive using digit-sequence which allows
digit separators.
There is some confusion in that bug about whether C++ is meant to
allow digit separators there or not, but the last comment there
suggests they are meant to be allowed, and the version of digit
separators accepted for C2X at the March meeting explicitly mentions
digit separators in the #line specification to avoid any ambiguity
there.
This patch thus adds code to handle digit separators in the line
number in #line, as part of the preparation for enabling digit
separators in C2X mode. The code changed does not contain any
conditionals for whether digit separators are supported in the chosen
language version, because that was handled earlier in pp-number lexing
and if they aren't supported they won't appear in the string passed to
that function. It does however make sure not to allow adjacent digit
separators because those are only handled at a later stage of lexing
at present. (Problems with how certain source character sequences
involving digit separators that don't actually match the pp-number
syntax get lexed as a pp-number and only diagnosed later, if at all,
are bugs 83873 and 97604, to be addressed separately.)
Making the change in this location will have the effect of allowing
digit separators in the "# <line-number> <file> <flags>" form of
directive as well as #line; I don't think that's a problem.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
libcpp/
PR preprocessor/82359
* directives.c (strtolinenum): Handle digit separators.
gcc/testsuite/
PR preprocessor/82359
* g++.dg/cpp1y/digit-sep-line.C,
g++.dg/cpp1y/digit-sep-line-neg.C: New tests.
This reverts a s/column_offset/column/ change in the fix for PR99446.
2021-04-19 Richard Biener <rguenther@suse.de>
PR preprocessor/100142
libcpp/
* line-map.c (linemap_position_for_loc_and_offset): Revert
unintended s/column_offset/column/ change.
gcc/testsuite/
* gcc.dg/pr100142.c: New testcase.
* g++.dg/diagnostic/pr72803.C: Revert last change.
This ICE was because when adjusting a column offset we could advance
into a linemap for a different file. We only checked the next line
map was not for a line further advanced in any file, forgetting that
it could be for an earlier line in a different file. The testcase
needed adjusting as column 512 was unrepresentable, once that was
taken into consideration.
PR preprocessor/99446
libcpp/
* line-map.c (line-map.c): Do not advance to linemaps for
different files.
gcc/testsuite/
* g++.dg/diagnostic/pr72803.C: Adjust expected column.
The problem is that the new IS_MACRO_LOC macro:
inline bool
IS_MACRO_LOC (location_t loc)
{
return !IS_ORDINARY_LOC (loc) && !IS_ADHOC_LOC (loc);
}
is not fully correct since the position of the macro lines is not fixed:
/* Returns the lowest location [of a token resulting from macro
expansion] encoded in this line table. */
inline location_t
LINEMAPS_MACRO_LOWEST_LOCATION (const line_maps *set)
{
return LINEMAPS_MACRO_USED (set)
? MAP_START_LOCATION (LINEMAPS_LAST_MACRO_MAP (set))
: MAX_LOCATION_T + 1;
}
In Ada, LINEMAPS_MACRO_USED is false so LINEMAPS_MACRO_LOWEST_LOCATION is
MAX_LOCATION_T + 1, but IS_MACRO_LOC nevertheless returns true for anything
in the range [LINE_MAP_MAX_LOCATION; MAX_LOCATION_T], thus yielding an ICE
in linemap_macro_map_lookup for very large files.
libcpp/
* include/line-map.h (IS_MACRO_LOC): Delete.
* line-map.c (linemap_location_from_macro_expansion_p): Test
LINEMAPS_MACRO_LOWEST_LOCATION of the linemap.
gcc/cp/
* module.cc (ordinary_loc_of): Test LINEMAPS_MACRO_LOWEST_LOCATION
of the linemap.
(module_state::write_location): Likewise.
PR c/99323 describes an ICE due to a failed assertion deep inside the
fix-it printing machinery, where the fix-it hints on one line have not
been properly sorted in layout's constructor.
The underlying issue occurs when multiple fix-it hints affect a line
wider that LINE_MAP_MAX_COLUMN_NUMBER, where the location_t values for
characters after that threshold fall back to having column zero.
It's not meaningful to try to handle fix-it hints without column
information, so this patch rejects them as they are added to the
rich_location, falling back to the "no fix-it hints on this diagnostic"
case, fixing the crash.
gcc/ChangeLog:
PR c/99323
* diagnostic-show-locus.c
(selftest::test_one_liner_many_fixits_2): Fix accidental usage of
column 0.
gcc/testsuite/ChangeLog:
PR c/99323
* gcc.dg/pr99323-1.c: New test.
* gcc.dg/pr99323-2.c: New test.
libcpp/ChangeLog:
PR c/99323
* line-map.c (rich_location::maybe_add_fixit): Reject fix-it hints
at column 0.
This fixes some issues with macro maps. We were incorrectly
calculating the number of macro expansions in a location span, and I
had a workaround that partially covered that up. Further, while macro
location spans are monotonic, that is not true of ordinary location
spans. Thus we need to insert an indirection array when binary
searching the latter. (We load ordinary locations before loading
imports, but macro locations afterwards. We make sure an import
location is de-macrofied, if needed.)
PR c++/98718
gcc/cp/
* module.cc (ool): New indirection vector.
(loc_spans::maybe_propagate): Location is not optional.
(loc_spans::open): Likewise. Assert monotonically advancing.
(module_for_ordinary_loc): Use ool indirection vector.
(module_state::write_prepare_maps): Do not count empty macro
expansions. Elide empty spans.
(module_state::write_macro_maps): Skip empty expansions.
(ool_cmp): New qsort comparator.
(module_state::write): Create and destroy ool vector.
(name_pending_imports): Fix dump push/pop.
(preprocess_module): Likewise. Add more dumping.
(preprocessed_module): Likewise.
libcpp/
* include/line-map.h
* line-map.c
gcc/testsuite/
* g++.dg/modules/pr98718_a.C: New.
* g++.dg/modules/pr98718_b.C: New.
When we read preprocessed source, we deal with a couple of special
location lines at the start of the file. These provide information
about the original filename of the source and the current directory,
so we can process the source in the same manner. When updating that
code, I had a somewhat philosophical question: Should the line table
contain evidence of the filename the user provided to the compiler? I
figured to leave it there, as it did no harm. But this defect shows
an issue. It's in the line table and our (non optimizing) line table
serializer emits that filename. Which means if one re-preprocesses
the original source to a differently-named intermediate file, the
resultant CMI is different. Boo. That's a difference that doesn't
matter, except the CRC matching then fails. We should elide the
filename, so that one can preprocess to mktemp intermediate filenames
for whatever reason.
This patch takes the approach of expunging it from the line table --
so the line table will end up with exactly the same form. That seems
a better bet than trying to fix up mismatching line tables in CMI
emission.
PR c++/99072
libcpp/
* init.c (read_original_filename): Expunge all evidence of the
original filename.
gcc/testsuite/
* g++.dg/modules/pr99072.H: New.
This defect really required building header-units and include translation
of pieces of the standard library. This adds smarts to the modules
test harness to do that -- accept .X files as the source file, but
provide '-x c++-system-header $HDR' in the options. The .X file will
be considered by the driver to be a linker script and ignored (with a
warning).
Using this we can add 2 tests that end up building list_initializer
and iostream, along with a test that iostream's build
include-translates list_initializer's #include. That discovered a set
of issues with the -flang-info-include-translate=HDR handling, also
fixed and documented here.
PR c++/99023
gcc/cp/
* module.cc (canonicalize_header_name): Use
cpp_probe_header_unit.
(maybe_translate_include): Fix note_includes comparison.
(init_modules): Fix note_includes string termination.
libcpp/
* include/cpplib.h (cpp_find_header_unit): Rename to ...
(cpp_probe_header_unit): ... this.
* internal.h (_cp_find_header_unit): Declare.
* files.c (cpp_find_header_unit): Break apart to ..
(test_header_unit): ... this, and ...
(_cpp_find_header_unit): ... and, or and ...
(cpp_probe_header_unit): ... this.
* macro.c (cpp_get_token_1): Call _cpp_find_header_unit.
gcc/
* doc/invoke.texi (flang-info-include-translate): Document header
lookup behaviour.
gcc/testsuite/
* g++.dg/modules/modules.exp: Bail on cross-testing. Add support
for .X files.
* g++.dg/modules/pr99023_a.X: New.
* g++.dg/modules/pr99023_b.X: New.
We make sure files end in \n by placing one at the limit of the buffer
(just past the end of what is read). We need to do the same for
buffers generated via include-translation. Fortunately they have
space.
libcpp/
* files.c (_cpp_stack_file): Make buffers end in unread \n.
gcc/testsuite/
* g++.dg/modules/pr99050_a.H: New.
* g++.dg/modules/pr99050_b.C: New.
PR preprocessor/96391 describes an ICE in the C++ frontend on:
#define CONST const
#define VOID void
typedef CONST VOID *PCVOID;
where the typedef line occurs after enough code has been compiled
that location_t values are beyond LINE_MAP_MAX_LOCATION_WITH_COLS,
and hence no column numbers are available.
The issue occurs in linemap_compare_locations when comparing the
locations of the "const" and "void" tokens.
Upon resolving the LRK_MACRO_EXPANSION_POINT, both have the same
location_t, the line of the "typedef" (with no column), and so
the l0 == l1 clause is triggered, but they are not from the
same macro expansion, leading first_map_in_common to return NULL
and triggering the "abort" condition.
This patch fixes the issue by checking when the two macro expansion
point location_t values are equal that the value
<= LINE_MAP_MAX_LOCATION_WITH_COLS and thus has column information,
fixing the issue.
gcc/testsuite/ChangeLog:
PR preprocessor/96391
* g++.dg/plugin/location-overflow-test-pr96391.c: New test.
* g++.dg/plugin/plugin.exp (plugin_test_list): Add it,
using the location_overflow_plugin.c from gcc.dg/plugin.
libcpp/ChangeLog:
PR preprocessor/96391
* line-map.c (linemap_compare_locations): Require that
the location be <= LINE_MAP_MAX_LOCATION_WITH_COLS when
treating locations as coming from the same macro expansion.
The following patch uses make_signed_t<size_t> instead of
make_signed<size_t>::type in the diagnostics, because the former is shorter.
It is true that one can't use make_signed<size_t>::type in C++11 code (which
is why I haven't changed it in the testcase which is c++11 effective
target), but the message talks about C++23 and make_signed_t is a C++14 and
later feature, so I think it is fine.
2021-02-04 Jakub Jelinek <jakub@redhat.com>
* expr.c (cpp_classify_number): Use make_signed_t<size_t> instead of
make_signed<size_t>::type in the diagnostics.
* g++.dg/warn/Wsize_t-literals.C: Expect make_signed_t<size_t> instead
of make_signed<size_t>::type in the diagnostics.
GCC 11 ICEs on all -fdirectives-only preprocessing when the files don't end
with a newline.
The problem is in the assertion, for empty TUs buffer->cur == buffer->rlimit
and so buffer->rlimit[-1] access triggers UB in the preprocessor, for
non-empty TUs it refers to the last character in the file, which can be
anything.
The preprocessor adds a '\n' character (or '\r', in particular if the
user file ends with '\r' then it adds another '\r' rather than '\n'), but
that is added after the limit, i.e. at buffer->rlimit[0].
Now, if the routine handles occassional bumping of pos to buffer->rlimit + 1,
I think it is just the assert that needs changing, usually we read from *pos
if pos < limit and then e.g. if it is '\r', look at the following character
(which could be one of those '\n' or '\r' at buffer->rlimit[0]). There is
also the case where for '\\' before the limit we read following character
and if it is '\n', do one thing, if it is '\r' read another character.
But in that case if '\\' was the last char in the TU, the limit char will be
'\n', so we are ok.
2021-02-03 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/98882
* lex.c (cpp_directive_only_process): Don't assert that rlimit[-1]
is a newline, instead assert that rlimit[0] is either newline or
carriage return. When seeing '\\' followed by '\r', check limit
before accessing pos[1].
* gcc.dg/cpp/pr98882.c: New test.
Integer literal suffixes for signed size ('z') and unsigned size
(some permutation od 'zu') are provided as a language addition.
gcc/c-family/ChangeLog:
* c-cppbuiltin.c (c_cpp_builtins): Define __cpp_size_t_suffix.
* c-lex.c (interpret_integer): Set node type for size literal.
libcpp/ChangeLog:
* expr.c (interpret_int_suffix): Detect 'z' integer suffix.
(cpp_classify_number): Compat warning for use of 'z' suffix.
* include/cpplib.h (struct cpp_options): New flag.
(enum cpp_warning_reason): New flag.
(CPP_N_USERDEF): Comment C++0x -> C++11.
(CPP_N_SIZE_T): New flag for cpp_classify_number.
* init.c (cpp_set_lang): Initialize new flag.
gcc/testsuite/ChangeLog:
* g++.dg/cpp0x/udlit-shadow-neg.C: Test for 'z' and 'zu' shadowing.
* g++.dg/cpp23/feat-cxx2b.C: New test.
* g++.dg/cpp23/size_t-literals.C: New test.
* g++.dg/warn/Wsize_t-literals.C: New test.
Derived from the changes that added C++2a support in 2017.
r8-3237-g026a79f70cf33f836ea5275eda72d4870a3041e5
No C++23 features are added here.
Use of -std=c++23 sets __cplusplus to 202100L.
$ g++ -std=c++23 -dM -E -x c++ - < /dev/null | grep cplusplus
#define __cplusplus 202100L
gcc/
* doc/cpp.texi (__cplusplus): Document value for -std=c++23
or -std=gnu++23.
* doc/invoke.texi: Document -std=c++23 and -std=gnu++23.
* dwarf2out.c (highest_c_language): Recognise C++20 and C++23.
(gen_compile_unit_die): Recognise C++23.
gcc/c-family/
* c-common.h (cxx_dialect): Add cxx23 as a dialect.
* c.opt: Add options for -std=c++23, std=c++2b, -std=gnu++23
and -std=gnu++2b
* c-opts.c (set_std_cxx23): New.
(c_common_handle_option): Set options when -std=c++23 is enabled.
(c_common_post_options): Adjust comments.
(set_std_cxx20): Likewise.
gcc/testsuite/
* lib/target-supports.exp (check_effective_target_c++2a):
Check for C++2a or C++23.
(check_effective_target_c++20_down): New.
(check_effective_target_c++23_only): New.
(check_effective_target_c++23): New.
* g++.dg/cpp23/cplusplus.C: New.
libcpp/
* include/cpplib.h (c_lang): Add CXX23 and GNUCXX23.
* init.c (lang_defaults): Add rows for CXX23 and GNUCXX23.
(cpp_init_builtins): Set __cplusplus to 202100L for C++23.
For deferred macros we also need a new field on the macro itself, so
that the module machinery can determine the macro was imported. Also
the documentation for the hashnode's deferred field was incomplete.
libcpp/
* include/cpplib.h (struct cpp_macro): Add imported_p field.
(struct cpp_hashnode): Tweak deferred field documentation.
* macro.c (_cpp_new_macro): Clear new field.
(cpp_get_deferred_macro, get_deferred_or_lazy_macro): Assert
more.
GCC Administrator
Joseph Myers
H.J. Lu
Richard Biener
Nathan Sidwell
Eric Botcazou
David Malcolm
Jakub Jelinek
Ed Smith-Rowland
liuhongt
Paul Fee
Jason Merrill