libcpp/ChangeLog
2019-09-19 Lewis Hyatt <lhyatt@gmail.com>
PR c/67224
* charset.c (_cpp_valid_utf8): New function to help lex UTF-8 tokens.
* internal.h (_cpp_valid_utf8): Declare.
* lex.c (forms_identifier_p): Use it to recognize UTF-8 identifiers.
(_cpp_lex_direct): Handle UTF-8 in identifiers and CPP_OTHER tokens.
Do all work in "default" case to avoid slowing down typical code paths.
Also handle $ and UCN in the default case for consistency.
gcc/Changelog
2019-09-19 Lewis Hyatt <lhyatt@gmail.com>
PR c/67224
* doc/cpp.texi: Document support for extended characters in
identifiers.
* doc/cppopts.texi: Likewise.
gcc/testsuite/ChangeLog
2019-09-19 Lewis Hyatt <lhyatt@gmail.com>
PR c/67224
* c-c++-common/cpp/ucnid-2011-1-utf8.c: New test.
* g++.dg/cpp/ucnid-1-utf8.C: New test.
* g++.dg/cpp/ucnid-2-utf8.C: New test.
* g++.dg/cpp/ucnid-3-utf8.C: New test.
* g++.dg/cpp/ucnid-4-utf8.C: New test.
* g++.dg/other/ucnid-1-utf8.C: New test.
* gcc.dg/cpp/ucnid-1-utf8.c: New test.
* gcc.dg/cpp/ucnid-10-utf8.c: New test.
* gcc.dg/cpp/ucnid-11-utf8.c: New test.
* gcc.dg/cpp/ucnid-12-utf8.c: New test.
* gcc.dg/cpp/ucnid-13-utf8.c: New test.
* gcc.dg/cpp/ucnid-14-utf8.c: New test.
* gcc.dg/cpp/ucnid-15-utf8.c: New test.
* gcc.dg/cpp/ucnid-2-utf8.c: New test.
* gcc.dg/cpp/ucnid-3-utf8.c: New test.
* gcc.dg/cpp/ucnid-4-utf8.c: New test.
* gcc.dg/cpp/ucnid-6-utf8.c: New test.
* gcc.dg/cpp/ucnid-7-utf8.c: New test.
* gcc.dg/cpp/ucnid-9-utf8.c: New test.
* gcc.dg/ucnid-1-utf8.c: New test.
* gcc.dg/ucnid-10-utf8.c: New test.
* gcc.dg/ucnid-11-utf8.c: New test.
* gcc.dg/ucnid-12-utf8.c: New test.
* gcc.dg/ucnid-13-utf8.c: New test.
* gcc.dg/ucnid-14-utf8.c: New test.
* gcc.dg/ucnid-15-utf8.c: New test.
* gcc.dg/ucnid-16-utf8.c: New test.
* gcc.dg/ucnid-2-utf8.c: New test.
* gcc.dg/ucnid-3-utf8.c: New test.
* gcc.dg/ucnid-4-utf8.c: New test.
* gcc.dg/ucnid-5-utf8.c: New test.
* gcc.dg/ucnid-6-utf8.c: New test.
* gcc.dg/ucnid-7-utf8.c: New test.
* gcc.dg/ucnid-8-utf8.c: New test.
* gcc.dg/ucnid-9-utf8.c: New test.
From-SVN: r275979
* lex.c (_cpp_lex_direct): Use CPP_DL_NOTE instead of CPP_DL_PEDWARN,
CPP_DL_WARNING or CPP_DL_ERROR for note that diagnostics for C++ style
comments is reported only once per file and guard those calls on the
preceding cpp_error returning true.
* gcc.dg/cpp/pr61854-c90.c (foo): Expect a note, rather than error.
* gcc.dg/cpp/pr61854-c94.c (foo): Likewise.
* gcc.dg/cpp/pr61854-4.c (foo): Likewise.
* gcc.dg/cpp/pr61854-8.c: New test.
From-SVN: r262832
gcc/testsuite:
PR preprocessor/84517
* g++.dg/cpp0x/udlit-macros.C: Expect a warning for ""__FILE__.
libcpp:
PR preprocessor/84517
* lex.c (is_macro_not_literal_suffix): New function.
(lex_raw_string, lex_string): Use is_macro_not_literal_suffix to
decide when to issue -Wliteral-suffix warnings.
From-SVN: r258069
This implements __VA_OPT__, a new preprocessor feature added in C++2A.
The paper can be found here:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0306r4.html
gcc/ChangeLog
* doc/cpp.texi (Variadic Macros): Document __VA_OPT__.
gcc/testsuite/ChangeLog
* c-c++-common/cpp/va-opt-pedantic.c: New file.
* c-c++-common/cpp/va-opt.c: New file.
* c-c++-common/cpp/va-opt-error.c: New file.
libcpp/ChangeLog
* pch.c (cpp_read_state): Set n__VA_OPT__.
* macro.c (vaopt_state): New class.
(_cpp_arguments_ok): Check va_opt flag.
(replace_args, create_iso_definition): Use vaopt_state.
* lex.c (lex_identifier_intern): Possibly issue errors for
__VA_OPT__.
(lex_identifier): Likewise.
(maybe_va_opt_error): New function.
* internal.h (struct lexer_state) <va_args_ok>: Update comment.
(struct spec_nodes) <n__VA_OPT__>: New field.
* init.c (struct lang_flags) <va_opt>: New field.
(lang_defaults): Add entries for C++2A. Update all entries for
va_opt.
(cpp_set_lang): Initialize va_opt.
* include/cpplib.h (struct cpp_options) <va_opt>: New field.
* identifiers.c (_cpp_init_hashtable): Initialize n__VA_OPT__.
From-SVN: r254707
/libcpp
2017-11-06 Mukesh Kapoor <mukesh.kapoor@oracle.com>
PR c++/80955
* lex.c (lex_string): When checking for a valid macro for the
warning related to -Wliteral-suffix (CPP_W_LITERAL_SUFFIX),
check that the macro name does not start with an underscore
before calling is_macro().
/gcc/testsuite
2017-11-06 Mukesh Kapoor <mukesh.kapoor@oracle.com>
PR c++/80955
* g++.dg/cpp0x/udlit-macros.C: New.
From-SVN: r254443
gcc/ChangeLog:
PR preprocessor/78680
PR preprocessor/78811
* input.c (struct selftest::lexer_test): Add field
m_implicitly_expect_EOF.
(selftest::lexer_error_sink): New class.
(selftest::lexer_error_sink::s_singleton): New global.
(selftest::lexer_test::lexer_test): Initialize new field
"m_implicitly_expect_EOF".
(selftest::lexer_test::~lexer_test): Conditionalize the
check for the EOF token on the new field.
(selftest::test_lexer_string_locations_raw_string_unterminated):
New function.
(selftest::input_c_tests): Call the new test.
libcpp/ChangeLog:
PR preprocessor/78680
PR preprocessor/78811
* lex.c (_cpp_lex_direct): Only determine the end-location of
the token and build a range for non-reserved start locations.
Do not do it for EOF tokens.
From-SVN: r243721
Fix for PR preprocessor/78680
PR preprocessor/78680 identifies a crash when attempting to issue
a -Wformat warning, where the format string includes a string token
split across multiple physical source lines via backslash-continued
lines.
The issue is that libcpp is generating bogus range information for
such tokens.
For example, in:
void fn1() {
__builtin_printf("\
%ld.\n\
2\n"); };
the range of the string token is printed as:
__builtin_printf("\
^~
whereas the range ought to be:
__builtin_printf("\
^~
%ld.\n\
~~~~~~~
2\n"); };
~~~~
The root cause is that the line notes expressing the update
of the buffer in lex.c aren't yet updated when the end-point of
the token is computed
3095 tok_range.m_finish
3096 = linemap_position_for_column (pfile->line_table,
3097 CPP_BUF_COLUMN (buffer, buffer->cur));
so that the physical line is still regarded as that of the start
of the token, and, where CPP_BUF_COLUMN uses (BUF)->line_base,
line_base is still the location of the first physical line in the
and hence the column information is too large (as if it were the
offset in the *logical* line).
(the printed range is somewhat misleading; the actual buggy range
extends beyond the "\ in the line, but within diagnostic-show-locus.c
layout::print_annotation_line only prints up to the xbound set by
layout::print_source_line and so truncates most of the buggy range).
The fix is to ensure that line notes are handled before calculating
the end-point of the token range.
This leads to the range for the string token being correctly
computed, as:
__builtin_printf("\
^~
%ld.\n\
~~~~~~~
2\n"); };
~~~~
and this leads to get_substring_ranges_for_loc failing gracefully,
rather than crashing.
gcc/testsuite/ChangeLog:
PR preprocessor/78680
* gcc.dg/format/pr78680.c: New test case.
* gcc.dg/plugin/diagnostic-test-expressions-1.c
(test_multiline_token): New function.
* gcc.dg/plugin/diagnostic-test-string-literals-1.c
(test_backslash_continued_logical_lines): New function.
libcpp/ChangeLog:
PR preprocessor/78680
* lex.c (_cpp_lex_direct): Ensure line notes are processed before
computing the end-point of the token.
From-SVN: r243567
* common.opt (Wimplicit-fallthrough) Turn into alias to
-Wimplicit-fallthrough=3. Remove EnabledBy.
(Wimplicit-fallthrough=): New option.
* gimplify.c (warn_implicit_fallthrough_r): Use
OPT_Wimplicit_fallthrough_ instead of OPT_Wimplicit_fallthrough.
* doc/invoke.texi (-Wimplicit-fallthrough): Document as alias
to -Wimplicit-fallthrough=3.
(-Wimplicit-fallthrough=): Document.
gcc/c-family/
* c.opt (Wextra): Add as C/C++/ObjC/ObjC++ option.
(Wimplicit-fallthrough=): Enable for these languages by -Wextra.
* c-opts.c (sanitize_cpp_opts): Initialize
cpp_opts->cpp_warn_implicit_fallthrough.
gcc/testsuite/
* c-c++-common/Wimplicit-fallthrough-25.c: New test.
* c-c++-common/Wimplicit-fallthrough-26.c: New test.
* c-c++-common/Wimplicit-fallthrough-27.c: New test.
* c-c++-common/Wimplicit-fallthrough-28.c: New test.
* c-c++-common/Wimplicit-fallthrough-29.c: New test.
* c-c++-common/Wimplicit-fallthrough-30.c: New test.
* c-c++-common/Wimplicit-fallthrough-31.c: New test.
* c-c++-common/Wimplicit-fallthrough-32.c: New test.
* c-c++-common/Wimplicit-fallthrough-33.c: New test.
libcpp/
* include/cpplib.h (struct cpp_options): Add
cpp_warn_implicit_fallthrough.
* init.c (cpp_create_reader): Initialize it to 0.
* lex.c (fallthrough_comment_p): Handle different
cpp_warn_implicit_fallthrough levels. Whitespace fixes.
From-SVN: r241013
* c-lex.c (c_lex_with_flags) <case CPP_COMMENT>: For CPP_COMMENT
token with PREV_FALLTHROUGH, skip all following CPP_PADDING and
CPP_COMMENT tokens and set add_flags to PREV_FALLTHROUGH afterwards.
* doc/invoke.texi (-Wimplicit-fallthrough): Document the accepted
FALLTHRU comment styles.
* lex.c (fallthrough_comment_p): Fix off-by-one size comparison
errors, cleanup.
(_cpp_lex_direct): Allow arbitrary comments in between
fallthrough_comment_p comment and following token.
* c-c++-common/Wimplicit-fallthrough-23.c: New test.
* c-c++-common/Wimplicit-fallthrough-24.c: New test.
From-SVN: r240884
libcpp/ChangeLog:
2016-10-04 Kelvin Nilsen <kelvin@gcc.gnu.org>
PR target/77847
* lex.c (search_line_fast): Add a FALLTHROUGH comment to correct
compiler error in the version of this function that is
conditionally compiled when GCC_VERSION >= 4005 and both
__ALTIVEC__ and __BIG_ENDIAN__ symbols are defined.
From-SVN: r240783
gcc/c-family/ChangeLog:
* c-common.c: Include "substring-locations.h".
(get_cpp_ttype_from_string_type): New function.
(g_string_concat_db): New global.
(substring_loc::get_range): New method.
* c-common.h (g_string_concat_db): New declaration.
(class substring_loc): New class.
* c-lex.c (lex_string): When concatenating strings, capture the
locations of all tokens using a new obstack, and record the
concatenation locations within g_string_concat_db.
* c-opts.c (c_common_init_options): Construct g_string_concat_db
on the ggc-heap.
gcc/ChangeLog:
* input.c (string_concat::string_concat): New constructor.
(string_concat_db::string_concat_db): New constructor.
(string_concat_db::record_string_concatenation): New method.
(string_concat_db::get_string_concatenation): New method.
(string_concat_db::get_key_loc): New method.
(class auto_cpp_string_vec): New class.
(get_substring_ranges_for_loc): New function.
(get_source_range_for_substring): New function.
(get_num_source_ranges_for_substring): New function.
(class selftest::lexer_test_options): New class.
(struct selftest::lexer_test): New struct.
(class selftest::ebcdic_execution_charset): New class.
(selftest::ebcdic_execution_charset::s_singleton): New variable.
(selftest::lexer_test::lexer_test): New constructor.
(selftest::lexer_test::~lexer_test): New destructor.
(selftest::lexer_test::get_token): New method.
(selftest::assert_char_at_range): New function.
(ASSERT_CHAR_AT_RANGE): New macro.
(selftest::assert_num_substring_ranges): New function.
(ASSERT_NUM_SUBSTRING_RANGES): New macro.
(selftest::assert_has_no_substring_ranges): New function.
(ASSERT_HAS_NO_SUBSTRING_RANGES): New macro.
(selftest::test_lexer_string_locations_simple): New function.
(selftest::test_lexer_string_locations_ebcdic): New function.
(selftest::test_lexer_string_locations_hex): New function.
(selftest::test_lexer_string_locations_oct): New function.
(selftest::test_lexer_string_locations_letter_escape_1): New function.
(selftest::test_lexer_string_locations_letter_escape_2): New function.
(selftest::test_lexer_string_locations_ucn4): New function.
(selftest::test_lexer_string_locations_ucn8): New function.
(selftest::uint32_from_big_endian): New function.
(selftest::test_lexer_string_locations_wide_string): New function.
(selftest::uint16_from_big_endian): New function.
(selftest::test_lexer_string_locations_string16): New function.
(selftest::test_lexer_string_locations_string32): New function.
(selftest::test_lexer_string_locations_u8): New function.
(selftest::test_lexer_string_locations_utf8_source): New function.
(selftest::test_lexer_string_locations_concatenation_1): New
function.
(selftest::test_lexer_string_locations_concatenation_2): New
function.
(selftest::test_lexer_string_locations_concatenation_3): New
function.
(selftest::test_lexer_string_locations_macro): New function.
(selftest::test_lexer_string_locations_stringified_macro_argument):
New function.
(selftest::test_lexer_string_locations_non_string): New function.
(selftest::test_lexer_string_locations_long_line): New function.
(selftest::test_lexer_char_constants): New function.
(selftest::input_c_tests): Call the new test functions once per
case within the line_table test matrix.
* input.h (struct string_concat): New struct.
(struct location_hash): New struct.
(class string_concat_db): New class.
* substring-locations.h: New header.
gcc/testsuite/ChangeLog:
* gcc.dg/plugin/diagnostic-test-string-literals-1.c: New file.
* gcc.dg/plugin/diagnostic-test-string-literals-2.c: New file.
* gcc.dg/plugin/diagnostic_plugin_test_string_literals.c: New file.
* gcc.dg/plugin/plugin.exp (plugin_test_list): Add the above new files.
libcpp/ChangeLog:
* charset.c (cpp_substring_ranges::cpp_substring_ranges): New
constructor.
(cpp_substring_ranges::~cpp_substring_ranges): New destructor.
(cpp_substring_ranges::add_range): New method.
(cpp_substring_ranges::add_n_ranges): New method.
(_cpp_valid_ucn): Add "char_range" and "loc_reader" params; if
they are non-NULL, read position information from *loc_reader
and update char_range->m_finish accordingly.
(convert_ucn): Add "char_range", "loc_reader", and "ranges"
params. If loc_reader is non-NULL, read location information from
it, and update *ranges accordingly, using char_range.
Conditionalize the conversion into tbuf on tbuf being non-NULL.
(convert_hex): Likewise, conditionalizing the call to
emit_numeric_escape on tbuf.
(convert_oct): Likewise.
(convert_escape): Add params "loc_reader" and "ranges". If
loc_reader is non-NULL, read location information from it, and
update *ranges accordingly. Conditionalize the conversion into
tbuf on tbuf being non-NULL.
(cpp_interpret_string): Rename to...
(cpp_interpret_string_1): ...this, adding params "loc_readers" and
"out". Use "to" to conditionalize the initialization and usage of
"tbuf", such as running the converter. If "loc_readers" is
non-NULL, use the instances within it, reading location
information from them, and passing them to convert_escape; likewise
write to "out" if loc_readers is non-NULL. Check for leading
quote and issue an error if it is not present. Update boundary
check from "== limit" to ">= limit" to protect against erroneous
location values to calls that are not parsing string literals.
(cpp_interpret_string): Reimplement in terms to
cpp_interpret_string_1.
(noop_error_cb): New function.
(cpp_interpret_string_ranges): New function.
(cpp_string_location_reader::cpp_string_location_reader): New
constructor.
(cpp_string_location_reader::get_next): New method.
* include/cpplib.h (class cpp_string_location_reader): New class.
(class cpp_substring_ranges): New class.
(cpp_interpret_string_ranges): New prototype.
* internal.h (_cpp_valid_ucn): Add params "char_range" and
"loc_reader".
* lex.c (forms_identifier_p): Pass NULL for new params to
_cpp_valid_ucn.
From-SVN: r239175
This patch combines:
[PATCH 05/10] Add ranges to libcpp tokens (via ad-hoc data, unoptimized)
[PATCH 06/10] Track expression ranges in C frontend
[PATCH 07/10] Add plugin to recursively dump the source-ranges in a tree (v2)
[PATCH 08/10] Wire things up so that libcpp users get token underlines
[PATCH 09/10] Delay some resolution of ad-hoc locations, preserving ranges
[PATCH 10/10] Compress short ranges into source_location
[PATCH] libcpp: add examples to source_location description
along with fixes for the nits identified during review.
gcc/ChangeLog:
* Makefile.in (OBJS): Add gcc-rich-location.o.
* diagnostic.c (diagnostic_append_note): Pass line_table to
rich_location ctor.
(emit_diagnostic): Likewise.
(inform): Likewise.
(inform_n): Likewise.
(warning): Likewise.
(warning_at): Likewise.
(warning_n): Likewise.
(pedwarn): Likewise.
(permerror): Likewise.
(error): Likewise.
(error_n): Likewise.
(error_at): Likewise.
(sorry): Likewise.
(fatal_error): Likewise.
(internal_error): Likewise.
(internal_error_no_backtrace): Likewise.
(source_range::debug): Likewise.
* gcc-rich-location.c: New file.
* gcc-rich-location.h: New file.
* genmatch.c (fatal_at): Pass line_table to rich_location ctor.
(warning_at): Likewise.
* gimple.h (gimple_set_block): Use set_block function.
* input.c (dump_line_table_statistics): Dump stats on how many
ranges were optimized vs how many needed ad-hoc table.
(write_digit_row): Add "map" param; use its range_bits
to calculate the per-character offset.
(dump_location_info): Print the range and column bits for each
ordinary map. Use the range bits to calculate the per-character
offset. Pass the map as a new param to the various calls to
write_digit_row. Eliminate uses of
ORDINARY_MAP_NUMBER_OF_COLUMN_BITS.
* print-tree.c (print_node): Print any source range information.
* rtl-error.c (diagnostic_for_asm): Likewise.
* toplev.c (general_init): Initialize line_table's
default_range_bits.
* tree-cfg.c (move_block_to_fn): Likewise.
(move_block_to_fn): Likewise.
* tree-inline.c (copy_phis_for_bb): Likewise.
* tree.c (tree_set_block): Likewise.
(get_pure_location): New function.
(set_source_range): New functions.
(set_block): New function.
(set_source_range): New functions.
* tree.h (CAN_HAVE_RANGE_P): New.
(EXPR_LOCATION_RANGE): New.
(EXPR_HAS_RANGE): New.
(get_expr_source_range): New inline function.
(DECL_LOCATION_RANGE): New.
(set_source_range): New decls.
(get_decl_source_range): New inline function.
gcc/ada/ChangeLog:
* gcc-interface/trans.c (Sloc_to_locus): Add line_table param when
calling linemap_position_for_line_and_column.
gcc/c-family/ChangeLog:
* c-common.c (c_fully_fold_internal): Capture existing souce_range,
and store it on the result.
* c-opts.c (c_common_init_options): Set
global_dc->colorize_source_p.
gcc/c/ChangeLog:
* c-decl.c (warn_defaults_to): Pass line_table to
rich_location ctor.
* c-errors.c (pedwarn_c99): Likewise.
(pedwarn_c90): Likewise.
* c-parser.c (set_c_expr_source_range): New functions.
(c_token::get_range): New method.
(c_token::get_finish): New method.
(c_parser_expr_no_commas): Call set_c_expr_source_range on the ret
based on the range from the start of the LHS to the end of the
RHS.
(c_parser_conditional_expression): Likewise, based on the range
from the start of the cond.value to the end of exp2.value.
(c_parser_binary_expression): Call set_c_expr_source_range on
the stack values for TRUTH_ANDIF_EXPR and TRUTH_ORIF_EXPR.
(c_parser_cast_expression): Call set_c_expr_source_range on ret
based on the cast_loc through to the end of the expr.
(c_parser_unary_expression): Likewise, based on the
op_loc through to the end of op.
(c_parser_sizeof_expression) Likewise, based on the start of the
sizeof token through to either the closing paren or the end of
expr.
(c_parser_postfix_expression): Likewise, using the token range,
or from the open paren through to the close paren for
parenthesized expressions.
(c_parser_postfix_expression_after_primary): Likewise, for
various kinds of expression.
* c-tree.h (struct c_expr): Add field "src_range".
(c_expr::get_start): New method.
(c_expr::get_finish): New method.
(set_c_expr_source_range): New decls.
* c-typeck.c (parser_build_unary_op): Call set_c_expr_source_range
on ret for prefix unary ops.
(parser_build_binary_op): Likewise, running from the start of
arg1.value through to the end of arg2.value.
gcc/cp/ChangeLog:
* error.c (pedwarn_cxx98): Pass line_table to rich_location ctor.
gcc/fortran/ChangeLog:
* error.c (gfc_warning): Pass line_table to rich_location ctor.
(gfc_warning_now_at): Likewise.
(gfc_warning_now): Likewise.
(gfc_error_now): Likewise.
(gfc_fatal_error): Likewise.
(gfc_error): Likewise.
(gfc_internal_error): Likewise.
gcc/testsuite/ChangeLog:
* gcc.dg/diagnostic-token-ranges.c: New file.
* gcc.dg/diagnostic-tree-expr-ranges-2.c: New file.
* gcc.dg/plugin/diagnostic-test-expressions-1.c: New file.
* gcc.dg/plugin/diagnostic-test-show-trees-1.c: New file.
* gcc.dg/plugin/diagnostic_plugin_show_trees.c: New file.
* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (get_loc): Add
line_table param when calling
linemap_position_for_line_and_column.
(test_show_locus): Pass line_table to rich_location ctors.
(plugin_init): Remove setting of global_dc->colorize_source_p.
* gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c:
New file.
* gcc.dg/plugin/plugin.exp (plugin_test_list): Add
diagnostic_plugin_test_tree_expression_range.c,
diagnostic-test-expressions-1.c, diagnostic_plugin_show_trees.c,
and diagnostic-test-show-trees-1.c.
libcpp/ChangeLog:
* errors.c (cpp_diagnostic): Pass pfile->line_table to
rich_location ctor.
(cpp_diagnostic_with_line): Likewise.
* include/cpplib.h (struct cpp_token): Update comment for src_loc
to indicate that the range of the token is "baked into" the
source_location.
* include/line-map.h (source_location): Update the descriptive
comment to reflect the packing scheme for short ranges, adding
worked examples of location encoding.
(struct line_map_ordinary): Drop field "column_bits" in favor
of field "m_column_and_range_bits"; add field "m_range_bits".
(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS): Delete.
(location_adhoc_data): Add source_range field.
(struct line_maps): Add fields "default_range_bits",
"num_optimized_ranges" and "num_unoptimized_ranges".
(get_combined_adhoc_loc): Add source_range param.
(get_range_from_loc): New declaration.
(pure_location_p): New prototype.
(COMBINE_LOCATION_DATA): Add source_range param.
(SOURCE_LINE): Update for renaming of column_bits.
(SOURCE_COLUMN): Likewise. Shift the column right by the map's
range_bits.
(LAST_SOURCE_LINE_LOCATION): Update for renaming of column_bits.
(linemap_position_for_line_and_column): Add line_maps * params.
(rich_location::rich_location): Likewise.
* lex.c (_cpp_lex_direct): Capture the range of the token, baking
it into token->src_loc via a call to COMBINE_LOCATION_DATA.
* line-map.c (LINE_MAP_MAX_COLUMN_NUMBER): Reduce from 1U << 17 to
1U << 12.
(location_adhoc_data_hash): Add the src_range into
the hash value.
(location_adhoc_data_eq): Require equality of the src_range
values.
(can_be_stored_compactly_p): New function.
(get_combined_adhoc_loc): Add src_range param, and store it,
via a bit-packing scheme for short ranges, otherwise within the
lookaside table. Remove the requirement that data is non-NULL.
(get_range_from_adhoc_loc): New function.
(get_range_from_loc): New function.
(pure_location_p): New function.
(linemap_add): Ensure that start_location has zero for the
range_bits, unless we're past LINE_MAP_MAX_LOCATION_WITH_COLS.
Initialize range_bits to zero. Assert that the start_location
is "pure".
(linemap_line_start): Assert that the
column_and_range_bits >= range_bits.
Update determinination of whether we need to start a new map
using the effective column bits, without the range bits.
Use the set's default_range_bits in new maps, apart from
those with column_bits == 0, which should also have 0 range_bits.
Increase the column bits for new maps by the range bits.
When adding lines to an existing map, use set->highest_line
directly rather than offsetting highest by SOURCE_COLUMN.
Add assertions to sanity-check the return value.
(linemap_position_for_column): Offset to_column by range_bits.
Update set->highest_location if necessary.
(linemap_position_for_line_and_column): Add line_maps * param.
Update the calculation to offset the column by range_bits, and
conditionalize it on being <= LINE_MAP_MAX_LOCATION_WITH_COLS.
Bound it by LINEMAPS_MACRO_LOWEST_LOCATION. Update
set->highest_location if necessary.
(linemap_position_for_loc_and_offset): Handle ad-hoc locations;
pass "set" to linemap_position_for_line_and_column.
(linemap_macro_map_loc_unwind_toward_spelling): Add line_maps
param. Handle ad-hoc locations.
(linemap_location_in_system_header_p): Pass on "set" to call to
linemap_macro_map_loc_unwind_toward_spelling.
(linemap_macro_loc_to_spelling_point): Retain ad-hoc locations.
Pass on "set" to call to
linemap_macro_map_loc_unwind_toward_spelling.
(linemap_resolve_location): Retain ad-hoc locations. Pass on
"set" to call to linemap_macro_map_loc_unwind_toward_spelling.
(linemap_unwind_toward_expansion): Pass on "set" to call to
linemap_macro_map_loc_unwind_toward_spelling.
(linemap_expand_location): Extract the data pointer before
extracting the location.
(rich_location::rich_location): Add line_maps param; use it to
extract the range from the source_location.
* location-example.txt: Regenerate, showing new representation.
From-SVN: r230331
PR preprocessor/61977
* lex.c (cpp_peek_token): If peektok is CPP_EOF, back it up
with all tokens peeked by the current function.
* gcc.dg/cpp/pr61977.c: New test.
From-SVN: r221882
libcpp/
2015-03-16 Edward Smith-Rowland <3dw4rd@verizon.net>
PR c++/64626
* lex.c (lex_number): If a number ends with digit-seps (') skip back
and let lex_string take them.
gcc/testsuite/
2015-03-16 Edward Smith-Rowland <3dw4rd@verizon.net>
PR c++/64626
g++.dg/cpp1y/pr64626-1.C: New.
g++.dg/cpp1y/pr64626-2.C: New.
g++.dg/cpp1y/digit-sep-neg.C: Adjust errors and warnings.
From-SVN: r221470
Fix PR65261
Running bootstrap-ubsan on ppc64le shows many instances of:
libcpp/lex.c:552:30: runtime error: load of misaligned address
0x01001f31d37a for type 'const uchar', which requires 16 byte alignment
But the unaligned vector loads are intended in this case, because they
are preferable to forced-alignment on POWER8. So just silence the ubsan
errors.
2015-03-02 Markus Trippelsdorf <markus@trippelsdorf.de>
include/
PR target/65261
* ansidecl.h (ATTRIBUTE_NO_SANITIZE_UNDEFINED): New macro.
libcpp/
PR target/65261
* lex.c (search_line_fast): Silence ubsan errors.
From-SVN: r221190
This patch makes cpplib track the original spellings of extended
identifiers, as well as the canonical UTF-8 version, in order to
follow standard semantics properly without needing a convoluted and
undocumented canonicalization in translation phase 1 (see bug 9449
comments 39-46 regarding such a canonicalization).
The spelling is tracked in cpp_identifier and cpp_macro_arg without
making cpp_token any larger. The original spelling is used for checks
of duplicate macro definitions, stringizing (see the C++ tests added;
this case is only an issue for C++ not C because C makes it
implementation-defined whether a \ is inserted before the \ of a UCN
in a string or character constant when stringizing, while C++ does
not), pasting (relevant when the result is then stringized for C++)
and when macro definitions are output as text (e.g. for -d options).
Once a macro has been defined, only the original spelling of the
argument names needs keeping in the argument list. While it is being
defined, however, both spellings are needed: the original one for
subsequent saving for checks of duplicate macro definitions, and the
canonical one which is the node marked specially to generate macro
argument tokens rather than normal identifier tokens. The buffer that
is used to save the original values of the identifier tokens is
changed so that it stores both those original values and a pointer to
the canonical hash nodes, so that those canonical nodes can be found
when their values need restoring after the macro definition has been
parsed.
I believe this covers the known standards issues in extended
identifiers support (the remaining unimplemented C99 areas in GCC all
being floating-point-related), except for C++ translation of extended
characters to UCNs in phase 1 (which I have no plans to work on).
There are however probably issues left with handling of extended
identifiers in other places, as listed in
<https://gcc.gnu.org/ml/gcc-patches/2014-11/msg00337.html> (those
issues are generally the sort of thing that could be addressed as bugs
outside development stage 1). (The bulk of the potential issues Zack
was concerned about in 2003-5, that resulted in extended identifiers
being disabled in the absence of -fextended-identifiers, were
effectively eliminated by the audit and fixes I did in 2009, however;
that todo list reflects what was left over after that audit.)
Bootstrapped with no regressions on x86_64-unknown-linux-gnu.
libcpp:
* include/cpp-id-data.h (struct cpp_macro): Update comment
regarding parameters.
* include/cpplib.h (struct cpp_macro_arg, struct cpp_identifier):
Add spelling fields.
(struct cpp_token): Update comment on macro_arg.
* internal.h (_cpp_save_parameter): Add extra argument.
(_cpp_spell_ident_ucns): New declaration.
* lex.c (lex_identifier): Add SPELLING argument. Set *SPELLING to
original spelling of identifier.
(_cpp_lex_direct): Update calls to lex_identifier.
(_cpp_spell_ident_ucns): New function, factored out of
cpp_spell_token.
(cpp_spell_token): Adjust FORSTRING argument semantics to return
original spelling of identifiers. Use _cpp_spell_ident_ucns in
!FORSTRING case.
(_cpp_equiv_tokens): Check spellings of identifiers and macro
arguments are identical.
* macro.c (macro_arg_saved_data): New structure.
(paste_tokens): Use original spellings of identifiers from
cpp_spell_token.
(_cpp_save_parameter): Add argument SPELLING. Save both canonical
node and its value.
(parse_params): Update calls to _cpp_save_parameter.
(lex_expansion_token): Save spelling of macro argument tokens.
(_cpp_create_definition): Extract canonical node from saved data.
(cpp_macro_definition): Use UCNs in spelling of macro name. Use
original spellings of macro argument tokens and identifiers.
* traditional.c (scan_parameters): Update call to
_cpp_save_parameter.
gcc:
* doc/invoke.texi (-std=c99, -std=c11): Don't refer to corner
cases of extended identifiers.
gcc/testsuite:
* g++.dg/cpp/ucnid-2.C, g++.dg/cpp/ucnid-3.C,
gcc.dg/cpp/ucnid-11.c, gcc.dg/cpp/ucnid-12.c,
gcc.dg/cpp/ucnid-13.c, gcc.dg/cpp/ucnid-14.c,
gcc.dg/cpp/ucnid-15.c: New tests.
From-SVN: r217202
2014-10-03 Bill Schmidt <wschmidt@linux.vnet.ibm.com>
* lex.c (search_line_fast): Add new version to be used for Power8
and later targets when Altivec is enabled. Restrict the existing
Altivec version to big-endian systems so that lvsr is not used on
little endian, where it is deprecated. Remove LE-specific code
from the now-BE-only version.
From-SVN: r215873
