Commit Graph

149 Commits

Author SHA1 Message Date
Nathan Sidwell 13f93cf533 preprocessor: Add deferred macros
Deferred macros are needed for C++ modules.  Header units may export
macro definitions and undefinitions.  These are resolved lazily at the
point of (potential) use.  (The language specifies that, it's not just
a useful optimization.)  Thus, identifier nodes grow a 'deferred'
field, which fortunately doesn't expand the structure on 64-bit
systems as there was padding there.  This is non-zero on NT_MACRO
nodes, if the macro is deferred.  When such an identifier is lexed, it
is resolved via a callback that I added recently.  That will either
provide the macro definition, or discover it there was an overriding
undef.  Either way the identifier is no longer a deferred macro.
Notice it is now possible for NT_MACRO nodes to have a NULL macro
expansion.

	libcpp/
	* include/cpplib.h (struct cpp_hashnode): Add deferred field.
	(cpp_set_deferred_macro): Define.
	(cpp_get_deferred_macro): Declare.
	(cpp_macro_definition): Reformat, add overload.
	(cpp_macro_definition_location): Deal with deferred macro.
	(cpp_alloc_token_string, cpp_compare_macro): Declare.
	* internal.h (_cpp_notify_macro_use): Return bool
	(_cpp_maybe_notify_macro_use): Likewise.
	* directives.c (do_undef): Check macro is not undef before
	warning.
	(do_ifdef, do_ifndef): Deal with deferred macro.
	* expr.c (parse_defined): Likewise.
	* lex.c (cpp_allocate_token_string): Break out of ...
	(create_literal): ... here.  Call it.
	(cpp_maybe_module_directive): Deal with deferred macro.
	* macro.c (cpp_get_token_1): Deal with deferred macro.
	(warn_of_redefinition): Deal with deferred macro.
	(compare_macros): Rename to ...
	(cpp_compare_macro): ... here.  Make extern.
	(cpp_get_deferred_macro): New.
	(_cpp_notify_macro_use): Deal with deferred macro, return bool
	indicating definedness.
	(cpp_macro_definition): Deal with deferred macro.
2020-11-24 08:31:03 -08:00
Nathan Sidwell bf425849f1 preprocessor: main-file cleanup
In preparing module patch 7 I realized there was a cleanup I could
make to simplify it.  This is that cleanup.  Also, when doing the
cleanup I noticed some macros had been turned into inline functions,
but not renamed to the preprocessors internal namespace
(_cpp_$INTERNAL rather than cpp_$USER).  Thus, this renames those
functions, deletes an internal field of the file structure, and
determines whether we're in the main file by comparing to
pfile->main_file, the _cpp_file of the main file.

	libcpp/
	* internal.h (cpp_in_system_header): Rename to ...
	(_cpp_in_system_header): ... here.
	(cpp_in_primary_file): Rename to ...
	(_cpp_in_main_source_file): ... here.  Compare main_file equality
	and check main_search value.
	* lex.c (maybe_va_opt_error, _cpp_lex_direct): Adjust for rename.
	* macro.c (_cpp_builtin_macro_text): Likewise.
	(replace_args): Likewise.
	* directives.c (do_include_next): Likewise.
	(do_pragma_once, do_pragma_system_header): Likewise.
	* files.c (struct _cpp_file): Delete main_file field.
	(pch_open): Check pfile->main_file equality.
	(make_cpp_file): Drop cpp_reader parm, don't set main_file.
	(_cpp_find_file): Adjust.
	(_cpp_stack_file): Check pfile->main_file equality.
	(struct report_missing_guard_data): Add cpp_reader field.
	(report_missing_guard): Check pfile->main_file equality.
	(_cpp_report_missing_guards): Adjust.
2020-11-19 04:47:00 -08:00
Nathan Sidwell c9c3d5f28a preprocessor: C++ module-directives
C++20 modules introduces a new kind of preprocessor directive -- a
module directive.  These are directives but without the leading '#'.
We have to detect them by sniffing the start of a logical line.  When
detected we replace the initial identifiers with unspellable tokens
and pass them through to the language parser the same way deferred
pragmas are.  There's a PRAGMA_EOL at the logical end of line too.

One additional complication is that we have to do header-name lexing
after the initial tokens, and that requires changes in the macro-aware
piece of the preprocessor.  The above sniffer sets a counter in the
lexer state, and that triggers at the appropriate point.  We then do
the same header-name lexing that occurs on a #include directive or
has_include pseudo-macro.  Except that the header name ends up in the
token stream.

A couple of token emitters need to deal with the new token possibility.

	gcc/c-family/
	* c-lex.c (c_lex_with_flags): CPP_HEADER_NAMEs can now be seen.
	libcpp/
	* include/cpplib.h (struct cpp_options): Add module_directives
	option.
	(NODE_MODULE): New node flag.
	(struct cpp_hashnode): Make rid-code a bitfield, increase bits in
	flags and swap with type field.
	* init.c (post_options): Create module-directive identifier nodes.
	* internal.h (struct lexer_state): Add directive_file_token &
	n_modules fields.  Add module node enumerator.
	* lex.c (cpp_maybe_module_directive): New.
	(_cpp_lex_token): Call it.
	(cpp_output_token): Add '"' around CPP_HEADER_NAME token.
	(do_peek_ident, do_peek_module): New.
	(cpp_directives_only): Detect module-directive lines.
	* macro.c (cpp_get_token_1): Deal with directive_file_token
	triggering.
2020-11-18 10:24:12 -08:00
Nathan Sidwell 8bd9a00f43 cpplib: EOF in pragmas
This patch moves the generation of PRAGMA_EOF earlier, to when we set
need_line, rather than when we try and get the next line.  It also
prevents peeking past a PRAGMA token.

	libcpp/
	* lex.c (cpp_peek_token): Do not peek past CPP_PRAGMA.
	(_cpp_lex_direct): Handle EOF in pragma when setting need_line,
	not when needing a line.
2020-11-03 10:07:20 -08:00
Nathan Sidwell 082a7b2390 cpplib: Fix off-by-one error
I noticed a fencepost error in the preprocessor.  We should be
checking if the next char is at the limit, not the current char (which
can't be, because we're looking at it).

	libcpp/
	* lex.c (_cpp_clean_line): Fix DOS off-by-one error.
2020-11-03 08:49:25 -08:00
Nathan Sidwell dbcc6b1577 preprocessor: Further fix for EOF in macro args [PR97471]
My previous attempt at fixing this was incorrect.  The problem occurs
earlier in that _cpp_lex_direct processes the unwinding EOF needs in
collect_args mode.  This patch changes it not to do that, in the same
way as directive parsing works.  Also collect_args shouldn't push_back
such fake EOFs, and neither should funlike_invocation_p.

	libcpp/
	* lex.c (_cpp_lex_direct): Do not complete EOF processing when
	parsing_args.
	* macro.c (collect_args): Do not unwind fake EOF.
	(funlike_invocation_p): Do not unwind fake EOF.
	(cpp_context): Replace abort with gcc_assert.
	gcc/testsuite/
	* gcc.dg/cpp/endif.c: Move to ...
	* c-c++-common/cpp/endif.c: ... here.
	* gcc.dg/cpp/endif.h: Move to ...
	* c-c++-common/cpp/endif.h: ... here.
	* c-c++-common/cpp/eof-2.c: Adjust diagnostic.
	* c-c++-common/cpp/eof-3.c: Adjust diagnostic.
2020-10-20 08:01:34 -07:00
Jakub Jelinek d00b1b023e powerpc, libcpp: Fix gcc build with clang on power8 [PR97163]
libcpp has two specialized altivec implementations of search_line_fast,
one for power8+ and the other one otherwise.
Both use __attribute__((altivec(vector))) and the GCC builtins rather than
altivec.h and the APIs from there, which is fine, but should be restricted
to when libcpp is built with GCC, so that it can be relied on.
The second elif is
and thus e.g. when built with clang it isn't picked, but the first one was
just guarded with
and so according to the bugreporter clang fails miserably on that.

The following patch fixes that by adding the same GCC_VERSION requirement
as the second version.  I don't know where the 4.5 in there comes from and
the exact version doesn't matter that much, as long as it is above 4.2 that
clang pretends to be and smaller or equal to 4.8 as the oldest gcc we
support as bootstrap compiler ATM.
Furthermore, the patch fixes the comment, the version it is talking about is
not pre-GCC 5, but actually the GCC 5+ one.

2020-09-26  Jakub Jelinek  <jakub@redhat.com>

	PR bootstrap/97163
	* lex.c (search_line_fast): Only use _ARCH_PWR8 Altivec version
	for GCC >= 4.5.
2020-09-26 10:07:41 +02:00
Jakub Jelinek ae49af9485 libcpp: Fix up raw string literal parsing error-recovery [PR96323]
For (invalid) newline inside of the raw string literal delimiter, doing
continue means we skip the needed processing of newlines.  Instead of
duplicating that, this patch just doesn't continue for those.

2020-07-28  Jakub Jelinek  <jakub@redhat.com>

	PR preprocessor/96323
	* lex.c (lex_raw_string): For c == '\n' don't continue after reporting
	an prefix delimiter error.

	* c-c++-common/cpp/pr96323.c: New test.
2020-07-28 15:40:15 +02:00
Nathan Sidwell ed63c387aa preprocessor: Reimplement raw string lexing [pr95149]
pr95149 is a false positive static analysis checker.  But it
encouranged me to fix raw string lexing, which does contain a
complicated macro and pointers to local variables.  The
reimplementation does away with that macro.  Part of the complication
is we need to undo some of the fresh line processing -- trigraph notes
and escaped line continuations.  But the undone characters need to go
through the raw string processing, as they can legitimately be part of
the prefix marker.  however, in this reformulation we only process one
line marker at a time[*], so there's a limited number of undone
characters.  We can arrange the buffering to make sure we don't split
such an append sequence, and then simply take the characters from the
append buffer.

The prefix scanner had a switch statement, which I discovered was not
optimized as well as an if of a bunch of explicit comparisons (pr
95208 filed).

Finally I adjusted the failure mode.  When we get a bad prefix, we lex
up until the next '"', thus often swallowing the whole raw string.
Previously we'd bail and then the lexer would usually generate stupid
tokens, particularly when meeting the ending '"'.

	libcpp/
	* lex.c (struct lit_accum): New.
	(bufring_append): Replace by lit_accum::append.
	(lex_raw_string): Reimplement, using fragments of the old version.
	(lex_string): Adjust lex_raw_string call.

	gcc/testsuite/
	* c-c++-common/raw-string-14.c: Adjust errors.
	* c-c++-common/raw-string-16.c: Likewise.
	* c-c++-common/raw-string-5.c: Likewise.
2020-05-19 11:39:15 -07:00
Jason Merrill b04445d4a8 c++: Replace "C++2a" with "C++20".
C++20 isn't final quite yet, but all that remains is formalities, so let's
go ahead and change all the references.

I think for the next C++ standard we can just call it C++23 rather than
C++2b, since the committee has been consistent about time-based releases
rather than feature-based.

gcc/c-family/ChangeLog
2020-05-13  Jason Merrill  <jason@redhat.com>

	* c.opt (std=c++20): Make c++2a the alias.
	(std=gnu++20): Likewise.
	* c-common.h (cxx_dialect): Change cxx2a to cxx20.
	* c-opts.c: Adjust.
	* c-cppbuiltin.c: Adjust.
	* c-ubsan.c: Adjust.
	* c-warn.c: Adjust.

gcc/cp/ChangeLog
2020-05-13  Jason Merrill  <jason@redhat.com>

	* call.c, class.c, constexpr.c, constraint.cc, decl.c, init.c,
	lambda.c, lex.c, method.c, name-lookup.c, parser.c, pt.c, tree.c,
	typeck2.c: Change cxx2a to cxx20.

libcpp/ChangeLog
2020-05-13  Jason Merrill  <jason@redhat.com>

	* include/cpplib.h (enum c_lang): Change CXX2A to CXX20.
	* init.c, lex.c: Adjust.
2020-05-13 15:16:49 -04:00
Nathan Sidwell 2a0225e478 preprocessor: EOF location is at end of file [PR95013]
My recent C++ parser change to pay attention to EOF location uncovered
a separate bug.  The preprocesor's EOF logic would set the EOF
location to be the beginning of the last line of text in the file --
not the 'line' after that, which contains no characters.  Mostly.
This fixes things so that when we attempt to read the last line of the
main file, we don't pop the buffer until the tokenizer has a chance to
create an EOF token with the correct location information.  It is then
responsible for popping the buffer.  As it happens, raw string literal
tokenizing contained a bug -- it would increment the line number
prematurely, because it cached buffer->cur in a local variable, but
checked buffer->cur before updating it to figure out if it was at end
of file.   We fix up that too.

The EOF token intentionally doesn't have a column number -- it's not a
position on a line, it's a non-existant line.

The testsuite churn is just correcting the EOF location diagnostics.

	libcpp/
	PR preprocessor/95013
	* lex.c (lex_raw_string): Process line notes before incrementing.
	Correct incrementing condition.  Adjust for new
	_cpp_get_fresh_line EOF behaviour.
	(_cpp_get_fresh_line): Do not pop buffer at EOF, increment line
	instead.
	(_cpp_lex_direct): Adjust for new _cpp_get_fresh_line behaviour.
	(cpp_directive_only_process): Assert we got a fresh line.
	* traditional.c (_cpp_read_logical_line_trad): Adjust for new
	_cpp_get_fresh_line behaviour.

	gcc/testsuite/
	* c-c++-common/goacc/pr79428-1.c: Adjust EOF diagnostic location.
	* c-c++-common/gomp/pr79428-2.c: Likewise.
	* g++.dg/cpp0x/decltype63.C: Likewise.
	* g++.dg/cpp0x/gen-attrs-64.C: Likewise.
	* g++.dg/cpp0x/pr68726.C: Likewise.
	* g++.dg/cpp0x/pr78341.C: Likewise.
	* g++.dg/cpp1y/pr65202.C: Likewise.
	* g++.dg/cpp1y/pr65340.C: Likewise.
	* g++.dg/cpp1y/pr68578.C: Likewise.
	* g++.dg/cpp1z/class-deduction44.C: Likewise.
	* g++.dg/diagnostic/unclosed-extern-c.C: Likewise.
	* g++.dg/diagnostic/unclosed-function.C: Likewise.
	* g++.dg/diagnostic/unclosed-namespace.C: Likewise.
	* g++.dg/diagnostic/unclosed-struct.C: Likewise.
	* g++.dg/ext/pr84598.C: Likewise.
	* g++.dg/other/switch4.C: Likewise.
	* g++.dg/parse/attr4.C: Likewise.
	* g++.dg/parse/cond4.C: Likewise.
	* g++.dg/parse/crash10.C: Likewise.
	* g++.dg/parse/crash18.C: Likewise.
	* g++.dg/parse/crash27.C: Likewise.
	* g++.dg/parse/crash34.C: Likewise.
	* g++.dg/parse/crash35.C: Likewise.
	* g++.dg/parse/crash52.C: Likewise.
	* g++.dg/parse/crash59.C: Likewise.
	* g++.dg/parse/crash61.C: Likewise.
	* g++.dg/parse/crash67.C: Likewise.
	* g++.dg/parse/error14.C: Likewise.
	* g++.dg/parse/error56.C: Likewise.
	* g++.dg/parse/invalid1.C: Likewise.
	* g++.dg/parse/parameter-declaration-1.C: Likewise.
	* g++.dg/parse/parser-pr28152-2.C: Likewise.
	* g++.dg/parse/parser-pr28152.C: Likewise.
	* g++.dg/parse/pr68722.C: Likewise.
	* g++.dg/pr46852.C: Likewise.
	* g++.dg/pr46868.C: Likewise.
	* g++.dg/template/crash115.C: Likewise.
	* g++.dg/template/crash43.C: Likewise.
	* g++.dg/template/crash90.C: Likewise.
	* g++.dg/template/error-recovery1.C: Likewise.
	* g++.dg/template/error57.C: Likewise.
	* g++.old-deja/g++.other/crash31.C: Likewise.
	* gcc.dg/empty-source-2.c: Likewise.
	* gcc.dg/empty-source-3.c: Likewise.
	* gcc.dg/noncompile/pr30552-3.c: Likewise.
	* gcc.dg/noncompile/pr35447-1.c: Likewise.
	* gcc.dg/pr20245-1.c: Likewise.
	* gcc.dg/pr28419.c: Likewise.
	* gcc.dg/rtl/truncated-rtl-file.c: Likewise.
	* gcc.dg/unclosed-init.c: Likewise.
	* obj-c++.dg/property/property-neg-6.mm: Likewise.
	* obj-c++.dg/syntax-error-10.mm: Likewise.
	* obj-c++.dg/syntax-error-8.mm: Likewise.
	* obj-c++.dg/syntax-error-9.mm: Likewise.
2020-05-12 13:40:29 -07:00
Nathan Sidwell b224c3763e preprocessor: Reimplement directives only processing, support raw literals.
The existing directives-only code (a) punched a hole through the
libcpp interface and (b) didn't support raw string literals.  This
reimplements this preprocessing mode.  I added a proper callback
interface, and adjusted c-ppoutput to use it.  Sadly I cannot get rid
of the libcpp/internal.h include for unrelated reasons.

The new scanner is in lex.x, and works doing some backwards scanning
when it finds a charater of interest.  This reduces the number of
cases one has to deal with in forward scanning.  It may have different
failure mode than forward scanning on bad tokenization.

Finally, Moved some cpp tests from the c-specific dg.gcc/cpp directory
to the c-c++-common/cpp shared directory,

	libcpp/
	* directives-only.c: Delete.
	* Makefile.in (libcpp_a_OBJS, libcpp_a_SOURCES): Remove it.
	* include/cpplib.h (enum CPP_DO_task): New enum.
	(cpp_directive_only_preprocess): Declare.
	* internal.h (_cpp_dir_only_callbacks): Delete.
	(_cpp_preprocess_dir_only): Delete.
	* lex.c (do_peek_backslask, do_peek_next, do_peek_prev): New.
	(cpp_directives_only_process): New implementation.

	gcc/c-family/
	Reimplement directives only processing.
	* c-ppoutput.c (token_streamer): Ne.
	(directives_only_cb): New.  Swallow ...
	(print_lines_directives_only): ... this.
	(scan_translation_unit_directives_only): Reimplment using the
	published interface.

	gcc/testsuite/
	* gcc.dg/cpp/counter-[23].c: Move to c-c+_-common/cpp.
	* gcc.dg/cpp/dir-only-*: Likewise.
	* c-c++-common/cpp/dir-only-[78].c: New.
2020-05-08 11:13:29 -07:00
Jakub Jelinek 8d9254fc8a Update copyright years.
From-SVN: r279813
2020-01-01 12:51:42 +01:00
Jason Merrill b7689b962d Implement C++20 operator<=>.
There are three major pieces to this support: scalar operator<=>,
synthesis of comparison operators, and rewritten/reversed overload
resolution (e.g. a < b becomes 0 > b <=> a).

Unlike other defaulted functions, where we use synthesized_method_walk to
semi-simulate what the definition of the function will be like, this patch
determines the characteristics of a comparison operator by trying to define
it.

My handling of non-dependent rewritten operators in templates can still use
some work: build_min_non_dep_op_overload can't understand the rewrites and
crashes, so I'm avoiding it for now by clearing *overload.  This means we'll
do name lookup again at instantiation time, which can incorrectly mean a
different result.  I'll poke at this more in stage 3.

I'm leaving out a fourth section ("strong structural equality") even though
I've implemented it, because it seems likely to change radically tomorrow.

Thanks to Tim van Deurzen and Jakub for implementing lexing of the <=>
operator, and Jonathan for the initial <compare> header.

gcc/cp/
	* cp-tree.h (struct lang_decl_fn): Add maybe_deleted bitfield.
	(DECL_MAYBE_DELETED): New.
	(enum special_function_kind): Add sfk_comparison.
	(LOOKUP_REWRITTEN, LOOKUP_REVERSED): New.
	* call.c (struct z_candidate): Add rewritten and reversed methods.
	(add_builtin_candidate): Handle SPACESHIP_EXPR.
	(add_builtin_candidates): Likewise.
	(add_candidates): Don't add a reversed candidate if the parms are
	the same.
	(add_operator_candidates): Split out from build_new_op_1.  Handle
	rewritten and reversed candidates.
	(add_candidate): Swap conversions of reversed candidate.
	(build_new_op_1): Swap them back.  Build a second operation for
	rewritten candidates.
	(extract_call_expr): Handle rewritten calls.
	(same_fn_or_template): New.
	(joust): Handle rewritten and reversed candidates.
	* class.c (add_implicitly_declared_members): Add implicit op==.
	(classtype_has_op, classtype_has_defaulted_op): New.
	* constexpr.c (cxx_eval_binary_expression): Handle SPACESHIP_EXPR.
	(cxx_eval_constant_expression, potential_constant_expression_1):
	Likewise.
	* cp-gimplify.c (genericize_spaceship): New.
	(cp_genericize_r): Use it.
	* cp-objcp-common.c (cp_common_init_ts): Handle SPACESHIP_EXPR.
	* decl.c (finish_function): Handle deleted function.
	* decl2.c (grokfield): SET_DECL_FRIEND_CONTEXT on defaulted friend.
	(mark_used): Check DECL_MAYBE_DELETED.  Remove assumption that
	defaulted functions are non-static members.
	* error.c (dump_expr): Handle SPACESHIP_EXPR.
	* method.c (type_has_trivial_fn): False for sfk_comparison.
	(enum comp_cat_tag, struct comp_cat_info_t): New types.
	(comp_cat_cache): New array variable.
	(lookup_comparison_result, lookup_comparison_category)
	(is_cat, cat_tag_for, spaceship_comp_cat)
	(spaceship_type, genericize_spaceship)
	(common_comparison_type, early_check_defaulted_comparison)
	(comp_info, build_comparison_op): New.
	(synthesize_method): Handle sfk_comparison.  Handle deleted.
	(get_defaulted_eh_spec, maybe_explain_implicit_delete)
	(explain_implicit_non_constexpr, implicitly_declare_fn)
	(defaulted_late_check, defaultable_fn_check): Handle sfk_comparison.
	* name-lookup.c (get_std_name_hint): Add comparison categories.
	* tree.c (special_function_p): Add sfk_comparison.
	* typeck.c (cp_build_binary_op): Handle SPACESHIP_EXPR.

2019-11-05  Tim van Deurzen  <tim@kompiler.org>

	Add new tree code for the spaceship operator.
gcc/cp/
	* cp-tree.def: Add new tree code.
	* operators.def: New binary operator.
	* parser.c: Add new token and tree code.
libcpp/
	* cpplib.h: Add spaceship operator for C++.
	* lex.c: Implement conditional lexing of spaceship operator for C++20.

2019-11-05  Jonathan Wakely  <jwakely@redhat.com>

libstdc++-v3/
	* libsupc++/compare: New header.
	* libsupc++/Makefile.am (std_HEADERS): Add compare.
	* include/std/version: Define __cpp_lib_three_way_comparison.
	* include/std/functional: #include <compare>.

From-SVN: r277865
2019-11-05 18:56:18 -05:00
Joseph Myers 93313b94fe Handle :: tokens in C for C2x.
As part of adding [[]]-style attributes, C2x adds the token :: for use
in scoped attribute names.

This patch adds corresponding support for that token in C to GCC.  The
token is supported both for C2x and for older gnu* standards (on the
basis that extensions are normally supported in older gnu* versions;
people will expect to be able to use [[]] attributes, before C2x is
the default, without needing to use -std=gnu2x).

There are no cases in older C standards where the token : can be
followed by a token starting with : in syntactically valid sources;
the only cases the :: token could break in older standard C thus are
ones involving concatenation of pp-tokens where the result does not
end up as tokens (e.g., gets stringized).  In GNU C extensions, the
main case where :: might appear in existing sources is in asm
statements, and the C parser is thus made to handle it like two
consecutive : tokens, which the C++ parser already does.  A limited
test of various positionings of :: in asm statements is added to the
testsuite (in particular, to cover the syntax error when :: means too
many colons but a single : would be OK), but existing tests cover a
variety of styles there anyway.

Technically there are cases in Objective-C and OpenMP for which this
also changes how previously valid code is lexed: the objc-selector-arg
syntax allows multiple consecutive : tokens (although I don't think
they are particularly useful there), while OpenMP syntax includes
array section syntax such as [:] which, before :: was a token, could
also be written as [::> (there might be other OpenMP cases potentially
affected, I didn't check all the OpenMP syntax in detail).  I don't
think either of those cases affects the basis for supporting the ::
token in all -std=gnu* modes, or that there is any obvious need to
special-case handling of CPP_SCOPE tokens for those constructs the way
there is for asm statements.

cpp_avoid_paste, which determines when spaces need adding between
tokens in preprocessed output where there wouldn't otherwise be
whitespace between them (e.g. if stringized), already inserts space
between : and : unconditionally, rather than only for C++, so no
change is needed there (but a C2x test is added that such space is
indeed inserted).

Bootstrapped with no regressions on x86-64-pc-linux-gnu.

gcc/c:
	* c-parser.c (c_parser_asm_statement): Handle CPP_SCOPE like two
	CPP_COLON tokens.

gcc/testsuite:
	* gcc.dg/asm-scope-1.c, gcc.dg/cpp/c11-scope-1.c,
	gcc.dg/cpp/c17-scope-1.c, gcc.dg/cpp/c2x-scope-1.c,
	gcc.dg/cpp/c2x-scope-2.c, gcc.dg/cpp/c90-scope-1.c,
	gcc.dg/cpp/c94-scope-1.c, gcc.dg/cpp/c99-scope-1.c,
	gcc.dg/cpp/gnu11-scope-1.c, gcc.dg/cpp/gnu17-scope-1.c,
	gcc.dg/cpp/gnu89-scope-1.c, gcc.dg/cpp/gnu99-scope-1.c: New tests.

libcpp:
	* include/cpplib.h (struct cpp_options): Add member scope.
	* init.c (struct lang_flags, lang_defaults): Likewise.
	(cpp_set_lang): Set scope member of pfile.
	* lex.c (_cpp_lex_direct): Test CPP_OPTION (pfile, scope) not
	CPP_OPTION (pfile, cplusplus) for creating CPP_SCOPE tokens.

From-SVN: r276434
2019-10-02 01:08:40 +01:00
Lewis Hyatt 7d112d6670 Support extended characters in C/C++ identifiers (PR c/67224)
libcpp/ChangeLog
2019-09-19  Lewis Hyatt  <lhyatt@gmail.com>

	PR c/67224
	* charset.c (_cpp_valid_utf8): New function to help lex UTF-8 tokens.
	* internal.h (_cpp_valid_utf8): Declare.
	* lex.c (forms_identifier_p): Use it to recognize UTF-8 identifiers.
	(_cpp_lex_direct): Handle UTF-8 in identifiers and CPP_OTHER tokens.
	Do all work in "default" case to avoid slowing down typical code paths.
	Also handle $ and UCN in the default case for consistency.

gcc/Changelog
2019-09-19  Lewis Hyatt  <lhyatt@gmail.com>

	PR c/67224
	* doc/cpp.texi: Document support for extended characters in
	identifiers.
	* doc/cppopts.texi: Likewise.

gcc/testsuite/ChangeLog
2019-09-19  Lewis Hyatt  <lhyatt@gmail.com>

	PR c/67224
	* c-c++-common/cpp/ucnid-2011-1-utf8.c: New test.
	* g++.dg/cpp/ucnid-1-utf8.C: New test.
	* g++.dg/cpp/ucnid-2-utf8.C: New test.
	* g++.dg/cpp/ucnid-3-utf8.C: New test.
	* g++.dg/cpp/ucnid-4-utf8.C: New test.
	* g++.dg/other/ucnid-1-utf8.C: New test.
	* gcc.dg/cpp/ucnid-1-utf8.c: New test.
	* gcc.dg/cpp/ucnid-10-utf8.c: New test.
	* gcc.dg/cpp/ucnid-11-utf8.c: New test.
	* gcc.dg/cpp/ucnid-12-utf8.c: New test.
	* gcc.dg/cpp/ucnid-13-utf8.c: New test.
	* gcc.dg/cpp/ucnid-14-utf8.c: New test.
	* gcc.dg/cpp/ucnid-15-utf8.c: New test.
	* gcc.dg/cpp/ucnid-2-utf8.c: New test.
	* gcc.dg/cpp/ucnid-3-utf8.c: New test.
	* gcc.dg/cpp/ucnid-4-utf8.c: New test.
	* gcc.dg/cpp/ucnid-6-utf8.c: New test.
	* gcc.dg/cpp/ucnid-7-utf8.c: New test.
	* gcc.dg/cpp/ucnid-9-utf8.c: New test.
	* gcc.dg/ucnid-1-utf8.c: New test.
	* gcc.dg/ucnid-10-utf8.c: New test.
	* gcc.dg/ucnid-11-utf8.c: New test.
	* gcc.dg/ucnid-12-utf8.c: New test.
	* gcc.dg/ucnid-13-utf8.c: New test.
	* gcc.dg/ucnid-14-utf8.c: New test.
	* gcc.dg/ucnid-15-utf8.c: New test.
	* gcc.dg/ucnid-16-utf8.c: New test.
	* gcc.dg/ucnid-2-utf8.c: New test.
	* gcc.dg/ucnid-3-utf8.c: New test.
	* gcc.dg/ucnid-4-utf8.c: New test.
	* gcc.dg/ucnid-5-utf8.c: New test.
	* gcc.dg/ucnid-6-utf8.c: New test.
	* gcc.dg/ucnid-7-utf8.c: New test.
	* gcc.dg/ucnid-8-utf8.c: New test.
	* gcc.dg/ucnid-9-utf8.c: New test.

From-SVN: r275979
2019-09-19 20:56:11 +01:00
Nathan Sidwell 056f95ec95 [preprocessor/91639] #includes at EOF
https://gcc.gnu.org/ml/gcc-patches/2019-09/msg00280.html
	libcpp/
	PR preprocessor/91639
	* directives.c (do_include_common): Tell lexer we're a #include.
	* files.c (_cpp_stack_file): Lexer will have always incremented.
	* internal.h (struct cpp_context): Extend in_directive's
	semantics.
	* lex.c (_cpp_lex_direct): Increment line for final \n when lexing
	for an ISO #include.
	* line-map.c (linemap_line_start): Remember if we overflowed.

	gcc/testsuite/
	PR preprocessor/91639
	* c-c++-common/cpp/pr91639.c: New.
	* c-c++-common/cpp/pr91639-one.h: New.
	* c-c++-common/cpp/pr91639-two.h: New.

From-SVN: r275402
2019-09-05 11:23:48 +00:00
Andrew Pinski 3f23e487f3 [PATCH] Fix PR 81721: ICE with PCH and Pragma warning and C++ operator
libcpp/ChangeLog:
2019-05-19  Andrew Pinski  <apinski@marvell.com>

        PR pch/81721
        * lex.c (cpp_token_val_index <case SPELL_OPERATOR>): If tok->flags
        has NAMED_OP set, then return CPP_TOKEN_FLD_NODE.

gcc/testsuite/ChangeLog:
2019-05-19  Andrew Pinski  <apinski@marvell.com>

        PR pch/81721
        * g++.dg/pch/operator-1.C: New testcase.
        * g++.dg/pch/operator-1.Hs: New file.

From-SVN: r271395
2019-05-19 23:59:06 -07:00
Jakub Jelinek a554497024 Update copyright years.
From-SVN: r267494
2019-01-01 13:31:55 +01:00
David Malcolm 620e594be5 Eliminate source_location in favor of location_t
Historically GCC used location_t, while libcpp used source_location.

This inconsistency has been annoying me for a while, so this patch
removes source_location in favor of location_t throughout
(as the latter is shorter).

gcc/ChangeLog:
	* builtins.c: Replace "source_location" with "location_t".
	* diagnostic-show-locus.c: Likewise.
	* diagnostic.c: Likewise.
	* dumpfile.c: Likewise.
	* gcc-rich-location.h: Likewise.
	* genmatch.c: Likewise.
	* gimple.h: Likewise.
	* gimplify.c: Likewise.
	* input.c: Likewise.
	* input.h: Likewise.  Eliminate the typedef.
	* omp-expand.c: Likewise.
	* selftest.h: Likewise.
	* substring-locations.h (get_source_location_for_substring):
	Rename to..
	(get_location_within_string): ...this.
	* tree-cfg.c: Replace "source_location" with "location_t".
	* tree-cfgcleanup.c: Likewise.
	* tree-diagnostic.c: Likewise.
	* tree-into-ssa.c: Likewise.
	* tree-outof-ssa.c: Likewise.
	* tree-parloops.c: Likewise.
	* tree-phinodes.c: Likewise.
	* tree-phinodes.h: Likewise.
	* tree-ssa-loop-ivopts.c: Likewise.
	* tree-ssa-loop-manip.c: Likewise.
	* tree-ssa-phiopt.c: Likewise.
	* tree-ssa-phiprop.c: Likewise.
	* tree-ssa-threadupdate.c: Likewise.
	* tree-ssa.c: Likewise.
	* tree-ssa.h: Likewise.
	* tree-vect-loop-manip.c: Likewise.

gcc/c-family/ChangeLog:
	* c-common.c (c_get_substring_location): Update for renaming of
	get_source_location_for_substring to get_location_within_string.
	* c-lex.c: Replace "source_location" with "location_t".
	* c-opts.c: Likewise.
	* c-ppoutput.c: Likewise.

gcc/c/ChangeLog:
	* c-decl.c: Replace "source_location" with "location_t".
	* c-tree.h: Likewise.
	* c-typeck.c: Likewise.
	* gimple-parser.c: Likewise.

gcc/cp/ChangeLog:
	* call.c: Replace "source_location" with "location_t".
	* cp-tree.h: Likewise.
	* cvt.c: Likewise.
	* name-lookup.c: Likewise.
	* parser.c: Likewise.
	* typeck.c: Likewise.

gcc/fortran/ChangeLog:
	* cpp.c: Replace "source_location" with "location_t".
	* gfortran.h: Likewise.

gcc/go/ChangeLog:
	* go-gcc-diagnostics.cc: Replace "source_location" with "location_t".
	* go-gcc.cc: Likewise.
	* go-linemap.cc: Likewise.
	* go-location.h: Likewise.
	* gofrontend/README: Likewise.

gcc/jit/ChangeLog:
	* jit-playback.c: Replace "source_location" with "location_t".

gcc/testsuite/ChangeLog:
	* g++.dg/plugin/comment_plugin.c: Replace "source_location" with
	"location_t".
	* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c: Likewise.

libcc1/ChangeLog:
	* libcc1plugin.cc: Replace "source_location" with "location_t".
	(plugin_context::get_source_location): Rename to...
	(plugin_context::get_location_t): ...this.
	* libcp1plugin.cc: Likewise.

libcpp/ChangeLog:
	* charset.c: Replace "source_location" with "location_t".
	* directives-only.c: Likewise.
	* directives.c: Likewise.
	* errors.c: Likewise.
	* expr.c: Likewise.
	* files.c: Likewise.
	* include/cpplib.h: Likewise.  Rename MAX_SOURCE_LOCATION to
	MAX_LOCATION_T.
	* include/line-map.h: Likewise.
	* init.c: Likewise.
	* internal.h: Likewise.
	* lex.c: Likewise.
	* line-map.c: Likewise.
	* location-example.txt: Likewise.
	* macro.c: Likewise.
	* pch.c: Likewise.
	* traditional.c: Likewise.

From-SVN: r266085
2018-11-13 20:05:03 +00:00
Nathan Sidwell f3f6029db2 [6/6] Preprocessor forced macro location
https://gcc.gnu.org/ml/gcc-patches/2018-10/msg02044.html
	libcpp/
	* internal.h (struct cpp_reader): Rename forced_token_location_p
	to forced_token_location and drop its pointerness.
	* include/cpplib.h (cpp_force_token_locations): Take location, not
	pointer to one.
	* init.c (cpp_create_reader): Adjust.
	* lex.c (cpp_read_main_file): 

	gcc/c-family/
	* c-opts.c (c_finish_options): Adjust cpp_force_token_locations call.

	gcc/fortran/
	* cpp.c (gfc_cpp_init): Adjust cpp_force_token_locations call.

From-SVN: r265692
2018-10-31 15:26:28 +00:00
Nathan Sidwell 10f04917ab [PATCH] Macro body is trailing array
https://gcc.gnu.org/ml/gcc-patches/2018-08/msg01037.html
	* include/cpplib.h (enum cpp_macro_kind): New.
	(struct cpp_macro): Make body trailing array.  Add kind field,
	delete traditional flag.
	* internal.h (_cpp_new_macro): Declare.
	(_cpp_reserve_room): New inline.
	(_cpp_commit_buf): Declare.
	(_cpp_create_trad_definition): Return new macro.
	* lex.c (_cpp_commit_buff): New.
	* macro.c (macro_real_token_count): Count backwards.
	(replace_args): Pointer equality not orderedness.
	(_cpp_save_parameter): Use _cpp_reserve_room.
	(alloc_expansion_token): Delete.
	(lex_expansion_token): Return macro pointer.  Use _cpp_reserve_room.
	(create_iso_definition): Allocate macro itself.  Adjust for
	different allocation ordering.
	(_cpp_new_macro): New.
	(_cpp_create_definition): Adjust for API changes.
	* traditional.c (push_replacement_text): Don't set traditional
	flag.
	(save_replacement_text): Likewise.
	(_cpp_create_trad_definition): Allocate macro itself, Adjust for
	different allocation ordering.

From-SVN: r263622
2018-08-17 16:07:19 +00:00
Nathan Sidwell 3f6677f418 [PATCH] CPP Macro predicates
https://gcc.gnu.org/ml/gcc-patches/2018-08/msg00897.html
	libcpp/
	* include/cpplib.h (cpp_user_macro_p, cpp_builtin_macro_p)
	(cpp_macro_p): New inlines.
	* directives.c (do_pragma_poison): Use cpp_macro_p.
	(do_ifdef, do_ifndef): Likewise.  Use _cpp_maybe_notify_macro_use.
	(cpp_pop_definition): Use cpp_macro_p.  Move _cpp_free_definition
	earlier.  Don't zap node directly.
	* expr.c (parse_defined): Use _cpp_maybe_notify_macro_use &
	cpp_macro_p.
	* files.c (should_stack_file): Use cpp_macro_p.
	* identifiers.c (cpp_defined): Likewise.
	* internal.h (_cpp_mark_macro): Use cpp_user_macro_p.
	(_cpp_notify_macro_use): Declare.
	(_cpp_maybe_notify_macro_use): New inline.
	* lex.c (is_macro): Use cpp_macro_p.
	* macro.c (_cpp_warn_if_unused_macro): Use cpp_user_macro_p.
	(enter_macro_context): Likewise.
	(_cpp_create_definition): Use cpp_builtin_macro_p,
	cpp_user_macro_p.  Move _cpp_free_definition earlier.
	(_cpp_notify_macro_use): New, broken out of multiple call sites.
	* traditional.c (fun_like_macro_p): Use cpp_builtin_macro_p.
	(maybe_start_funlike, _cpp_scan_out_logical_line)
	(push_replacement_text): Likewise.
	gcc/c-family/
	* c-ada-spec.c (count_ada_macro): Use cpp_user_macro_p.
	(store_ada_macro): Likewise.
	* c-ppoutput.c (cb_used_define, dump_macro): Likewise.
	* c-spellcheck.cc (should-suggest_as_macro_p): Likewise,
	gcc/
	* config/rs6000/rs6000-c.c (rs6000_macro_to_expend): Use cpp_macro_p.
	* config/powerpcspc/powerpcspe-c.c (rs6000_macro_to_expend): Likewise.
	gcc/cp/
	* name-lookup.c (lookup_name_fuzzy): Likewise.
	gcc/fortran/
	* cpp.c (dump_macro): Use cpp_user_macro_p.

From-SVN: r263587
2018-08-16 13:51:38 +00:00
Jakub Jelinek 0c86a39db2 lex.c (_cpp_lex_direct): Use CPP_DL_NOTE instead of CPP_DL_PEDWARN...
* lex.c (_cpp_lex_direct): Use CPP_DL_NOTE instead of CPP_DL_PEDWARN,
	CPP_DL_WARNING or CPP_DL_ERROR for note that diagnostics for C++ style
	comments is reported only once per file and guard those calls on the
	preceding cpp_error returning true.

	* gcc.dg/cpp/pr61854-c90.c (foo): Expect a note, rather than error.
	* gcc.dg/cpp/pr61854-c94.c (foo): Likewise.
	* gcc.dg/cpp/pr61854-4.c (foo): Likewise.
	* gcc.dg/cpp/pr61854-8.c: New test.

From-SVN: r262832
2018-07-17 20:10:57 +02:00
Jonathan Wakely b44f8ad8b2 PR preprocessor/84517 allow double-underscore macros after string literals
gcc/testsuite:

	PR preprocessor/84517
	* g++.dg/cpp0x/udlit-macros.C: Expect a warning for ""__FILE__.

libcpp:

	PR preprocessor/84517
	* lex.c (is_macro_not_literal_suffix): New function.
	(lex_raw_string, lex_string): Use is_macro_not_literal_suffix to
	decide when to issue -Wliteral-suffix warnings.

From-SVN: r258069
2018-02-28 15:27:17 +00:00
Kelvin Nilsen a3a821c903 rs6000-p8swap.c (rs6000_sum_of_two_registers_p): New function.
gcc/ChangeLog:

2018-01-10  Kelvin Nilsen  <kelvin@gcc.gnu.org>

	* config/rs6000/rs6000-p8swap.c (rs6000_sum_of_two_registers_p):
	New function.
	(rs6000_quadword_masked_address_p): Likewise.
	(quad_aligned_load_p): Likewise.
	(quad_aligned_store_p): Likewise.
	(const_load_sequence_p): Add comment to describe the outer-most loop.
	(mimic_memory_attributes_and_flags): New function.
	(rs6000_gen_stvx): Likewise.
	(replace_swapped_aligned_store): Likewise.
	(rs6000_gen_lvx): Likewise.
	(replace_swapped_aligned_load): Likewise.
	(replace_swapped_load_constant): Capitalize argument name in
	comment describing this function.
	(rs6000_analyze_swaps): Add a third pass to search for vector loads
	and stores that access quad-word aligned addresses and replace
	with stvx or lvx instructions when appropriate.
	* config/rs6000/rs6000-protos.h (rs6000_sum_of_two_registers_p):
	New function prototype.
	(rs6000_quadword_masked_address_p): Likewise.
	(rs6000_gen_lvx): Likewise.
	(rs6000_gen_stvx): Likewise.
	* config/rs6000/vsx.md (*vsx_le_perm_load_<mode>): For modes
	VSX_D (V2DF, V2DI), modify this split to select lvx instruction
	when memory address is aligned.
	(*vsx_le_perm_load_<mode>): For modes VSX_W (V4SF, V4SI), modify
	this split to select lvx instruction when memory address is aligned.
	(*vsx_le_perm_load_v8hi): Modify this split to select lvx
	instruction when memory address is aligned.
	(*vsx_le_perm_load_v16qi): Likewise.
	(four unnamed splitters): Modify to select the stvx instruction
	when memory is aligned.

gcc/testsuite/ChangeLog:

2018-01-10  Kelvin Nilsen  <kelvin@gcc.gnu.org>

	* gcc.target/powerpc/pr48857.c: Modify dejagnu directives to look
	for lvx and stvx instead of lxvd2x and stxvd2x and require
	little-endian target.  Add comments.
	* gcc.target/powerpc/swaps-p8-28.c: Add functions for more
	comprehensive testing.
	* gcc.target/powerpc/swaps-p8-29.c: Likewise.
	* gcc.target/powerpc/swaps-p8-30.c: Likewise.
	* gcc.target/powerpc/swaps-p8-31.c: Likewise.
	* gcc.target/powerpc/swaps-p8-32.c: Likewise.
	* gcc.target/powerpc/swaps-p8-33.c: Likewise.
	* gcc.target/powerpc/swaps-p8-34.c: Likewise.
	* gcc.target/powerpc/swaps-p8-35.c: Likewise.
	* gcc.target/powerpc/swaps-p8-36.c: Likewise.
	* gcc.target/powerpc/swaps-p8-37.c: Likewise.
	* gcc.target/powerpc/swaps-p8-38.c: Likewise.
	* gcc.target/powerpc/swaps-p8-39.c: Likewise.
	* gcc.target/powerpc/swaps-p8-40.c: Likewise.
	* gcc.target/powerpc/swaps-p8-41.c: Likewise.
	* gcc.target/powerpc/swaps-p8-42.c: Likewise.
	* gcc.target/powerpc/swaps-p8-43.c: Likewise.
	* gcc.target/powerpc/swaps-p8-44.c: Likewise.
	* gcc.target/powerpc/swaps-p8-45.c: Likewise.
	* gcc.target/powerpc/vec-extract-2.c: Add comment and remove
	scan-assembler-not directives that forbid lvx and xxpermdi.
	* gcc.target/powerpc/vec-extract-3.c: Likewise.
	* gcc.target/powerpc/vec-extract-5.c: Likewise.
	* gcc.target/powerpc/vec-extract-6.c: Likewise.
	* gcc.target/powerpc/vec-extract-7.c: Likewise.
	* gcc.target/powerpc/vec-extract-8.c: Likewise.
	* gcc.target/powerpc/vec-extract-9.c: Likewise.
	* gcc.target/powerpc/vsx-vector-6-le.c: Change
	scan-assembler-times directives to reflect different numbers of
	expected xxlnor, xxlor, xvcmpgtdp, and xxland instructions.

libcpp/ChangeLog:

2018-01-10  Kelvin Nilsen  <kelvin@gcc.gnu.org>

	* lex.c (search_line_fast): Remove illegal coercion of an
	unaligned pointer value to vector pointer type and replace with
	use of __builtin_vec_vsx_ld () built-in function, which operates
	on unaligned pointer values.

From-SVN: r256656
2018-01-14 05:19:29 +00:00
Jakub Jelinek 85ec4feb11 Update copyright years.
From-SVN: r256169
2018-01-03 11:03:58 +01:00
Michael Weiser 35c4515b8b [PATCH, PR83492] Fix selection of aarch64 big-endian shift parameters based on __AARCH64EB__
2017-12-20  Michael Weiser  <michael.weiser@gmx.de>

	PR preprocessor/83492
	* lex.c (search_line_fast) [__ARM_NEON && __ARM_64BIT_STATE]:
	Fix selection of big-endian shift parameters by using
	__ARM_BIG_ENDIAN.

From-SVN: r255896
2017-12-20 15:07:01 +00:00
Tom Tromey fb771b9dad Implement __VA_OPT__
This implements __VA_OPT__, a new preprocessor feature added in C++2A.
The paper can be found here:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0306r4.html

gcc/ChangeLog

        * doc/cpp.texi (Variadic Macros): Document __VA_OPT__.

gcc/testsuite/ChangeLog

        * c-c++-common/cpp/va-opt-pedantic.c: New file.
        * c-c++-common/cpp/va-opt.c: New file.
        * c-c++-common/cpp/va-opt-error.c: New file.

libcpp/ChangeLog

        * pch.c (cpp_read_state): Set n__VA_OPT__.
        * macro.c (vaopt_state): New class.
        (_cpp_arguments_ok): Check va_opt flag.
        (replace_args, create_iso_definition): Use vaopt_state.
        * lex.c (lex_identifier_intern): Possibly issue errors for
        __VA_OPT__.
        (lex_identifier): Likewise.
        (maybe_va_opt_error): New function.
        * internal.h (struct lexer_state) <va_args_ok>: Update comment.
        (struct spec_nodes) <n__VA_OPT__>: New field.
        * init.c (struct lang_flags) <va_opt>: New field.
        (lang_defaults): Add entries for C++2A.  Update all entries for
        va_opt.
        (cpp_set_lang): Initialize va_opt.
        * include/cpplib.h (struct cpp_options) <va_opt>: New field.
        * identifiers.c (_cpp_init_hashtable): Initialize n__VA_OPT__.

From-SVN: r254707
2017-11-13 20:17:42 +00:00
Mukesh Kapoor 7d19c460ed re PR c++/80955 (Macros expanded in definition of user-defined literals)
/libcpp
2017-11-06  Mukesh Kapoor  <mukesh.kapoor@oracle.com>

	PR c++/80955
	* lex.c (lex_string): When checking for a valid macro for the
	warning related to -Wliteral-suffix (CPP_W_LITERAL_SUFFIX),
	check that the macro name does not start with an underscore
	before calling is_macro().

/gcc/testsuite
2017-11-06  Mukesh Kapoor  <mukesh.kapoor@oracle.com>

	PR c++/80955
	* g++.dg/cpp0x/udlit-macros.C: New.

From-SVN: r254443
2017-11-06 10:33:41 +00:00
Tom de Vries c830c7d5c7 [libcpp] Remove semicolon after do {} while (0) in BUF_APPEND
2017-11-05  Tom de Vries  <tom@codesourcery.com>

	PR other/82784
	* lex.c (BUF_APPEND): Remove semicolon after
	"do {} while (0)".

From-SVN: r254424
2017-11-05 09:58:16 +00:00
David Malcolm 05945a1b83 libcpp: add callback for comment-handling
gcc/testsuite/ChangeLog:
	* g++.dg/plugin/comment_plugin.c: New test plugin.
	* g++.dg/plugin/comments-1.C: New test file.
	* g++.dg/plugin/plugin.exp (plugin_test_list): Add the above.

libcpp/ChangeLog:
	* include/cpplib.h (struct cpp_callbacks): Add "comment"
	callback.
	* lex.c (_cpp_lex_direct): Call the comment callback if non-NULL.

From-SVN: r248901
2017-06-05 20:53:06 +00:00
Jonathan Wakely 5764ee3c84 Fix numerous typos in comments
gcc:

	* alias.c (base_alias_check): Fix typo in comment.
	* cgraph.h (class ipa_polymorphic_call_context): Likewise.
	* cgraphunit.c (symbol_table::compile): Likewise.
	* collect2.c (maybe_run_lto_and_relink): Likewise.
	* config/arm/arm.c (arm_thumb1_mi_thunk): Likewise.
	* config/avr/avr-arch.h (avr_arch_info_t): Likewise.
	* config/avr/avr.c (avr_map_op_t): Likewise.
	* config/cr16/cr16.h (DATA_ALIGNMENT): Likewise.
	* config/epiphany/epiphany.c (TARGET_ARG_PARTIAL_BYTES): Likewise.
	* config/epiphany/epiphany.md (movcc): Likewise.
	* config/i386/i386.c (legitimize_pe_coff_extern_decl): Likewise.
	* config/m68k/m68k.c (struct _sched_ib, m68k_sched_variable_issue):
	Likewise.
	* config/mips/mips.c (mips_save_restore_reg): Likewise.
	* config/rx/rx.c (rx_is_restricted_memory_address): Likewise.
	* config/s390/s390.c (Z10_EARLYLOAD_DISTANCE): Likewise.
	* config/sh/sh.c (sh_rtx_costs): Likewise.
	* fold-const.c (fold_truth_andor): Likewise.
	* genautomata.c (collapse_flag): Likewise.
	* gengtype.h (struct type::u::s): Likewise.
	* gensupport.c (has_subst_attribute, add_mnemonic_string): Likewise.
	* input.c (FORMAT_AMOUNT): Likewise.
	* ipa-cp.c (class ipcp_lattice, agg_replacements_to_vector)
	(known_aggs_to_agg_replacement_list): Likewise.
	* ipa-inline-analysis.c: Likewise.
	* ipa-inline.h (estimate_edge_time, estimate_edge_hints): Likewise.
	* ipa-polymorphic-call.c
	(ipa_polymorphic_call_context::restrict_to_inner_class): Likewise.
	* loop-unroll.c (analyze_insn_to_expand_var): Likewise.
	* lra.c (lra_optional_reload_pseudos, lra_subreg_reload_pseudos):
	Likewise.
	* modulo-sched.c (apply_reg_moves): Likewise.
	* omp-expand.c (build_omp_regions_1): Likewise.
	* trans-mem.c (struct tm_wrapper_hasher): Likewise.
	* tree-ssa-loop-ivopts.c (may_eliminate_iv): Likewise.
	* tree-ssa-loop-niter.c (maybe_lower_iteration_bound): Likewise.
	* tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Likewise.
	* value-prof.c: Likewise.
	* var-tracking.c (val_reset): Likewise.

gcc/ada:

	* doc/gnat_ugn/gnat_and_program_execution.rst: Fix typo.
	* g-socket.adb (To_Host_Entry): Fix typo in comment.
	* gnat_ugn.texi: Fix typo.
	* raise.c (_gnat_builtin_longjmp): Fix capitalization in comment.
	* s-stposu.adb (Allocate_Any_Controlled): Fix typo in comment.
	* sem_ch3.adb (Build_Derived_Record_Type): Likewise.
	* sem_util.adb (Mark_Coextensions): Likewise.
	* sem_util.ads (Available_Full_View_Of_Component): Likewise.

gcc/c:

	* c-array-notation.c: Fix typo in comment.

gcc/c-family:

	* c-warn.c (do_warn_double_promotion): Fix typo in comment.

gcc/cp:

        * class.c (update_vtable_entry_for_fn): Fix typo in comment.
	* decl2.c (one_static_initialization_or_destruction): Likewise.
	* name-lookup.c (store_bindings): Likewise.
	* parser.c (make_call_declarator): Likewise.
	* pt.c (check_explicit_specialization): Likewise.

gcc/testsuite:

	* g++.old-deja/g++.benjamin/scope02.C: Fix typo in comment.
	* gcc.dg/20031012-1.c: Likewise.
	* gcc.dg/ipa/ipcp-1.c: Likewise.
	* gcc.dg/torture/matrix-3.c: Likewise.
	* gcc.target/powerpc/ppc-spe.c: Likewise.
	* gcc.target/rx/zero-width-bitfield.c: Likewise.

libcpp:

	* include/line-map.h (LINEMAPS_MACRO_MAPS): Fix typo in comment.
	* lex.c (search_line_fast): Likewise.
	* pch.h (cpp_valid_state): Likewise.

libdecnumber:

	* decCommon.c (decFloatFromPackedChecked): Fix typo in comment.
	* decNumber.c (decNumberPower, decMultiplyOp): Likewise.

libgcc:

	* config/c6x/pr-support.c (__gnu_unwind_execute): Fix typo in comment.

libitm:

	* libitm_i.h (sutrct gtm_thread): Fix typo in comment.

From-SVN: r246664
2017-04-03 23:30:56 +01:00
Andreas Schwab 8c00ae2406 Fix search_line_fast for aarch64/ILP32
* lex.c (search_line_fast) [__ARM_NEON && __ARM_64BIT_STATE]:
	Convert 64-bit value to boolean before passing to
	__builtin_expect.

From-SVN: r246312
2017-03-21 11:10:17 +00:00
Jakub Jelinek cbe34bb5ed Update copyright years.
From-SVN: r243994
2017-01-01 13:07:43 +01:00
David Malcolm a3998c2fb1 Fix use-after-free lexing unterminated raw strings (PR preprocessor/78811)
gcc/ChangeLog:
	PR preprocessor/78680
	PR preprocessor/78811
	* input.c (struct selftest::lexer_test): Add field
	m_implicitly_expect_EOF.
	(selftest::lexer_error_sink): New class.
	(selftest::lexer_error_sink::s_singleton): New global.
	(selftest::lexer_test::lexer_test): Initialize new field
	"m_implicitly_expect_EOF".
	(selftest::lexer_test::~lexer_test): Conditionalize the
	check for the EOF token on the new field.
	(selftest::test_lexer_string_locations_raw_string_unterminated):
	New function.
	(selftest::input_c_tests): Call the new test.

libcpp/ChangeLog:
	PR preprocessor/78680
	PR preprocessor/78811
	* lex.c (_cpp_lex_direct): Only determine the end-location of
	the token and build a range for non-reserved start locations.
	Do not do it for EOF tokens.

From-SVN: r243721
2016-12-15 18:05:05 +00:00
David Malcolm 470a60b2c4 re PR preprocessor/78680 (ICE in get_substring_ranges_for_loc, at input.c:1398)
Fix for PR preprocessor/78680

PR preprocessor/78680 identifies a crash when attempting to issue
a -Wformat warning, where the format string includes a string token
split across multiple physical source lines via backslash-continued
lines.

The issue is that libcpp is generating bogus range information for
such tokens.

For example, in:

void fn1() {
  __builtin_printf("\
     %ld.\n\
        2\n"); };

the range of the string token is printed as:

   __builtin_printf("\
                    ^~

whereas the range ought to be:

  __builtin_printf("\
                   ^~
     %ld.\n\
     ~~~~~~~
        2\n"); };
        ~~~~

The root cause is that the line notes expressing the update
of the buffer in lex.c aren't yet updated when the end-point of
the token is computed

3095	    tok_range.m_finish
3096	      = linemap_position_for_column (pfile->line_table,
3097					     CPP_BUF_COLUMN (buffer, buffer->cur));

so that the physical line is still regarded as that of the start
of the token, and, where CPP_BUF_COLUMN uses (BUF)->line_base,
line_base is still the location of the first physical line in the
and hence the column information is too large (as if it were the
offset in the *logical* line).

(the printed range is somewhat misleading; the actual buggy range
extends beyond the "\ in the line, but within diagnostic-show-locus.c
layout::print_annotation_line only prints up to the xbound set by
layout::print_source_line and so truncates most of the buggy range).

The fix is to ensure that line notes are handled before calculating
the end-point of the token range.

This leads to the range for the string token being correctly
computed, as:

  __builtin_printf("\
                   ^~
     %ld.\n\
     ~~~~~~~
        2\n"); };
        ~~~~

and this leads to get_substring_ranges_for_loc failing gracefully,
rather than crashing.

gcc/testsuite/ChangeLog:
	PR preprocessor/78680
	* gcc.dg/format/pr78680.c: New test case.
	* gcc.dg/plugin/diagnostic-test-expressions-1.c
	(test_multiline_token): New function.
	* gcc.dg/plugin/diagnostic-test-string-literals-1.c
	(test_backslash_continued_logical_lines): New function.

libcpp/ChangeLog:
	PR preprocessor/78680
	* lex.c (_cpp_lex_direct): Ensure line notes are processed before
	computing the end-point of the token.

From-SVN: r243567
2016-12-12 17:37:48 +00:00
Richard Earnshaw a6ac871cdf [AArch64] Optimized implementation of search_line_fast for the CPP lexer
* lex.c (search_line_fast): New implementation for AArch64.

From-SVN: r241964
2016-11-08 13:29:32 +00:00
Jakub Jelinek 70f6d5e19b gcc/
* common.opt (Wimplicit-fallthrough) Turn into alias to
	-Wimplicit-fallthrough=3.  Remove EnabledBy.
	(Wimplicit-fallthrough=): New option.
	* gimplify.c (warn_implicit_fallthrough_r): Use
	OPT_Wimplicit_fallthrough_ instead of OPT_Wimplicit_fallthrough.
	* doc/invoke.texi (-Wimplicit-fallthrough): Document as alias
	to -Wimplicit-fallthrough=3.
	(-Wimplicit-fallthrough=): Document.
gcc/c-family/
	* c.opt (Wextra): Add as C/C++/ObjC/ObjC++ option.
	(Wimplicit-fallthrough=): Enable for these languages by -Wextra.
	* c-opts.c (sanitize_cpp_opts): Initialize
	cpp_opts->cpp_warn_implicit_fallthrough.
gcc/testsuite/
	* c-c++-common/Wimplicit-fallthrough-25.c: New test.
	* c-c++-common/Wimplicit-fallthrough-26.c: New test.
	* c-c++-common/Wimplicit-fallthrough-27.c: New test.
	* c-c++-common/Wimplicit-fallthrough-28.c: New test.
	* c-c++-common/Wimplicit-fallthrough-29.c: New test.
	* c-c++-common/Wimplicit-fallthrough-30.c: New test.
	* c-c++-common/Wimplicit-fallthrough-31.c: New test.
	* c-c++-common/Wimplicit-fallthrough-32.c: New test.
	* c-c++-common/Wimplicit-fallthrough-33.c: New test.
libcpp/
	* include/cpplib.h (struct cpp_options): Add
	cpp_warn_implicit_fallthrough.
	* init.c (cpp_create_reader): Initialize it to 0.
	* lex.c (fallthrough_comment_p): Handle different
	cpp_warn_implicit_fallthrough levels.  Whitespace fixes.

From-SVN: r241013
2016-10-12 01:19:06 +02:00
Jakub Jelinek ee19ef45ba invoke.texi: Document accepting Else, fallthrough.
* doc/invoke.texi: Document accepting Else, fallthrough.

	* lex.c (fallthrough_comment_p): Accept Else, fallthrough.

	* c-c++-common/Wimplicit-fallthrough-23.c (foo): Add further tests.

From-SVN: r240886
2016-10-08 12:54:27 +02:00
Jakub Jelinek 81b02905b0 invoke.texi (-Wimplicit-fallthrough): Document FALLTHRU comment style changes.
* doc/invoke.texi (-Wimplicit-fallthrough): Document FALLTHRU comment
	style changes.

	* lex.c (fallthrough_comment_p): Extend to handle more common FALLTHRU
	comment styles.

	* c-c++-common/Wimplicit-fallthrough-23.c (foo): Add further tests.

From-SVN: r240885
2016-10-08 12:53:05 +02:00
Jakub Jelinek 7bad794aa0 c-lex.c (c_lex_with_flags): For CPP_COMMENT token with PREV_FALLTHROUGH...
* c-lex.c (c_lex_with_flags) <case CPP_COMMENT>: For CPP_COMMENT
	token with PREV_FALLTHROUGH, skip all following CPP_PADDING and
	CPP_COMMENT tokens and set add_flags to PREV_FALLTHROUGH afterwards.

	* doc/invoke.texi (-Wimplicit-fallthrough): Document the accepted
	FALLTHRU comment styles.

	* lex.c (fallthrough_comment_p): Fix off-by-one size comparison
	errors, cleanup.
	(_cpp_lex_direct): Allow arbitrary comments in between
	fallthrough_comment_p comment and following token.

	* c-c++-common/Wimplicit-fallthrough-23.c: New test.
	* c-c++-common/Wimplicit-fallthrough-24.c: New test.

From-SVN: r240884
2016-10-08 12:48:54 +02:00
Kelvin Nilsen 67ef83c61f re PR target/77847 (PowerPC big endian power7/power8 do not bootstrap due to fall through error)
libcpp/ChangeLog:

2016-10-04  Kelvin Nilsen  <kelvin@gcc.gnu.org>

	PR target/77847
	* lex.c (search_line_fast): Add a FALLTHROUGH comment to correct
	compiler error in the version of this function that is
	conditionally compiled when GCC_VERSION >= 4005 and both
	__ALTIVEC__ and __BIG_ENDIAN__ symbols are defined.

From-SVN: r240783
2016-10-05 12:36:48 +00:00
Marek Polacek 81fea426da Implement -Wimplicit-fallthrough.
Co-Authored-By: Jakub Jelinek <jakub@redhat.com>

From-SVN: r240485
2016-09-26 09:42:50 +00:00
Marek Polacek 191816a36b re PR c/7652 (-Wswitch-break : Warn if a switch case falls through)
PR c/7652
gcc/
	* alias.c (find_base_value): Adjust fall through comment.
	* cfgexpand.c (expand_debug_expr): Likewise.
	* combine.c (find_split_point): Likewise.
	(expand_compound_operation): Likewise.  Add FALLTHRU.
	(make_compound_operation): Adjust fall through comment.
	(canon_reg_for_combine): Add FALLTHRU.
	(force_to_mode): Adjust fall through comment.
	(simplify_shift_const_1): Likewise.
	(simplify_comparison): Likewise.
	* config/aarch64/aarch64-builtins.c (aarch64_simd_expand_args): Add
	FALLTHRU.
	* config/aarch64/predicates.md: Likewise.
	* config/i386/i386.c (function_arg_advance_32): Likewise.
	(ix86_gimplify_va_arg): Likewise.
	(print_reg): Likewise.
	(ix86_print_operand): Likewise.
	(ix86_build_const_vector): Likewise.
	(ix86_expand_branch): Likewise.
	(ix86_sched_init_global): Adjust fall through comment.
	(ix86_expand_args_builtin): Add FALLTHRU.
	(ix86_expand_builtin): Likewise.
	(ix86_expand_vector_init_one_var): Likewise.
	* config/rs6000/rs6000.c (rs6000_emit_vector_compare_inner): Likewise.
	(rs6000_adjust_cost): Likewise.
	(insn_must_be_first_in_group): Likewise.
	* config/rs6000/rs6000.md: Likewise.  Adjust fall through comment.
	* dbxout.c (dbxout_symbol): Adjust fall through comment.
	* df-scan.c (df_uses_record): Likewise.
	* dojump.c (do_jump): Add FALLTHRU.
	* dwarf2out.c (mem_loc_descriptor): Likewise.  Adjust fall through
	comment.
	(resolve_args_picking_1): Adjust fall through comment.
	(loc_list_from_tree_1): Likewise.
	* expmed.c (make_tree): Likewise.
	* expr.c (expand_expr_real_2): Add FALLTHRU.
	(expand_expr_real_1): Likewise.  Adjust fall through comment.
	* fold-const.c (const_binop): Adjust fall through comment.
	(fold_truth_not_expr): Likewise.
	(fold_cond_expr_with_comparison): Add FALLTHRU.
	(fold_binary_loc): Likewise.
	(contains_label_1): Adjust fall through comment.
	(multiple_of_p): Likewise.
	* gcov-tool.c (process_args): Add FALLTHRU.
	* genattrtab.c (check_attr_test): Likewise.
	(write_test_expr): Likewise.
	* genconfig.c (walk_insn_part): Likewise.
	* genpreds.c (validate_exp): Adjust fall through comment.
	(needs_variable): Likewise.
	* gensupport.c (get_alternatives_number): Add FALLTHRU.
	(subst_dup): Likewise.
	* gimple-pretty-print.c (dump_gimple_assign): Likewise.
	* gimplify.c (gimplify_addr_expr): Adjust fall through comment.
	(gimplify_scan_omp_clauses): Add FALLTHRU.
	(goa_stabilize_expr): Likewise.
	* graphite-isl-ast-to-gimple.c (substitute_ssa_name): Adjust fall
	through comment.
	* hsa-gen.c (get_address_from_value): Likewise.
	* ipa-icf.c (sem_function::hash_stmt): Likewise.
	* ira.c (ira_setup_alts): Add FALLTHRU.
	* lra-eliminations.c (lra_eliminate_regs_1): Adjust fall through
	comment.
	* lto-streamer-out.c (lto_output_tree_ref): Add FALLTHRU.
	* opts.c (common_handle_option): Likewise.
	* read-rtl.c (read_rtx_code): Likewise.
	* real.c (round_for_format): Likewise.
	* recog.c (asm_operand_ok): Likewise.
	* reginfo.c (reg_scan_mark_refs): Adjust fall through comment.
	* reload1.c (set_label_offsets): Likewise.
	(eliminate_regs_1): Likewise.
	(reload_reg_reaches_end_p): Likewise.
	* rtlanal.c (commutative_operand_precedence): Add FALLTHRU.
	(rtx_cost): Likewise.
	* sched-rgn.c (is_exception_free): Likewise.
	* simplify-rtx.c (simplify_rtx): Adjust fall through comment.
	* stor-layout.c (int_mode_for_mode): Likewise.
	* toplev.c (print_to_asm_out_file): Likewise.
	(print_to_stderr): Likewise.
	* tree-cfg.c (gimple_verify_flow_info): Likewise.
	* tree-chrec.c (chrec_fold_plus_1): Add FALLTHRU.
	(chrec_fold_multiply): Likewise.
	(evolution_function_is_invariant_rec_p): Likewise.
	(for_each_scev_op): Likewise.
	* tree-data-ref.c (siv_subscript_p): Likewise.
	(get_references_in_stmt): Likewise.
	* tree.c (find_placeholder_in_expr): Adjust fall through comment.
	(substitute_in_expr): Likewise.
	(type_cache_hasher::equal): Likewise.
	(walk_type_fields): Likewise.
	* var-tracking.c (adjust_mems): Add FALLTHRU.
	(set_dv_changed): Adjust fall through comment.
	* varasm.c (default_function_section): Add FALLTHRU.
gcc/c-family/
	* c-common.c (scalar_to_vector): Adjust fall through comment.
	* c-opts.c (c_common_handle_option): Likewise.
	* c-pragma.c (handle_pragma_pack): Add FALLTHRU.
	* c-pretty-print.c (c_pretty_printer::postfix_expression): Adjust
	fall through comment.
	* cilk.c (extract_free_variables): Add FALLTHRU.
gcc/c/
	* c-parser.c (c_parser_external_declaration): Add FALLTHRU.
	(c_parser_postfix_expression): Likewise.
	* c-typeck.c (build_unary_op): Adjust fall through comment.
	(c_mark_addressable): Likewise.
gcc/cp/
	* call.c (add_builtin_candidate): Add FALLTHRU.
	(build_integral_nontype_arg_conv): Adjust fall through comment.
	(build_new_op_1): Add FALLTHRU.
	(convert_like_real): Adjust fall through comment.
	* class.c (fixed_type_or_null): Likewise.
	* constexpr.c (cxx_eval_constant_expression): Likewise.
	(potential_constant_expression_1): Likewise.  Add FALLTHRU.
	* cp-gimplify.c (cp_gimplify_expr): Adjust fall through comment.
	(cp_fold): Add FALLTHRU.
	* cvt.c (build_expr_type_conversion): Adjust fall through comment.
	* cxx-pretty-print.c (pp_cxx_unqualified_id): Add FALLTHRU.
	(pp_cxx_qualified_id): Likewise.
	(cxx_pretty_printer::constant): Adjust fall through comment.
	(cxx_pretty_printer::primary_expression): Add FALLTHRU.
	(pp_cxx_pm_expression): Adjust fall through comment.
	(cxx_pretty_printer::expression): Add FALLTHRU.
	(cxx_pretty_printer::declaration_specifiers): Reformat code.
	(pp_cxx_type_specifier_seq): Adjust fall through comment.
	(pp_cxx_ptr_operator): Likewise.  Add FALLTHRU.
	* error.c (dump_type): Adjust fall through comment.
	(dump_decl): Likewise.
	* mangle.c (write_type): Likewise.
	* method.c (synthesized_method_walk): Add FALLTHRU.
	* name-lookup.c (arg_assoc_type): Likewise.
	* parser.c (cp_lexer_print_token): Adjust fall through comment.
	(cp_parser_primary_expression): Add FALLTHRU.
	(cp_parser_operator): Likewise.
	* pt.c (find_parameter_packs_r): Likewise.
	(tsubst_aggr_type): Adjust fall through comment.
	* semantics.c (finish_omp_clauses): Add FALLTHRU.
	* tree.c (lvalue_kind): Likewise.
gcc/fortran/
	* decl.c (match_attr_spec): Add FALLTHRU.
	* primary.c (match_arg_list_function): Likewise.
	* resolve.c (resolve_operator): Adjust fall through comment.
	(fixup_charlen): Add FALLTHRU.
	(resolve_allocate_expr): Adjust fall through comment.
	* trans-array.c (gfc_conv_ss_startstride): Add FALLTHRU.
	* trans-intrinsic.c (gfc_conv_intrinsic_len): Adjust fall through
	comment.
gcc/java/
	* expr.c (java_truthvalue_conversion): Adjust fall through comment.
	* jcf-io.c (verify_constant_pool): Likewise.
	* typeck.c (promote_type): Likewise.
gcc/objc/
	* objc-encoding.c (encode_type): Add FALLTHRU.
libcpp/
	* lex.c (search_line_fast): Add FALLTHRU.
	(_cpp_lex_direct): Likewise.
	(cpp_token_val_index): Adjust fall through comment.
	* macro.c (parse_params): Add FALLTHRU.
	* pch.c (count_defs): Adjust fall through comment.
	(write_defs): Likewise.
libiberty/
	* cp-demangle.c (d_print_mod): Add FALLTHRU.

From-SVN: r239410
2016-08-12 10:30:47 +00:00
David Malcolm 88fa5555a3 On-demand locations within string-literals
gcc/c-family/ChangeLog:
	* c-common.c: Include "substring-locations.h".
	(get_cpp_ttype_from_string_type): New function.
	(g_string_concat_db): New global.
	(substring_loc::get_range): New method.
	* c-common.h (g_string_concat_db): New declaration.
	(class substring_loc): New class.
	* c-lex.c (lex_string): When concatenating strings, capture the
	locations of all tokens using a new obstack, and record the
	concatenation locations within g_string_concat_db.
	* c-opts.c (c_common_init_options): Construct g_string_concat_db
	on the ggc-heap.

gcc/ChangeLog:
	* input.c (string_concat::string_concat): New constructor.
	(string_concat_db::string_concat_db): New constructor.
	(string_concat_db::record_string_concatenation): New method.
	(string_concat_db::get_string_concatenation): New method.
	(string_concat_db::get_key_loc): New method.
	(class auto_cpp_string_vec): New class.
	(get_substring_ranges_for_loc): New function.
	(get_source_range_for_substring): New function.
	(get_num_source_ranges_for_substring): New function.
	(class selftest::lexer_test_options): New class.
	(struct selftest::lexer_test): New struct.
	(class selftest::ebcdic_execution_charset): New class.
	(selftest::ebcdic_execution_charset::s_singleton): New variable.
	(selftest::lexer_test::lexer_test): New constructor.
	(selftest::lexer_test::~lexer_test): New destructor.
	(selftest::lexer_test::get_token): New method.
	(selftest::assert_char_at_range): New function.
	(ASSERT_CHAR_AT_RANGE): New macro.
	(selftest::assert_num_substring_ranges): New function.
	(ASSERT_NUM_SUBSTRING_RANGES): New macro.
	(selftest::assert_has_no_substring_ranges): New function.
	(ASSERT_HAS_NO_SUBSTRING_RANGES): New macro.
	(selftest::test_lexer_string_locations_simple): New function.
	(selftest::test_lexer_string_locations_ebcdic): New function.
	(selftest::test_lexer_string_locations_hex): New function.
	(selftest::test_lexer_string_locations_oct): New function.
	(selftest::test_lexer_string_locations_letter_escape_1): New function.
	(selftest::test_lexer_string_locations_letter_escape_2): New function.
	(selftest::test_lexer_string_locations_ucn4): New function.
	(selftest::test_lexer_string_locations_ucn8): New function.
	(selftest::uint32_from_big_endian): New function.
	(selftest::test_lexer_string_locations_wide_string): New function.
	(selftest::uint16_from_big_endian): New function.
	(selftest::test_lexer_string_locations_string16): New function.
	(selftest::test_lexer_string_locations_string32): New function.
	(selftest::test_lexer_string_locations_u8): New function.
	(selftest::test_lexer_string_locations_utf8_source): New function.
	(selftest::test_lexer_string_locations_concatenation_1): New
	function.
	(selftest::test_lexer_string_locations_concatenation_2): New
	function.
	(selftest::test_lexer_string_locations_concatenation_3): New
	function.
	(selftest::test_lexer_string_locations_macro): New function.
	(selftest::test_lexer_string_locations_stringified_macro_argument):
	New function.
	(selftest::test_lexer_string_locations_non_string): New function.
	(selftest::test_lexer_string_locations_long_line): New function.
	(selftest::test_lexer_char_constants): New function.
	(selftest::input_c_tests): Call the new test functions once per
	case within the line_table test matrix.
	* input.h (struct string_concat): New struct.
	(struct location_hash): New struct.
	(class string_concat_db): New class.
	* substring-locations.h: New header.

gcc/testsuite/ChangeLog:
	* gcc.dg/plugin/diagnostic-test-string-literals-1.c: New file.
	* gcc.dg/plugin/diagnostic-test-string-literals-2.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_test_string_literals.c: New file.
	* gcc.dg/plugin/plugin.exp (plugin_test_list): Add the above new files.

libcpp/ChangeLog:
	* charset.c (cpp_substring_ranges::cpp_substring_ranges): New
	constructor.
	(cpp_substring_ranges::~cpp_substring_ranges): New destructor.
	(cpp_substring_ranges::add_range): New method.
	(cpp_substring_ranges::add_n_ranges): New method.
	(_cpp_valid_ucn): Add "char_range" and "loc_reader" params; if
	they are non-NULL, read position information from *loc_reader
	and update char_range->m_finish accordingly.
	(convert_ucn): Add "char_range", "loc_reader", and "ranges"
	params.  If loc_reader is non-NULL, read location information from
	it, and update *ranges accordingly, using char_range.
	Conditionalize the conversion into tbuf on tbuf being non-NULL.
	(convert_hex): Likewise, conditionalizing the call to
	emit_numeric_escape on tbuf.
	(convert_oct): Likewise.
	(convert_escape): Add params "loc_reader" and "ranges".  If
	loc_reader is non-NULL, read location information from it, and
	update *ranges accordingly.  Conditionalize the conversion into
	tbuf on tbuf being non-NULL.
	(cpp_interpret_string): Rename to...
	(cpp_interpret_string_1): ...this, adding params "loc_readers" and
	"out".  Use "to" to conditionalize the initialization and usage of
	"tbuf", such as running the converter.  If "loc_readers" is
	non-NULL, use the instances within it, reading location
	information from them, and passing them to convert_escape; likewise
	write to "out" if loc_readers is non-NULL.  Check for leading
	quote and issue an error if it is not present.  Update boundary
	check from "== limit" to ">= limit" to protect against erroneous
	location values to calls that are not parsing string literals.
	(cpp_interpret_string): Reimplement in terms to
	cpp_interpret_string_1.
	(noop_error_cb): New function.
	(cpp_interpret_string_ranges): New function.
	(cpp_string_location_reader::cpp_string_location_reader): New
	constructor.
	(cpp_string_location_reader::get_next): New method.
	* include/cpplib.h (class cpp_string_location_reader): New class.
	(class cpp_substring_ranges): New class.
	(cpp_interpret_string_ranges): New prototype.
	* internal.h (_cpp_valid_ucn): Add params "char_range" and
	"loc_reader".
	* lex.c (forms_identifier_p): Pass NULL for new params to
	_cpp_valid_ucn.

From-SVN: r239175
2016-08-05 18:08:33 +00:00
Martin Liska ceb17928e5 Change ENABLE_VALGRIND_CHECKING to
* config.in: Regenerated.
	* configure: Likewise.
	* configure.ac: Handle --enable-valgrind-annotations.
	* lex.c (new_buff): Use ENABLE_VALGRIND_ANNOTATIONS instead
	of ENABLE_VALGRIND_CHECKING.
	(_cpp_free_buff): Likewise.

From-SVN: r236496
2016-05-20 08:03:25 +00:00
Jakub Jelinek 818ab71a41 Update copyright years.
From-SVN: r232055
2016-01-04 15:30:50 +01:00
David Malcolm ebedc9a341 Source range tracking in libcpp and C FE, with bit-packing optimization
This patch combines:
  [PATCH 05/10] Add ranges to libcpp tokens (via ad-hoc data, unoptimized)
  [PATCH 06/10] Track expression ranges in C frontend
  [PATCH 07/10] Add plugin to recursively dump the source-ranges in a tree (v2)
  [PATCH 08/10] Wire things up so that libcpp users get token underlines
  [PATCH 09/10] Delay some resolution of ad-hoc locations, preserving ranges
  [PATCH 10/10] Compress short ranges into source_location
  [PATCH] libcpp: add examples to source_location description
along with fixes for the nits identified during review.

gcc/ChangeLog:
	* Makefile.in (OBJS): Add gcc-rich-location.o.
	* diagnostic.c (diagnostic_append_note): Pass line_table to
	rich_location ctor.
	(emit_diagnostic): Likewise.
	(inform): Likewise.
	(inform_n): Likewise.
	(warning): Likewise.
	(warning_at): Likewise.
	(warning_n): Likewise.
	(pedwarn): Likewise.
	(permerror): Likewise.
	(error): Likewise.
	(error_n): Likewise.
	(error_at): Likewise.
	(sorry): Likewise.
	(fatal_error): Likewise.
	(internal_error): Likewise.
	(internal_error_no_backtrace): Likewise.
	(source_range::debug): Likewise.
	* gcc-rich-location.c: New file.
	* gcc-rich-location.h: New file.
	* genmatch.c (fatal_at): Pass line_table to rich_location ctor.
	(warning_at): Likewise.
	* gimple.h (gimple_set_block): Use set_block function.
	* input.c (dump_line_table_statistics): Dump stats on how many
	ranges were optimized vs how many needed ad-hoc table.
	(write_digit_row): Add "map" param; use its range_bits
	to calculate the per-character offset.
	(dump_location_info): Print the range and column bits for each
	ordinary map.  Use the range bits to calculate the per-character
	offset.  Pass the map as a new param to the various calls to
	write_digit_row.  Eliminate uses of
	ORDINARY_MAP_NUMBER_OF_COLUMN_BITS.
	* print-tree.c (print_node): Print any source range information.
	* rtl-error.c (diagnostic_for_asm): Likewise.
	* toplev.c (general_init): Initialize line_table's
	default_range_bits.
	* tree-cfg.c (move_block_to_fn): Likewise.
	(move_block_to_fn): Likewise.
	* tree-inline.c (copy_phis_for_bb): Likewise.
	* tree.c (tree_set_block): Likewise.
	(get_pure_location): New function.
	(set_source_range): New functions.
	(set_block): New function.
	(set_source_range): New functions.
	* tree.h (CAN_HAVE_RANGE_P): New.
	(EXPR_LOCATION_RANGE): New.
	(EXPR_HAS_RANGE): New.
	(get_expr_source_range): New inline function.
	(DECL_LOCATION_RANGE): New.
	(set_source_range): New decls.
	(get_decl_source_range): New inline function.

gcc/ada/ChangeLog:
	* gcc-interface/trans.c (Sloc_to_locus): Add line_table param when
	calling linemap_position_for_line_and_column.

gcc/c-family/ChangeLog:
	* c-common.c (c_fully_fold_internal): Capture existing souce_range,
	and store it on the result.
	* c-opts.c (c_common_init_options): Set
	global_dc->colorize_source_p.

gcc/c/ChangeLog:
	* c-decl.c (warn_defaults_to): Pass line_table to
	rich_location ctor.
	* c-errors.c (pedwarn_c99): Likewise.
	(pedwarn_c90): Likewise.
	* c-parser.c (set_c_expr_source_range): New functions.
	(c_token::get_range): New method.
	(c_token::get_finish): New method.
	(c_parser_expr_no_commas): Call set_c_expr_source_range on the ret
	based on the range from the start of the LHS to the end of the
	RHS.
	(c_parser_conditional_expression): Likewise, based on the range
	from the start of the cond.value to the end of exp2.value.
	(c_parser_binary_expression): Call set_c_expr_source_range on
	the stack values for TRUTH_ANDIF_EXPR and TRUTH_ORIF_EXPR.
	(c_parser_cast_expression): Call set_c_expr_source_range on ret
	based on the cast_loc through to the end of the expr.
	(c_parser_unary_expression): Likewise, based on the
	op_loc through to the end of op.
	(c_parser_sizeof_expression) Likewise, based on the start of the
	sizeof token through to either the closing paren or the end of
	expr.
	(c_parser_postfix_expression): Likewise, using the token range,
	or from the open paren through to the close paren for
	parenthesized expressions.
	(c_parser_postfix_expression_after_primary): Likewise, for
	various kinds of expression.
	* c-tree.h (struct c_expr): Add field "src_range".
	(c_expr::get_start): New method.
	(c_expr::get_finish): New method.
	(set_c_expr_source_range): New decls.
	* c-typeck.c (parser_build_unary_op): Call set_c_expr_source_range
	on ret for prefix unary ops.
	(parser_build_binary_op): Likewise, running from the start of
	arg1.value through to the end of arg2.value.

gcc/cp/ChangeLog:
	* error.c (pedwarn_cxx98): Pass line_table to rich_location ctor.

gcc/fortran/ChangeLog:
	* error.c (gfc_warning): Pass line_table to rich_location ctor.
	(gfc_warning_now_at): Likewise.
	(gfc_warning_now): Likewise.
	(gfc_error_now): Likewise.
	(gfc_fatal_error): Likewise.
	(gfc_error): Likewise.
	(gfc_internal_error): Likewise.

gcc/testsuite/ChangeLog:
	* gcc.dg/diagnostic-token-ranges.c: New file.
	* gcc.dg/diagnostic-tree-expr-ranges-2.c: New file.
	* gcc.dg/plugin/diagnostic-test-expressions-1.c: New file.
	* gcc.dg/plugin/diagnostic-test-show-trees-1.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_show_trees.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (get_loc): Add
	line_table param when calling
	linemap_position_for_line_and_column.
	(test_show_locus): Pass line_table to rich_location ctors.
	(plugin_init): Remove setting of global_dc->colorize_source_p.
	* gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c:
	New file.
	* gcc.dg/plugin/plugin.exp (plugin_test_list): Add
	diagnostic_plugin_test_tree_expression_range.c,
	diagnostic-test-expressions-1.c, diagnostic_plugin_show_trees.c,
	and diagnostic-test-show-trees-1.c.

libcpp/ChangeLog:
	* errors.c (cpp_diagnostic): Pass pfile->line_table to
	rich_location ctor.
	(cpp_diagnostic_with_line): Likewise.
	* include/cpplib.h (struct cpp_token): Update comment for src_loc
	to indicate that the range of the token is "baked into" the
	source_location.
	* include/line-map.h (source_location): Update the descriptive
	comment to reflect the packing scheme for short ranges, adding
	worked examples of location encoding.
	(struct line_map_ordinary): Drop field "column_bits" in favor
	of field "m_column_and_range_bits"; add field "m_range_bits".
	(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS): Delete.
	(location_adhoc_data): Add source_range field.
	(struct line_maps): Add fields "default_range_bits",
	"num_optimized_ranges" and "num_unoptimized_ranges".
	(get_combined_adhoc_loc): Add source_range param.
	(get_range_from_loc): New declaration.
	(pure_location_p): New prototype.
	(COMBINE_LOCATION_DATA):  Add source_range param.
	(SOURCE_LINE): Update for renaming of column_bits.
	(SOURCE_COLUMN): Likewise.  Shift the column right by the map's
	range_bits.
	(LAST_SOURCE_LINE_LOCATION): Update for renaming of column_bits.
	(linemap_position_for_line_and_column): Add line_maps * params.
	(rich_location::rich_location): Likewise.
	* lex.c (_cpp_lex_direct): Capture the range of the token, baking
	it into token->src_loc via a call to COMBINE_LOCATION_DATA.
	* line-map.c (LINE_MAP_MAX_COLUMN_NUMBER): Reduce from 1U << 17 to
	1U << 12.
	(location_adhoc_data_hash): Add the src_range into
	the hash value.
	(location_adhoc_data_eq): Require equality of the src_range
	values.
	(can_be_stored_compactly_p): New function.
	(get_combined_adhoc_loc): Add src_range param, and store it,
	via a bit-packing scheme for short ranges, otherwise within the
	lookaside table.  Remove the requirement that data is non-NULL.
	(get_range_from_adhoc_loc): New function.
	(get_range_from_loc): New function.
	(pure_location_p): New function.
	(linemap_add): Ensure that start_location has zero for the
	range_bits, unless we're past LINE_MAP_MAX_LOCATION_WITH_COLS.
	Initialize range_bits to zero.  Assert that the start_location
	is "pure".
	(linemap_line_start): Assert that the
	column_and_range_bits >= range_bits.
	Update determinination of whether we need to start a new map
	using the effective column bits, without the range bits.
	Use the set's default_range_bits in new maps, apart from
	those with column_bits == 0, which should also have 0 range_bits.
	Increase the column bits for new maps by the range bits.
	When adding lines to an existing map, use set->highest_line
	directly rather than offsetting highest by SOURCE_COLUMN.
	Add assertions to sanity-check the return value.
	(linemap_position_for_column): Offset to_column by range_bits.
	Update set->highest_location if necessary.
	(linemap_position_for_line_and_column): Add line_maps * param.
	Update the calculation to offset the column by range_bits, and
	conditionalize it on being <= LINE_MAP_MAX_LOCATION_WITH_COLS.
	Bound it by LINEMAPS_MACRO_LOWEST_LOCATION.  Update
	set->highest_location if necessary.
	(linemap_position_for_loc_and_offset): Handle ad-hoc locations;
	pass "set" to linemap_position_for_line_and_column.
	(linemap_macro_map_loc_unwind_toward_spelling): Add line_maps
	param.  Handle ad-hoc locations.
	(linemap_location_in_system_header_p): Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_macro_loc_to_spelling_point): Retain ad-hoc locations.
	Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_resolve_location): Retain ad-hoc locations.  Pass on
	"set" to call to linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_unwind_toward_expansion):  Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_expand_location): Extract the data pointer before
	extracting the location.
	(rich_location::rich_location): Add line_maps param; use it to
	extract the range from the source_location.
	* location-example.txt: Regenerate, showing new representation.

From-SVN: r230331
2015-11-13 16:29:59 +00:00
Uros Bizjak 49445904cf lex.c (search_line_sse42): Correctly advance the pointer to an aligned address.
* lex.c (search_line_sse42): Correctly advance the pointer to an
	aligned address.

From-SVN: r229718
2015-11-03 20:05:50 +01:00