Commit Graph

42 Commits

Author SHA1 Message Date
David Malcolm
bbd6fcf320 Provide location information for terminator characters (PR preprocessor/77672)
substring_loc::get_location currently fails for the final terminator
character in a STRING_CST from the C frontend, so that format_warning_va
falls back to using the location of the string as a whole.

This patch tweaks things [1] so that we use the final closing quote
as the location of the terminator character, as requested in
PR preprocessor/77672.

[1] specifically, cpp_interpret_string_1.

gcc/ChangeLog:
	PR preprocessor/77672
	* input.c (selftest::test_lexer_string_locations_simple): Update
	test to expect location information of the terminator character
	at the location of the final closing quote.
	(selftest::test_lexer_string_locations_hex): Likewise.
	(selftest::test_lexer_string_locations_oct): Likewise.
	(selftest::test_lexer_string_locations_letter_escape_1): Likewise.
	(selftest::test_lexer_string_locations_letter_escape_2): Likewise.
	(selftest::test_lexer_string_locations_ucn4): Likewise.
	(selftest::test_lexer_string_locations_ucn8): Likewise.
	(selftest::test_lexer_string_locations_u8): Likewise.
	(selftest::test_lexer_string_locations_utf8_source): Likewise.
	(selftest::test_lexer_string_locations_concatenation_1): Likewise.
	(selftest::test_lexer_string_locations_concatenation_2): Likewise.
	(selftest::test_lexer_string_locations_concatenation_3): Likewise.
	(selftest::test_lexer_string_locations_macro): Likewise.
	(selftest::test_lexer_string_locations_long_line): Likewise.

gcc/testsuite/ChangeLog:
	PR preprocessor/77672
	* gcc.dg/plugin/diagnostic-test-string-literals-1.c
	(test_terminator_location): New function.

libcpp/ChangeLog:
	PR preprocessor/77672
	* charset.c (cpp_interpret_string_1): Add a source_range for the
	NUL-terminator, using the location of the trailing quote of the
	final string.

From-SVN: r240434
2016-09-23 14:14:52 +00:00
David Malcolm
e7864d68ee Fix crash in selftest::test_lexer_string_locations_ucn4 (PR bootstrap/72823)
libcpp/ChangeLog:
	PR bootstrap/72823
	* charset.c (_cpp_valid_ucn): Replace overzealous assert with one
	that allows for char_range to be non-NULL when loc_reader is NULL.

From-SVN: r239211
2016-08-06 18:06:30 +00:00
David Malcolm
88fa5555a3 On-demand locations within string-literals
gcc/c-family/ChangeLog:
	* c-common.c: Include "substring-locations.h".
	(get_cpp_ttype_from_string_type): New function.
	(g_string_concat_db): New global.
	(substring_loc::get_range): New method.
	* c-common.h (g_string_concat_db): New declaration.
	(class substring_loc): New class.
	* c-lex.c (lex_string): When concatenating strings, capture the
	locations of all tokens using a new obstack, and record the
	concatenation locations within g_string_concat_db.
	* c-opts.c (c_common_init_options): Construct g_string_concat_db
	on the ggc-heap.

gcc/ChangeLog:
	* input.c (string_concat::string_concat): New constructor.
	(string_concat_db::string_concat_db): New constructor.
	(string_concat_db::record_string_concatenation): New method.
	(string_concat_db::get_string_concatenation): New method.
	(string_concat_db::get_key_loc): New method.
	(class auto_cpp_string_vec): New class.
	(get_substring_ranges_for_loc): New function.
	(get_source_range_for_substring): New function.
	(get_num_source_ranges_for_substring): New function.
	(class selftest::lexer_test_options): New class.
	(struct selftest::lexer_test): New struct.
	(class selftest::ebcdic_execution_charset): New class.
	(selftest::ebcdic_execution_charset::s_singleton): New variable.
	(selftest::lexer_test::lexer_test): New constructor.
	(selftest::lexer_test::~lexer_test): New destructor.
	(selftest::lexer_test::get_token): New method.
	(selftest::assert_char_at_range): New function.
	(ASSERT_CHAR_AT_RANGE): New macro.
	(selftest::assert_num_substring_ranges): New function.
	(ASSERT_NUM_SUBSTRING_RANGES): New macro.
	(selftest::assert_has_no_substring_ranges): New function.
	(ASSERT_HAS_NO_SUBSTRING_RANGES): New macro.
	(selftest::test_lexer_string_locations_simple): New function.
	(selftest::test_lexer_string_locations_ebcdic): New function.
	(selftest::test_lexer_string_locations_hex): New function.
	(selftest::test_lexer_string_locations_oct): New function.
	(selftest::test_lexer_string_locations_letter_escape_1): New function.
	(selftest::test_lexer_string_locations_letter_escape_2): New function.
	(selftest::test_lexer_string_locations_ucn4): New function.
	(selftest::test_lexer_string_locations_ucn8): New function.
	(selftest::uint32_from_big_endian): New function.
	(selftest::test_lexer_string_locations_wide_string): New function.
	(selftest::uint16_from_big_endian): New function.
	(selftest::test_lexer_string_locations_string16): New function.
	(selftest::test_lexer_string_locations_string32): New function.
	(selftest::test_lexer_string_locations_u8): New function.
	(selftest::test_lexer_string_locations_utf8_source): New function.
	(selftest::test_lexer_string_locations_concatenation_1): New
	function.
	(selftest::test_lexer_string_locations_concatenation_2): New
	function.
	(selftest::test_lexer_string_locations_concatenation_3): New
	function.
	(selftest::test_lexer_string_locations_macro): New function.
	(selftest::test_lexer_string_locations_stringified_macro_argument):
	New function.
	(selftest::test_lexer_string_locations_non_string): New function.
	(selftest::test_lexer_string_locations_long_line): New function.
	(selftest::test_lexer_char_constants): New function.
	(selftest::input_c_tests): Call the new test functions once per
	case within the line_table test matrix.
	* input.h (struct string_concat): New struct.
	(struct location_hash): New struct.
	(class string_concat_db): New class.
	* substring-locations.h: New header.

gcc/testsuite/ChangeLog:
	* gcc.dg/plugin/diagnostic-test-string-literals-1.c: New file.
	* gcc.dg/plugin/diagnostic-test-string-literals-2.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_test_string_literals.c: New file.
	* gcc.dg/plugin/plugin.exp (plugin_test_list): Add the above new files.

libcpp/ChangeLog:
	* charset.c (cpp_substring_ranges::cpp_substring_ranges): New
	constructor.
	(cpp_substring_ranges::~cpp_substring_ranges): New destructor.
	(cpp_substring_ranges::add_range): New method.
	(cpp_substring_ranges::add_n_ranges): New method.
	(_cpp_valid_ucn): Add "char_range" and "loc_reader" params; if
	they are non-NULL, read position information from *loc_reader
	and update char_range->m_finish accordingly.
	(convert_ucn): Add "char_range", "loc_reader", and "ranges"
	params.  If loc_reader is non-NULL, read location information from
	it, and update *ranges accordingly, using char_range.
	Conditionalize the conversion into tbuf on tbuf being non-NULL.
	(convert_hex): Likewise, conditionalizing the call to
	emit_numeric_escape on tbuf.
	(convert_oct): Likewise.
	(convert_escape): Add params "loc_reader" and "ranges".  If
	loc_reader is non-NULL, read location information from it, and
	update *ranges accordingly.  Conditionalize the conversion into
	tbuf on tbuf being non-NULL.
	(cpp_interpret_string): Rename to...
	(cpp_interpret_string_1): ...this, adding params "loc_readers" and
	"out".  Use "to" to conditionalize the initialization and usage of
	"tbuf", such as running the converter.  If "loc_readers" is
	non-NULL, use the instances within it, reading location
	information from them, and passing them to convert_escape; likewise
	write to "out" if loc_readers is non-NULL.  Check for leading
	quote and issue an error if it is not present.  Update boundary
	check from "== limit" to ">= limit" to protect against erroneous
	location values to calls that are not parsing string literals.
	(cpp_interpret_string): Reimplement in terms to
	cpp_interpret_string_1.
	(noop_error_cb): New function.
	(cpp_interpret_string_ranges): New function.
	(cpp_string_location_reader::cpp_string_location_reader): New
	constructor.
	(cpp_string_location_reader::get_next): New method.
	* include/cpplib.h (class cpp_string_location_reader): New class.
	(class cpp_substring_ranges): New class.
	(cpp_interpret_string_ranges): New prototype.
	* internal.h (_cpp_valid_ucn): Add params "char_range" and
	"loc_reader".
	* lex.c (forms_identifier_p): Pass NULL for new params to
	_cpp_valid_ucn.

From-SVN: r239175
2016-08-05 18:08:33 +00:00
Jakub Jelinek
b5c1c98852 re PR c++/69628 (Conditional jump or move depends on uninitialised value(s) in lex_charconst(cpp_token const*) (c-lex.c:1252))
PR c++/69628
	* charset.c (cpp_interpret_charconst): Clear *PCHARS_SEEN
	and *UNSIGNEDP if bailing out early due to errors.

	* g++.dg/parse/pr69628.C: New test.

From-SVN: r233186
2016-02-05 20:39:48 +01:00
Jakub Jelinek
818ab71a41 Update copyright years.
From-SVN: r232055
2016-01-04 15:30:50 +01:00
Paolo Carlini
fbb22910cf re PR preprocessor/53690 ([C++11] \u0000 and \U00000000 are wrongly encoded as U+0001.)
/libcpp
2015-07-02  Paolo Carlini  <paolo.carlini@oracle.com>

	PR c++/53690
	* charset.c (_cpp_valid_ucn): Add cppchar_t * parameter and change
	return type to bool.  Fix encoding of \u0000 and \U00000000 in C++.
	(convert_ucn): Adjust call.
	* lex.c (forms_identifier_p): Likewise.
	* internal.h (_cpp_valid_ucn): Adjust declaration.

/gcc/testsuite
2015-07-02  Paolo Carlini  <paolo.carlini@oracle.com>

	PR c++/53690
	* g++.dg/cpp/pr53690.C: New.

From-SVN: r225353
2015-07-02 18:54:41 +00:00
Edward Smith-Rowland
fe95b0366a Implement N4197 - Adding u8 character literals
libcpp:

2015-06-30  Edward Smith-Rowland  <3dw4rd@verizon.net>

	Implement N4197 - Adding u8 character literals
	* include/cpplib.h (UTF8CHAR, UTF8CHAR_USERDEF): New cpp tokens;
	(struct cpp_options): Add utf8_char_literals.
	* init.c (struct lang_flags): Add utf8_char_literals;
	(struct lang_flags lang_defaults): Add column for utf8_char_literals.
	* macro.c (stringify_arg()): Treat CPP_UTF8CHAR token; 
	* expr.c (cpp_userdef_char_remove_type(), cpp_userdef_char_add_type()):
	Treat CPP_UTF8CHAR_USERDEF, CPP_UTF8CHAR tokens;
	(cpp_userdef_char_p()): Treat CPP_UTF8CHAR_USERDEF token;
	(eval_token(), _cpp_parse_expr()): Treat CPP_UTF8CHAR token.
	* lex.c (lex_string(), _cpp_lex_direct()): Include CPP_UTF8CHAR tokens.
	* charset.c (converter_for_type(), cpp_interpret_charconst()):
	Treat CPP_UTF8CHAR token.


gcc/c-family:

2015-06-30  Edward Smith-Rowland  <3dw4rd@verizon.net>

	Implement N4197 - Adding u8 character literals
	* c-family/c-ada-spec.c (print_ada_macros()): Treat CPP_UTF8CHAR
	like CPP_CHAR.
	* c-family/c-common.c (c_parse_error()): print CPP_UTF8CHAR
	and CPP_UTF8CHAR_USERDEF tokens.
	* c-family/c-lex.c (c_lex_with_flags()): Treat CPP_UTF8CHAR_USERDEF
	and CPP_UTF8CHAR tokens; (lex_charconst()): Treat CPP_UTF8CHAR token.


gcc/cp:

2015-06-30  Edward Smith-Rowland  <3dw4rd@verizon.net>

	Implement N4197 - Adding u8 character literals
	* parser.c (cp_parser_primary_expression()): Treat CPP_UTF8CHAR
	and CPP_UTF8CHAR_USERDEF tokens;
	(cp_parser_parenthesized_expression_list()): Treat CPP_UTF8CHAR token.


gcc/testsuite:

2015-06-30  Edward Smith-Rowland  <3dw4rd@verizon.net>

	Implement N4197 - Adding u8 character literals
	* g++.dg/cpp1z/utf8.C: New.
	* g++.dg/cpp1z/utf8-neg.C: New.
	* g++.dg/cpp1z/udlit-utf8char.C: New.

From-SVN: r225185
2015-06-30 12:58:48 +00:00
Jakub Jelinek
5624e564d2 Update copyright years.
From-SVN: r219188
2015-01-05 13:33:28 +01:00
Joseph Myers
81fee4a708 Fix off-by-one bug in utf16 conversion (PR preprocessor/41698).
libcpp:
2014-11-29  John Schmerge  <jbschmerge@gmail.com>

	PR preprocessor/41698
	* charset.c (one_utf8_to_utf16): Do not produce surrogate pairs
	for 0xffff.

gcc/testsuite:
2014-11-29  Joseph Myers  <joseph@codesourcery.com>

	PR preprocessor/41698
	* gcc/testsuite/g++.dg/cpp/utf16-pr41698-1.C: New test.

From-SVN: r218179
2014-11-29 01:56:06 +00:00
Bernd Edlinger
dc257367bb charset.c (convert_no_conversion): Reallocate memory with 25% headroom.
2014-10-02  Bernd Edlinger  <bernd.edlinger@hotmail.de>
            Jeff Law  <law@redhat.com>

        * charset.c (convert_no_conversion): Reallocate memory with 25%
        headroom.

Co-Authored-By: Jeff Law <law@redhat.com>

From-SVN: r215785
2014-10-02 00:06:28 +00:00
Jan Hubicka
d87fc69983 charset.c (conversion): Rename to ...
* charset.c (conversion): Rename to ...
	(cpp_conversion): ... this one; update.
	* files.c (file_hash_entry): Rename to ...
	(cpp_file_hash_entry): ... this one ; update.

From-SVN: r215482
2014-09-22 19:43:02 +00:00
Marek Polacek
177cce463d c-opts.c (sanitize_cpp_opts): Make warn_long_long be set according to warn_c90_c99_compat.
gcc/c-family/
	* c-opts.c (sanitize_cpp_opts): Make warn_long_long be set according
	to warn_c90_c99_compat.
	* c.opt (Wc90-c99-compat, Wdeclaration-after-statement): Initialize
	to -1.
gcc/c/
	* c-decl.c (warn_variable_length_array): Pass OPT_Wvla unconditionally
	to pedwarn_c90.
	* c-errors.c: Include "opts.h".
	(pedwarn_c90): Rewrite to handle -Wno-c90-c99-compat better.
	* c-parser.c (disable_extension_diagnostics): Handle negative value
	of warn_c90_c99_compat, too.
	(restore_extension_diagnostics): Likewise.
	(c_parser_compound_statement_nostart): Pass
	OPT_Wdeclaration_after_statement unconditionally to pedwarn_c90.
gcc/testsuite/
	* gcc.dg/Wc90-c99-compat-4.c: Remove all dg-warnings.
	* gcc.dg/Wc90-c99-compat-5.c: Remove all dg-errors.
	* gcc.dg/Wc90-c99-compat-7.c: New test.
	* gcc.dg/Wc90-c99-compat-8.c: New test.
	* gcc.dg/Wdeclaration-after-statement-4.c: New test.
libcpp/
	* charset.c (_cpp_valid_ucn): Warn only if -Wc90-c99-compat.
	* lex.c (_cpp_lex_direct): Likewise.
	* macro.c (replace_args): Likewise.
	(parse_params): Likewise.
	* include/cpplib.h (cpp_options): Change cpp_warn_c90_c99_compat
	to char.

From-SVN: r214131
2014-08-19 05:34:31 +00:00
Marek Polacek
f3bede7188 re PR c/51849 (-Wc99-compat would be considered useful)
PR c/51849
gcc/
	* gcc/doc/invoke.texi: Document -Wc90-c99-compat.
gcc/c-family/
	* c-opts.c (sanitize_cpp_opts): Pass warn_c90_c99_compat to libcpp.
	* c.opt (Wc90-c99-compat): Add option.
gcc/c/
	* c-decl.c (build_array_declarator): Remove check for !flag_isoc99.
	Call pedwarn_c90 instead of pedwarn.
	(check_bitfield_type_and_width): Likewise.
	(declspecs_add_qual): Likewise.
	(declspecs_add_type): Likewise.
	(warn_variable_length_array): Unify function for -pedantic and -Wvla.
	Adjust to only call pedwarn_c90.
	(grokdeclarator): Remove pedantic && !flag_isoc99 check.  Call
	pedwarn_c90 instead of pedwarn.
	* c-errors.c (pedwarn_c90): Handle -Wc90-c99-compat.
	* c-parser.c (disable_extension_diagnostics): Handle
	warn_c90_c99_compat.
	(restore_extension_diagnostics): Likewise.
	(c_parser_enum_specifier): Remove check for !flag_isoc99.  Call
	pedwarn_c90 instead of pedwarn.
	(c_parser_initelt): Likewise.
	(c_parser_postfix_expression): Likewise.
	(c_parser_postfix_expression_after_paren_type): Likewise.
	(c_parser_compound_statement_nostart): Remove check for !flag_isoc99.
	* c-tree.h: Fix formatting.
	* c-typeck.c (build_array_ref): Remove check for !flag_isoc99.  Call
	pedwarn_c90 instead of pedwarn.
gcc/testsuite/
	* gcc.dg/Wc90-c99-compat-1.c: New test.
	* gcc.dg/Wc90-c99-compat-2.c: New test.
	* gcc.dg/Wc90-c99-compat-3.c: New test.
	* gcc.dg/Wc90-c99-compat-4.c: New test.
	* gcc.dg/Wc90-c99-compat-5.c: New test.
	* gcc.dg/Wc90-c99-compat-6.c: New test.
	* gcc.dg/wvla-1.c: Adjust dg-warning.
	* gcc.dg/wvla-2.c: Adjust dg-warning.
	* gcc.dg/wvla-4.c: Adjust dg-warning.
	* gcc.dg/wvla-6.c: Adjust dg-warning.
libcpp/
	* lex.c (_cpp_lex_direct): Warn when -Wc90-c99-compat is in effect.
	* charset.c (_cpp_valid_ucn): Likewise.
	* include/cpplib.h (cpp_options): Add cpp_warn_c90_c99_compat.
	* macro.c (replace_args): Warn when -Wc90-c99-compat is in effect.
	(parse_params): Likewise.

From-SVN: r213786
2014-08-10 06:10:49 +00:00
Richard Sandiford
35c3d610e3 Update copyright years in libcpp/
From-SVN: r206293
2014-01-02 22:24:45 +00:00
Joseph Myers
d3f4ff8b51 ucnid-2011-1.c: New test.
gcc/testsuite:
	* c-c++-common/cpp/ucnid-2011-1.c: New test.

libcpp:
	* ucnid.tab: Add C11 and C11NOSTART data.
	* makeucnid.c (digit): Rename enum value to N99.
	(C11, N11, all_languages): New enum values.
	(NUM_CODE_POINTS, MAX_CODE_POINT): New macros.
	(flags, decomp, combining_value): Use NUM_CODE_POINTS as array
	size.
	(decomp): Use unsigned int as element type.
	(all_decomp): New array.
	(read_ucnid): Handle C11 and C11NOSTART.  Use MAX_CODE_POINT.
	(read_table): Use MAX_CODE_POINT.  Store all decompositions in
	all_decomp.
	(read_derived): Use MAX_CODE_POINT.
	(write_table): Use NUM_CODE_POINTS.  Print N99, C11 and N11
	flags.  Print whole array variable declaration rather than just
	array contents.
	(char_id_valid, write_context_switch): New functions.
	(main): Call write_context_switch.
	* ucnid.h: Regenerate.
	* include/cpplib.h (struct cpp_options): Add c11_identifiers.
	* init.c (struct lang_flags): Add c11_identifiers.
	(cpp_set_lang): Set c11_identifiers option from selected language.
	* internal.h (struct normalize_state): Document "previous" as
	previous starter character.
	(NORMALIZE_STATE_UPDATE_IDNUM): Take character as argument.
	* charset.c (DIG): Rename enum value to N99.
	(C11, N11): New enum values.
	(struct ucnrange): Give name to struct.  Use short for flags and
	unsigned int for end of range.  Include ucnid.h for whole variable
	declaration.
	(ucn_valid_in_identifier): Allow for characters up to 0x10FFFF.
	Allow for C11 in determining valid characters and valid start
	characters.  Use check_nfc for non-Hangul context-dependent
	checks.  Only store starter characters in nst->previous.
	(_cpp_valid_ucn): Pass new argument to
	NORMALIZE_STATE_UPDATE_IDNUM.
	* lex.c (lex_identifier): Pass new argument to
	NORMALIZE_STATE_UPDATE_IDNUM.  Call NORMALIZE_STATE_UPDATE_IDNUM
	after initial non-UCN part of identifier.
	(lex_number): Pass new argument to NORMALIZE_STATE_UPDATE_IDNUM.

From-SVN: r204886
2013-11-16 00:05:08 +00:00
Richard Sandiford
500f3ed906 Update copyright years in libcpp.
From-SVN: r195162
2013-01-14 18:13:59 +00:00
Jakub Jelinek
f41e5bd19d re PR bootstrap/55380 (All search_line_fast implementations read beyond buffer)
PR bootstrap/55380
	PR other/54691
	* files.c (read_file_guts): Allocate extra 16 bytes instead of
	1 byte at the end of buf.  Pass size + 16 instead of size
	to _cpp_convert_input.
	* charset.c (_cpp_convert_input): Reallocate if there aren't
	at least 16 bytes beyond to.len in the buffer.  Clear 16 bytes
	at to.text + to.len.

From-SVN: r194102
2012-12-03 18:19:47 +01:00
Jakub Jelinek
d652f226fc Update Copyright years for files modified in 2010.
From-SVN: r168438
2011-01-03 21:52:22 +01:00
Simon Baldwin
87cf065171 diagnostic.h (diagnostic_override_option_index): New macro to set a diagnostic's option_index.
* diagnostic.h (diagnostic_override_option_index): New macro to
	set a diagnostic's option_index.
	* c-tree.h (c_cpp_error): Add warning reason argument.
	* opts.c (_warning_as_error_callback): New.
	(register_warning_as_error_callback): Store callback for
	warnings enabled via enable_warning_as_error.
	(enable_warning_as_error): Call callback, minor code tidy.
	* opts.h (register_warning_as_error_callback): Declare.
	* c-opts.c (warning_as_error_callback): New, set cpp_opts flag in
	response to -Werror=.
	(c_common_init_options): Register warning_as_error_callback in opts.c.
	* common.opt: Add -Wno-cpp option.
	* c-common.c (struct reason_option_codes_t): Map cpp warning
	reason codes to gcc option indexes.
	* (c_option_controlling_cpp_error): New function, lookup the gcc
	option index for a cpp warning reason code.
	* (c_cpp_error): Add warning reason argument, call
	c_option_controlling_cpp_error for diagnostic_override_option_index.
	* doc/invoke.texi: Document -Wno-cpp.

	* cpp.c (cb_cpp_error): Add warning reason argument, set a value
	for diagnostic_override_option_index if CPP_W_WARNING_DIRECTIVE.

	* directives.c (do_diagnostic): Add warning reason argument,
	call appropriate error reporting function for code.
	(directive_diagnostics): Call specific warning functions with
	warning reason where appropriate.
	(do_error, do_warning, do_pragma_dependency): Add warning reason
	argument to do_diagnostic calls.
	* macro.c (_cpp_warn_if_unused_macro, enter_macro_context,
	_cpp_create_definition): Call specific warning functions with
        warning reason where appropriate.
	* Makefile.in: Add new diagnostic functions to gettext translations.
	* include/cpplib.h (struct cpp_callbacks): Add warning reason code
	to error callback.
	(CPP_DL_WARNING, CPP_DL_WARNING_SYSHDR, CPP_DL_PEDWARN, CPP_DL_ERROR,
	CPP_DL_ICE, CPP_DL_NOTE, CPP_DL_FATAL): Replace macros with enums.
	(CPP_W_NONE, CPP_W_DEPRECATED, CPP_W_COMMENTS,
	CPP_W_MISSING_INCLUDE_DIRS, CPP_W_TRIGRAPHS, CPP_W_MULTICHAR,
	CPP_W_TRADITIONAL, CPP_W_LONG_LONG, CPP_W_ENDIF_LABELS,
	CPP_W_NUM_SIGN_CHANGE, CPP_W_VARIADIC_MACROS,
	CPP_W_BUILTIN_MACRO_REDEFINED, CPP_W_DOLLARS, CPP_W_UNDEF,
	CPP_W_UNUSED_MACROS, CPP_W_CXX_OPERATOR_NAMES, CPP_W_NORMALIZE,
	CPP_W_INVALID_PCH, CPP_W_WARNING_DIRECTIVE): New enums for cpp
	warning reason codes.
	(cpp_warning, cpp_pedwarning, cpp_warning_syshdr,
	cpp_warning_with_line, cpp_pedwarning_with_line,
	cpp_warning_with_line_syshdr): New specific error reporting functions.
	* pch.c (cpp_valid_state): Call specific warning functions with
        warning reason where appropriate.
	* errors.c (cpp_diagnostic, cpp_diagnostic_with_line): New central
	diagnostic handlers.
	(cpp_warning, cpp_pedwarning, cpp_warning_syshdr,
	cpp_warning_with_line, cpp_pedwarning_with_line,
	cpp_warning_with_line_syshdr): New specific error reporting functions.
	* expr.c (cpp_classify_number, eval_token, num_unary_op): Call
	specific warning functions with warning reason where appropriate.
	* lex.c (_cpp_process_line_notes, _cpp_skip_block_comment,
	warn_about_normalization, lex_identifier_intern, lex_identifier,
	_cpp_lex_direct): Ditto.
	* charset.c (_cpp_valid_ucn, convert_hex, convert_escape,
	narrow_str_to_charconst): Ditto.

	* gcc.dg/cpp/warn-undef-2.c: New.
	* gcc.dg/cpp/warn-traditional-2.c: New.
	* gcc.dg/cpp/warn-comments-2.c: New.
	* gcc.dg/cpp/warning-directive-1.c: New.
	* gcc.dg/cpp/warn-long-long.c: New.
	* gcc.dg/cpp/warn-traditional.c: New.
	* gcc.dg/cpp/warn-variadic-2.c: New.
	* gcc.dg/cpp/warn-undef.c: New.
	* gcc.dg/cpp/warn-normalized-1.c: New.
	* gcc.dg/cpp/warning-directive-2.c: New.
	* gcc.dg/cpp/warn-long-long-2.c: New.
	* gcc.dg/cpp/warn-variadic.c: New.
	* gcc.dg/cpp/warn-normalized-2.c: New.
	* gcc.dg/cpp/warning-directive-3.c: New.
	* gcc.dg/cpp/warn-deprecated-2.c: New.
	* gcc.dg/cpp/warn-trigraphs-1.c: New.
	* gcc.dg/cpp/warn-multichar-2.c: New.
	* gcc.dg/cpp/warn-normalized-3.c: New.
	* gcc.dg/cpp/warning-directive-4.c: New.
	* gcc.dg/cpp/warn-unused-macros.c: New.
	* gcc.dg/cpp/warn-trigraphs-2.c: New.
	* gcc.dg/cpp/warn-cxx-compat-2.c: New.
	* gcc.dg/cpp/warn-cxx-compat.c: New.
	* gcc.dg/cpp/warn-redefined.c: New.
	* gcc.dg/cpp/warn-trigraphs-3.c: New.
	* gcc.dg/cpp/warn-unused-macros-2.c: New.
	* gcc.dg/cpp/warn-deprecated.c: New.
	* gcc.dg/cpp/warn-trigraphs-4.c: New.
	* gcc.dg/cpp/warn-redefined-2.c: New.
	* gcc.dg/cpp/warn-comments.c: New.
	* gcc.dg/cpp/warn-multichar.c: New.
	* g++.dg/cpp/warning-directive-1.C: New.
	* g++.dg/cpp/warning-directive-2.C: New.
	* g++.dg/cpp/warning-directive-3.C: New.
	* g++.dg/cpp/warning-directive-4.C: New.
	* gfortran.dg/warning-directive-1.F90: New.
	* gfortran.dg/warning-directive-3.F90: New.
	* gfortran.dg/warning-directive-2.F90: New.
	* gfortran.dg/warning-directive-4.F90: New.

From-SVN: r158079
2010-04-07 17:18:10 +00:00
Jason Merrill
00a81b8b9d More N3077 raw string changes
More N3077 raw string changes
	* charset.c (cpp_interpret_string): Don't transform UCNs in raw
	strings.
	* lex.c (bufring_append): Split out from...
	(lex_raw_string): ...here.  Undo trigraph and line splicing
	transformations.  Do process line notes in multi-line literals.
	(_cpp_process_line_notes): Ignore notes that were already handled.

From-SVN: r157804
2010-03-29 16:07:29 -04:00
Jason Merrill
521506258f Some raw string changes from N3077
Some raw string changes from N3077
	* charset.c (cpp_interpret_string): Change inner delimiters to ().
	* lex.c (lex_raw_string): Likewise.  Also disallow '\' in delimiter.

From-SVN: r157797
2010-03-29 11:00:43 -04:00
Jakub Jelinek
2c6e3f5540 charset.c (cpp_init_iconv): Initialize utf8_cset_desc.
* charset.c (cpp_init_iconv): Initialize utf8_cset_desc.
	(_cpp_destroy_iconv): Destroy utf8_cset_desc, char16_cset_desc
	and char32_cset_desc.
	(converter_for_type): Handle CPP_UTF8STRING.
	(cpp_interpret_string): Handle CPP_UTF8STRING and raw-strings.
	* directives.c (get__Pragma_string): Handle CPP_UTF8STRING.
	(parse_include): Reject raw strings.
	* include/cpplib.h (CPP_UTF8STRING): New token type.
	* internal.h (struct cpp_reader): Add utf8_cset_desc field.
	* lex.c (lex_raw_string): New function.
	(lex_string): Handle u8 string literals, call lex_raw_string
	for raw string literals.
	(_cpp_lex_direct): Call lex_string even for u8" and {,u,U,L,u8}R"
	sequences.
	* macro.c (stringify_arg): Handle CPP_UTF8STRING.

	* c-common.c (c_parse_error): Handle CPP_UTF8STRING.
	* c-lex.c (c_lex_with_flags): Likewise.  Test C_LEX_STRING_NO_JOIN
	instead of C_LEX_RAW_STRINGS.
	(lex_string): Handle CPP_UTF8STRING.
	* c-parser.c (c_parser_postfix_expression): Likewise.
	* c-pragma.h (C_LEX_RAW_STRINGS): Rename to ...
	(C_LEX_STRING_NO_JOIN): ... this.

	* parser.c (cp_lexer_print_token, cp_parser_is_string_literal,
	cp_parser_string_literal, cp_parser_primary_expression): Likewise.
	(cp_lexer_get_preprocessor_token): Use C_LEX_STRING_JOIN instead
	of C_LEX_RAW_STRINGS.

	* gcc.dg/raw-string-1.c: New test.
	* gcc.dg/raw-string-2.c: New test.
	* gcc.dg/raw-string-3.c: New test.
	* gcc.dg/raw-string-4.c: New test.
	* gcc.dg/raw-string-5.c: New test.
	* gcc.dg/raw-string-6.c: New test.
	* gcc.dg/raw-string-7.c: New test.
	* gcc.dg/utf8-1.c: New test.
	* gcc.dg/utf8-2.c: New test.
	* gcc.dg/utf-badconcat2.c: New test.
	* gcc.dg/utf-dflt2.c: New test.
	* gcc.dg/cpp/include6.c: New test.
	* g++.dg/ext/raw-string-1.C: New test.
	* g++.dg/ext/raw-string-2.C: New test.
	* g++.dg/ext/raw-string-3.C: New test.
	* g++.dg/ext/raw-string-4.C: New test.
	* g++.dg/ext/raw-string-5.C: New test.
	* g++.dg/ext/raw-string-6.C: New test.
	* g++.dg/ext/raw-string-7.C: New test.
	* g++.dg/ext/utf8-1.C: New test.
	* g++.dg/ext/utf8-2.C: New test.
	* g++.dg/ext/utf-badconcat2.C: New test.
	* g++.dg/ext/utf-dflt2.C: New test.

From-SVN: r152995
2009-10-19 23:41:15 +02:00
Jason Merrill
30c99a9e19 * charset.c (_cpp_valid_ucn): Update C++0x restrictions.
From-SVN: r152614
2009-10-09 20:39:46 -04:00
Tom Tromey
709a22df79 re PR preprocessor/41067 (Inconsistency in warnings on invalid \-escapes)
PR preprocessor/41067:
	* charset.c (convert_escape): Add missing ":" to error text.

From-SVN: r150854
2009-08-17 17:34:53 +00:00
Joseph Myers
9e322bc1a5 charset.c (one_utf8_to_cppchar): Correct mask used for 5-byte UTF-8 sequences.
libcpp:
	* charset.c (one_utf8_to_cppchar): Correct mask used for 5-byte
	UTF-8 sequences.

gcc/testsuite:
	* gcc.dg/cpp/utf8-5byte-1.c: New test.

From-SVN: r147073
2009-05-03 12:59:26 +01:00
Jakub Jelinek
748086b7b2 Licensing changes to GPLv3 resp. GPLv3 with GCC Runtime Exception.
From-SVN: r145841
2009-04-09 17:00:19 +02:00
H.J. Lu
0b7c73cc04 re PR preprocessor/36479 (Short buffer in libcpp)
2008-06-12  H.J. Lu  <hongjiu.lu@intel.com>

	PR preprocessor/36479
	* charset.c (cpp_interpret_string_notranslate): Also set
	narrow_cset_desc.width.

From-SVN: r136714
2008-06-12 10:03:41 -07:00
Tom Tromey
688e7a5344 re PR preprocessor/33415 (Can't compile .cpp file with UTF-8 BOM.)
libcpp
	PR libcpp/33415:
	* charset.c (_cpp_convert_input): Add buffer_start argument.
	Ignore UTF-8 BOM if seen.
	* internal.h (_cpp_convert_input): Add argument.
	* files.c (struct _cpp_file) <buffer_start>: New field.
	(destroy_cpp_file): Free buffer_start, not buffer.
	(_cpp_pop_file_buffer): Likewise.
	(read_file_guts): Update.
gcc/testsuite
	PR libcpp/33415:
	* gcc.dg/cpp/pr33415.c: New file.

From-SVN: r134507
2008-04-21 14:02:00 +00:00
Kris Van Hees
b6baa67d79 cpp-id-data.h (UC): Was U, conflicts with U...
libcpp/ChangeLog:
2008-04-14  Kris Van Hees <kris.van.hees@oracle.com>

* include/cpp-id-data.h (UC): Was U, conflicts with U... literal.
* include/cpplib.h (CHAR16, CHAR32, STRING16, STRING32): New tokens.
(struct cpp_options): Added uliterals.
(cpp_interpret_string): Update prototype.
(cpp_interpret_string_notranslate): Idem.
* charset.c (init_iconv_desc): New width member in cset_converter.
(cpp_init_iconv): Add support for char{16,32}_cset_desc.
(convert_ucn): Idem.
(emit_numeric_escape): Idem.
(convert_hex): Idem.
(convert_oct): Idem.
(convert_escape): Idem.
(converter_for_type): New function.
(cpp_interpret_string): Use converter_for_type, support u and U prefix.
(cpp_interpret_string_notranslate): Match changed prototype.
(wide_str_to_charconst): Use converter_for_type.
(cpp_interpret_charconst): Add support for CPP_CHAR{16,32}.
* directives.c (linemarker_dir): Macro U changed to UC.
(parse_include): Idem.
(register_pragma_1): Idem.
(restore_registered_pragmas): Idem.
(get__Pragma_string): Support CPP_STRING{16,32}.
* expr.c (eval_token): Support CPP_CHAR{16,32}.
* init.c (struct lang_flags): Added uliterals.
(lang_defaults): Idem.
* internal.h (struct cset_converter) <width>: New field.
(struct cpp_reader) <char16_cset_desc>: Idem.
(struct cpp_reader) <char32_cset_desc>: Idem.
* lex.c (digraph_spellings): Macro U changed to UC.
(OP, TK): Idem.
(lex_string): Add support for u'...', U'...', u... and U....
(_cpp_lex_direct): Idem.
* macro.c (_cpp_builtin_macro_text): Macro U changed to UC.
(stringify_arg): Support CPP_CHAR{16,32} and CPP_STRING{16,32}.

gcc/ChangeLog:
2008-04-14  Kris Van Hees <kris.van.hees@oracle.com>
  
* c-common.c (CHAR16_TYPE, CHAR32_TYPE): New macros.
(fname_as_string): Match updated cpp_interpret_string prototype.
(fix_string_type): Support char16_t* and char32_t*.
(c_common_nodes_and_builtins): Add char16_t and char32_t (and
derivative) nodes.  Register as builtin if C++0x.
(c_parse_error): Support CPP_CHAR{16,32}.
* c-common.h (RID_CHAR16, RID_CHAR32): New elements. 
(enum c_tree_index) <CTI_CHAR16_TYPE, CTI_SIGNED_CHAR16_TYPE,
CTI_UNSIGNED_CHAR16_TYPE, CTI_CHAR32_TYPE, CTI_SIGNED_CHAR32_TYPE,
CTI_UNSIGNED_CHAR32_TYPE, CTI_CHAR16_ARRAY_TYPE,
CTI_CHAR32_ARRAY_TYPE>: New elements.
(char16_type_node, signed_char16_type_node, unsigned_char16_type_node,
char32_type_node, signed_char32_type_node, char16_array_type_node,
char32_array_type_node): New defines.
* c-lex.c (cb_ident): Match updated cpp_interpret_string prototype.
(c_lex_with_flags): Support CPP_CHAR{16,32} and CPP_STRING{16,32}.
(lex_string): Support CPP_STRING{16,32}, match updated
cpp_interpret_string and cpp_interpret_string_notranslate prototypes.
(lex_charconst): Support CPP_CHAR{16,32}.
* c-parser.c (c_parser_postfix_expression): Support CPP_CHAR{16,32}
and CPP_STRING{16,32}.

gcc/cp/ChangeLog:
2008-04-14  Kris Van Hees <kris.van.hees@oracle.com>

* cvt.c (type_promotes_to): Support char16_t and char32_t.
* decl.c (grokdeclarator): Disallow signed/unsigned/short/long on
char16_t and char32_t.
* lex.c (reswords): Add char16_t and char32_t (for c++0x).
* mangle.c (write_builtin_type): Mangle char16_t/char32_t as vendor
extended builtin type u8char32_t.
* parser.c (cp_lexer_next_token_is_decl_specifier_keyword): Support
RID_CHAR{16,32}.
(cp_lexer_print_token): Support CPP_STRING{16,32}.
(cp_parser_is_string_literal): Idem.
(cp_parser_string_literal): Idem.
(cp_parser_primary_expression): Support CPP_CHAR{16,32} and
CPP_STRING{16,32}.
(cp_parser_simple_type_specifier): Support RID_CHAR{16,32}. 
* tree.c (char_type_p): Support char16_t and char32_t as char types.
* typeck.c (string_conv_p): Support char16_t and char32_t.

gcc/testsuite/ChangeLog:
2008-04-14  Kris Van Hees <kris.van.hees@oracle.com>

Tests for char16_t and char32_t support.
* g++.dg/ext/utf-cvt.C: New
* g++.dg/ext/utf-cxx0x.C: New
* g++.dg/ext/utf-cxx98.C: New
* g++.dg/ext/utf-dflt.C: New
* g++.dg/ext/utf-gnuxx0x.C: New
* g++.dg/ext/utf-gnuxx98.C: New
* g++.dg/ext/utf-mangle.C: New
* g++.dg/ext/utf-typedef-cxx0x.C: New
* g++.dg/ext/utf-typedef-
* g++.dg/ext/utf-typespec.C: New
* g++.dg/ext/utf16-1.C: New
* g++.dg/ext/utf16-2.C: New
* g++.dg/ext/utf16-3.C: New
* g++.dg/ext/utf16-4.C: New
* g++.dg/ext/utf32-1.C: New
* g++.dg/ext/utf32-2.C: New
* g++.dg/ext/utf32-3.C: New
* g++.dg/ext/utf32-4.C: New
* gcc.dg/utf-cvt.c: New
* gcc.dg/utf-dflt.c: New
* gcc.dg/utf16-1.c: New
* gcc.dg/utf16-2.c: New
* gcc.dg/utf16-3.c: New
* gcc.dg/utf16-4.c: New
* gcc.dg/utf32-1.c: New
* gcc.dg/utf32-2.c: New
* gcc.dg/utf32-3.c: New
* gcc.dg/utf32-4.c: New

libiberty/ChangeLog:
2008-04-14  Kris Van Hees <kris.van.hees@oracle.com>

* testsuite/demangle-expected: Added tests for char16_t and char32_t.

From-SVN: r134438
2008-04-18 09:58:08 -04:00
DJ Delorie
675575f57a charset.c (convert_using_iconv): Close out any shift states, returning to the initial state.
* charset.c (convert_using_iconv): Close out any shift states,
returning to the initial state.

From-SVN: r130785
2007-12-11 20:38:10 -05:00
Tom Tromey
30b0edc057 re PR preprocessor/30001 (out-of-bounds access when processing empty file)
PR preprocessor/30001:
	* charset.c (_cpp_convert_input): Check that to.len is greater
	than zero.

From-SVN: r120245
2006-12-28 18:45:48 +00:00
Joseph Myers
178b58b59e re PR c++/17964 (cpp error messages contain wrong line in C++)
gcc:
	PR c++/17964
	* diagnostic.c (diagnostic_set_info_translated): New function.
	(diagnostic_set_info): Use it.  Add comment.
	* diagnostic.h (diagnostic_set_info_translated): Declare.

gcc/cp:
	* error.c (cp_cpp_error): New function.
	* cp-tree.h (cp_cpp_error): Declare.
	* parser.c (cp_lexer_new_main): Set CPP option client_diagnostic
	and error callback after lexing.

gcc/testsuite:
	* g++.dg/cpp/string-1.C: New test.

libcpp:
	* include/cpplib.h (struct cpp_options): Add client_diagnostic.
	(struct cpp_callbacks): Add error.
	* errors.c (cpp_error): If client_diagnostic, use error callback.
	* charset.c (convert_escape): Don't use %03o in diagnostic.

From-SVN: r106454
2005-11-03 23:08:18 +00:00
Kelley Cook
200031d1d5 all files: Update FSF address in copyright headers.
2005-06-29  Kelley Cook  <kcook@gcc.gnu.org>

	* all files: Update FSF address in copyright headers.
	* makeucnid.c (write_copyright): Update outputted FSF address.

From-SVN: r101413
2005-06-29 02:34:39 +00:00
Gabriel Dos Reis
c3f829c1a6 configure.ac: Check declarations for asprintf and vasprintf.
* configure.ac: Check declarations for asprintf and vasprintf.
	* config.in: Regenerate.
	* configure: Likewise.

	* charset.c (conversion_loop): Use XRESIZEVEC.
	(convert_no_conversion): Likewise.
	(convert_using_iconv): Likewise.
	(init_iconv_desc): Cast return value of alloca.
	(cpp_host_to_exec_charset): Use XNEWVEC.
	(emit_numeric_escape): Use XRESIZEVEC.
	(cpp_interpret_string): Use XNEWVEC.
	(cpp_interpret_string): Use XRESIZEVEC.
	(_cpp_interpret_identifier): Cast return value of alloca.
	(_cpp_convert_input): Use XNEWVEC and XRESIZEVEC.
	* directives.c (glue_header_name): Use XNEWVEC and XRESIZEVEC.
	(parse_include): Use XNEWVEC.
	(insert_pragma_entry): Rename local variable "new" to
	"new_entry".
	(save_registered_pragmas): Cast return value of xmemdup.
	(destringize_and_run): Same for alloca.
	(parse_assertion): Likewise.
	(do_assert): Cast allocated storage to proper type.
	(cpp_define): Likewise.
	(_cpp_define_builtin): Likewise.
	(cpp_undef): Likewise.
	(handle_assertion): Likewise.
	(cpp_push_buffer): Rename local variable "new" to "new_buffer".
	* expr.c (CPP_UPLUS): Cast value to type cpp_ttype.
	(CPP_UMINUS): Likewise.
	(struct cpp_operator): Rename from struct operator.
	(_cpp_expand_op_stack): Use XRESIZEVEC.
	* files.c (pch_open_file): Use XNEWVEC.
	(pch_open_file): Use XRESIZEVEC.
	(read_file_guts): Use XNEWVEC and XRESIZEVEC.
	(dir_name_of_file): Use XNEWVEC.
	(make_cpp_file): Use XCNEW.
	(make_cpp_dir): Likewise.
	(allocate_file_hash_entries): USE XNEWVEC.
	(cpp_included): Cast return value of htab_find_with_hash.
	(append_file_to_dir): Use XNEWVEC.
	(read_filename_string): Likewise. Use XRESIZEVEC too.
	(read_name_map): Cast return value of alloca.  Use XRESIZEVEC.
	(remap_filename): Use XNEWVEC.
	(struct pchf_entry): Move definition out of struct pchf_data.
	(_cpp_save_file_entries): Use XCNEWVAR.
	(_cpp_read_file_entries): Use XNEWVAR.
	* identifiers.c (alloc_node): Use XOBNEW.
	* init.c (cpp_create_reader): Use XCNEW.
	(cpp_init_builtins): Cast of b->value to enum builtin_type.
	(read_original_directory): Cast return value of alloca.
	* lex.c (add_line_note): Use XRESIZEVEC.
	(warn_about_normalization): Use XNEWVEC.
	(_cpp_lex_direct): Cast node->directive_index to (enum cpp_ttype).
	(new_buff): Use XNEWVEC.
	* line-map.c (linemap_add): Use XRESIZEVEC.
	* macro.c (builtin_macro): Cast return value of alloca.
	(paste_tokens): Likewise.
	(expand_arg): Use XNEWVEC and XRESIZEVEC.
	(_cpp_save_parameter): Use XRESIZEVEC.
	(create_iso_definition): Cast allocated storage to proper type.
	(_cpp_create_definition): Likewise.
	(cpp_macro_definition): Use XRESIZEVEC.
	* makedepend.c (add_clm): Use XNEW.
	(add_dir): Likewise.
	* mkdeps.c (munge): Use XNEWVEC.
	(deps_init): Use XCNEW.
	(deps_add_target): Use XRESIZEVEC.
	(deps_add_default_target): Cast return value of alloca.
	(deps_add_dep): Use XRESIZEVEC.
	(deps_add_vpath): Likewise.  Use XNEWVEC too.
	(deps_restore): Likewise.
	* pch.c (save_idents): Use XNEW and XNEWVEC.
	(cpp_save_state): Use XNEW.
	(count_defs): Cast return value of htab_find.
	(write_defs): Likewise.
	(cpp_write_pch_deps): Use XNEWVEC.
	(collect_ht_nodes): Use XRESIZEVEC.
	(cpp_valid_state): Use XNEWVEC.
	(save_macros): Use XRESIZEVEC.  Cast return value of xmemdup.
	* symtab.c (ht_create): Use XCNEW.
	(ht_lookup_with_hash): Cast return value of obstack_copy0.
	(ht_expand): Use XCNEWVEC.
	* system.h (HAVE_DESIGNATED_INITIALIZERS): False if __cplusplus.
	(bool): Do not define if __cplusplus.

From-SVN: r100295
2005-05-28 15:52:48 +00:00
Geoffrey Keating
c79e602b63 charset.c (_cpp_valid_ucn): In identifiers, reject a partial UCN rather than printing an error.
2005-03-15  Geoffrey Keating  <geoffk@apple.com>

	* charset.c (_cpp_valid_ucn): In identifiers, reject a partial
	UCN rather than printing an error.

Index: gcc/testsuite/ChangeLog
2005-03-15  Geoffrey Keating  <geoffk@apple.com>

	* gcc.dg/cpp/ucnid-7.c: New.

From-SVN: r96546
2005-03-16 00:59:31 +00:00
Geoffrey Keating
50668cf626 Index: gcc/ChangeLog
2005-03-14  Geoffrey Keating  <geoffk@apple.com>

	* doc/cppopts.texi (-fexec-charset): Add concept index entry.
	(-fwide-exec-charset): Likewise.
	(-finput-charset): Likewise.
	* doc/invoke.texi (Warning Options): Document -Wnormalized=.
	* c-opts.c (c_common_handle_option): Handle -Wnormalized=.
	* c.opt (Wnormalized): New.

Index: libcpp/ChangeLog
2005-03-14  Geoffrey Keating  <geoffk@apple.com>

	* init.c (cpp_create_reader): Default warn_normalize to normalized_C.
	* charset.c: Update for new format of ucnid.h.
	(ucn_valid_in_identifier): Update for new format of ucnid.h.
	Add NST parameter, and update it; update callers.
	(cpp_valid_ucn): Add NST parameter, update callers.  Replace abort
	with cpp_error.
	(convert_ucn): Pass normalize_state to cpp_valid_ucn.
	* internal.h (struct normalize_state): New.
	(INITIAL_NORMALIZE_STATE): New.
	(NORMALIZE_STATE_RESULT): New.
	(NORMALIZE_STATE_UPDATE_IDNUM): New.
	(_cpp_valid_ucn): New.
	* lex.c (warn_about_normalization): New.
	(forms_identifier_p): Add normalize_state parameter, update callers.
	(lex_identifier): Add normalize_state parameter, update callers.  Keep
	the state current.
	(lex_number): Likewise.
	(_cpp_lex_direct): Pass normalize_state to subroutines.  Check
	it with warn_about_normalization.
	* makeucnid.c: New.
	* ucnid.h: Replace.
	* ucnid.pl: Remove.
	* ucnid.tab: Make appropriate for input to makeucnid.c.  Remove
	comments about obsolete version of C++.
	* include/cpplib.h (enum cpp_normalize_level): New.
	(struct cpp_options): Add warn_normalize field.

Index: gcc/testsuite/ChangeLog
2005-03-14  Geoffrey Keating  <geoffk@apple.com>

	* gcc.dg/cpp/normalize-1.c: New.
	* gcc.dg/cpp/normalize-2.c: New.
	* gcc.dg/cpp/normalize-3.c: New.
	* gcc.dg/cpp/normalize-4.c: New.
	* gcc.dg/cpp/ucnid-4.c: New.
	* gcc.dg/cpp/ucnid-5.c: New.
	* g++.dg/cpp/normalize-1.C: New.
	* g++.dg/cpp/ucnid-1.C: New.

From-SVN: r96459
2005-03-15 00:36:33 +00:00
Geoffrey Keating
47e204910a Index: libcpp/ChangeLog
2005-03-12  Geoffrey Keating  <geoffk@apple.com>

	* directives.c (glue_header_name): Update call to cpp_spell_token.
	* internal.h (_cpp_interpret_identifier): New.
	* charset.c (_cpp_interpret_identifier): New.
	(_cpp_valid_ucn): Allow UCN version of '$'.
	* lex.c (lex_identifier): Add extra parameter to indicate if initial
	character was '$' or '\'.  Support identifiers with UCNs.
	(forms_identifier_p): Allow UCNs.
	(_cpp_lex_direct): Pass extra parameter to lex_identifier.
	(utf8_to_ucn): New.
	(cpp_spell_token): Add FORSTRING parameter.  Use it.
	(cpp_token_as_text): Update call to cpp_spell_token.
	(cpp_output_token): Write UCNs back out.
	(stringify_arg): Update call to cpp_spell_token.
	(paste_tokens): Likewise.
	(cpp_macro_definition): Likewise.
	* macro.c (stringify_arg): Likewise.
	(paste_tokens): Likewise.
	(cpp_macro_definition): Likewise.
	* include/cpplib.h: Add parameter to cpp_spell_token.

Index: gcc/ChangeLog
2005-03-12  Geoffrey Keating  <geoffk@apple.com>

	* c-lex.c (c_lex_with_flags): Add parameter to call to
	cpp_spell_token.

Index: gcc/testsuite/ChangeLog
2005-03-12  Geoffrey Keating  <geoffk@apple.com>

	* gcc.dg/ucnid-1.c: New.
	* gcc.dg/ucnid-2.c: New.
	* gcc.dg/ucnid-3.c: New.
	* gcc.dg/ucnid-4.c: New.
	* gcc.dg/ucnid-5.c: New.
	* gcc.dg/ucnid-6.c: New.
	* gcc.dg/cpp/ucnid-1.c: New.
	* gcc.dg/cpp/ucnid-2.c: New.
	* gcc.dg/cpp/ucnid-3.c: New.
	* g++.dg/other/ucnid-1.C: New.

From-SVN: r96333
2005-03-12 10:44:06 +00:00
Zack Weinberg
c5ff069dc4 re PR middle-end/18785 (isdigit builtin function fails with EBCDIC character sets)
PR 18785
libcpp:
	* charset.c (LAST_POSSIBLY_BASIC_SOURCE_CHAR): New helper macro.
	(cpp_host_to_exec_charset): New function.
	* include/cpplib.h: Declare cpp_host_to_exec_charset.

gcc:
	* langhooks.h (struct lang_hooks): Add to_target_charset.
	* langhooks.c (lhd_to_target_charset): New function.
	* langhooks-def.h: Declare lhd_to_target_charset.
	(LANG_HOOKS_TO_TARGET_CHARSET): New macro.
	(LANG_HOOKS_INITIALIZER): Update.
	* c-common.c (c_common_to_target_charset): New function.
	* c-common.h: Declare it.
	* c-objc-common.h (LANG_HOOKS_TO_TARGET_CHARSET): Set to
	c_common_to_target_charset.

	* defaults.c (TARGET_BELL, TARGET_BS, TARGET_CR, TARGET_DIGIT0)
	(TARGET_ESC, TARGET_FF, TARGET_NEWLINE, TARGET_TAB, TARGET_VT):
	Delete definitions.
	* system.h: Poison them.
	* doc/tm.texi: Don't discuss them.
	* builtins.c (fold_builtin_isdigit): Use lang_hooks.to_target_charset.
	* c-pretty-print.c (pp_c_integer_constant): Don't use pp_c_char.
	(pp_c_char): Do not attempt to generate letter escapes for
	newline, tab, etc.
	* config/arm/arm.c (output_ascii_pseudo_op): Likewise.
	* config/mips/mips.c (mips_output_ascii): Likewise.
gcc/cp:
	* cp-objcp-common.h (LANG_HOOKS_TO_TARGET_CHARSET): Set to
	c_common_to_target_charset.  Delete bogus comment.
gcc/testsuite:
	* gcc.dg/charset/builtin1.c: New test.

From-SVN: r95304
2005-02-20 17:01:32 +00:00
Devang Patel
04c90eea07 charset.c (_cpp_convert_input): Check '\r' before inserting '\n' at the end.
* charset.c (_cpp_convert_input): Check '\r' before inserting
      '\n' at the end.
      * gcc.dg/cpp/mac-eol-at-eof.c: New test.

From-SVN: r95289
2005-02-19 11:48:02 -08:00
Zack Weinberg
a29f62d9c3 * charset.c (_cpp_destroy_iconv, emit_numeric_escape)
(_cpp_convert_input, _cpp_default_encoding): Add comments.
	Some other comments in this file also tweaked.

From-SVN: r87688
2004-09-18 00:56:19 +00:00
Zack Weinberg
c6e8380069 Makefile.am: Add makedepend.
libcpp:
	* Makefile.am: Add makedepend.
	* Makefile.in, aclocal.m4: Regenerate.
	* charset.c: Insert a space to avoid a warning.
	* directives.c: Include mkdeps.h.
	(_cpp_handle_directive): Reenable macro expander if appropriate.
	(undefine_macros): Inline body of _cpp_free_definition for speed.
	Do not call undef callback or _cpp_warn_if_unused_macro.
	(cpp_get_deps): New interface.
	* files.c (search_cache): Add pfile argument.  Check for file
	that would be found by "" or <> search here...
	(_cpp_find_file): ...not here.  Correct recorded start_dir of
	files found by directory-of-current-file search that would be
	found by "" or <> search.
	* init.c (cpp_add_dependency_target): Delete.
	* internal.h (struct lexer_state): Add discarding_output flag.
	* lex.c (lex_identifier): Compute hash function while scanning.
	* macro.c (cpp_scan_nooutput): Disable macro expansion outside
	directives.
	* makedepend.c: New file.
	* mkdeps.c (struct deps): Add vpath vector.
	(apply_vpath, deps_add_vpath): New function.
	(deps_free): Free vpath vector.
	(deps_add_dep, deps_add_target): Use apply_vpath.
	* symtab.c (calc_hash): Use HT_HASHSTEP and HT_FINISH.
	(ht_lookup_with_hash): New function.
	* cpplib.h, mkdeps.h: Update prototypes.
	* symtab.h: Update prototypes.
	(HT_HASHSTEP, HT_FINISH): New macros.

gcc:
	* Makefile.in (MKDEPS_H): New shorthand.
	(c-opts.o): Update dependencies.
	* c-opts.c: Include mkdeps.h.
	(handle_deferred_opts): Use cpp_get_deps and deps_add_target,
	not cpp_add_dependency_target.

From-SVN: r82654
2004-06-05 20:58:06 +00:00
Paolo Bonzini
4f4e53dd85 Makefile.def (host_modules): add libcpp.
ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	* Makefile.def (host_modules): add libcpp.
	* Makefile.tpl: Add dependencies on and for libcpp.
	* Makefile.in: Regenerate.
	* configure.in: Add libcpp host module.
	* configure: Regenerate.

config/ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	* acx.m4 (ACX_HEADER_STDBOOL, ACX_HEADER_STRING):
	From gcc.

gcc/ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	Move libcpp to the toplevel.
	* Makefile.in: Remove references to libcpp files,
	use CPPLIBS instead of libcpp.a.  Define SYMTAB_H
	and change hashtable.h to that.
	* aclocal.m4 (gcc_AC_HEADER_STDBOOL,
	gcc_AC_HEADER_STRING, gcc_AC_C__BOOL): Remove.
	* configure.ac (gcc_AC_C__BOOL, HAVE_UCHAR): Remove tests.
	* configure: Regenerate.
	* config.in: Regenerate.
	* c-ppoutput.c: Include ../libcpp/internal.h instead of cpphash.h.
	* cppcharset.c: Removed.
	* cpperror.c: Removed.
	* cppexp.c: Removed.
	* cppfiles.c: Removed.
	* cpphash.c: Removed.
	* cpphash.h: Removed.
	* cppinit.c: Removed.
	* cpplex.c: Removed.
	* cpplib.c: Removed.
	* cpplib.h: Removed.
	* cppmacro.c: Removed.
	* cpppch.c: Removed.
	* cpptrad.c: Removed.
	* cppucnid.h: Removed.
	* cppucnid.pl: Removed.
	* cppucnid.tab: Removed.
	* hashtable.c: Removed.
	* hashtable.h: Removed.
	* line-map.c: Removed.
	* line-map.h: Removed.
	* mkdeps.c: Removed.
	* mkdeps.h: Removed.
	* stringpool.h: Include symtab.h instead of hashtable.h.
	* tree.h: Include symtab.h instead of hashtable.h.
	* system.h (O_NONBLOCK, O_NOCTTY): Do not define.

gcc/cp/ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	* Make-lang.in: No need to specify $(LIBCPP).

gcc/java/ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	* Make-lang.in: Link in $(LIBCPP) instead of mkdeps.o.

libcpp/ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	Moved libcpp from the gcc subdirectory to the toplevel.
	* Makefile.am: New file.
	* Makefile.in: Regenerate.
	* configure.ac: New file.
	* configure: Regenerate.
	* config.in: Regenerate.
	* charset.c: Moved from gcc/cppcharset.c.  Add note about
	brokenness of input charset detection.  Adjust for change
	in name of cppucnid.h.
	* errors.c: Moved from gcc/cpperror.c.  Do not include intl.h.
	* expr.c: Moved from gcc/cppexp.c.
	* files.c: Moved from gcc/cppfiles.c.  Do not include intl.h.
	Remove #define of O_BINARY, it is in system.h.
	* identifiers.c: Moved from gcc/cpphash.c.
	* internal.h: Moved from gcc/cpphash.h.  Change header
	guard name.  All other files adjusted to match name change.
	* init.c: Moved from gcc/cppinit.c.
	(init_library) [ENABLE_NLS]: Call bindtextdomain.
	* lex.c: Moved from gcc/cpplex.c.
	* directives.c: Moved from gcc/cpplib.c.
	* macro.c: Moved from gcc/cppmacro.c.
	* pch.c: Moved from gcc/cpppch.c.  Do not include intl.h.
	* traditional.c: Moved from gcc/cpptrad.c.
	* ucnid.h: Moved from gcc/cppucnid.h.  Change header
	guard name.
	* ucnid.pl: Moved from gcc/cppucnid.pl.
	* ucnid.tab: Moved from gcc/cppucnid.tab.  Change header
	guard name.
	* symtab.c: Moved from gcc/hashtable.c.
	* line-map.c: Moved from gcc.  Do not include intl.h.
	* mkdeps.c: Moved from gcc.
	* system.h: New file.

libcpp/include/ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	* cpplib.h: Moved from gcc.  Change header guard name.
	* line-map.h: Moved from gcc.  Change header guard name.
	* mkdeps.h: Moved from gcc.  Change header guard name.
	* symtab.h: Moved from gcc/hashtable.h.  Change header
	guard name.

libcpp/po/ChangeLog:

2004-05-23  Paolo Bonzini  <bonzini@gnu.org>

	* be.po: Extracted from gcc/po/be.po.
	* ca.po: Extracted from gcc/po/ca.po.
	* da.po: Extracted from gcc/po/da.po.
	* de.po: Extracted from gcc/po/de.po.
	* el.po: Extracted from gcc/po/el.po.
	* es.po: Extracted from gcc/po/es.po.
	* fr.po: Extracted from gcc/po/fr.po.
	* ja.po: Extracted from gcc/po/ja.po.
	* nl.po: Extracted from gcc/po/nl.po.
	* sv.po: Extracted from gcc/po/sv.po.
	* tr.po: Extracted from gcc/po/tr.po.

From-SVN: r82199
2004-05-24 10:50:45 +00:00