libcpp/
* include/internal.h (_cpp_remaining_tokens_num_in_context): Take the
context to act upon.
* lex.c (_cpp_remaining_tokens_num_in_context): Likewise. Update
comment.
(cpp_token_from_context_at): Likewise.
(cpp_peek_token): Use the context to peek tokens from.
From-SVN: r180328
This second instalment uses the infrastructure of the previous patch
to allocate a macro map for each macro expansion and assign a virtual
location to each token resulting from the expansion.
To date when cpp_get_token comes across a token that happens to be a
macro, the macro expander kicks in, expands the macro, pushes the
resulting tokens onto a "token context" and returns a dummy padding
token. The next call to cpp_get_token goes look into the token context
for the next token [which is going to result from the previous macro
expansion] and returns it. If the token is a macro, the macro expander
kicks in and you know the story.
This patch piggy-backs on that macro expansion process, so to speak.
First it modifies the macro expander to make it create a macro map for
each macro expansion. It then allocates a virtual location for each
resulting token. Virtual locations of tokens resulting from macro
expansions are then stored on a special kind of context called an
"expanded tokens context". In other words, in an expanded tokens
context, there are tokens resulting from macro expansion and their
associated virtual locations. cpp_get_token_with_location is modified
to return the virtual location of tokens resulting from macro
expansion. Note that once all tokens from an expanded token context have
been consumed and the context and is freed, the memory used to store the
virtual locations of the tokens held in that context is freed as well.
This helps reducing the overall peak memory consumption.
The client code that was getting macro expansion point location from
cpp_get_token_with_location now gets virtual location from it. Those
virtual locations can in turn be resolved into the different
interesting physical locations thanks to the linemap API exposed by
the previous patch.
Expensive progress. Possibly. So this whole virtual location
allocation business is switched off by default. So by default no
extended token is created. No extended token context is created
either. One has to use -ftrack-macro-expansion to switch this on. This
complicates the code but I believe it can be useful as some of our
friends found out at http://llvm.org/bugs/show_bug.cgi?id=5610
The patch tries to reduce the memory consumption by freeing some token
context memory that was being reused before. I didn't notice any
compilation slow down due to this immediate freeing on my GNU/Linux
system.
As no client code tries to resolve virtual locations to anything but
what was being done before, no new test case has been added.
Co-Authored-By: Dodji Seketeli <dodji@redhat.com>
From-SVN: r180082
This is the first instalment of a set which goal is to track locations
of tokens across macro expansions. Tom Tromey did the original work
and attached the patch to PR preprocessor/7263. This opus is a
derivative of that original work.
This patch modifies the linemap module of libcpp to add virtual
locations support.
A virtual location is a mapped location that can resolve to several
different physical locations. It can always resolve to the spelling
location of a token. For tokens resulting from macro expansion it can
resolve to:
- either the location of the expansion point of the macro.
- or the location of the token in the definition of the
macro
- or, if the token is an argument of a function-like macro,
the location of the use of the matching macro parameter in
the definition of the macro
The patch creates a new type of line map called a macro map. For every
single macro expansion, there is a macro map that generates a virtual
location for every single resulting token of the expansion.
The good old type of line map we all know is now called an ordinary
map. That one still encodes spelling locations as it has always had.
As a result linemap_lookup as been extended to return a macro map when
given a virtual location resulting from a macro expansion. The layout
of structs line_map has changed to support this new type of map. So
did the layout of struct line_maps. Accessor macros have been
introduced to avoid messing with the implementation details of these
datastructures directly. This helped already as we have been testing
different ways of arranging these datastructure. Having to constantly
adjust client code that is too tied with the internals of line_map and
line_maps would have been even more painful.
Of course, many new public functions have been added to the linemap
module to handle the resolution of virtual locations.
This patch introduces the infrastructure but no part of the compiler
uses virtual locations yet.
However the client code of the linemap data structures has been
adjusted as per the changes. E.g, it's not anymore reliable for a
client code to manipulate struct line_map directly if it just wants to
deal with spelling locations, because struct line_map can now
represent a macro map as well. In that case, it's better to use the
convenient API to resolve the initial (possibly virtual) location to a
spelling location (or to an ordinary map) and use that.
This is the reason why the patch adjusts the Java, Ada and Fortran
front ends.
Also, note that virtual locations are not supposed to be ordered for
relations '<' and '>' anymore. To test if a virtual location appears
"before" another one, one has to use a new operator exposed by the
line map interface. The patch updates the only spot (in the
diagnostics module) I have found that was making the assumption that
locations were ordered for these relations. This is the only change
that introduces a use of the new line map API in this patch, so I am
adding a regression test for it only.
From-SVN: r180081
Use it to force BUILTINS_LOCATION when declaring builtins instead of creating a <built-in> entry in the line_table which is wrong.
* c-opts.c (c_finish_options): Force BUILTINS_LOCATION for tokens
defined in cpp_init_builtins and c_cpp_builtins.
gcc/fortran/ChangeLog
* cpp.c (gfc_cpp_init): Force BUILTINS_LOCATION for tokens
defined in cpp_define_builtins.
libcpp/ChangeLog
* init.c (cpp_create_reader): Inititalize forced_token_location_p.
* internal.h (struct cpp_reader): Add field forced_token_location_p.
* lex.c (_cpp_lex_direct): Use forced_token_location_p.
(cpp_force_token_locations): New.
(cpp_stop_forcing_token_locations): New.
From-SVN: r177973
2010-09-29 Kai Tietz <kai.tietz@onevision.com>
PR preprocessor/45362
* directives.c (cpp_pop_definition): Make static.
(do_pragma_push_macro): Reworked to store text
definition.
(do_pragma_pop_macro): Add free text definition.
(cpp_push_definition): Removed.
* include/cpplib.h (cpp_push_definition): Removed.
(cpp_pop_definition): Likewise.
* internal.h (def_pragma_macro): Remove member 'value'
and add new members 'definition', 'line',
'syshdr', 'sued' and 'is_undef'.
* pch.c (_cpp_restore_pushed_macros): Rework to work
on text definition and store additional macro flags.
(_cpp_save_pushed_macros): Likewise.
From-SVN: r164729
More N3077 raw string changes
* charset.c (cpp_interpret_string): Don't transform UCNs in raw
strings.
* lex.c (bufring_append): Split out from...
(lex_raw_string): ...here. Undo trigraph and line splicing
transformations. Do process line notes in multi-line literals.
(_cpp_process_line_notes): Ignore notes that were already handled.
From-SVN: r157804
2008-07-21 Manuel Lopez-Ibanez <manu@gcc.gnu.org>
* include/line-map.h (linenum_type): New typedef.
(struct line_map): Use it.
(SOURCE_LINE): Second arguments is a LOCATION not a LINE.
(SOURCE_COLUMN): Likewise.
* macro.c (_cpp_builtin_macro_text): Use linenum_type. Don't store
source_location values in a variable of type linenum_type.
* directives.c (struct if_stack): Use linenum_type.
(strtoul_for_line): Rename as strtolinenum.
(do_line): Use linenum_type.
(do_linemarker): Use linenum_type and strtolinenum.
(_cpp_do_file_change): Use linenum_t.
* line-map.c (linemap_add): Likewise.
(linemap_line_start): Likewise.
* traditional.c (struct fun_macro): 'line' is a source_location.
* errors.c (print_location): Use linenum_type.
* directives-only.c (_cpp_preprocess_dir_only): Likewise.
* internal.h (CPP_INCREMENT_LINE): Likewise.
* lex.c (_cpp_skip_block_comment): Use source_location.
From-SVN: r138026
libcpp/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* include/cpp-id-data.h (UC): Was U, conflicts with U... literal.
* include/cpplib.h (CHAR16, CHAR32, STRING16, STRING32): New tokens.
(struct cpp_options): Added uliterals.
(cpp_interpret_string): Update prototype.
(cpp_interpret_string_notranslate): Idem.
* charset.c (init_iconv_desc): New width member in cset_converter.
(cpp_init_iconv): Add support for char{16,32}_cset_desc.
(convert_ucn): Idem.
(emit_numeric_escape): Idem.
(convert_hex): Idem.
(convert_oct): Idem.
(convert_escape): Idem.
(converter_for_type): New function.
(cpp_interpret_string): Use converter_for_type, support u and U prefix.
(cpp_interpret_string_notranslate): Match changed prototype.
(wide_str_to_charconst): Use converter_for_type.
(cpp_interpret_charconst): Add support for CPP_CHAR{16,32}.
* directives.c (linemarker_dir): Macro U changed to UC.
(parse_include): Idem.
(register_pragma_1): Idem.
(restore_registered_pragmas): Idem.
(get__Pragma_string): Support CPP_STRING{16,32}.
* expr.c (eval_token): Support CPP_CHAR{16,32}.
* init.c (struct lang_flags): Added uliterals.
(lang_defaults): Idem.
* internal.h (struct cset_converter) <width>: New field.
(struct cpp_reader) <char16_cset_desc>: Idem.
(struct cpp_reader) <char32_cset_desc>: Idem.
* lex.c (digraph_spellings): Macro U changed to UC.
(OP, TK): Idem.
(lex_string): Add support for u'...', U'...', u... and U....
(_cpp_lex_direct): Idem.
* macro.c (_cpp_builtin_macro_text): Macro U changed to UC.
(stringify_arg): Support CPP_CHAR{16,32} and CPP_STRING{16,32}.
gcc/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* c-common.c (CHAR16_TYPE, CHAR32_TYPE): New macros.
(fname_as_string): Match updated cpp_interpret_string prototype.
(fix_string_type): Support char16_t* and char32_t*.
(c_common_nodes_and_builtins): Add char16_t and char32_t (and
derivative) nodes. Register as builtin if C++0x.
(c_parse_error): Support CPP_CHAR{16,32}.
* c-common.h (RID_CHAR16, RID_CHAR32): New elements.
(enum c_tree_index) <CTI_CHAR16_TYPE, CTI_SIGNED_CHAR16_TYPE,
CTI_UNSIGNED_CHAR16_TYPE, CTI_CHAR32_TYPE, CTI_SIGNED_CHAR32_TYPE,
CTI_UNSIGNED_CHAR32_TYPE, CTI_CHAR16_ARRAY_TYPE,
CTI_CHAR32_ARRAY_TYPE>: New elements.
(char16_type_node, signed_char16_type_node, unsigned_char16_type_node,
char32_type_node, signed_char32_type_node, char16_array_type_node,
char32_array_type_node): New defines.
* c-lex.c (cb_ident): Match updated cpp_interpret_string prototype.
(c_lex_with_flags): Support CPP_CHAR{16,32} and CPP_STRING{16,32}.
(lex_string): Support CPP_STRING{16,32}, match updated
cpp_interpret_string and cpp_interpret_string_notranslate prototypes.
(lex_charconst): Support CPP_CHAR{16,32}.
* c-parser.c (c_parser_postfix_expression): Support CPP_CHAR{16,32}
and CPP_STRING{16,32}.
gcc/cp/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* cvt.c (type_promotes_to): Support char16_t and char32_t.
* decl.c (grokdeclarator): Disallow signed/unsigned/short/long on
char16_t and char32_t.
* lex.c (reswords): Add char16_t and char32_t (for c++0x).
* mangle.c (write_builtin_type): Mangle char16_t/char32_t as vendor
extended builtin type u8char32_t.
* parser.c (cp_lexer_next_token_is_decl_specifier_keyword): Support
RID_CHAR{16,32}.
(cp_lexer_print_token): Support CPP_STRING{16,32}.
(cp_parser_is_string_literal): Idem.
(cp_parser_string_literal): Idem.
(cp_parser_primary_expression): Support CPP_CHAR{16,32} and
CPP_STRING{16,32}.
(cp_parser_simple_type_specifier): Support RID_CHAR{16,32}.
* tree.c (char_type_p): Support char16_t and char32_t as char types.
* typeck.c (string_conv_p): Support char16_t and char32_t.
gcc/testsuite/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
Tests for char16_t and char32_t support.
* g++.dg/ext/utf-cvt.C: New
* g++.dg/ext/utf-cxx0x.C: New
* g++.dg/ext/utf-cxx98.C: New
* g++.dg/ext/utf-dflt.C: New
* g++.dg/ext/utf-gnuxx0x.C: New
* g++.dg/ext/utf-gnuxx98.C: New
* g++.dg/ext/utf-mangle.C: New
* g++.dg/ext/utf-typedef-cxx0x.C: New
* g++.dg/ext/utf-typedef-
* g++.dg/ext/utf-typespec.C: New
* g++.dg/ext/utf16-1.C: New
* g++.dg/ext/utf16-2.C: New
* g++.dg/ext/utf16-3.C: New
* g++.dg/ext/utf16-4.C: New
* g++.dg/ext/utf32-1.C: New
* g++.dg/ext/utf32-2.C: New
* g++.dg/ext/utf32-3.C: New
* g++.dg/ext/utf32-4.C: New
* gcc.dg/utf-cvt.c: New
* gcc.dg/utf-dflt.c: New
* gcc.dg/utf16-1.c: New
* gcc.dg/utf16-2.c: New
* gcc.dg/utf16-3.c: New
* gcc.dg/utf16-4.c: New
* gcc.dg/utf32-1.c: New
* gcc.dg/utf32-2.c: New
* gcc.dg/utf32-3.c: New
* gcc.dg/utf32-4.c: New
libiberty/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* testsuite/demangle-expected: Added tests for char16_t and char32_t.
From-SVN: r134438
* internal.h (struct cpp_reader): Add new fields:
nonexistent_file_hash and nonexistent_file_ob.
* files.c: Include "obstack.h".
(find_file_in_dir): Before trying to open the file, look up the
path name in the hash table of nonexistent files. After failing
to open the file, add the path name to the hash table.
(_cpp_find_file): Cache the results of looking up the file name
starting with the quote and bracket chain heads, if we can.
(nonexistent_file_hash_eq): New static function.
(_cpp_init_files): Initialize pfile->nonexistent_file_hash and
pfile->nonexistent_file_ob.
(_cpp_cleanup_files): Free pfile->nonexistent_file_hash and
pfile->nonexistent_file_ob.
From-SVN: r124929
PR preprocessor/15220
* files.c (_cpp_find_file): New parameter angle_brackets. Fix all
callers. Pass to open_file_failed.
(open_file_failed): New parameter angle_brackets. Fix all callers.
Use in print_dep assignment.
* init.c (cpp_read_main_file): Pass additional arg to _cpp_find_file.
* internal.h (_cpp_find_file): Add new parm to declaration.
From-SVN: r105757
* internal.h: Replace all uses of uchar with unsigned char.
* include/cpp-id-data.h: Likewise. Guard typedef of uchar
with !IN_GCC, so uchar is only defined whilst building libcpp.
Co-Authored-By: Zack Weinberg <zack@codesourcery.com>
From-SVN: r91394
2004-09-09 Matt Austern <austern@apple.com>
Zack Weinberg <zack@codesourcery.com>
* include/cpplib.h (TTYPE_TABLE): Remove CPP_ and SPELL_
prefixes throughout. Add entry for PRAGMA. Remove
unnecessary "= 0" from EQ.
(enum cpp_ttype): Adjust OP and TK definitions to restore
prefixes, via token-paste.
(CPP_LAST_EQ, CPP_FIRST_DIGRAPH, CPP_LAST_PUNCTUATOR, CPP_LAST_CPP_OP):
Change from #defines to additional cpp_ttype enumerators.
(struct cpp_options): Add defer_pragmas.
(cpp_handle_deferred_pragma): Prototype new interface.
* internal.h (struct cpp_reader): Add directive_result.
* directives.c (struct pragma_entry): Add is_internal field;
give boolean fields type bool.
(start_directive): Initialize pfile->directive_result.type.
(_cpp_do__Pragma): Likewise.
(run_directive): Do not crash if pfile->buffer->prev is NULL.
(insert_pragma_entry): Add 'internal' argument; set new->is_internal
from it.
(register_pragma): New static function, bulk of former
cpp_register_pragma here; add 'internal' argument, pass along
to insert_pragma_entry.
(cpp_register_pragma): Now a wrapper around register_pragma which
always passes false for 'internal' argument.
(_cpp_init_internal_pragmas): Call register_pragma directly, passing
true for 'internal'.
(do_pragma): If CPP_OPTION (pfile, defer_pragmas) and this isn't
an internal pragma, save text till the end of the line as a CPP_PRAGMA
token instead of executing the pragma.
(cpp_handle_deferred_pragma): New interface.
* lex.c (token_spellings): Adjust OP and TK definitions to
match changes to cpplib.h.
(_cpp_lex_token): Check for a directive-result token and
return it if present.
(cpp_token_val_index): Handle CPP_PRAGMA.
* macro.c (cpp_builtin_macro_text): Correct comment.
(builtin_macro): Handle directive-result tokens from _cpp_do__Pragma.
From-SVN: r87247
2004-07-16 Andris Pavenis <pavenis@latnet.lv>
PR preprocessor/16366
* internal.h (struct cpp_reader): New field dir_hash.
* files.c (make_cpp_dir): Use dir_hash, not file_hash.
(_cpp_init_files, _cpp_cleanup_files): Update for new field.
From-SVN: r84821
libcpp:
* Makefile.am: Add makedepend.
* Makefile.in, aclocal.m4: Regenerate.
* charset.c: Insert a space to avoid a warning.
* directives.c: Include mkdeps.h.
(_cpp_handle_directive): Reenable macro expander if appropriate.
(undefine_macros): Inline body of _cpp_free_definition for speed.
Do not call undef callback or _cpp_warn_if_unused_macro.
(cpp_get_deps): New interface.
* files.c (search_cache): Add pfile argument. Check for file
that would be found by "" or <> search here...
(_cpp_find_file): ...not here. Correct recorded start_dir of
files found by directory-of-current-file search that would be
found by "" or <> search.
* init.c (cpp_add_dependency_target): Delete.
* internal.h (struct lexer_state): Add discarding_output flag.
* lex.c (lex_identifier): Compute hash function while scanning.
* macro.c (cpp_scan_nooutput): Disable macro expansion outside
directives.
* makedepend.c: New file.
* mkdeps.c (struct deps): Add vpath vector.
(apply_vpath, deps_add_vpath): New function.
(deps_free): Free vpath vector.
(deps_add_dep, deps_add_target): Use apply_vpath.
* symtab.c (calc_hash): Use HT_HASHSTEP and HT_FINISH.
(ht_lookup_with_hash): New function.
* cpplib.h, mkdeps.h: Update prototypes.
* symtab.h: Update prototypes.
(HT_HASHSTEP, HT_FINISH): New macros.
gcc:
* Makefile.in (MKDEPS_H): New shorthand.
(c-opts.o): Update dependencies.
* c-opts.c: Include mkdeps.h.
(handle_deferred_opts): Use cpp_get_deps and deps_add_target,
not cpp_add_dependency_target.
From-SVN: r82654
ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* Makefile.def (host_modules): add libcpp.
* Makefile.tpl: Add dependencies on and for libcpp.
* Makefile.in: Regenerate.
* configure.in: Add libcpp host module.
* configure: Regenerate.
config/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* acx.m4 (ACX_HEADER_STDBOOL, ACX_HEADER_STRING):
From gcc.
gcc/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
Move libcpp to the toplevel.
* Makefile.in: Remove references to libcpp files,
use CPPLIBS instead of libcpp.a. Define SYMTAB_H
and change hashtable.h to that.
* aclocal.m4 (gcc_AC_HEADER_STDBOOL,
gcc_AC_HEADER_STRING, gcc_AC_C__BOOL): Remove.
* configure.ac (gcc_AC_C__BOOL, HAVE_UCHAR): Remove tests.
* configure: Regenerate.
* config.in: Regenerate.
* c-ppoutput.c: Include ../libcpp/internal.h instead of cpphash.h.
* cppcharset.c: Removed.
* cpperror.c: Removed.
* cppexp.c: Removed.
* cppfiles.c: Removed.
* cpphash.c: Removed.
* cpphash.h: Removed.
* cppinit.c: Removed.
* cpplex.c: Removed.
* cpplib.c: Removed.
* cpplib.h: Removed.
* cppmacro.c: Removed.
* cpppch.c: Removed.
* cpptrad.c: Removed.
* cppucnid.h: Removed.
* cppucnid.pl: Removed.
* cppucnid.tab: Removed.
* hashtable.c: Removed.
* hashtable.h: Removed.
* line-map.c: Removed.
* line-map.h: Removed.
* mkdeps.c: Removed.
* mkdeps.h: Removed.
* stringpool.h: Include symtab.h instead of hashtable.h.
* tree.h: Include symtab.h instead of hashtable.h.
* system.h (O_NONBLOCK, O_NOCTTY): Do not define.
gcc/cp/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* Make-lang.in: No need to specify $(LIBCPP).
gcc/java/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* Make-lang.in: Link in $(LIBCPP) instead of mkdeps.o.
libcpp/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
Moved libcpp from the gcc subdirectory to the toplevel.
* Makefile.am: New file.
* Makefile.in: Regenerate.
* configure.ac: New file.
* configure: Regenerate.
* config.in: Regenerate.
* charset.c: Moved from gcc/cppcharset.c. Add note about
brokenness of input charset detection. Adjust for change
in name of cppucnid.h.
* errors.c: Moved from gcc/cpperror.c. Do not include intl.h.
* expr.c: Moved from gcc/cppexp.c.
* files.c: Moved from gcc/cppfiles.c. Do not include intl.h.
Remove #define of O_BINARY, it is in system.h.
* identifiers.c: Moved from gcc/cpphash.c.
* internal.h: Moved from gcc/cpphash.h. Change header
guard name. All other files adjusted to match name change.
* init.c: Moved from gcc/cppinit.c.
(init_library) [ENABLE_NLS]: Call bindtextdomain.
* lex.c: Moved from gcc/cpplex.c.
* directives.c: Moved from gcc/cpplib.c.
* macro.c: Moved from gcc/cppmacro.c.
* pch.c: Moved from gcc/cpppch.c. Do not include intl.h.
* traditional.c: Moved from gcc/cpptrad.c.
* ucnid.h: Moved from gcc/cppucnid.h. Change header
guard name.
* ucnid.pl: Moved from gcc/cppucnid.pl.
* ucnid.tab: Moved from gcc/cppucnid.tab. Change header
guard name.
* symtab.c: Moved from gcc/hashtable.c.
* line-map.c: Moved from gcc. Do not include intl.h.
* mkdeps.c: Moved from gcc.
* system.h: New file.
libcpp/include/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* cpplib.h: Moved from gcc. Change header guard name.
* line-map.h: Moved from gcc. Change header guard name.
* mkdeps.h: Moved from gcc. Change header guard name.
* symtab.h: Moved from gcc/hashtable.h. Change header
guard name.
libcpp/po/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* be.po: Extracted from gcc/po/be.po.
* ca.po: Extracted from gcc/po/ca.po.
* da.po: Extracted from gcc/po/da.po.
* de.po: Extracted from gcc/po/de.po.
* el.po: Extracted from gcc/po/el.po.
* es.po: Extracted from gcc/po/es.po.
* fr.po: Extracted from gcc/po/fr.po.
* ja.po: Extracted from gcc/po/ja.po.
* nl.po: Extracted from gcc/po/nl.po.
* sv.po: Extracted from gcc/po/sv.po.
* tr.po: Extracted from gcc/po/tr.po.
From-SVN: r82199
