This second instalment uses the infrastructure of the previous patch
to allocate a macro map for each macro expansion and assign a virtual
location to each token resulting from the expansion.
To date when cpp_get_token comes across a token that happens to be a
macro, the macro expander kicks in, expands the macro, pushes the
resulting tokens onto a "token context" and returns a dummy padding
token. The next call to cpp_get_token goes look into the token context
for the next token [which is going to result from the previous macro
expansion] and returns it. If the token is a macro, the macro expander
kicks in and you know the story.
This patch piggy-backs on that macro expansion process, so to speak.
First it modifies the macro expander to make it create a macro map for
each macro expansion. It then allocates a virtual location for each
resulting token. Virtual locations of tokens resulting from macro
expansions are then stored on a special kind of context called an
"expanded tokens context". In other words, in an expanded tokens
context, there are tokens resulting from macro expansion and their
associated virtual locations. cpp_get_token_with_location is modified
to return the virtual location of tokens resulting from macro
expansion. Note that once all tokens from an expanded token context have
been consumed and the context and is freed, the memory used to store the
virtual locations of the tokens held in that context is freed as well.
This helps reducing the overall peak memory consumption.
The client code that was getting macro expansion point location from
cpp_get_token_with_location now gets virtual location from it. Those
virtual locations can in turn be resolved into the different
interesting physical locations thanks to the linemap API exposed by
the previous patch.
Expensive progress. Possibly. So this whole virtual location
allocation business is switched off by default. So by default no
extended token is created. No extended token context is created
either. One has to use -ftrack-macro-expansion to switch this on. This
complicates the code but I believe it can be useful as some of our
friends found out at http://llvm.org/bugs/show_bug.cgi?id=5610
The patch tries to reduce the memory consumption by freeing some token
context memory that was being reused before. I didn't notice any
compilation slow down due to this immediate freeing on my GNU/Linux
system.
As no client code tries to resolve virtual locations to anything but
what was being done before, no new test case has been added.
Co-Authored-By: Dodji Seketeli <dodji@redhat.com>
From-SVN: r180082
2008-07-21 Manuel Lopez-Ibanez <manu@gcc.gnu.org>
* include/line-map.h (linenum_type): New typedef.
(struct line_map): Use it.
(SOURCE_LINE): Second arguments is a LOCATION not a LINE.
(SOURCE_COLUMN): Likewise.
* macro.c (_cpp_builtin_macro_text): Use linenum_type. Don't store
source_location values in a variable of type linenum_type.
* directives.c (struct if_stack): Use linenum_type.
(strtoul_for_line): Rename as strtolinenum.
(do_line): Use linenum_type.
(do_linemarker): Use linenum_type and strtolinenum.
(_cpp_do_file_change): Use linenum_t.
* line-map.c (linemap_add): Likewise.
(linemap_line_start): Likewise.
* traditional.c (struct fun_macro): 'line' is a source_location.
* errors.c (print_location): Use linenum_type.
* directives-only.c (_cpp_preprocess_dir_only): Likewise.
* internal.h (CPP_INCREMENT_LINE): Likewise.
* lex.c (_cpp_skip_block_comment): Use source_location.
From-SVN: r138026
libcpp
2008-01-07 Fred Fish <fnf@specifix.com>
PR preprocessor/30363:
* traditional.c (replace_args_and_push): Add local variable
cxtquote, calculate the replacement text size assuming a
worst case of every input character quoted with backslash,
and properly handle output quoting of quote characters in
actual arguments used in function-like macros.
gcc/testsuite
2008-01-07 Fred Fish <fnf@specifix.com>
PR preprocessor/30363:
* gcc.dg/cpp/trad/macroargs.c: Add code to test quoting in
macro expansions.
From-SVN: r131379
* traditional.c (_cpp_scan_out_logical_line): Initialize
fmacro.args, fmacro.node, fmacro.offset, fmacro.line and
fmacro.args to prevent 'may be used uninitialized' warning.
From-SVN: r126311
* configure.ac: Check declarations for asprintf and vasprintf.
* config.in: Regenerate.
* configure: Likewise.
* charset.c (conversion_loop): Use XRESIZEVEC.
(convert_no_conversion): Likewise.
(convert_using_iconv): Likewise.
(init_iconv_desc): Cast return value of alloca.
(cpp_host_to_exec_charset): Use XNEWVEC.
(emit_numeric_escape): Use XRESIZEVEC.
(cpp_interpret_string): Use XNEWVEC.
(cpp_interpret_string): Use XRESIZEVEC.
(_cpp_interpret_identifier): Cast return value of alloca.
(_cpp_convert_input): Use XNEWVEC and XRESIZEVEC.
* directives.c (glue_header_name): Use XNEWVEC and XRESIZEVEC.
(parse_include): Use XNEWVEC.
(insert_pragma_entry): Rename local variable "new" to
"new_entry".
(save_registered_pragmas): Cast return value of xmemdup.
(destringize_and_run): Same for alloca.
(parse_assertion): Likewise.
(do_assert): Cast allocated storage to proper type.
(cpp_define): Likewise.
(_cpp_define_builtin): Likewise.
(cpp_undef): Likewise.
(handle_assertion): Likewise.
(cpp_push_buffer): Rename local variable "new" to "new_buffer".
* expr.c (CPP_UPLUS): Cast value to type cpp_ttype.
(CPP_UMINUS): Likewise.
(struct cpp_operator): Rename from struct operator.
(_cpp_expand_op_stack): Use XRESIZEVEC.
* files.c (pch_open_file): Use XNEWVEC.
(pch_open_file): Use XRESIZEVEC.
(read_file_guts): Use XNEWVEC and XRESIZEVEC.
(dir_name_of_file): Use XNEWVEC.
(make_cpp_file): Use XCNEW.
(make_cpp_dir): Likewise.
(allocate_file_hash_entries): USE XNEWVEC.
(cpp_included): Cast return value of htab_find_with_hash.
(append_file_to_dir): Use XNEWVEC.
(read_filename_string): Likewise. Use XRESIZEVEC too.
(read_name_map): Cast return value of alloca. Use XRESIZEVEC.
(remap_filename): Use XNEWVEC.
(struct pchf_entry): Move definition out of struct pchf_data.
(_cpp_save_file_entries): Use XCNEWVAR.
(_cpp_read_file_entries): Use XNEWVAR.
* identifiers.c (alloc_node): Use XOBNEW.
* init.c (cpp_create_reader): Use XCNEW.
(cpp_init_builtins): Cast of b->value to enum builtin_type.
(read_original_directory): Cast return value of alloca.
* lex.c (add_line_note): Use XRESIZEVEC.
(warn_about_normalization): Use XNEWVEC.
(_cpp_lex_direct): Cast node->directive_index to (enum cpp_ttype).
(new_buff): Use XNEWVEC.
* line-map.c (linemap_add): Use XRESIZEVEC.
* macro.c (builtin_macro): Cast return value of alloca.
(paste_tokens): Likewise.
(expand_arg): Use XNEWVEC and XRESIZEVEC.
(_cpp_save_parameter): Use XRESIZEVEC.
(create_iso_definition): Cast allocated storage to proper type.
(_cpp_create_definition): Likewise.
(cpp_macro_definition): Use XRESIZEVEC.
* makedepend.c (add_clm): Use XNEW.
(add_dir): Likewise.
* mkdeps.c (munge): Use XNEWVEC.
(deps_init): Use XCNEW.
(deps_add_target): Use XRESIZEVEC.
(deps_add_default_target): Cast return value of alloca.
(deps_add_dep): Use XRESIZEVEC.
(deps_add_vpath): Likewise. Use XNEWVEC too.
(deps_restore): Likewise.
* pch.c (save_idents): Use XNEW and XNEWVEC.
(cpp_save_state): Use XNEW.
(count_defs): Cast return value of htab_find.
(write_defs): Likewise.
(cpp_write_pch_deps): Use XNEWVEC.
(collect_ht_nodes): Use XRESIZEVEC.
(cpp_valid_state): Use XNEWVEC.
(save_macros): Use XRESIZEVEC. Cast return value of xmemdup.
* symtab.c (ht_create): Use XCNEW.
(ht_lookup_with_hash): Cast return value of obstack_copy0.
(ht_expand): Use XCNEWVEC.
* system.h (HAVE_DESIGNATED_INITIALIZERS): False if __cplusplus.
(bool): Do not define if __cplusplus.
From-SVN: r100295
ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* Makefile.def (host_modules): add libcpp.
* Makefile.tpl: Add dependencies on and for libcpp.
* Makefile.in: Regenerate.
* configure.in: Add libcpp host module.
* configure: Regenerate.
config/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* acx.m4 (ACX_HEADER_STDBOOL, ACX_HEADER_STRING):
From gcc.
gcc/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
Move libcpp to the toplevel.
* Makefile.in: Remove references to libcpp files,
use CPPLIBS instead of libcpp.a. Define SYMTAB_H
and change hashtable.h to that.
* aclocal.m4 (gcc_AC_HEADER_STDBOOL,
gcc_AC_HEADER_STRING, gcc_AC_C__BOOL): Remove.
* configure.ac (gcc_AC_C__BOOL, HAVE_UCHAR): Remove tests.
* configure: Regenerate.
* config.in: Regenerate.
* c-ppoutput.c: Include ../libcpp/internal.h instead of cpphash.h.
* cppcharset.c: Removed.
* cpperror.c: Removed.
* cppexp.c: Removed.
* cppfiles.c: Removed.
* cpphash.c: Removed.
* cpphash.h: Removed.
* cppinit.c: Removed.
* cpplex.c: Removed.
* cpplib.c: Removed.
* cpplib.h: Removed.
* cppmacro.c: Removed.
* cpppch.c: Removed.
* cpptrad.c: Removed.
* cppucnid.h: Removed.
* cppucnid.pl: Removed.
* cppucnid.tab: Removed.
* hashtable.c: Removed.
* hashtable.h: Removed.
* line-map.c: Removed.
* line-map.h: Removed.
* mkdeps.c: Removed.
* mkdeps.h: Removed.
* stringpool.h: Include symtab.h instead of hashtable.h.
* tree.h: Include symtab.h instead of hashtable.h.
* system.h (O_NONBLOCK, O_NOCTTY): Do not define.
gcc/cp/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* Make-lang.in: No need to specify $(LIBCPP).
gcc/java/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* Make-lang.in: Link in $(LIBCPP) instead of mkdeps.o.
libcpp/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
Moved libcpp from the gcc subdirectory to the toplevel.
* Makefile.am: New file.
* Makefile.in: Regenerate.
* configure.ac: New file.
* configure: Regenerate.
* config.in: Regenerate.
* charset.c: Moved from gcc/cppcharset.c. Add note about
brokenness of input charset detection. Adjust for change
in name of cppucnid.h.
* errors.c: Moved from gcc/cpperror.c. Do not include intl.h.
* expr.c: Moved from gcc/cppexp.c.
* files.c: Moved from gcc/cppfiles.c. Do not include intl.h.
Remove #define of O_BINARY, it is in system.h.
* identifiers.c: Moved from gcc/cpphash.c.
* internal.h: Moved from gcc/cpphash.h. Change header
guard name. All other files adjusted to match name change.
* init.c: Moved from gcc/cppinit.c.
(init_library) [ENABLE_NLS]: Call bindtextdomain.
* lex.c: Moved from gcc/cpplex.c.
* directives.c: Moved from gcc/cpplib.c.
* macro.c: Moved from gcc/cppmacro.c.
* pch.c: Moved from gcc/cpppch.c. Do not include intl.h.
* traditional.c: Moved from gcc/cpptrad.c.
* ucnid.h: Moved from gcc/cppucnid.h. Change header
guard name.
* ucnid.pl: Moved from gcc/cppucnid.pl.
* ucnid.tab: Moved from gcc/cppucnid.tab. Change header
guard name.
* symtab.c: Moved from gcc/hashtable.c.
* line-map.c: Moved from gcc. Do not include intl.h.
* mkdeps.c: Moved from gcc.
* system.h: New file.
libcpp/include/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* cpplib.h: Moved from gcc. Change header guard name.
* line-map.h: Moved from gcc. Change header guard name.
* mkdeps.h: Moved from gcc. Change header guard name.
* symtab.h: Moved from gcc/hashtable.h. Change header
guard name.
libcpp/po/ChangeLog:
2004-05-23 Paolo Bonzini <bonzini@gnu.org>
* be.po: Extracted from gcc/po/be.po.
* ca.po: Extracted from gcc/po/ca.po.
* da.po: Extracted from gcc/po/da.po.
* de.po: Extracted from gcc/po/de.po.
* el.po: Extracted from gcc/po/el.po.
* es.po: Extracted from gcc/po/es.po.
* fr.po: Extracted from gcc/po/fr.po.
* ja.po: Extracted from gcc/po/ja.po.
* nl.po: Extracted from gcc/po/nl.po.
* sv.po: Extracted from gcc/po/sv.po.
* tr.po: Extracted from gcc/po/tr.po.
From-SVN: r82199
Tom Tromey
Jakub Jelinek
Manuel López-Ibáñez
Fred Fish
Uros Bizjak
Kelley Cook
Gabriel Dos Reis
Kazu Hirata
Geoffrey Keating
Paolo Bonzini