This is another bug in computing the fastmap. It was reported by a user
of sed because it usually does not happen with !_LIBC. However, it is
there in that case too.
The bug is that whenever we have a range at the beginning of the regex,
the regex must be tested on any possible multibyte character. The reason
why _LIBC masks it, is that in general there is a collation symbol for
each possible multibyte-character lead byte, so all the lead bytes are
in general already part of the fastmap.
The tests use cyrillic characters as an example. With _LIBC, they pass
without the patch too, but you can make them fail by removing collation
symbols handling.
2008-03-29 Eric Blake <ebb9@byu.net>
Rewrite string searches to O(n) rather than O(n^2).
* string/str-two-way.h: New file. For linear fixed-allocation
string searching.
* string/memmem.c: New implementation.
* string/strstr.c: New implementation.
* string/strcasestr.c: New implementation.
* sysdeps/posix/getaddrinfo.c (getaddrinfo): Call _res_hconf_init
2007-02-05 Jakub Jelinek <jakub@redhat.com>
[BZ #3957]
* posix/regcomp.c (parse_bracket_exp): Set '\n' bit rather than '\0'
bit for RE_HAT_LISTS_NOT_NEWLINE.
(build_charclass_op): Remove bogus comment.
* posix/Makefile (tests): Add bug-regex27 and bug-regex28.
* posix/bug-regex27.c: New test.
* posix/bug-regex28.c: New test.
2005-08-20 Paul Eggert <eggert@cs.ucla.edu>
[BZ #1224]
* posix/regcomp.c (init_word_char, duplicate_node_closure,
fetch_token, peek_token_bracket, build_range_exp,
build_collating_symbol): Remove forward decls; no longer needed
now that we use prototypes.
2005-09-06 Paul Eggert <eggert@cs.ucla.edu>
Ulrich Drepper <drepper@redhat.com>
[BZ #1302]
Change bitset word type from unsigned int to unsigned long int,
as this has better performance on typical 64-bit hosts. Change
bitset type name to bitset_t.
* posix/regcomp.c (build_equiv_class, build_charclass):
(build_range_exp, build_collating_symbol):
Prefer bitset_t to re_bitset_ptr_t in prototypes, when the actual
argument is a bitset. This is merely a style issue, but it makes
it clearer that an entire array is expected.
(re_compile_fastmap_iter, init_dfa, init_word_char, optimize_subexps,
lower_subexp): Adjust for new bitset_t definition.
(lower_subexp, parse_bracket_exp, built_charclass_op): Likewise.
* posix/regex_internal.h (bitset_set, bitset_clear, bitset_contain,
bitset_not, bitset_merge, bitset_set_all, bitset_mask): Likewise.
* posix/regexec.c (check_dst_limits_calc_pos_1,
check_subexp_matching_top, build_trtable, group_nodes_into_DFAstates):
Likewise.
* posix/regcomp.c (utf8_sb_map): Don't assume initializer
== 0xffffffff.
* posix/regex_internal.h (BITSET_WORD_BITS): Renamed from UINT_BITS.
All uses changed.
(BITSET_WORDS): Renamed from BITSET_UINTS. All uses changed.
(bitset_word_t): New type, replacing 'unsigned int' for bitset uses.
All uses changed.
(BITSET_WORD_MAX): New macro.
(bitset_set, bitset_clear, bitset_contain, bitset_empty,
(bitset_set_all, bitset_copy): Adjust for bitset_t change.
(bitset_empty, bitset_copy):
Prefer sizeof (bitset_t) to multiplying it out ourselves.
(bitset_not_merge): Remove; unused.
(bitset_contain): Return bool, not unsigned int with one bit on.
All callers changed.
* posix/regexec.c (build_trtable): Don't assume bitset_t has no
stricter alignment than re_node_set; do this by defining a new
internal type struct dests_alloc and using it to allocate memory.
mbrtowc for very simple UTF-8 case.
2005-09-01 Paul Eggert <eggert@cs.ucla.edu>
* posix/regex_internal.c (build_wcs_upper_buffer): Fix portability
bugs in int versus size_t comparisons.
2005-09-06 Ulrich Drepper <drepper@redhat.com>
* posix/regex_internal.c (re_acquire_state): Make DFA pointer arg
a pointer-to-const.
(re_acquire_state_context): Likewise.
* posix/regex_internal.h: Adjust prototypes.
2005-08-31 Jim Meyering <jim@meyering.net>
* posix/regcomp.c (search_duplicated_node): Make first pointer arg
a pointer-to-const.
* posix/regex_internal.c (create_ci_newstate, create_cd_newstate,
register_state): Likewise.
* posix/regexec.c (search_cur_bkref_entry, check_dst_limits):
(check_dst_limits_calc_pos_1, check_dst_limits_calc_pos):
(group_nodes_into_DFAstates): Likewise.
* posix/regexec.c (re_search_internal): Simplify update of
rm_so and rm_eo by replacing "if (A == B) A += C - B;"
with the equivalent of "if (A == B) A = C;".
2005-09-06 Ulrich Drepper <drepper@redhat.com>
* posix/regcomp.c (re_compile_internal): Change third parameter type
to size_t.
(init_dfa): Likewise. Make sure that arithmetic on pat_len doesn't
overflow.
* posix/regex_internal.h (struct re_dfa_t): Change type of nodes_alloc
and nodes_len to size_t.
* posix/regex_internal.c (re_dfa_add_node): Use size_t as type for
new_nodes_alloc. Check for overflow.
2005-08-31 Paul Eggert <eggert@cs.ucla.edu>
* posix/regcomp.c (re_compile_fastmap_iter, init_dfa, init_word_char):
(optimize_subexps, lower_subexp):
Don't assume 1<<31 has defined behavior on hosts with 32-bit int,
since the signed shift might overflow. Use 1u<<31 instead.
* posix/regex_internal.h (bitset_set, bitset_clear, bitset_contain):
Likewise.
* posix/regexec.c (check_dst_limits_calc_pos_1): Likewise.
(check_subexp_matching_top): Likewise.
* posix/regcomp.c (optimize_subexps, lower_subexp):
Use CHAR_BIT rather than 8, for clarity.
* posix/regexec.c (check_dst_limits_calc_pos_1):
(check_subexp_matching_top): Likewise.
* posix/regcomp.c (init_dfa): Make table_size unsigned, so that we
don't have to worry about portability issues when shifting it left.
Remove no-longer-needed test for table_size > 0.
* posix/regcomp.c (parse_sub_exp): Do not shift more bits than there
are in a word, as the resulting behavior is undefined.
* posix/regexec.c (check_dst_limits_calc_pos_1): Likewise;
in one case, a <= should have been an <, and in another case the
whole test was missing.
* posix/regex_internal.h (BYTE_BITS): Remove. All uses changed to
the standard name CHAR_BIT.
next_last_offset.
(struct re_dfa_t): Remove unused member states_alloc.
* posix/regcomp.c (init_dfa): Don't initialize unused members.
2005-08-25 Paul Eggert <eggert@cs.ucla.edu>
* posix/regexec.c (set_regs): Don't alloca with an unbounded size.
alloca modernization/simplification for regex.
* posix/regex.c: Remove portability cruft for alloca. This no longer
needs to be at the start of the file, and can be moved into
regex_internal.h and simplified.
* posix/regex_internal.h: Include <alloca.h>.
(__libc_use_alloca) [!defined _LIBC]: New macro.
* posix/regexec.c (build_trtable): Remove "#ifdef _LIBC",
since the code now works outside glibc.
2005-09-06 Ulrich Drepper <drepper@redhat.com>
* include/regex.h: Remove use of _RE_ARGS.
2005-08-25 Paul Eggert <eggert@cs.ucla.edu>
* posix/regexec.c (find_recover_state): Change "err" to "*err".
2005-08-24 Paul Eggert <eggert@cs.ucla.edu>
* posix/regcomp.c (regerror): Pointer args are 'restrict',
as per POSIX.
* posix/regex.h (regerror): Likewise.
* manual/pattern.texi (POSIX Regexp Compilation): Likewise.
Similarly for regcomp and regexec. Also, first 2 args of regexec
and 2nd arg of regerror are const.
* posix/regex.c: Do not include <sys/types.h>, as POSIX no longer
requires this. (The code never needed it.)
2005-08-20 Paul Eggert <eggert@cs.ucla.edu>
* posix/regexec.c (sift_states_bkref): re_node_set_insert returns
int, not reg_errcode_t.
* posix/regex_internal.c (calc_state_hash): Put 'inline' before type,
since some broken compilers warn about it otherwise.
* posix/regcomp.c (create_initial_state): Remove duplicate decl.
2005-08-20 Paul Eggert <eggert@cs.ucla.edu>
* posix/regex.h (_RE_ARGS): Remove. No longer needed, since we assume
C89 or better. All uses removed.
2005-09-06 Ulrich Drepper <drepper@redhat.com>
* posix/regex.c: Prevent using C++ compilers.
2005-08-19 Paul Eggert <eggert@cs.ucla.edu>
* posix/regcomp.c (duplicate_node): Return new index, not an error
code, and let the caller return REG_ESPACE if out of space. This
removes an uninitialied-variable warning with GCC 4.0.1, and also
avoids taking the address of a local variable. All callers
changed.
2005-09-06 Ulrich Drepper <drepper@redhat.com>
* include/time.h (__strptime_internal): Rename parameter to avoid
bogus compiler warning.
2005-08-19 Jim Meyering <jim@meyering.net>
* posix/regexec.c (proceed_next_node): Redo local variables to
avoid GCC shadowing warnings.
2005-09-06 Ulrich Drepper <drepper@redhat.com>
* posix/regex_internal.c (re_acquire_state): Minor code rearrangement.
(re_acquire_state_context): Likewise.
2005-08-19 Paul Eggert <eggert@cs.ucla.edu>
* posix/regex_internal.c (re_string_realloc_buffers):
(re_node_set_insert, re_node_set_insert_last, re_dfa_add_node):
Rename local variables to avoid GCC shadowing warnings.
2005-07-08 Eric Blake <ebb9@byu.net>
Paul Eggert <eggert@cs.ucla.edu>
* posix/regcomp.c (init_dfa): Store __btowc value in wint_t, not
wchar_t. Remove now-unnecessary cast.
(build_range_exp): Likewise.
Update.
2005-01-27 Paolo Bonzini <bonzini@gnu.org>
[BZ #558]
* posix/regcomp.c (calc_inveclosure): Return reg_errcode_t.
Initialize the node sets in dfa->inveclosures.
(analyze): Initialize inveclosures only if it is needed.
Check errors from calc_inveclosure.
* posix/regex_internal.c (re_dfa_add_node): Do not initialize
the inveclosure node set.
* posix/regexec.c (re_search_internal): If nmatch includes unused
subexpressions, reset them to { rm_so: -1, rm_eo: -1 } here.
* posix/regcomp.c (parse_bracket_exp) [!RE_ENABLE_I18N]:
Do build a SIMPLE_BRACKET token.
* posix/regexec.c (transit_state_mb): Do not examine nodes
where ACCEPT_MB is not set.
Update.
2004-12-13 Paolo Bonzini <bonzini@gnu.org>
Separate parsing and creation of the NFA. Avoided recursion on
the (very unbalanced) parse tree.
[BZ #611]
* posix/regcomp.c (struct subexp_optimize, analyze_tree, calc_epsdest,
re_dfa_add_tree_node, mark_opt_subexp_iter): Removed.
(optimize_subexps, duplicate_tree, calc_first, calc_next,
mark_opt_subexp): Rewritten.
(preorder, postorder, lower_subexps, lower_subexp, link_nfa_nodes,
create_token_tree, free_tree, free_token): New.
(analyze): Accept a regex_t *. Invoke the passes via the preorder and
postorder generic visitors. Do not initialize the fields in the
re_dfa_t that represent the transitions.
(free_dfa_content): Use free_token.
(re_compile_internal): Analyze before UTF-8 optimizations. Do not
include optimization of subexpressions.
(create_initial_state): Fetch the DFA node index from the first node's
bin_tree_t *.
(optimize_utf8): Abort on unexpected nodes, including OP_DUP_QUESTION.
Return on COMPLEX_BRACKET.
(duplicate_node_closure): Fix comment.
(duplicate_node): Do not initialize the fields in the
re_dfa_t that represent the transitions.
(calc_eclosure, calc_inveclosure): Do not handle OP_DELETED_SUBEXP.
(create_tree): Remove final argument. All callers adjusted. Rewritten
to use create_token_tree.
(parse_reg_exp, parse_branch, parse_expression, parse_bracket_exp,
build_charclass_op): Use create_tree or create_token_tree instead
of re_dfa_add_tree_node.
(parse_dup_op): Likewise. Also free the tree using free_tree for
"<re>{0}", and lower OP_DUP_QUESTION to OP_ALT: "a?" is equivalent
to "a|". Adjust invocation of mark_opt_subexp.
(parse_sub_exp): Create a single SUBEXP node.
* posix/regex_internal.c (re_dfa_add_node): Remove last parameter,
always perform as if it was 1. Do not initialize OPT_SUBEXP and
DUPLICATED, and initialize the DFA fields representing the transitions.
* posix/regex_internal.h (re_dfa_add_node): Adjust prototype.
(re_token_type_t): Move OP_DUP_PLUS and OP_DUP_QUESTION to the tokens
section. Add a tree-only code SUBEXP. Remove OP_DELETED_SUBEXP.
(bin_tree_t): Include a full re_token_t for TOKEN. Turn FIRST and
NEXT into pointers to trees. Remove ECLOSURE.
2004-12-28 Paolo Bonzini <bonzini@gnu.org >
[BZ #605]
* posix/regcomp.c (parse_bracket_exp): Do not modify DFA nodes
that were already created.
* posix/regex_internal.c (re_dfa_add_node): Set accept_mb field
in the token if needed.
(create_ci_newstate, create_cd_newstate): Set accept_mb field
from the tokens' field.
* posix/regex_internal.h (re_token_t): Add accept_mb field.
(ACCEPT_MB_NODE): Removed.
* posix/regexec.c (proceed_next_node, transit_states_mb,
build_sifted_states, check_arrival_add_next_nodes): Use
accept_mb instead of ACCEPT_MB_NODE.
2004-11-23 Paolo Bonzini <bonzini@gnu.org>
* posix/regcomp.c (analyze_tree): Always call calc_epsdest.
(calc_inveclosure): Use re_node_set_insert_last.
(parse_dup_op): Lower X{1,5} to (X(X(X(XX?)?)?)?)?
rather than X?X?X?X?X?.
* posix/regex_internal.h (re_node_set_insert_last): New declaration.
* posix/regex_internal.c (re_node_set_insert_last): New function.
* posix/PCRE.tests: Add testcases.
Update.
2004-11-18 Jakub Jelinek <jakub@redhat.com>
[BZ #544]
* posix/regex.h (RE_NO_SUB): New define.
* posix/regex_internal.h (OP_DELETED_SUBEXP): New.
(re_dfa_t): Add subexp_map.
* posix/regcomp.c (struct subexp_optimize): New type.
(optimize_subexps): New routine.
(re_compile_internal): Call it.
(re_compile_pattern): Set preg->no_sub to 1 if RE_NO_SUB.
(free_dfa_content): Free subexp_map.
(calc_inveclosure, calc_eclosure): Skip OP_DELETED_SUBEXP
nodes.
* posix/regexec.c (re_search_internal): If subexp_map
is not NULL, duplicate registers as needed.
* posix/Makefile: Add rules to build and run tst-regex2.
* posix/tst-regex2.c: New test.
* posix/rxspencer/tests: Fix last two tests (\0 -> \1).
Add some new tests for nested subexpressions.
2004-11-08 Ulrich Drepper <drepper@redhat.com>
* posix/regcomp.c (utf8_sb_map): Define.
(free_dfa_content): Don't free dfa->sb_char if it's a pointer to
utf8_sb_map.
(init_dfa): Use utf8_sb_map instead of initializing memory when the
encoding is UTF-8.
* posix/regcomp.c (init_dfa): Get the codeset name outside glibc as
well. Check if it is spelled UTF8 as well as UTF-8, and check
case-insensitively. Set dfa->map_notascii manually when outside
glibc.
* posix/regex_internal.c (build_wcs_upper_buffer) [!_LIBC]: Enable
optimizations based on map_notascii.
* posix/regex_internal.h [HAVE_LANGINFO_H || HAVE_LANGINFO_CODESET
|| _LIBC]: Include langinfo.h.
* posix/regex_internal.h (struct re_backref_cache_entry): Add "more"
field.
* posix/regexec.c (check_dst_limits): Hoist computation of the source
and destination bkref_idx out of the loop. Pass it to
check_dst_limits_calc_pos.
(check_dst_limits_calc_pos_1): New function, containing the recursive
loop of check_dst_limits_calc_pos; uses the "more" field of
struct re_backref_cache to control the loop.
(check_dst_limits_calc_pos): Store into "boundaries" the position
relative to lim's start and end positions. Do not accept eclosures,
accept bkref_idx instead. Call check_dst_limits_calc_pos_1 to do the
work.
(sift_states_bkref): Use the "more" field of struct re_backref_cache
to control the loop. A big "if" was turned into a continue and the
function was reindented.
(get_subexp): Use the "more" field of struct re_backref_cache
to control the loop.
(match_ctx_add_entry): Initialize the bkref_ents' "more" field.
(search_cur_bkref_entry): Return -1 if out of bounds.
* posix/regexec.c (empty_set): Remove.
(sift_states_backward): Remove cur_src variable. Move inner loop
to build_sifted_states.
(build_sifted_states): Extract from sift_states_backward. Do not
use empty_set.
(update_cur_sifted_state): Do not use empty_set. Special case
dest_nodes->nelem == 0.
2004-01-12 Paolo Bonzini <bonzini@gnu.org>
* posix/regcomp.c [_LIBC && !RE_ENABLE_I18N]:
Drop code to support this, it is never true.
(build_range_exp) [!_LIBC]: Do not create a range
in MBCSET for a single-byte character set.
(build_range_exp) [_LIBC]: Do not create a range
in MBCSET for a single-byte character set without
collation elements.
(init_dfa): Do not conditionalize on _LIBC, it
just makes the code less clear.
(parse_bracket_exp): Use NON_MATCH variable in
addition to "mbcset->non_match", not as an
alternative.
(build_charclass_op): rename NOT parameter to
NON_MATCH, use it instead of declaring a variable.
(parse_bracket_exp) [!_LIBC]: Pass NULL for MBCSET
if the character set is single-byte.
2004-01-14 Jakub Jelinek <jakub@redhat.com>
* posix/regcomp.c (peek_token_bracket): Check remaining
string length before re_string_peek_byte (x, 1).
(parse_bracket_symbol): Likewise.
* posix/regex_internal.h (re_string_is_single_byte_char): Return
true at last byte in the string.
* posix/bug-regex22.c (main): Add new test.
Ulrich Drepper
Ulrich Drepper
Paul Eggert
Paolo Bonzini
Jakub Jelinek
Roland McGrath
Andreas Jaeger