Linux: Skip thread_db thread event reporting if PTRACE_EVENT_CLONE is supported
[A test I wrote stumbled on a libthread_db issue related to thread
event breakpoints. See glibc PR17705:
[nptl_db: stale thread create/death events if debugger detaches]
https://sourceware.org/bugzilla/show_bug.cgi?id=17705
This patch avoids that whole issue by making GDB stop using thread
event breakpoints in the first place, which is good for other reasons
as well, anyway.]
Before PTRACE_EVENT_CLONE (Linux 2.6), the only way to learn about new
threads in the inferior (to attach to them) or to learn about thread
exit was to coordinate with the inferior's glibc/runtime, using
libthread_db. That works by putting a breakpoint at a magic address
which is called when a new thread is spawned, or when a thread is
about to exit. When that breakpoint is hit, all threads are stopped,
and then GDB coordinates with libthread_db to read data structures out
of the inferior to learn about what happened. Then the breakpoint is
single-stepped, and then all threads are re-resumed. This isn't very
efficient (stops all threads) and is more fragile (inferior's thread
list in memory may be corrupt; libthread_db bugs, etc.) than ideal.
When the kernel supports PTRACE_EVENT_CLONE (which we already make use
of), there's really no need to use libthread_db's event reporting
mechanism to learn about new LWPs. And if the kernel supports that,
then we learn about LWP exits through regular WIFEXITED wait statuses,
so no need for the death event breakpoint either.
GDBserver has been likewise skipping the thread_db events for a long
while:
https://sourceware.org/ml/gdb-patches/2007-10/msg00547.html
There's one user-visible difference: we'll no longer print about
threads being created and exiting while the program is running, like:
[Thread 0x7ffff7dbb700 (LWP 30670) exited]
[New Thread 0x7ffff7db3700 (LWP 30671)]
[Thread 0x7ffff7dd3700 (LWP 30667) exited]
[New Thread 0x7ffff7dab700 (LWP 30672)]
[Thread 0x7ffff7db3700 (LWP 30671) exited]
[Thread 0x7ffff7dcb700 (LWP 30668) exited]
This is exactly the same behavior as when debugging against remote
targets / gdbserver. I actually think that's a good thing (and as
such have listed this in the local/remote parity wiki page a while
ago), as the printing slows down the inferior. It's also a
distraction to keep bothering the user about short-lived threads that
she won't be able to interact with anyway. Instead, the user (and
frontend) will be informed about new threads that currently exist in
the program when the program next stops:
(gdb) c
...
* ctrl-c *
[New Thread 0x7ffff7963700 (LWP 7797)]
[New Thread 0x7ffff796b700 (LWP 7796)]
Program received signal SIGINT, Interrupt.
[Switching to Thread 0x7ffff796b700 (LWP 7796)]
clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:81
81 testq %rax,%rax
(gdb) info threads
A couple of tests had assumptions on GDB thread numbers that no longer
hold.
Tested on x86_64 Fedora 20.
gdb/
2014-01-09 Pedro Alves <palves@redhat.com>
Skip enabling event reporting if the kernel supports
PTRACE_EVENT_CLONE.
* linux-thread-db.c: Include "nat/linux-ptrace.h".
(thread_db_use_events): New function.
(try_thread_db_load_1): Check thread_db_use_events before enabling
event reporting.
(update_thread_state): New function.
(attach_thread): Use it. Check thread_db_use_events before
enabling event reporting.
(thread_db_detach): Check thread_db_use_events before disabling
event reporting.
(find_new_threads_callback): Check thread_db_use_events before
enabling event reporting. Update the thread's state if not using
libthread_db events.
gdb/testsuite/
2014-01-09 Pedro Alves <palves@redhat.com>
* gdb.threads/fork-thread-pending.exp: Switch to the main thread
instead of to thread 2.
* gdb.threads/signal-command-multiple-signals-pending.c (main):
Add barrier around each pthread_create call instead of around all
calls.
* gdb.threads/signal-command-multiple-signals-pending.exp (test):
Set a break on thread_function and have the child threads hit it
one at at a time.
2014-12-16 17:12:25 +01:00
|
|
|
|
2014-01-09 Pedro Alves <palves@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
Skip enabling event reporting if the kernel supports
|
|
|
|
|
|
PTRACE_EVENT_CLONE.
|
|
|
|
|
|
* linux-thread-db.c: Include "nat/linux-ptrace.h".
|
|
|
|
|
|
(thread_db_use_events): New function.
|
|
|
|
|
|
(try_thread_db_load_1): Check thread_db_use_events before enabling
|
|
|
|
|
|
event reporting.
|
|
|
|
|
|
(update_thread_state): New function.
|
|
|
|
|
|
(attach_thread): Use it. Check thread_db_use_events before
|
|
|
|
|
|
enabling event reporting.
|
|
|
|
|
|
(thread_db_detach): Check thread_db_use_events before disabling
|
|
|
|
|
|
event reporting.
|
|
|
|
|
|
(find_new_threads_callback): Check thread_db_use_events before
|
|
|
|
|
|
enabling event reporting. Update the thread's state if not using
|
|
|
|
|
|
libthread_db events.
|
|
|
|
|
|
|
libthread_db: Skip attaching to terminated and joined threads
I wrote a test that attaches to a program that constantly spawns
short-lived threads, which exposed several issues. This is one of
them.
On GNU/Linux, attaching to a multi-threaded program sometimes prints
out warnings like:
...
[New LWP 20700]
warning: unable to open /proc file '/proc/-1/status'
[New LWP 20850]
[New LWP 21019]
...
That happens because when a thread exits, and is joined, glibc does:
nptl/pthread_join.c:
pthread_join ()
{
...
if (__glibc_likely (result == 0))
{
/* We mark the thread as terminated and as joined. */
pd->tid = -1;
...
/* Free the TCB. */
__free_tcb (pd);
}
So if we attach or interrupt the program (which does an implicit "info
threads") at just the right (or rather, wrong) time, we can find and
return threads in the libthread_db/pthreads thread list with kernel
thread ID -1. I've filed glibc PR nptl/17707 for this. You'll find
more info there.
This patch handles this as a special case in GDB.
This is actually more than just a cosmetic issue. lin_lwp_attach_lwp
will think that this -1 is an LWP we're not attached to yet, and after
failing to attach will try to check we were already attached to the
process, using a waitpid call, which in this case ends up being
"waitpid (-1, ...", which obviously results in GDB potentially
discarding an event when it shouldn't...
Tested on x86_64 Fedora 20, native and gdbserver.
gdb/gdbserver/
2015-01-09 Pedro Alves <palves@redhat.com>
* thread-db.c (find_new_threads_callback): Ignore thread if the
kernel thread ID is -1.
gdb/
2015-01-09 Pedro Alves <palves@redhat.com>
* linux-nat.c (lin_lwp_attach_lwp): Assert that the lwp id we're
about to wait for is > 0.
* linux-thread-db.c (find_new_threads_callback): Ignore thread if
the kernel thread ID is -1.
2014-12-16 17:12:24 +01:00
|
|
|
|
2015-01-09 Pedro Alves <palves@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
* linux-nat.c (lin_lwp_attach_lwp): Assert that the lwp id we're
|
|
|
|
|
|
about to wait for is > 0.
|
|
|
|
|
|
* linux-thread-db.c (find_new_threads_callback): Ignore thread if
|
|
|
|
|
|
the kernel thread ID is -1.
|
|
|
|
|
|
|
Linux: on attach, attach to lwps listed under /proc/$pid/task/
... instead of relying on libthread_db.
I wrote a test that attaches to a program that constantly spawns
short-lived threads, which exposed several issues. This is one of
them.
On Linux, we need to attach to all threads of a process (thread group)
individually. We currently rely on libthread_db to list the threads,
but that is problematic, because libthread_db relies on reading data
structures out of the inferior (which may well be corrupted). If
threads are being created or exiting just while we try to attach, we
may trip on inconsistencies in the inferior's thread list. To work
around that, when we see a seemingly corrupt list, we currently retry
a few times:
static void
thread_db_find_new_threads_2 (ptid_t ptid, int until_no_new)
{
...
if (until_no_new)
{
/* Require 4 successive iterations which do not find any new threads.
The 4 is a heuristic: there is an inherent race here, and I have
seen that 2 iterations in a row are not always sufficient to
"capture" all threads. */
...
That heuristic may well fail, and when it does, we end up with threads
in the program that aren't under GDB's control. That's obviously bad
and results in quite mistifying failures, like e.g., the process dying
for seeminly no reason when a thread that wasn't attached trips on a
breakpoint.
There's really no reason to rely on libthread_db for this nowadays
when we have /proc mounted. In that case, which is the usual case, we
can list the LWPs from /proc/PID/task/. In fact, GDBserver is already
doing this. The patch factors out that code that knows to walk the
task/ directory out of GDBserver, and makes GDB use it too.
Like GDBserver, the patch makes GDB attach to LWPs and _not_ wait for
them to stop immediately. Instead, we just tag the LWP as having an
expected stop. Because we can only set the ptrace options when the
thread stops, we need a new flag in the lwp structure to keep track of
whether we've already set the ptrace options, just like in GDBserver.
Note that nothing issues any ptrace command to the threads between the
PTRACE_ATTACH and the stop, so this is safe (unlike one scenario
described in gdbserver's linux-low.c).
When we attach to a program that has threads exiting while we attach,
it's easy to race with a thread just exiting as we try to attach to
it, like:
#1 - get current list of threads
#2 - attach to each listed thread
#3 - ooops, attach failed, thread is already gone
As this is pretty normal, we shouldn't be issuing a scary warning in
step #3.
When #3 happens, PTRACE_ATTACH usually fails with ESRCH, but sometimes
we'll see EPERM as well. That happens when the kernel still has the
thread in its task list, but the thread is marked as dead.
Unfortunately, EPERM is ambiguous and we'll get it also on other
scenarios where the thread isn't dead, and in those cases, it's useful
to get a warning. To distiguish the cases, when we get an EPERM
failure, we open /proc/PID/status, and check the thread's state -- if
the /proc file no longer exists, or the state is "Z (Zombie)" or "X
(Dead)", we ignore the EPERM error silently; otherwise, we'll warn.
Unfortunately, there seems to be a kernel race here. Sometimes I get
EPERM, and then the /proc state still indicates "R (Running)"... If
we wait a bit and retry, we do end up seeing X or Z state, or get an
ESRCH. I thought of making GDB retry the attach a few times, but even
with a 500ms wait and 4 retries, I still see the warning sometimes. I
haven't been able to identify the kernel path that causes this yet,
but in any case, it looks like a kernel bug to me. As this just
results failure to suppress a warning that we've been printing since
about forever anyway, I'm just making the test cope with it, and issue
an XFAIL.
gdb/gdbserver/
2015-01-09 Pedro Alves <palves@redhat.com>
* linux-low.c (linux_attach_fail_reason_string): Move to
nat/linux-ptrace.c, and rename.
(linux_attach_lwp): Update comment.
(attach_proc_task_lwp_callback): New function.
(linux_attach): Adjust to rename and use
linux_proc_attach_tgid_threads.
(linux_attach_fail_reason_string): Delete declaration.
gdb/
2015-01-09 Pedro Alves <palves@redhat.com>
* linux-nat.c (attach_proc_task_lwp_callback): New function.
(linux_nat_attach): Use linux_proc_attach_tgid_threads.
(wait_lwp, linux_nat_filter_event): If not set yet, set the lwp's
ptrace option flags.
* linux-nat.h (struct lwp_info) <must_set_ptrace_flags>: New
field.
* nat/linux-procfs.c: Include <dirent.h>.
(linux_proc_get_int): New parameter "warn". Handle it.
(linux_proc_get_tgid): Adjust.
(linux_proc_get_tracerpid): Rename to ...
(linux_proc_get_tracerpid_nowarn): ... this.
(linux_proc_pid_get_state): New function, factored out from
(linux_proc_pid_has_state): ... this. Add new parameter "warn"
and handle it.
(linux_proc_pid_is_gone): New function.
(linux_proc_pid_is_stopped): Adjust.
(linux_proc_pid_is_zombie_maybe_warn)
(linux_proc_pid_is_zombie_nowarn): New functions.
(linux_proc_pid_is_zombie): Use
linux_proc_pid_is_zombie_maybe_warn.
(linux_proc_attach_tgid_threads): New function.
* nat/linux-procfs.h (linux_proc_get_tgid): Update comment.
(linux_proc_get_tracerpid): Rename to ...
(linux_proc_get_tracerpid_nowarn): ... this, and update comment.
(linux_proc_pid_is_gone): New declaration.
(linux_proc_pid_is_zombie): Update comment.
(linux_proc_pid_is_zombie_nowarn): New declaration.
(linux_proc_attach_lwp_func): New typedef.
(linux_proc_attach_tgid_threads): New declaration.
* nat/linux-ptrace.c (linux_ptrace_attach_fail_reason): Adjust to
use nowarn functions.
(linux_ptrace_attach_fail_reason_string): Move here from
gdbserver/linux-low.c and rename.
(ptrace_supports_feature): If the current ptrace options are not
known yet, check them now, instead of asserting.
* nat/linux-ptrace.h (linux_ptrace_attach_fail_reason_string):
Declare.
2014-12-16 17:12:24 +01:00
|
|
|
|
2015-01-09 Pedro Alves <palves@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
* linux-nat.c (attach_proc_task_lwp_callback): New function.
|
|
|
|
|
|
(linux_nat_attach): Use linux_proc_attach_tgid_threads.
|
|
|
|
|
|
(wait_lwp, linux_nat_filter_event): If not set yet, set the lwp's
|
|
|
|
|
|
ptrace option flags.
|
|
|
|
|
|
* linux-nat.h (struct lwp_info) <must_set_ptrace_flags>: New
|
|
|
|
|
|
field.
|
|
|
|
|
|
* nat/linux-procfs.c: Include <dirent.h>.
|
|
|
|
|
|
(linux_proc_get_int): New parameter "warn". Handle it.
|
|
|
|
|
|
(linux_proc_get_tgid): Adjust.
|
|
|
|
|
|
(linux_proc_get_tracerpid): Rename to ...
|
|
|
|
|
|
(linux_proc_get_tracerpid_nowarn): ... this.
|
|
|
|
|
|
(linux_proc_pid_get_state): New function, factored out from
|
|
|
|
|
|
(linux_proc_pid_has_state): ... this. Add new parameter "warn"
|
|
|
|
|
|
and handle it.
|
|
|
|
|
|
(linux_proc_pid_is_gone): New function.
|
|
|
|
|
|
(linux_proc_pid_is_stopped): Adjust.
|
|
|
|
|
|
(linux_proc_pid_is_zombie_maybe_warn)
|
|
|
|
|
|
(linux_proc_pid_is_zombie_nowarn): New functions.
|
|
|
|
|
|
(linux_proc_pid_is_zombie): Use
|
|
|
|
|
|
linux_proc_pid_is_zombie_maybe_warn.
|
|
|
|
|
|
(linux_proc_attach_tgid_threads): New function.
|
|
|
|
|
|
* nat/linux-procfs.h (linux_proc_get_tgid): Update comment.
|
|
|
|
|
|
(linux_proc_get_tracerpid): Rename to ...
|
|
|
|
|
|
(linux_proc_get_tracerpid_nowarn): ... this, and update comment.
|
|
|
|
|
|
(linux_proc_pid_is_gone): New declaration.
|
|
|
|
|
|
(linux_proc_pid_is_zombie): Update comment.
|
|
|
|
|
|
(linux_proc_pid_is_zombie_nowarn): New declaration.
|
|
|
|
|
|
(linux_proc_attach_lwp_func): New typedef.
|
|
|
|
|
|
(linux_proc_attach_tgid_threads): New declaration.
|
|
|
|
|
|
* nat/linux-ptrace.c (linux_ptrace_attach_fail_reason): Adjust to
|
|
|
|
|
|
use nowarn functions.
|
|
|
|
|
|
(linux_ptrace_attach_fail_reason_string): Move here from
|
|
|
|
|
|
gdbserver/linux-low.c and rename.
|
|
|
|
|
|
(ptrace_supports_feature): If the current ptrace options are not
|
|
|
|
|
|
known yet, check them now, instead of asserting.
|
|
|
|
|
|
* nat/linux-ptrace.h (linux_ptrace_attach_fail_reason_string):
|
|
|
|
|
|
Declare.
|
|
|
|
|
|
|
2014-12-16 17:12:23 +01:00
|
|
|
|
2015-01-09 Pedro Alves <palves@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
* linux-thread-db.c (thread_db_find_new_threads_silently)
|
|
|
|
|
|
(try_thread_db_load_1, try_thread_db_load, thread_db_load_search)
|
|
|
|
|
|
(find_new_threads_once): Print debug output on gdb_stdlog.
|
|
|
|
|
|
|
2015-01-09 11:09:03 +01:00
|
|
|
|
2015-01-09 Chen Gang <gang.chen.5i5j@gmail.com>
|
|
|
|
|
|
Pedro Alves <palves@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
* compile/compile.c: Include "gdb_wait.h".
|
|
|
|
|
|
(do_rmdir): Check return value, and free 'zap'.
|
|
|
|
|
|
|
2014-12-28 09:12:53 +01:00
|
|
|
|
2015-01-08 Pedro Alves <palves@redhat.com>
|
|
|
|
|
|
Yao Qi <yao@codesourcery.com>
|
|
|
|
|
|
|
|
|
|
|
|
* dwarf2loc.c (indirect_pieced_value): Don't call
|
|
|
|
|
|
gdb_sign_extend. Call extract_signed_integer instead.
|
|
|
|
|
|
* utils.c (gdb_sign_extend): Remove.
|
|
|
|
|
|
* utils.h (gdb_sign_extend): Remove declaration.
|
|
|
|
|
|
|
2015-01-08 08:53:26 +01:00
|
|
|
|
2015-01-07 Pierre Muller <muller@sourceware.org>
|
|
|
|
|
|
|
|
|
|
|
|
PR symtab/17811
|
|
|
|
|
|
* stabsread.c (define_symbol): Set language for C++ special symbols.
|
|
|
|
|
|
|
2015-01-07 22:23:39 +01:00
|
|
|
|
2015-01-07 Patrick Palka <patrick@parcs.ath.cx>
|
|
|
|
|
|
|
|
|
|
|
|
* inflow.c (initial_gdb_ttystate): Tweak comment.
|
|
|
|
|
|
|
2015-01-07 15:49:49 +01:00
|
|
|
|
2015-01-07 Joel Brobecker <brobecker@adacore.com>
|
|
|
|
|
|
|
|
|
|
|
|
* inflow.c (set_initial_gdb_ttystate): Add empty line after
|
|
|
|
|
|
comment documenting function.
|
|
|
|
|
|
|
Don't propagate our current terminal state to the inferior
Currently when we start an inferior we have the inferior inherit our
terminal state. Under TUI, our terminal is highly modified by ncurses
and readline. So when starting an inferior under TUI, the inferior will
have a highly modified terminal state which will interfere with standard
I/O. For example,
$ gdb gdb
(gdb) break main
(gdb) run
(gdb) print puts ("a\nb")
a
b
$1 = 4
(gdb) [enter TUI mode]
(gdb) run
(gdb) [exit TUI mode]
(gdb) print puts ("a\nb")
a
b
$2 = 4
(gdb) print puts ("a\r\nb\r")
a
b
$3 = 6
As you can see, when we start the inferior under the regular interface,
puts() prints the text properly. But when we start the inferior under
TUI, puts() does not print the text properly. This is because when we
start the inferior under TUI it inherits our current terminal state
which has been modified by ncurses to, among other things, require an
explicit \r\n to print a new line. As a result the inferior performs
standard I/O in an unexpected way.
Because of this discrepancy, it doesn't seem like a good idea to have
the inferior inherit our _current_ terminal state for it may have been
modified by readline and/or ncurses. Instead, we should have the
inferior inherit a pristine snapshot of our terminal state taken before
readline or ncurses have had a chance to alter it. This enables the
inferior to run in a more accurate way, more closely mimicking the
program's behavior had it run standalone. And it fixes the above
mentioned issue.
Tested on x86_64-unknown-linux-gnu.
gdb/ChangeLog:
* terminal.h (set_initial_gdb_ttystate): Declare.
* inflow.c (initial_gdb_ttystate): New static variable.
(set_initial_gdb_ttystate): New setter.
(child_terminal_init_with_pgrp): Copy initial_gdb_ttystate
instead of our current terminal state.
* top.c (gdb_init): Call set_initial_gdb_ttystate.
2014-11-22 20:12:49 +01:00
|
|
|
|
2015-01-07 Patrick Palka <patrick@parcs.ath.cx>
|
|
|
|
|
|
|
|
|
|
|
|
* terminal.h (set_initial_gdb_ttystate): Declare.
|
|
|
|
|
|
* inflow.c (initial_gdb_ttystate): New static variable.
|
|
|
|
|
|
(set_initial_gdb_ttystate): New setter.
|
|
|
|
|
|
(child_terminal_init_with_pgrp): Copy initial_gdb_ttystate
|
|
|
|
|
|
instead of our current terminal state.
|
|
|
|
|
|
* top.c (gdb_init): Call set_initial_gdb_ttystate.
|
|
|
|
|
|
|
2015-01-07 04:34:29 +01:00
|
|
|
|
2015-01-07 Joel Brobecker <brobecker@adacore.com>
|
|
|
|
|
|
|
|
|
|
|
|
* guile/scm-type.c (tyscm_array_1): Add comment.
|
|
|
|
|
|
* python/py-type.c (typy_array_1): Add comment.
|
|
|
|
|
|
|
2015-01-06 15:37:53 +01:00
|
|
|
|
2015-01-06 Joel Brobecker <brobecker@adacore.com>
|
|
|
|
|
|
|
|
|
|
|
|
* guile/scm-type.c (tyscm_array_1): Do not raise out-of-range
|
|
|
|
|
|
error if N2 is equal to N1 - 1.
|
|
|
|
|
|
|
2015-01-06 15:30:53 +01:00
|
|
|
|
2015-01-06 Joel Brobecker <brobecker@adacore.com>
|
|
|
|
|
|
|
|
|
|
|
|
* python/py-type.c (typy_array_1): Do not raise negative-length
|
|
|
|
|
|
exception if N2 is equal to N1 - 1.
|
|
|
|
|
|
|
2015-01-03 21:01:29 +01:00
|
|
|
|
2015-01-03 Doug Evans <xdje42@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
* c-exp.y: Whitespace cleanup.
|
|
|
|
|
|
(classify_inner_name): Remove extra ;.
|
|
|
|
|
|
|
2015-01-03 00:36:05 +01:00
|
|
|
|
2015-01-02 Maciej W. Rozycki <macro@codesourcery.com>
|
|
|
|
|
|
|
|
|
|
|
|
* mips-tdep.c (mips32_scan_prologue): Keep the extracted stack
|
|
|
|
|
|
offset signed.
|
|
|
|
|
|
|
2015-01-02 20:49:14 +01:00
|
|
|
|
2015-01-02 Doug Evans <dje@google.com>
|
|
|
|
|
|
|
|
|
|
|
|
* dwarf2read.c (setup_type_unit_groups): Remove outdated comment.
|
|
|
|
|
|
|
2015-01-02 20:02:31 +01:00
|
|
|
|
2015-01-02 Doug Evans <dje@google.com>
|
|
|
|
|
|
|
|
|
|
|
|
* symtab.h (struct symbol): Fix typo in comment.
|
|
|
|
|
|
|
2015-01-01 10:32:14 +01:00
|
|
|
|
2015-01-01 Joel Brobecker <brobecker@adacore.com>
|
|
|
|
|
|
|
|
|
|
|
|
Update year range in copyright notice of all files.
|
|
|
|
|
|
|
2015-01-01 10:24:41 +01:00
|
|
|
|
2015-01-01 Joel Brobecker <brobecker@adacore.com>
|
|
|
|
|
|
|
|
|
|
|
|
* top.c (print_gdb_version): Update copyright year to 2015.
|
|
|
|
|
|
|
2015-01-01 10:21:14 +01:00
|
|
|
|
2015-01-01 Joel Brobecker <brobecker@adacore.com>
|
2014-12-30 08:36:53 +01:00
|
|
|
|
|
2015-01-01 10:21:14 +01:00
|
|
|
|
* config/djgpp/fnchange.lst: Add entry for gdb/ChangeLog-2014.
|
2014-12-30 08:36:53 +01:00
|
|
|
|
|
2015-01-01 10:21:14 +01:00
|
|
|
|
For older changes see ChangeLog-2014.
|
1999-04-16 03:35:26 +02:00
|
|
|
|
|
|
|
|
|
|
Local Variables:
|
|
|
|
|
|
mode: change-log
|
|
|
|
|
|
left-margin: 8
|
|
|
|
|
|
fill-column: 74
|
|
|
|
|
|
version-control: never
|
2007-08-10 00:44:38 +02:00
|
|
|
|
coding: utf-8
|
1999-04-16 03:35:26 +02:00
|
|
|
|
End:
|
