OpenE2K
/
glibc
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
glibc/benchtests/Makefile

132 lines
3.9 KiB
Makefile
Raw Normal View History

Framework for performance benchmarking of functions See benchtests/Makefile to know how to use it.
2013-03-15 08:00:03 +01:00
# Copyright (C) 2013 Free Software Foundation, Inc.
# This file is part of the GNU C Library.
# The GNU C Library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 2.1 of the License, or (at your option) any later version.
# The GNU C Library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
# You should have received a copy of the GNU Lesser General Public
# License along with the GNU C Library; if not, see
# <http://www.gnu.org/licenses/>.
# Makefile for benchmark tests. The only useful target here is `bench`.
Sort benchmark functions
2013-05-22 07:35:04 +02:00
# Add benchmark functions in alphabetical order.
Framework for performance benchmarking of functions See benchtests/Makefile to know how to use it.
2013-03-15 08:00:03 +01:00
subdir := benchtests
benchmark inputs for exp2, log2, log and tan
2013-12-12 05:01:53 +01:00
bench := acos acosh asin asinh atan atanh cos cosh exp exp2 log log2 modf pow \
rint sin sincos sinh sqrt tan tanh
Sort benchmark functions
2013-05-22 07:35:04 +02:00
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
# String function benchmarks.
benchtests/Makefile: Run benchmark for memcpy. The benchmark for memcpy got disabled accidentally. Re-enable it. ChangeLog: 2013-09-06 Will Newton <will.newton@linaro.org> * benchtests/Makefile (string-bench): Add memcpy.
2013-09-04 10:11:58 +02:00
string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
mempcpy memset rawmemchr stpcpy stpncpy strcasecmp strcasestr \
strcat strchr strchrnul strcmp strcpy strcspn strlen \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
benchtests: Add strsep benchmark
2013-11-18 13:03:07 +01:00
strspn strstr strcpy_chk stpcpy_chk memrchr strsep
Remove duplicate ifunc benchtests.
2013-11-26 12:48:33 +01:00
string-bench-all := $(string-bench)
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
benchtests: Add strtod benchmark
2013-11-11 14:23:02 +01:00
stdlib-bench := strtod
benchset := $(string-bench-all) $(stdlib-bench)
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
benchtests/Makefile: Use LDLIBS instead of LDFLAGS. LDFLAGS puts the library too early in the command line if --as-needed is being used. Use LDLIBS instead. ChangeLog: 2013-09-04 Will Newton <will.newton@linaro.org> * benchtests/Makefile: Use LDLIBS instead of LDFLAGS.
2013-09-02 15:00:45 +02:00
LDLIBS-bench-acos = -lm
LDLIBS-bench-acosh = -lm
LDLIBS-bench-asin = -lm
LDLIBS-bench-asinh = -lm
LDLIBS-bench-atan = -lm
LDLIBS-bench-atanh = -lm
LDLIBS-bench-cos = -lm
LDLIBS-bench-cosh = -lm
LDLIBS-bench-exp = -lm
benchmark inputs for exp2, log2, log and tan
2013-12-12 05:01:53 +01:00
LDLIBS-bench-exp2 = -lm
benchtests/Makefile: Use LDLIBS instead of LDFLAGS. LDFLAGS puts the library too early in the command line if --as-needed is being used. Use LDLIBS instead. ChangeLog: 2013-09-04 Will Newton <will.newton@linaro.org> * benchtests/Makefile: Use LDLIBS instead of LDFLAGS.
2013-09-02 15:00:45 +02:00
LDLIBS-bench-log = -lm
benchmark inputs for exp2, log2, log and tan
2013-12-12 05:01:53 +01:00
LDLIBS-bench-log2 = -lm
benchtests/Makefile: Use LDLIBS instead of LDFLAGS. LDFLAGS puts the library too early in the command line if --as-needed is being used. Use LDLIBS instead. ChangeLog: 2013-09-04 Will Newton <will.newton@linaro.org> * benchtests/Makefile: Use LDLIBS instead of LDFLAGS.
2013-09-02 15:00:45 +02:00
LDLIBS-bench-pow = -lm
LDLIBS-bench-rint = -lm
LDLIBS-bench-sin = -lm
LDLIBS-bench-sinh = -lm
Benchmark test for sqrt function.
2013-12-02 18:37:18 +01:00
LDLIBS-bench-sqrt = -lm
benchtests/Makefile: Use LDLIBS instead of LDFLAGS. LDFLAGS puts the library too early in the command line if --as-needed is being used. Use LDLIBS instead. ChangeLog: 2013-09-04 Will Newton <will.newton@linaro.org> * benchtests/Makefile: Use LDLIBS instead of LDFLAGS.
2013-09-02 15:00:45 +02:00
LDLIBS-bench-tan = -lm
LDLIBS-bench-tanh = -lm
Add benchmark inputs for sincos
2013-09-19 13:25:27 +02:00
LDLIBS-bench-sincos = -lm
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
# Rules to build and execute the benchmarks. Do not put any benchmark
# parameters beyond this point.
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
# We don't want the benchmark programs to run in parallel since that could
# affect their performance.
.NOTPARALLEL:
Framework for performance benchmarking of functions See benchtests/Makefile to know how to use it.
2013-03-15 08:00:03 +01:00
include ../Makeconfig
include ../Rules
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
binaries-bench := $(addprefix $(objpfx)bench-,$(bench))
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
binaries-benchset := $(addprefix $(objpfx)bench-,$(benchset))
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
Maintain runtime of each benchmark at ~10 seconds The idea to run benchmarks for a constant number of iterations is problematic. While the benchmarks may run for 10 seconds on x86_64, they could run for about 30 seconds on powerpc and worse, over 3 minutes on arm. Besides that, adding a new benchmark is cumbersome since one needs to find out the number of iterations needed for a sufficient runtime. A better idea would be to run each benchmark for a specific amount of time. This patch does just that. The run time defaults to 10 seconds and it is configurable at command line: make BENCH_DURATION=5 bench
2013-04-30 10:40:20 +02:00
# The default duration: 10 seconds.
ifndef BENCH_DURATION
BENCH_DURATION := 10
endif
Avoid overwriting earlier flags in CPPFLAGS-nonlib in benchtests When setting BENCH_DURATION in CPPFLAGS-nonlib, append to the variable instead of assigning to it, to avoid overwriting earlier set flags, notably the -DNOT_IN_libc=1 flag.
2013-06-10 06:38:46 +02:00
CPPFLAGS-nonlib += -DDURATION=$(BENCH_DURATION)
Maintain runtime of each benchmark at ~10 seconds The idea to run benchmarks for a constant number of iterations is problematic. While the benchmarks may run for 10 seconds on x86_64, they could run for about 30 seconds on powerpc and worse, over 3 minutes on arm. Besides that, adding a new benchmark is cumbersome since one needs to find out the number of iterations needed for a sufficient runtime. A better idea would be to run each benchmark for a specific amount of time. This patch does just that. The run time defaults to 10 seconds and it is configurable at command line: make BENCH_DURATION=5 bench
2013-04-30 10:40:20 +02:00
Use HP_TIMING for benchmarks if available HP_TIMING uses native timestamping instructions if available, thus greatly reducing the overhead of recording start and end times for function calls. For architectures that don't have HP_TIMING available, we fall back to the clock_gettime bits. One may also override this by invoking the benchmark as follows: make USE_CLOCK_GETTIME=1 bench and get the benchmark results using clock_gettime. One has to do `make bench-clean` to ensure that the benchmark programs are rebuilt.
2013-05-13 10:14:32 +02:00
# Use clock_gettime to measure performance of functions. The default is to use
# HP_TIMING if it is available.
ifdef USE_CLOCK_GETTIME
CPPFLAGS-nonlib += -DUSE_CLOCK_GETTIME
endif
Define NOT_IN_libc when compiling benchmark programs
2013-04-16 15:04:03 +02:00
# This makes sure CPPFLAGS-nonlib and CFLAGS-nonlib are passed
# for all these modules.
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
cpp-srcs-left := $(binaries-benchset:=.c) $(binaries-bench:=.c)
Define NOT_IN_libc when compiling benchmark programs
2013-04-16 15:04:03 +02:00
lib := nonlib
include $(patsubst %,$(..)cppflags-iterator.mk,$(cpp-srcs-left))
Use HP_TIMING for benchmarks if available HP_TIMING uses native timestamping instructions if available, thus greatly reducing the overhead of recording start and end times for function calls. For architectures that don't have HP_TIMING available, we fall back to the clock_gettime bits. One may also override this by invoking the benchmark as follows: make USE_CLOCK_GETTIME=1 bench and get the benchmark results using clock_gettime. One has to do `make bench-clean` to ensure that the benchmark programs are rebuilt.
2013-05-13 10:14:32 +02:00
bench-deps := bench-skeleton.c bench-timing.h Makefile
Rebuild benchmark sources when Makefile is updated Benchmark programs are generated using parameters from the Makefile, so it is necessary to rebuild them whenever the parameters in the Makefile are updated. Hence, added a dependency for the generated C source on the Makefile so that it gets regenerated when the Makefile is updated.
2013-04-15 07:47:01 +02:00
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
run-bench = $(test-wrapper-env) \
GCONV_PATH=$(common-objpfx)iconvdata LC_ALL=C \
$($*-ENV) $(rtld-prefix) $${run}
Add target bench-clean
2013-04-16 10:37:21 +02:00
bench-clean:
rm -f $(binaries-bench) $(addsuffix .o,$(binaries-bench))
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
rm -f $(binaries-benchset) $(addsuffix .o,$(binaries-benchset))
bench: bench-set bench-func
bench-set: $(binaries-benchset)
for run in $^; do \
echo "Running $${run}"; \
$(run-bench) > $${run}.out; \
done
Add target bench-clean
2013-04-16 10:37:21 +02:00
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
bench-func: $(binaries-bench)
Write to bench.out-tmp only once Appending benchmark program output on every run could result in a case where the benchmark run was cancelled, resulting in a partially written file. This file gets used again on the next run, resulting in results being appended to old results. It could have been possible to remove the file before every benchmark run, but it is easier to just write the output to bench.out-tmp only once.
2013-04-15 10:22:26 +02:00
{ for run in $^; do \
echo "Running $${run}" >&2; \
$(run-bench); \
done; } > $(objpfx)bench.out-tmp; \
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
if [ -f $(objpfx)bench.out ]; then \
mv -f $(objpfx)bench.out $(objpfx)bench.out.old; \
fi; \
mv -f $(objpfx)bench.out-tmp $(objpfx)bench.out
Begin porting string performance tests to benchtests This is the initial support for string function performance tests, along with copying tests for memcpy and memcpy-ifunc as proof of concept. The string function benchmarks perform operations at different alignments and for different sizes and compare performance between plain operations and the optimized string operations. Due to this their output is incompatible with the function benchmarks where we're interested in fastest time, throughput, etc. In future, the correctness checks in the benchmark tests can be removed. Same goes for the performance measurements in the string/test-*.
2013-04-16 14:07:24 +02:00
$(binaries-bench) $(binaries-benchset): %: %.o \
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
$(sort $(filter $(common-objpfx)lib%,$(link-libc))) \
$(addprefix $(csu-objpfx),start.o) $(+preinit) $(+postinit)
$(+link)
Rebuild benchmark sources when Makefile is updated Benchmark programs are generated using parameters from the Makefile, so it is necessary to rebuild them whenever the parameters in the Makefile are updated. Hence, added a dependency for the generated C source on the Makefile so that it gets regenerated when the Makefile is updated.
2013-04-15 07:47:01 +02:00
$(objpfx)bench-%.c: %-inputs $(bench-deps)
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
{ if [ -n "$($*-INCLUDE)" ]; then \
cat $($*-INCLUDE); \
fi; \
Add more directives to benchmark input files This patch adds some more directives to the benchmark inputs file, moving functionality from the Makefile and making the code generation script a bit cleaner. The function argument and return types that were earlier added as variables in the makefile and passed to the script via command line arguments are now the 'args' and 'ret' directive respectively. 'args' should be a colon separated list of argument types (skipped if the function doesn't accept any arguments) and 'ret' should be the return type. Additionally, an 'includes' directive may have a comma separated list of headers to include in the source. For example, the pow input file now looks like this: 42.0, 42.0 1.0000000000000020, 1.5 I did this to unclutter the benchtests Makefile a bit and eventually eliminate dependency of the tests on the Makefile and have tests depend on their respective include files only.
2013-10-07 08:21:24 +02:00
$(..)scripts/bench.pl $(patsubst %-inputs,%,$<); } > $@-tmp
Move bench target to benchtests The bench target will only be used within the benchtests directory.
2013-04-12 11:31:44 +02:00
mv -f $@-tmp $@