OpenE2K
/
qemu-e2k
13
4
Fork 0
Code Issues 4 Pull Requests Projects 2 Releases Activity

tests/qtest/npcm7xx_rng-test: Disable randomness tests 

The randomness tests in the NPCM7xx RNG test fail intermittently
but fairly frequently. On my machine running the test in a loop:
 while QTEST_QEMU_BINARY=./qemu-system-aarch64 ./tests/qtest/npcm7xx_rng-test; do true; done

will fail in less than a minute with an error like:
ERROR:../../tests/qtest/npcm7xx_rng-test.c:256:test_first_byte_runs:
assertion failed (calc_runs_p(buf.l, sizeof(buf) * BITS_PER_BYTE) > 0.01): (0.00286205989 > 0.01)

(Failures have been observed on all 4 of the randomness tests,
not just first_byte_runs.)

It's not clear why these tests are failing like this, but intermittent
failures make CI and merge testing awkward, so disable running them
unless a developer specifically sets QEMU_TEST_FLAKY_RNG_TESTS when
running the test suite, until we work out the cause.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20201102152454.8287-1-peter.maydell@linaro.org
Reviewed-by: Havard Skinnemoen <hskinnemoen@google.com>
This commit is contained in:
Peter Maydell 2020-11-02 16:52:18 +00:00
parent 5a4b0d4112
commit ffb4fbf90a
1 changed files with 10 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
tests/qtest/npcm7xx_rng-test.c
View File

@ -265,10 +265,16 @@ int main(int argc, char **argv)
qtest_add_func("npcm7xx_rng/enable_disable", test_enable_disable);
qtest_add_func("npcm7xx_rng/rosel", test_rosel);
qtest_add_func("npcm7xx_rng/continuous/monobit", test_continuous_monobit);
qtest_add_func("npcm7xx_rng/continuous/runs", test_continuous_runs);
qtest_add_func("npcm7xx_rng/first_byte/monobit", test_first_byte_monobit);
qtest_add_func("npcm7xx_rng/first_byte/runs", test_first_byte_runs);
/*
* These tests fail intermittently; only run them on explicit
* request until we figure out why.
*/
if (getenv("QEMU_TEST_FLAKY_RNG_TESTS")) {
qtest_add_func("npcm7xx_rng/continuous/monobit", test_continuous_monobit);
qtest_add_func("npcm7xx_rng/continuous/runs", test_continuous_runs);
qtest_add_func("npcm7xx_rng/first_byte/monobit", test_first_byte_monobit);
qtest_add_func("npcm7xx_rng/first_byte/runs", test_first_byte_runs);
}
qtest_start("-machine npcm750-evb");
ret = g_test_run();