Commit Graph

106739 Commits

Author SHA1 Message Date
Richard Henderson
9358fbbf6e tcg: Add tcg_out_tb_start backend hook
This hook may emit code at the beginning of the TB.

Suggested-by: Jordan Niethe <jniethe5@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:16 +00:00
Richard Henderson
722460652b fpu: Handle m68k extended precision denormals properly
Motorola treats denormals with explicit integer bit set as
having unbiased exponent 0, unlike Intel which treats it as
having unbiased exponent 1 (more like all other IEEE formats
that have no explicit integer bit).

Add a flag on FloatFmt to differentiate the behaviour.

Reported-by: Keith Packard <keithp@keithp.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:16 +00:00
LIU Zhiwei
00f9ef8f3d fpu: Add conversions between bfloat16 and [u]int8
We missed these functions when upstreaming the bfloat16 support.

Signed-off-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com>
Message-Id: <20230531065458.2082-1-zhiwei_liu@linux.alibaba.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
1f9823cea2 accel/tcg: Introduce do_st16_mmio_leN
Split out int_st_mmio_leN, to be used by both do_st_mmio_leN
and do_st16_mmio_leN.  Move the locks down into the two
functions, since each one now covers all accesses to once page.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
8bf6726741 accel/tcg: Introduce do_ld16_mmio_beN
Split out int_ld_mmio_beN, to be used by both do_ld_mmio_beN
and do_ld16_mmio_beN.  Move the locks down into the two
functions, since each one now covers all accesses to once page.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
5646d6a70f accel/tcg: Merge io_writex into do_st_mmio_leN
Avoid multiple calls to io_prepare for unaligned acceses.
One call to do_st_mmio_leN will never cross pages.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
13e617475d accel/tcg: Merge io_readx into do_ld_mmio_beN
Avoid multiple calls to io_prepare for unaligned acceses.
One call to do_ld_mmio_beN will never cross pages.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
d89c64f6fd accel/tcg: Replace direct use of io_readx/io_writex in do_{ld,st}_1
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
bef0c21678 accel/tcg: Merge cpu_transaction_failed into io_failed
Push computation down into the if statements to the point
the data is used.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
405c02d85d plugin: Simplify struct qemu_plugin_hwaddr
Rather than saving MemoryRegionSection and offset,
save phys_addr and MemoryRegion.  This matches up
much closer with the plugin api.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
0e1144400f accel/tcg: Use CPUTLBEntryFull.phys_addr in io_failed
Since the introduction of CPUTLBEntryFull, we can recover
the full cpu address space physical address without having
to examine the MemoryRegionSection.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
fb3cb376e9 accel/tcg: Split out io_prepare and io_failed
These are common code from io_readx and io_writex.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
da6aef48d9 accel/tcg: Simplify tlb_plugin_lookup
Now that we defer address space update and tlb_flush until
the next async_run_on_cpu, the plugin run at the end of the
instruction no longer has to contend with a flushed tlb.
Therefore, delete SavedIOTLB entirely.

Properly return false from tlb_plugin_lookup when we do
not have a tlb match.

Fixes a bug in which SavedIOTLB had stale data, because
there were multiple i/o accesses within a single insn.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
e8967b6152 target/arm: Use tcg_gen_gvec_cmpi for compare vs 0
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20230831030904.1194667-3-richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Richard Henderson
9622c697d1 tcg: Add gvec compare with immediate and scalar operand
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20230831030904.1194667-2-richard.henderson@linaro.org>
2023-09-16 14:57:15 +00:00
Jiajie Chen
58f8961285 tcg/loongarch64: Implement 128-bit load & store
If LSX is available, use LSX instructions to implement 128-bit load &
store when MO_128 is required, otherwise use two 64-bit loads & stores.

Signed-off-by: Jiajie Chen <c@jia.je>
Message-Id: <20230908022302.180442-17-c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-16 14:57:10 +00:00
Richard Henderson
055c99015a host/include/aarch64: Implement clmul.h
Detect PMULL in cpuinfo; implement the accel hook.

Acked-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
d6493dbb46 host/include/i386: Implement clmul.h
Detect PCLMUL in cpuinfo; implement the accel hook.

Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
7bdbf233d9 target/ppc: Use clmul_64
Use generic routine for 64-bit carry-less multiply.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
ef73fe7cf1 target/s390x: Use clmul_64
Use the generic routine for 64-bit carry-less multiply.
Remove our local version of galois_multiply64.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
7fcb505455 target/i386: Use clmul_64
Use generic routine for 64-bit carry-less multiply.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
a50cfdf0be target/arm: Use clmul_64
Use generic routine for 64-bit carry-less multiply.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
00f463b38a crypto: Add generic 64-bit carry-less multiply routine
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
f56d3c1a14 target/ppc: Use clmul_32* routines
Use generic routines for 32-bit carry-less multiply.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
653aab27ba target/s390x: Use clmul_32* routines
Use generic routines for 32-bit carry-less multiply.
Remove our local version of galois_multiply32.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
bae25f648e target/arm: Use clmul_32* routines
Use generic routines for 32-bit carry-less multiply.
Remove our local version of pmull_d.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
9a65a570fa crypto: Add generic 32-bit carry-less multiply routines
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
a2c67342ee target/ppc: Use clmul_16* routines
Use generic routines for 16-bit carry-less multiply.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
25c304e936 target/s390x: Use clmul_16* routines
Use generic routines for 16-bit carry-less multiply.
Remove our local version of galois_multiply16.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
c6f0dcb1fd target/arm: Use clmul_16* routines
Use generic routines for 16-bit carry-less multiply.
Remove our local version of pmull_w.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
cf1b2cab83 crypto: Add generic 16-bit carry-less multiply routines
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
cec4090d94 target/ppc: Use clmul_8* routines
Use generic routines for 8-bit carry-less multiply.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
2d8bc6815e target/s390x: Use clmul_8* routines
Use generic routines for 8-bit carry-less multiply.
Remove our local version of galois_multiply8.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:57:00 +00:00
Richard Henderson
8e3da4c716 target/arm: Use clmul_8* routines
Use generic routines for 8-bit carry-less multiply.
Remove our local version of pmull_h.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:56:59 +00:00
Richard Henderson
07f348d77c crypto: Add generic 8-bit carry-less multiply routines
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 13:56:59 +00:00
Jiajie Chen
561b001aef tcg/loongarch64: Lower rotli_vec to vrotri
Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-16-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
0765cce114 tcg/loongarch64: Lower rotv_vec ops to LSX
Lower the following ops:

- rotrv_vec
- rotlv_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-15-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
2931527b4d tcg/loongarch64: Lower vector shift integer ops
Lower the following ops:

- shli_vec
- shrv_vec
- sarv_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-14-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
c8b859b45e tcg/loongarch64: Lower bitsel_vec to vbitsel
Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-13-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
94304d7b3d tcg/loongarch64: Lower vector shift vector ops
Lower the following ops:

- shlv_vec
- shrv_vec
- sarv_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-12-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
5256ea1176 tcg/loongarch64: Lower vector saturated ops
Lower the following ops:

- ssadd_vec
- usadd_vec
- sssub_vec
- ussub_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-11-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
b2f84adc00 tcg/loongarch64: Lower vector min max ops
Lower the following ops:

- smin_vec
- smax_vec
- umin_vec
- umax_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-10-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
76d20c205d tcg/loongarch64: Lower mul_vec to vmul
Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-9-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
7d577c3ecd tcg/loongarch64: Lower neg_vec to vneg
Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-8-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
24c42fde52 tcg/loongarch64: Lower vector bitwise operations
Lower the following ops:

- and_vec
- andc_vec
- or_vec
- orc_vec
- xor_vec
- nor_vec
- not_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-7-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
e9d7c8cf95 tcg/loongarch64: Lower add/sub_vec to vadd/vsub
Lower the following ops:

- add_vec
- sub_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-6-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
d8b6fa593d tcg/loongarch64: Lower cmp_vec to vseq/vsle/vslt
Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-5-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
ebe92db2cc tcg: pass vece to tcg_target_const_match()
Pass vece to tcg_target_const_match() to allow correct interpretation of
const args of vector ops.

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230908022302.180442-4-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:51 -07:00
Jiajie Chen
16288ded94 tcg/loongarch64: Lower basic tcg vec ops to LSX
LSX support on host cpu is detected via hwcap.

Lower the following ops to LSX:

- dup_vec
- dupi_vec
- dupm_vec
- ld_vec
- st_vec

Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-3-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:50 -07:00
Jiajie Chen
af88a28414 tcg/loongarch64: Import LSX instructions
Add opcodes and encoder functions for LSX.

Generated from
https://github.com/jiegec/loongarch-opcodes/tree/qemu-lsx.

Signed-off-by: Jiajie Chen <c@jia.je>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230908022302.180442-2-c@jia.je>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-15 05:26:50 -07:00