OpenE2K
/
linux
6
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
linux/arch/sparc64/kernel/sun4v_tlb_miss.S

427 lines
10 KiB
ArmAsm
Raw Normal View History

[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
/* sun4v_tlb_miss.S: Sun4v TLB miss handlers.
*
* Copyright (C) 2006 <davem@davemloft.net>
*/
.text
.align 32
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
/* Load ITLB fault information into VADDR and CTX, using BASE. */
#define LOAD_ITLB_INFO(BASE, VADDR, CTX) \
ldx [BASE + HV_FAULT_I_ADDR_OFFSET], VADDR; \
ldx [BASE + HV_FAULT_I_CTX_OFFSET], CTX;
/* Load DTLB fault information into VADDR and CTX, using BASE. */
#define LOAD_DTLB_INFO(BASE, VADDR, CTX) \
ldx [BASE + HV_FAULT_D_ADDR_OFFSET], VADDR; \
ldx [BASE + HV_FAULT_D_CTX_OFFSET], CTX;
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
/* DEST = (VADDR >> 22)
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
*
[SPARC64]: Fix sun4v mna winfixup handling. We were clobbering a base register before we were done using it. Fix a comment typo while we're here. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-03 05:42:53 +01:00
* Branch to ZERO_CTX_LABEL if context is zero.
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
*/
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
#define COMPUTE_TAG_TARGET(DEST, VADDR, CTX, ZERO_CTX_LABEL) \
srlx VADDR, 22, DEST; \
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
brz,pn CTX, ZERO_CTX_LABEL; \
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
nop;
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
/* Create TSB pointer. This is something like:
*
* index_mask = (512 << (tsb_reg & 0x7UL)) - 1UL;
* tsb_base = tsb_reg & ~0x7UL;
[SPARC64]: Add a secondary TSB for hugepage mappings. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 09:49:59 +01:00
* tsb_index = ((vaddr >> HASH_SHIFT) & tsb_mask);
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
* tsb_ptr = tsb_base + (tsb_index * 16);
*/
[SPARC64]: Add a secondary TSB for hugepage mappings. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 09:49:59 +01:00
#define COMPUTE_TSB_PTR(TSB_PTR, VADDR, HASH_SHIFT, TMP1, TMP2) \
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
and TSB_PTR, 0x7, TMP1; \
mov 512, TMP2; \
andn TSB_PTR, 0x7, TSB_PTR; \
sllx TMP2, TMP1, TMP2; \
[SPARC64]: Add a secondary TSB for hugepage mappings. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 09:49:59 +01:00
srlx VADDR, HASH_SHIFT, TMP1; \
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
sub TMP2, 1, TMP2; \
and TMP1, TMP2, TMP1; \
sllx TMP1, 4, TMP1; \
add TSB_PTR, TMP1, TSB_PTR;
sun4v_itlb_miss:
/* Load MMU Miss base into %g2. */
ldxa [%g0] ASI_SCRATCHPAD, %g2
/* Load UTSB reg into %g1. */
mov SCRATCHPAD_UTSBREG1, %g1
ldxa [%g1] ASI_SCRATCHPAD, %g1
LOAD_ITLB_INFO(%g2, %g4, %g5)
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
COMPUTE_TAG_TARGET(%g6, %g4, %g5, kvmap_itlb_4v)
[SPARC64]: Add a secondary TSB for hugepage mappings. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 09:49:59 +01:00
COMPUTE_TSB_PTR(%g1, %g4, PAGE_SHIFT, %g3, %g7)
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
/* Load TSB tag/pte into %g2/%g3 and compare the tag. */
[SPARC64]: Don't use ASI_QUAD_LDD_PHYS on SUN4V. Need to use ASI_QUAD_LDD_PHYS_4V instead. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-16 06:21:17 +01:00
ldda [%g1] ASI_QUAD_LDD_PHYS_4V, %g2
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
cmp %g2, %g6
bne,a,pn %xcc, tsb_miss_page_table_walk
mov FAULT_CODE_ITLB, %g3
[SPARC64]: Fix _PAGE_EXEC handling. First of all, use the known _PAGE_EXEC_{4U,4V} value instead of loading _PAGE_EXEC from memory. We either know which one to use by context, or we can code patch the test. Next, we need to check executability of a PTE in the generic TSB miss handler. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-02 07:42:18 +01:00
andcc %g3, _PAGE_EXEC_4V, %g0
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
be,a,pn %xcc, tsb_do_fault
mov FAULT_CODE_ITLB, %g3
/* We have a valid entry, make hypervisor call to load
* I-TLB and return from trap.
*
* %g3: PTE
* %g4: vaddr
*/
sun4v_itlb_load:
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g6
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
mov %o0, %g1 ! save %o0
mov %o1, %g2 ! save %o1
mov %o2, %g5 ! save %o2
mov %o3, %g7 ! save %o3
mov %g4, %o0 ! vaddr
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldx [%g6 + HV_FAULT_I_CTX_OFFSET], %o1 ! ctx
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
mov %g3, %o2 ! PTE
mov HV_MMU_IMMU, %o3 ! flags
ta HV_MMU_MAP_ADDR_TRAP
[SPARC64]: Fix some SUN4V TLB handling bugs. 1) Add error return checking for TLB load hypervisor calls. 2) Don't fallthru to dtlb tsb miss handler from itlb tsb miss handler, oops. 3) On window fixups, propagate fault information to fixup handler correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-17 23:58:02 +01:00
brnz,pn %o0, sun4v_itlb_error
mov %g2, %o1 ! restore %o1
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
mov %g1, %o0 ! restore %o0
mov %g5, %o2 ! restore %o2
mov %g7, %o3 ! restore %o3
retry
sun4v_dtlb_miss:
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
/* Load MMU Miss base into %g2. */
ldxa [%g0] ASI_SCRATCHPAD, %g2
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
/* Load UTSB reg into %g1. */
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
mov SCRATCHPAD_UTSBREG1, %g1
[SPARC64]: Fix some SUN4V TLB miss bugs. Code patching did not sign extend negative branch offsets correctly. Kernel TLB miss path needs patching and %g4 register preservation in order to handle SUN4V correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 21:21:20 +01:00
ldxa [%g1] ASI_SCRATCHPAD, %g1
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
LOAD_DTLB_INFO(%g2, %g4, %g5)
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
COMPUTE_TAG_TARGET(%g6, %g4, %g5, kvmap_dtlb_4v)
[SPARC64]: Add a secondary TSB for hugepage mappings. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 09:49:59 +01:00
COMPUTE_TSB_PTR(%g1, %g4, PAGE_SHIFT, %g3, %g7)
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
/* Load TSB tag/pte into %g2/%g3 and compare the tag. */
[SPARC64]: Don't use ASI_QUAD_LDD_PHYS on SUN4V. Need to use ASI_QUAD_LDD_PHYS_4V instead. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-16 06:21:17 +01:00
ldda [%g1] ASI_QUAD_LDD_PHYS_4V, %g2
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
cmp %g2, %g6
bne,a,pn %xcc, tsb_miss_page_table_walk
[SPARC64]: Fix typo in SUN4V D-TLB miss handler. Should put FAULT_CODE_DTLB into %g3 not FAULT_CODE_ITLB. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-02 07:27:09 +01:00
mov FAULT_CODE_DTLB, %g3
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
/* We have a valid entry, make hypervisor call to load
* D-TLB and return from trap.
*
* %g3: PTE
* %g4: vaddr
*/
sun4v_dtlb_load:
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g6
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
mov %o0, %g1 ! save %o0
mov %o1, %g2 ! save %o1
mov %o2, %g5 ! save %o2
mov %o3, %g7 ! save %o3
mov %g4, %o0 ! vaddr
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldx [%g6 + HV_FAULT_D_CTX_OFFSET], %o1 ! ctx
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
mov %g3, %o2 ! PTE
mov HV_MMU_DMMU, %o3 ! flags
ta HV_MMU_MAP_ADDR_TRAP
[SPARC64]: Fix some SUN4V TLB handling bugs. 1) Add error return checking for TLB load hypervisor calls. 2) Don't fallthru to dtlb tsb miss handler from itlb tsb miss handler, oops. 3) On window fixups, propagate fault information to fixup handler correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-17 23:58:02 +01:00
brnz,pn %o0, sun4v_dtlb_error
mov %g2, %o1 ! restore %o1
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
mov %g1, %o0 ! restore %o0
mov %g5, %o2 ! restore %o2
mov %g7, %o3 ! restore %o3
retry
sun4v_dtlb_prot:
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
SET_GL(1)
[SPARC64]: Fix unaligned access winfxup handling on SUN4V. Another case where we have to force ourselves into global register level one. Also make sure the arguments passed to sun4v_do_mna() are correct. This area actually needs some more work, for example spill fixup is not necessarily going to do the right thing for this case. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-19 01:39:39 +01:00
/* Load MMU Miss base into %g5. */
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g5
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldx [%g5 + HV_FAULT_D_ADDR_OFFSET], %g5
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
rdpr %tl, %g1
cmp %g1, 1
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
bgu,pn %xcc, winfix_trampoline
mov FAULT_CODE_DTLB | FAULT_CODE_WRITE, %g4
[SPARC64]: Set g4/g5 properly in sun4v dtlb-prot handling. Mirror the logic in the sun4u handler, we have to update both registers even when we branch out to window fault fixup handling. The way it works is that if we are in etrap processing a fault already, g4/g5 holds the original fault information. If we take a window spill fault while doing etrap, then we put the window spill fault info into g4/g5 and this is what the top-level fault handler ends up processing first. Then we retry the originally faulting instruction, and process the original fault at that time. This is all necessary because of how constrained the trap registers are in these code paths. These cases trigger very rarely, so even if there is some performance implication it's doesn't happen very often. In fact the rarity is why it took so long to trigger and find this particular bug. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-01-27 03:48:16 +01:00
ba,pt %xcc, sparc64_realfault_common
nop
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
[SPARC64]: Fix unaligned access winfxup handling on SUN4V. Another case where we have to force ourselves into global register level one. Also make sure the arguments passed to sun4v_do_mna() are correct. This area actually needs some more work, for example spill fixup is not necessarily going to do the right thing for this case. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-19 01:39:39 +01:00
/* Called from trap table:
* %g4: vaddr
* %g5: context
* %g6: TAG TARGET
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
*/
sun4v_itsb_miss:
[SPARC64]: Fix some SUN4V TLB miss bugs. Code patching did not sign extend negative branch offsets correctly. Kernel TLB miss path needs patching and %g4 register preservation in order to handle SUN4V correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 21:21:20 +01:00
mov SCRATCHPAD_UTSBREG1, %g1
ldxa [%g1] ASI_SCRATCHPAD, %g1
brz,pn %g5, kvmap_itlb_4v
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
mov FAULT_CODE_ITLB, %g3
[SPARC64]: Fix some SUN4V TLB handling bugs. 1) Add error return checking for TLB load hypervisor calls. 2) Don't fallthru to dtlb tsb miss handler from itlb tsb miss handler, oops. 3) On window fixups, propagate fault information to fixup handler correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-17 23:58:02 +01:00
ba,a,pt %xcc, sun4v_tsb_miss_common
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
[SPARC64]: Fix unaligned access winfxup handling on SUN4V. Another case where we have to force ourselves into global register level one. Also make sure the arguments passed to sun4v_do_mna() are correct. This area actually needs some more work, for example spill fixup is not necessarily going to do the right thing for this case. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-19 01:39:39 +01:00
/* Called from trap table:
* %g4: vaddr
* %g5: context
* %g6: TAG TARGET
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
*/
sun4v_dtsb_miss:
[SPARC64]: Fix some SUN4V TLB miss bugs. Code patching did not sign extend negative branch offsets correctly. Kernel TLB miss path needs patching and %g4 register preservation in order to handle SUN4V correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 21:21:20 +01:00
mov SCRATCHPAD_UTSBREG1, %g1
ldxa [%g1] ASI_SCRATCHPAD, %g1
brz,pn %g5, kvmap_dtlb_4v
mov FAULT_CODE_DTLB, %g3
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
[SPARC64]: Fix unaligned access winfxup handling on SUN4V. Another case where we have to force ourselves into global register level one. Also make sure the arguments passed to sun4v_do_mna() are correct. This area actually needs some more work, for example spill fixup is not necessarily going to do the right thing for this case. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-19 01:39:39 +01:00
/* fallthrough */
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
sun4v_tsb_miss_common:
[SPARC64]: Add a secondary TSB for hugepage mappings. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 09:49:59 +01:00
COMPUTE_TSB_PTR(%g1, %g4, PAGE_SHIFT, %g5, %g7)
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
sub %g2, TRAP_PER_CPU_FAULT_INFO, %g2
[SPARC64]: Add a secondary TSB for hugepage mappings. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 09:49:59 +01:00
#ifdef CONFIG_HUGETLB_PAGE
mov SCRATCHPAD_UTSBREG2, %g5
ldxa [%g5] ASI_SCRATCHPAD, %g5
cmp %g5, -1
be,pt %xcc, 80f
nop
COMPUTE_TSB_PTR(%g5, %g4, HPAGE_SHIFT, %g2, %g7)
/* That clobbered %g2, reload it. */
ldxa [%g0] ASI_SCRATCHPAD, %g2
sub %g2, TRAP_PER_CPU_FAULT_INFO, %g2
80: stx %g5, [%g2 + TRAP_PER_CPU_TSB_HUGE_TEMP]
#endif
[SPARC64]: Simplify sun4v TLB handling using macros. There was also a bug in sun4v_itlb_miss, it loaded the MMU Fault Status base into %g3 instead of %g2. This pointed out a fast path for TSB miss processing, since we have %g2 with the MMU Fault Status base, we can use that to quickly load up the PGD phys address. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 09:29:34 +01:00
ba,pt %xcc, tsb_miss_page_table_walk_sun4v_fastpath
ldx [%g2 + TRAP_PER_CPU_PGD_PADDR], %g7
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
[SPARC64]: Fix some SUN4V TLB handling bugs. 1) Add error return checking for TLB load hypervisor calls. 2) Don't fallthru to dtlb tsb miss handler from itlb tsb miss handler, oops. 3) On window fixups, propagate fault information to fixup handler correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-17 23:58:02 +01:00
sun4v_itlb_error:
sethi %hi(sun4v_err_itlb_vaddr), %g1
stx %g4, [%g1 + %lo(sun4v_err_itlb_vaddr)]
sethi %hi(sun4v_err_itlb_ctx), %g1
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g6
ldx [%g6 + HV_FAULT_I_CTX_OFFSET], %o1
[SPARC64]: Fix some SUN4V TLB handling bugs. 1) Add error return checking for TLB load hypervisor calls. 2) Don't fallthru to dtlb tsb miss handler from itlb tsb miss handler, oops. 3) On window fixups, propagate fault information to fixup handler correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-17 23:58:02 +01:00
stx %o1, [%g1 + %lo(sun4v_err_itlb_ctx)]
sethi %hi(sun4v_err_itlb_pte), %g1
stx %g3, [%g1 + %lo(sun4v_err_itlb_pte)]
sethi %hi(sun4v_err_itlb_error), %g1
stx %o0, [%g1 + %lo(sun4v_err_itlb_error)]
rdpr %tl, %g4
cmp %g4, 1
ble,pt %icc, 1f
sethi %hi(2f), %g7
ba,pt %xcc, etraptl1
or %g7, %lo(2f), %g7
1: ba,pt %xcc, etrap
2: or %g7, %lo(2b), %g7
call sun4v_itlb_error_report
add %sp, PTREGS_OFF, %o0
/* NOTREACHED */
sun4v_dtlb_error:
sethi %hi(sun4v_err_dtlb_vaddr), %g1
stx %g4, [%g1 + %lo(sun4v_err_dtlb_vaddr)]
sethi %hi(sun4v_err_dtlb_ctx), %g1
[SPARC64]: More TLB/TSB handling fixes. The SUN4V convention with non-shared TSBs is that the context bit of the TAG is clear. So we have to choose an "invalid" bit and initialize new TSBs appropriately. Otherwise a zero TAG looks "valid". Make sure, for the window fixup cases, that we use the right global registers and that we don't potentially trample on the live global registers in etrap/rtrap handling (%g2 and %g6) and that we put the missing virtual address properly in %g5. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-18 03:01:02 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g6
ldx [%g6 + HV_FAULT_D_CTX_OFFSET], %o1
[SPARC64]: Fix some SUN4V TLB handling bugs. 1) Add error return checking for TLB load hypervisor calls. 2) Don't fallthru to dtlb tsb miss handler from itlb tsb miss handler, oops. 3) On window fixups, propagate fault information to fixup handler correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-17 23:58:02 +01:00
stx %o1, [%g1 + %lo(sun4v_err_dtlb_ctx)]
sethi %hi(sun4v_err_dtlb_pte), %g1
stx %g3, [%g1 + %lo(sun4v_err_dtlb_pte)]
sethi %hi(sun4v_err_dtlb_error), %g1
stx %o0, [%g1 + %lo(sun4v_err_dtlb_error)]
rdpr %tl, %g4
cmp %g4, 1
ble,pt %icc, 1f
sethi %hi(2f), %g7
ba,pt %xcc, etraptl1
or %g7, %lo(2f), %g7
1: ba,pt %xcc, etrap
2: or %g7, %lo(2b), %g7
call sun4v_dtlb_error_report
add %sp, PTREGS_OFF, %o0
/* NOTREACHED */
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
/* Instruction Access Exception, tl0. */
sun4v_iacc:
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g2
ldx [%g2 + HV_FAULT_I_TYPE_OFFSET], %g3
ldx [%g2 + HV_FAULT_I_ADDR_OFFSET], %g4
ldx [%g2 + HV_FAULT_I_CTX_OFFSET], %g5
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
sllx %g3, 16, %g3
or %g5, %g3, %g5
ba,pt %xcc, etrap
rd %pc, %g7
mov %l4, %o1
mov %l5, %o2
call sun4v_insn_access_exception
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
/* Instruction Access Exception, tl1. */
sun4v_iacc_tl1:
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g2
ldx [%g2 + HV_FAULT_I_TYPE_OFFSET], %g3
ldx [%g2 + HV_FAULT_I_ADDR_OFFSET], %g4
ldx [%g2 + HV_FAULT_I_CTX_OFFSET], %g5
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
sllx %g3, 16, %g3
or %g5, %g3, %g5
ba,pt %xcc, etraptl1
rd %pc, %g7
mov %l4, %o1
mov %l5, %o2
call sun4v_insn_access_exception_tl1
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
/* Data Access Exception, tl0. */
sun4v_dacc:
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g2
ldx [%g2 + HV_FAULT_D_TYPE_OFFSET], %g3
ldx [%g2 + HV_FAULT_D_ADDR_OFFSET], %g4
ldx [%g2 + HV_FAULT_D_CTX_OFFSET], %g5
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
sllx %g3, 16, %g3
or %g5, %g3, %g5
ba,pt %xcc, etrap
rd %pc, %g7
mov %l4, %o1
mov %l5, %o2
call sun4v_data_access_exception
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
/* Data Access Exception, tl1. */
sun4v_dacc_tl1:
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g2
ldx [%g2 + HV_FAULT_D_TYPE_OFFSET], %g3
ldx [%g2 + HV_FAULT_D_ADDR_OFFSET], %g4
ldx [%g2 + HV_FAULT_D_CTX_OFFSET], %g5
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
sllx %g3, 16, %g3
or %g5, %g3, %g5
ba,pt %xcc, etraptl1
rd %pc, %g7
mov %l4, %o1
mov %l5, %o2
call sun4v_data_access_exception_tl1
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
/* Memory Address Unaligned. */
sun4v_mna:
[SPARC64]: Fix unaligned access winfxup handling on SUN4V. Another case where we have to force ourselves into global register level one. Also make sure the arguments passed to sun4v_do_mna() are correct. This area actually needs some more work, for example spill fixup is not necessarily going to do the right thing for this case. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-19 01:39:39 +01:00
/* Window fixup? */
rdpr %tl, %g2
cmp %g2, 1
ble,pt %icc, 1f
nop
SET_GL(1)
[SPARC64]: Fix sun4v mna winfixup handling. We were clobbering a base register before we were done using it. Fix a comment typo while we're here. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-03 05:42:53 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g2
ldx [%g2 + HV_FAULT_D_ADDR_OFFSET], %g5
[SPARC64]: Fix unaligned access winfxup handling on SUN4V. Another case where we have to force ourselves into global register level one. Also make sure the arguments passed to sun4v_do_mna() are correct. This area actually needs some more work, for example spill fixup is not necessarily going to do the right thing for this case. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-19 01:39:39 +01:00
mov HV_FAULT_TYPE_UNALIGNED, %g3
[SPARC64]: Fix sun4v mna winfixup handling. We were clobbering a base register before we were done using it. Fix a comment typo while we're here. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-03 05:42:53 +01:00
ldx [%g2 + HV_FAULT_D_CTX_OFFSET], %g4
[SPARC64]: Fix unaligned access winfxup handling on SUN4V. Another case where we have to force ourselves into global register level one. Also make sure the arguments passed to sun4v_do_mna() are correct. This area actually needs some more work, for example spill fixup is not necessarily going to do the right thing for this case. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-19 01:39:39 +01:00
sllx %g3, 16, %g3
or %g4, %g3, %g4
ba,pt %xcc, winfix_mna
rdpr %tpc, %g3
/* not reached */
1: ldxa [%g0] ASI_SCRATCHPAD, %g2
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
mov HV_FAULT_TYPE_UNALIGNED, %g3
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
ldx [%g2 + HV_FAULT_D_ADDR_OFFSET], %g4
ldx [%g2 + HV_FAULT_D_CTX_OFFSET], %g5
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
sllx %g3, 16, %g3
or %g5, %g3, %g5
ba,pt %xcc, etrap
rd %pc, %g7
mov %l4, %o1
mov %l5, %o2
[SPARC64]: Fix C-function name called by sun4v_mna trap code. The trap code was calling itself :-) Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-15 01:39:22 +01:00
call sun4v_do_mna
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
/* Privileged Action. */
sun4v_privact:
ba,pt %xcc, etrap
rd %pc, %g7
call do_privact
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
/* Unaligned ldd float, tl0. */
sun4v_lddfmna:
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g2
ldx [%g2 + HV_FAULT_D_TYPE_OFFSET], %g3
ldx [%g2 + HV_FAULT_D_ADDR_OFFSET], %g4
ldx [%g2 + HV_FAULT_D_CTX_OFFSET], %g5
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
sllx %g3, 16, %g3
or %g5, %g3, %g5
ba,pt %xcc, etrap
rd %pc, %g7
mov %l4, %o1
mov %l5, %o2
call handle_lddfmna
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
/* Unaligned std float, tl0. */
sun4v_stdfmna:
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 00:39:51 +01:00
ldxa [%g0] ASI_SCRATCHPAD, %g2
ldx [%g2 + HV_FAULT_D_TYPE_OFFSET], %g3
ldx [%g2 + HV_FAULT_D_ADDR_OFFSET], %g4
ldx [%g2 + HV_FAULT_D_CTX_OFFSET], %g5
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
sllx %g3, 16, %g3
or %g5, %g3, %g5
ba,pt %xcc, etrap
rd %pc, %g7
mov %l4, %o1
mov %l5, %o2
call handle_stdfmna
add %sp, PTREGS_OFF, %o0
ba,a,pt %xcc, rtrap_clr_l6
[SPARC64]: Implement sun4v TSB miss handlers. When we register a TSB with the hypervisor, so that it or hardware can handle TLB misses and do the TSB walk for us, the hypervisor traps down to these trap when it incurs a TSB miss. Processing is simple, we load the missing virtual address and context, and do a full page table walk. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 01:12:22 +01:00
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
#define BRANCH_ALWAYS 0x10680000
#define NOP 0x01000000
#define SUN4V_DO_PATCH(OLD, NEW) \
sethi %hi(NEW), %g1; \
or %g1, %lo(NEW), %g1; \
sethi %hi(OLD), %g2; \
or %g2, %lo(OLD), %g2; \
sub %g1, %g2, %g1; \
sethi %hi(BRANCH_ALWAYS), %g3; \
[SPARC64]: Fix some SUN4V TLB miss bugs. Code patching did not sign extend negative branch offsets correctly. Kernel TLB miss path needs patching and %g4 register preservation in order to handle SUN4V correctly. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 21:21:20 +01:00
sll %g1, 11, %g1; \
srl %g1, 11 + 2, %g1; \
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
or %g3, %lo(BRANCH_ALWAYS), %g3; \
or %g3, %g1, %g3; \
stw %g3, [%g2]; \
sethi %hi(NOP), %g3; \
or %g3, %lo(NOP), %g3; \
stw %g3, [%g2 + 0x4]; \
flush %g2;
.globl sun4v_patch_tlb_handlers
.type sun4v_patch_tlb_handlers,#function
sun4v_patch_tlb_handlers:
SUN4V_DO_PATCH(tl0_iamiss, sun4v_itlb_miss)
SUN4V_DO_PATCH(tl1_iamiss, sun4v_itlb_miss)
SUN4V_DO_PATCH(tl0_damiss, sun4v_dtlb_miss)
SUN4V_DO_PATCH(tl1_damiss, sun4v_dtlb_miss)
SUN4V_DO_PATCH(tl0_daprot, sun4v_dtlb_prot)
SUN4V_DO_PATCH(tl1_daprot, sun4v_dtlb_prot)
[SPARC64]: SUN4V memory exception trap handlers. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-10 05:20:34 +01:00
SUN4V_DO_PATCH(tl0_iax, sun4v_iacc)
SUN4V_DO_PATCH(tl1_iax, sun4v_iacc_tl1)
SUN4V_DO_PATCH(tl0_dax, sun4v_dacc)
SUN4V_DO_PATCH(tl1_dax, sun4v_dacc_tl1)
SUN4V_DO_PATCH(tl0_mna, sun4v_mna)
SUN4V_DO_PATCH(tl1_mna, sun4v_mna)
SUN4V_DO_PATCH(tl0_lddfmna, sun4v_lddfmna)
SUN4V_DO_PATCH(tl0_stdfmna, sun4v_stdfmna)
SUN4V_DO_PATCH(tl0_privact, sun4v_privact)
[SPARC64]: Initial sun4v TLB miss handling infrastructure. Things are a little tricky because, unlike sun4u, we have to: 1) do a hypervisor trap to do the TLB load. 2) do the TSB lookup calculations by hand Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 08:44:37 +01:00
retl
nop
.size sun4v_patch_tlb_handlers,.-sun4v_patch_tlb_handlers