Commit Graph

90 Commits

Author SHA1 Message Date
Ralf Baechle d9cdc901af MIPS: cache: Provide cache flush operations for XFS
Until now flush_kernel_vmap_range() and invalidate_kernel_vmap_range() did
not exist on MIPS resulting in heavy cache corruption on XFS filesystems.

Left for the post-3.0 time: optimization and make this work with highmem,
too.  Since the combination of highmem + cache aliases atm doesn't work
this isn't a regression.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Patchwork: https://patchwork.linux-mips.org/patch/2505/
2011-10-20 15:00:18 +01:00
Kevin Cernekee d0023c4a0a MIPS: Add SYNC after cacheflush
On processors with deep write buffers, it is likely that many cycles
will pass between a CACHE instruction and the time the data actually
gets written out to DRAM.  Add a SYNC instruction to ensure that the
buffers get emptied before the flush functions return.

Actual problem seen in the wild:

1) dma_alloc_coherent() allocates cached memory

2) memset() is called to clear the new pages

3) dma_cache_wback_inv() is called to flush the zero data out to memory

4) dma_alloc_coherent() returns an uncached (kseg1) pointer to the
freshly allocated pages

5) Caller writes data through the kseg1 pointer

6) Buffered writeback data finally gets flushed out to DRAM

7) Part of caller's data is inexplicably zeroed out

This patch adds SYNC between steps 3 and 4, which fixed the problem.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: 
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2011-07-25 17:26:53 +01:00
Linus Torvalds 57d19e80f4 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (39 commits)
  b43: fix comment typo reqest -> request
  Haavard Skinnemoen has left Atmel
  cris: typo in mach-fs Makefile
  Kconfig: fix copy/paste-ism for dell-wmi-aio driver
  doc: timers-howto: fix a typo ("unsgined")
  perf: Only include annotate.h once in tools/perf/util/ui/browsers/annotate.c
  md, raid5: Fix spelling error in comment ('Ofcourse' --> 'Of course').
  treewide: fix a few typos in comments
  regulator: change debug statement be consistent with the style of the rest
  Revert "arm: mach-u300/gpio: Fix mem_region resource size miscalculations"
  audit: acquire creds selectively to reduce atomic op overhead
  rtlwifi: don't touch with treewide double semicolon removal
  treewide: cleanup continuations and remove logging message whitespace
  ath9k_hw: don't touch with treewide double semicolon removal
  include/linux/leds-regulator.h: fix syntax in example code
  tty: fix typo in descripton of tty_termios_encode_baud_rate
  xtensa: remove obsolete BKL kernel option from defconfig
  m68k: fix comment typo 'occcured'
  arch:Kconfig.locks Remove unused config option.
  treewide: remove extra semicolons
  ...
2011-05-23 09:12:26 -07:00
Jayachandran C efa0f81c11 MIPS: Netlogic: Cache, TLB support and feature overrides for XLR
CPU_XLR case added to mm/tlbex.c
CPU_XLR case added to mm/c-r4k.c for PINDEX attribute
Feature overrides for XLR cpu.

Signed-off-by: Jayachandran C <jayachandranc@netlogicmicro.com>
To: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/2333/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2011-05-19 09:55:40 +01:00
Ralf Baechle 71271aab8c MIPS: c-r4k: Fix GCC 4.6.0 build error
CC      arch/mips/mm/c-r4k.o
arch/mips/mm/c-r4k.c: In function 'probe_scache':
arch/mips/mm/c-r4k.c:1078:6: error: variable 'tmp' set but not used [-Werror=unused-but-set-variable]
cc1: all warnings being treated as errors

Older GCC versions didn't warn about the unused variable tmp because it was
getting initialized.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2011-05-10 18:15:22 +01:00
Justin P. Mattock 79add62773 update David Miller's old email address
Signed-off-by: Justin P. Mattock <justinmattock@gmail.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2011-04-06 06:19:38 -07:00
Ralf Baechle 48a26e60c6 MIPS: Remove wait argument of r4k_on_each_cpu
All callers were passing in 1 anyway.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2010-10-29 19:08:25 +01:00
Ralf Baechle c8c5f3fd9f MIPS: More detailed description of r4k_on_each_cpu
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2010-10-29 19:08:25 +01:00
Ralf Baechle 631330f584 MIPS: Build fix - include <linux/smp.h> into all smp_processor_id() users.
Some of the were relying into smp.h being dragged in by another header
which of course is fragile.  <asm/cpu-info.h> uses smp_processor_id()
only in macros and including smp.h there leads to an include loop, so
don't change cpu-info.h.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-06-24 18:34:39 +01:00
Kevin Cernekee 605b7ef7b7 MIPS: Support 64-byte D-cache line size
Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-06-17 11:06:24 +01:00
Ralf Baechle 7fc7316aa8 MIPS: Print the actual detected I-cache associativity on bootup.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-05-14 13:50:25 +01:00
Manuel Lauss 270717a8a0 MIPS: Alchemy: unify CPU model constants.
This patch removes the various CPU_AU1??? model constants in favor of
a single CPU_ALCHEMY one.

All currently existing Alchemy models are identical in terms of cpu
core and cache size/organization.  The parts of the mips kernel which
need to know the exact CPU revision extract it from the c0_prid register
already; and finally nothing else in-tree depends on those any more.

Should a new variant with slightly different "company options" and/or
"processor revision" bits in c0_prid appear, it will be supported
immediately (minus an exact model string in cpuinfo).

Signed-off-by: Manuel Lauss <mano@roarinelk.homelinux.net>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-03-30 14:49:45 +02:00
Shinya Kuribayashi 5864810bc5 MIPS: VR5500: Enable prefetch
Signed-off-by: Shinya Kuribayashi <shinya.kuribayashi@necel.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-03-23 23:38:04 +01:00
Ralf Baechle a8ca8b64e3 MIPS: Avoid destructive invalidation on partial cachelines.
See discussion e9c3a7c20901051031y528d0d31r18d44c5096c59e0@mail.gmail.com.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-01-30 21:32:57 +00:00
Thomas Bogendoerfer e0cee3eea7 [MIPS] Fix WARNING: at kernel/smp.c:290
trap_init issues flush_icache_range(), which uses ipi functions to
get icache flushing done on all cpus. But this is done before interrupts
are enabled and caused WARN_ON messages. This changeset introduces
a new local_flush_icache_range() and uses it before interrupts (and
additional CPUs) are enabled to avoid this problem.

Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-09-05 21:24:11 +01:00
Jens Axboe 8691e5a8f6 smp_call_function: get rid of the unused nonatomic/retry argument
It's never used and the comments refer to nonatomic and retry
interchangably. So get rid of it.

Acked-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-06-26 11:24:35 +02:00
Ralf Baechle c9c5023d83 [MIPS] Fix buggy use of kmap_coherent.
Assuming the call of kmap_coherent in local_r4k_flush_cache_page doesn't
need fixing this was skipped in fcae549295bcae801ac48fc1c2030ab8cc487020.
Turns out it needed the same change after all.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-06-16 15:14:48 +01:00
Ralf Baechle 89052bd7b3 [MIPS] Fix build for PNX platforms.
Build error was caused by commit 351336929c.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-06-16 15:14:47 +01:00
Ralf Baechle 39b8d52542 [MIPS] Add support for MIPS CMP platform.
Signed-off-by: Chris Dearman <chris@mips.com>
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-04-28 17:14:26 +01:00
Chris Dearman 351336929c [MIPS] Allow setting of the cache attribute at run time.
Slightly tacky, but there is a precedent in the sparc archirecture code.

Signed-off-by: Chris Dearman <chris@mips.com>
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-04-28 17:14:25 +01:00
Ralf Baechle 9c5a3d729c [MIPS] Handle aliases in vmalloc correctly.
flush_cache_vmap / flush_cache_vunmap were calling flush_cache_all which -
having been deprecated - turned into a nop ...

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-04-07 22:31:04 +01:00
Ralf Baechle 234fcd1484 [MIPS] Fix loads of section missmatches
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-03-12 14:14:41 +00:00
Ralf Baechle 2eaa7ec286 [MIPS] Handle I-cache coherency in flush_cache_range()
So far flush_cache_range() did't consider the I-cache largely because it
did rarely ever matter to real world code.  This was working primarily
because normally code and data are don't share the same pages - with the
exception of MIPS16 code which uses address constants embedded between
the code.   The following sequence of events may break the code:

 o MIPS16 executable being loaded
 o dynamic linker relocates the address constants embedded into the code:
 o   Uses mprotect(2) to make code pages PROT_READ|PROT_WRITE
 o   Performs the actual relocations by writing to the pages which likely
     are COW.  Because no PROT_EXEC is set I-cache coherence will not be
     considered.
 o   Uses mprotect(2) to switch code pages back to PROT_READ|PROT_EXEC.
     This results in a call to flush_cache_range() which also does not
     consider I-caches.
 o => executing the page just having been relocated may now result in the
   I-cache getting refilled with stale data from memory.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-02-19 17:01:31 +00:00
Joe Perches 603e82edf7 arch/mips/: Spelling fixes
Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Adrian Bunk <bunk@kernel.org>
2008-02-03 16:54:53 +02:00
Manuel Lauss 237cfee1db [MIPS] Alchemy: Au1210/Au1250 CPU support
This patch adds IDs for new Au1200 variants: Au1210 and Au1250.
They are essentially identical to the Au1200 except for the Au1210
which has a different SoC-ID in the PRId register [bits 31:24].
The Au1250 is a "Au1200 V0.2".

Signed-off-by: Manuel Lauss <mano@roarinelk.homelinux.net>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-01-29 10:14:59 +00:00
Thomas Bogendoerfer e9c33572a9 [MIPS] Use real cache invalidate
R10k non coherent machines need a real dma cache invalidate to get rid of
speculative stores in cache.  For other machines this promises a slight
speedup.

Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-01-29 10:14:57 +00:00
Ralf Baechle 33202349ef [MIPS] Remove useless S-cache flushes.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-01-29 10:14:56 +00:00
Ralf Baechle 526af35e5d [MIPS] Use pte_present instead of open coded test for _PAGE_PRESENT.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-01-29 10:14:55 +00:00
Ralf Baechle a754f70886 [MIPS] Sibyte: resurrect old cache hack.
The recent switch of the Sibyte SOCs from the processor specific cache
managment code in c-sb1.c to c-r4k.c lost this old hack

    [MIPS] Hack for SB1 cache issues

    Removing flush_icache_page a while ago broke SB1 which was using an empty
    flush_data_cache_page function.  This glues things well enough so a more
    efficient but also more intrusive solution can be found later.

    Signed-Off-By: Thiemo Seufer <ths@networkno.de>
    Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

in the hope it was no longer needed.  As it turns it still is so resurrect
it until there is a better solution.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-11-15 23:21:48 +00:00
Ralf Baechle a76ab5c10d [MIPS] MT: Fix bug in multithreaded kernels.
When GDB writes a breakpoint into address area of inferior process the
kernel needs to invalidate the modified memory in the inferior which
is done by calling flush_cache_page which in turns calls
r4k_flush_cache_page and local_r4k_flush_cache_page for VSMP or SMTC
kernel via r4k_on_each_cpu().

As the VSMP and SMTC SMP kernels for 34K are running on a single shared
caches it is possible to get away without interprocessor function calls.
This optimization is implemented in r4k_on_each_cpu, so
local_r4k_flush_cache_page is only ever called on the local CPU.

This is where the following code in local_r4k_flush_cache_page() strikes:

        /*
         * If ownes no valid ASID yet, cannot possibly have gotten
         * this page into the cache.
         */
        if (cpu_context(smp_processor_id(), mm) == 0)
                return;

On VSMP and SMTC had a function of cpu_context() for each CPU(TC).

So in case another CPU than the CPU executing local_r4k_cache_flush_page
has not accessed the mm but one of the other CPUs has there may be data
to be flushed in the cache yet local_r4k_cache_flush_page will falsely
return leaving the I-cache inconsistent for the breakpoint.

While the issue was discovered with GDB it also exists in
local_r4k_flush_cache_range() and local_r4k_flush_cache().

Fixed by introducing a new function has_valid_asid which on MT kernels
returns true if a mm is active on any processor in the system.

This is relativly expensive since for memory acccesses in that loop
cache misses have to be assumed but it seems the most viable solution
for 2.6.23 and older -stable kernels.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-10-29 19:35:37 +00:00
Ralf Baechle 64bfca5cd8 [MIPS] Cache: Provide more information on cache policy on bootup.
This should help making bug reports for the gadzillion of cores with all
their configuration and synthesis options more useful.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-10-16 18:23:49 +01:00
Ralf Baechle 21a151d8ca [MIPS] checkfiles: Fix "need space after that ','" errors.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-10-11 23:46:15 +01:00
Ralf Baechle 10cc352907 [MIPS] Allow hardwiring of the CPU type to a single type for optimization.
This saves a few k on systems which only ever ship with a single CPU type.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-10-11 23:46:15 +01:00
Ralf Baechle db813fe5a7 [MIPS] Avoid indexed cacheops.
On MP configurations it's highly dubious what this code will actually
affect since blasting away cachelines may or may not do the right
thing wrt. cache coherency.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-10-11 23:46:12 +01:00
Ralf Baechle 641e97f318 [MIPS] Sibyte: Replace SB1 cachecode with standard R4000 class cache code.
It may not be perfect yet but the SB1 code is badly borken and has
horrible performance issues.

Downside: This seriously breaks support for pass 1 parts of the BCM1250
where indexed cacheops don't work quite reliable but I seem to be the
last one on the planet with a pass 1 part anyway.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-10-11 23:46:05 +01:00
Ralf Baechle e001e52801 [MIPS] Replace use of stext with _stext.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-07-31 21:35:28 +01:00
Fuxin Zhang 2a21c7300b [MIPS] define Hit_Invalidate_I to Index_Invalidate_I for loongson2
Signed-off-by: Fuxin Zhang <zhangfx@lemote.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-07-10 17:33:02 +01:00
Ralf Baechle 617667ba72 [MIPS] Avoid dupliate D-cache flush on R400C / R4400 SC and MC variants.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-11-30 01:14:48 +00:00
Atsushi Nemoto 0550d9d13e [MIPS] Remove redundant r4k_blast_icache() calls
r4k_flush_cache_all() and r4k_flush_cache_mm() case: these are noop if
the CPU did not have dc_aliases.  It would mean we do not need to care
about icache here.

r4k_flush_cache_range case: if r4k_flush_cache_mm() did not need to
care about icache, it would be same for this function.

Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-11-30 01:14:48 +00:00
Atsushi Nemoto c59a0f15be [MIPS] Remove __flush_icache_page
__flash_icache_page is unused, so kill it.

Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-10-01 23:16:58 +01:00
Ralf Baechle a00f631018 [MIPS] c-r4k: Convert init functions from inline to __init.
With more recent compilers inline doesn't necessarily means a function
will always be inlined.  So leave that decission to the compiler and
make the function as __init.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-09-27 13:37:44 +01:00
Atsushi Nemoto f6502791d7 [MIPS] Do not use drop_mmu_context to flusing other task's VIPT I-cache.
c-r4k.c and c-sb1.c use drop_mmu_context() to flush virtually tagged
I-caches, but this does not work for flushing other task's icache.  This
is for example triggered by copy_to_user_page() called from ptrace(2).
Use indexed flush for such cases.

Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-09-27 13:37:37 +01:00
Ralf Baechle 585fa72493 [MIPS] Retire flush_icache_page from mm use.
On the 34K the redundant cache operations were causing excessive stalls
resulting in realtime code running on the second VPE missing its deadline.
For all other platforms this patch is just a significant performance
improvment as illustrated by below benchmark numbers.

Processor, Processes - times in microseconds - smaller is better
------------------------------------------------------------------------------
Host                 OS  Mhz null null      open slct sig  sig  fork exec sh
                             call  I/O stat clos TCP  inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
25Kf      2.6.18-rc4     533 0.49 1.16 7.57 33.4 30.5 1.34 12.4 5497 17.K 54.K
25Kf      2.6.18-rc4-p   533 0.49 1.16 6.68 23.0 30.7 1.36 8.55 5030 16.K 48.K
4Kc       2.6.18-rc4      80 4.21 15.0 131. 289. 261. 16.5 258. 18.K 70.K 227K
4Kc       2.6.18-rc4-p    80 4.34 13.1 128. 285. 262. 18.2 258. 12.K 52.K 176K
34Kc      2.6.18-rc4      40 5.01 14.0 61.6 90.0 477. 17.9 94.7 29.K 108K 342K
34Kc      2.6.18-rc4-p    40 4.98 13.9 61.2 89.7 475. 17.6 93.7 8758 44.K 158K
BCM1480   2.6.18-rc4     700 0.28 0.60 3.68 5.92 16.0 0.78 5.08 931. 3163 15.K
BCM1480   2.6.18-rc4-p   700 0.28 0.61 3.65 5.85 16.0 0.79 5.20 395. 1464 8385
TX49-16K  2.6.18-rc3     197 0.73 2.41 19.0 37.8 82.9 2.94 17.5 4438 14.K 56.K
TX49-16K  2.6.18-rc3-p   197 0.73 2.40 19.9 36.3 82.9 2.94 23.4 2577 9103 38.K
TX49-32K  2.6.18-rc3     396 0.36 1.19 6.80 11.8 41.0 1.46 8.17 2738 8465 32.K
TX49-32K  2.6.18-rc3-p   396 0.36 1.19 6.82 10.2 41.0 1.46 8.18 1330 4638 18.K
    
Original patch by me with enhancements by Atsushi Nemoto.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
2006-09-27 13:37:34 +01:00
Ralf Baechle df586d59a4 [MIPS] c-r4k: Typo fix.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-09-27 13:37:23 +01:00
Yoichi Yuasa 2874fe5533 [MIPS] vr41xx: Replace magic number for P4K bit with symbol.
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-07-13 21:26:11 +01:00
Yoichi Yuasa 1058ecda9b [MIPS] vr41xx: Changed workaround to recommended method
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-07-13 21:26:11 +01:00
Yoichi Yuasa 4e8ab36182 [MIPS] VR41xx: Set VR41_CONF_BP only for PrId 0x0c80.
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-07-13 21:26:06 +01:00
Ralf Baechle fc5d2d279f [MIPS] Use the proper technical term for naming some of the cache macros.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-07-13 21:26:04 +01:00
Jörn Engel 6ab3d5624e Remove obsolete #include <linux/config.h>
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
2006-06-30 19:25:36 +02:00
Ralf Baechle 2e78ae3f48 [MIPS] 74K: Assume it will also have an AR bit in config7
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-06-29 21:10:54 +01:00