Age | Commit message (Collapse) | Author |
|
Loongson-3A6000 and newer processors have hardware page table walker
(PTW) support. PTW can handle all fastpaths of TLBI/TLBL/TLBS/TLBM
exceptions by hardware, software only need to handle slowpaths (page
faults).
BTW, PTW doesn't append _PAGE_MODIFIED for page table entries, so we
change pmd_dirty() and pte_dirty() to also check _PAGE_DIRTY for the
"dirty" attribute.
Signed-off-by: Liang Gao <gaoliang@loongson.cn>
Signed-off-by: Jun Yi <yijun@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|
|
When exception is triggered, code flow go handle_\exception in some
cases. One of stackframe in this case as follows,
high -> +-------+
| REGS | <- a pt_regs
| |
| | <- ex trigger
| REGS | <- ex pt_regs <-+
| | |
| | |
low -> +-------+ ->unwind-+
When unwinder unwinds to handler_\exception it cannot go on prologue
analysis. Because it is an asynchronous code flow, we should get the
next frame PC from regs->csr_era rather than regs->regs[1]. At init time
we copy the handlers to eentry and also copy them to NUMA-affine memory
named pcpu_handlers if NUMA is enabled. Thus, unwinder cannot unwind
normally. To solve this, we try to give some hints in handler_\exception
and fixup unwinders in unwind_next_frame().
Reported-by: Qing Zhang <zhangqing@loongson.cn>
Signed-off-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|
|
Move local_flush_tlb_all() earlier (just after setup_ptwalker() and
before page allocation). This can avoid stale TLB entries misguiding
the later page allocation. Without this patch the second kernel of
kexec/kdump fails to boot SMP.
BTW, move output_pgtable_bits_defines() into tlb_init() since it has
nothing to do with tlb handler setup.
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|
|
setup_tlb_handler() is expected to set per-cpu exception handlers, but
it only set the TLBRENTRY successfully because of copy & paste errors,
so fix it.
Reviewed-by: WANG Xuerui <git@xen0n.name>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|
|
Since setup_tlb_handler() is executed in atomic context, we should use
GFP_ATOMIC instead of GFP_KERNEL to alloc pages. Otherwise we will get
a "sleeping in atomic context" error:
[ 0.013118] BUG: sleeping function called from invalid context at mm/page_alloc.c:5158
[ 0.013126] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/1
[ 0.013131] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.19-rc3+ #1008 1a223086d14d07967cc427f15d52139422271360
[ 0.013136] Hardware name: Loongson Loongson-3A5000-7A1000-1w-V0.1-CRB/Loongson-LS3A5000-7A1000-1w-EVB-V1.21, BIOS Loongson-UDK2018-V2.0.04082-beta7 04/27
[ 0.013140] Stack : 90000000015fc990 9000000100493c18 9000000000df3370 9000000100490000
[ 0.013151] 9000000100493b50 0000000000000000 9000000100493b58 9000000001417ef0
[ 0.013160] 900000000199e54e 0000000000000040 9000000100493c18 90000000015f7a98
[ 0.013168] ffffffffffffffff 6de72f8b42179d1e 9000000100403b80 90000000015f7890
[ 0.013176] 0000000000000001 00000000fffff175 9000000000eb9860 9000000001530b4b
[ 0.013184] 9000000000e99e60 0000000000000013 0000000006ecc000 0000000000000001
[ 0.013193] 90000000015f7a98 9000000001417ef0 0000000000000004 0000000000000000
[ 0.013201] 0000000000000cc0 0000000000000000 0000000000000001 90000000015fc990
[ 0.013209] 9000000000217e74 9000000001603b6b 9000000000208640 0000000000000000
[ 0.013217] 00000000000000b0 0000000000000004 0000000000000000 0000000000070000
[ 0.013225] ...
[ 0.013229] Call Trace:
[ 0.013230] [<9000000000208640>] show_stack+0x4c/0x14c
[ 0.013240] [<9000000000df3370>] dump_stack_lvl+0x70/0xac
[ 0.013246] [<9000000000270c8c>] ___might_sleep+0x104/0x124
[ 0.013253] [<9000000000477e84>] __alloc_pages+0x240/0x464
[ 0.013260] [<9000000000214214>] setup_tlb_handler+0x104/0x1e8
[ 0.013265] [<9000000000214324>] tlb_init+0x2c/0x3c
[ 0.013270] [<9000000000208b74>] per_cpu_trap_init+0xec/0x108
[ 0.013275] [<9000000000202850>] cpu_probe+0x400/0x8a4
[ 0.013279] [<900000000020d160>] start_secondary+0x5c/0x3d4
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|
|
Add Non-Uniform Memory Access (NUMA) support for LoongArch. LoongArch
has 48-bit physical address, but the HyperTransport I/O bus only support
40-bit address, so we need a custom phys_to_dma() and dma_to_phys() to
extract the 4-bit node id (bit 44~47) from Loongson-3's 48-bit physical
address space and embed it into 40-bit. In the 40-bit dma address, node
id offset can be read from the LS7A_DMA_CFG register.
Reviewed-by: WANG Xuerui <git@xen0n.name>
Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|
|
Add memory management support for LoongArch, including: cache and tlb
management, page fault handling and ioremap/mmap support.
Reviewed-by: WANG Xuerui <git@xen0n.name>
Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|