summaryrefslogtreecommitdiff
path: root/arch/s390
AgeCommit message (Collapse)Author
2023-02-07KVM: s390: Refactor absolute vm mem_op functionJanis Schoetterl-Glausch
Remove code duplication with regards to the CHECK_ONLY flag. Decrease the number of indents. No functional change indented. Suggested-by: Janosch Frank <frankja@linux.ibm.com> Signed-off-by: Janis Schoetterl-Glausch <scgl@linux.ibm.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Janosch Frank <frankja@linux.ibm.com> Link: https://lore.kernel.org/r/20230206164602.138068-11-scgl@linux.ibm.com Message-Id: <20230206164602.138068-11-scgl@linux.ibm.com> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
2023-02-07KVM: s390: Dispatch to implementing function at top level of vm mem_opJanis Schoetterl-Glausch
Instead of having one function covering all mem_op operations, have a function implementing absolute access and dispatch to that function in its caller, based on the operation code. This way additional future operations can be implemented by adding an implementing function without changing existing operations. Suggested-by: Janosch Frank <frankja@linux.ibm.com> Signed-off-by: Janis Schoetterl-Glausch <scgl@linux.ibm.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Janosch Frank <frankja@linux.ibm.com> Link: https://lore.kernel.org/r/20230206164602.138068-10-scgl@linux.ibm.com Message-Id: <20230206164602.138068-10-scgl@linux.ibm.com> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
2023-02-07KVM: s390: Move common code of mem_op functions into functionJanis Schoetterl-Glausch
The vcpu and vm mem_op ioctl implementations share some functionality. Move argument checking into a function and call it from both implementations. This allows code reuse in case of additional future mem_op operations. Suggested-by: Janosch Frank <frankja@linux.ibm.com> Signed-off-by: Janis Schoetterl-Glausch <scgl@linux.ibm.com> Reviewed-by: Janosch Frank <frankja@linux.ibm.com> Link: https://lore.kernel.org/r/20230206164602.138068-9-scgl@linux.ibm.com Message-Id: <20230206164602.138068-9-scgl@linux.ibm.com> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
2023-02-07KVM: s390: disable migration mode when dirty tracking is disabledNico Boehr
Migration mode is a VM attribute which enables tracking of changes in storage attributes (PGSTE). It assumes dirty tracking is enabled on all memslots to keep a dirty bitmap of pages with changed storage attributes. When enabling migration mode, we currently check that dirty tracking is enabled for all memslots. However, userspace can disable dirty tracking without disabling migration mode. Since migration mode is pointless with dirty tracking disabled, disable migration mode whenever userspace disables dirty tracking on any slot. Also update the documentation to clarify that dirty tracking must be enabled when enabling migration mode, which is already enforced by the code in kvm_s390_vm_start_migration(). Also highlight in the documentation for KVM_S390_GET_CMMA_BITS that it can now fail with -EINVAL when dirty tracking is disabled while migration mode is on. Move all the error codes to a table so this stays readable. To disable migration mode, slots_lock should be held, which is taken in kvm_set_memory_region() and thus held in kvm_arch_prepare_memory_region(). Restructure the prepare code a bit so all the sanity checking is done before disabling migration mode. This ensures migration mode isn't disabled when some sanity check fails. Cc: stable@vger.kernel.org Fixes: 190df4a212a7 ("KVM: s390: CMMA tracking, ESSA emulation, migration mode") Signed-off-by: Nico Boehr <nrb@linux.ibm.com> Reviewed-by: Janosch Frank <frankja@linux.ibm.com> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com> Link: https://lore.kernel.org/r/20230127140532.230651-2-nrb@linux.ibm.com Message-Id: <20230127140532.230651-2-nrb@linux.ibm.com> [frankja@linux.ibm.com: fixed commit message typo, moved api.rst error table upwards] Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
2023-02-07Merge remote-tracking branch 'l390-korg/cmpxchg_user_key' into kvm-nextJanosch Frank
2023-02-06Merge branch 'fixes' into featuresHeiko Carstens
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/boot: avoid potential amode31 truncationVasily Gorbik
Fixes: bb1520d581a3 ("s390/mm: start kernel with DAT enabled") Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/boot: move detect_facilities() after cmd line parsingVasily Gorbik
Facilities setup has to be done after "facilities" command line option parsing, it might set extra or remove existing facilities bits for testing purposes. Fixes: bb1520d581a3 ("s390/mm: start kernel with DAT enabled") Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/kasan: avoid mapping KASAN shadow for standby memoryVasily Gorbik
KASAN common code is able to handle memory hotplug and create KASAN shadow memory on a fly. Online memory ranges are available from mem_detect, use this information to avoid mapping KASAN shadow for standby memory. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/boot: avoid page tables memory in kaslrVasily Gorbik
If kernel is build without KASAN support there is a chance that kernel image is going to be positioned by KASLR code to overlap with identity mapping page tables. When kernel is build with KASAN support enabled memory which is potentially going to be used for page tables and KASAN shadow mapping is accounted for in KASLR with the use of kasan_estimate_memory_needs(). Split this function and introduce vmem_estimate_memory_needs() to cover decompressor's vmem identity mapping page tables. Fixes: bb1520d581a3 ("s390/mm: start kernel with DAT enabled") Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/mem_detect: add get_mem_detect_online_total()Vasily Gorbik
Add a function to get online memory in total. It is supposed to be used in the decompressor as well as during early kernel startup. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/mem_detect: handle online memory limit just onceVasily Gorbik
Introduce mem_detect_truncate() to cut any online memory ranges above established identity mapping size, so that mem_detect users wouldn't have to do it over and over again. Suggested-by: Alexander Gordeev <agordeev@linux.ibm.com> Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/boot: fix mem_detect extended area allocationVasily Gorbik
Allocation of mem_detect extended area was not considered neither in commit 9641b8cc733f ("s390/ipl: read IPL report at early boot") nor in commit b2d24b97b2a9 ("s390/kernel: add support for kernel address space layout randomization (KASLR)"). As a result mem_detect extended theoretically may overlap with ipl report or randomized kernel image position. But as mem_detect code will allocate extended area only upon exceeding 255 online regions (which should alternate with offline memory regions) it is not seen in practice. To make sure mem_detect extended area does not overlap with ipl report or randomized kernel position extend usage of "safe_addr". Make initrd handling and mem_detect extended area allocation code move it further right and make KASLR takes in into consideration as well. Fixes: 9641b8cc733f ("s390/ipl: read IPL report at early boot") Fixes: b2d24b97b2a9 ("s390/kernel: add support for kernel address space layout randomization (KASLR)") Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/mem_detect: rely on diag260() if sclp_early_get_memsize() failsVasily Gorbik
In case sclp_early_get_memsize() fails but diag260() succeeds make sure some sane value is returned. This error scenario is highly unlikely, but this change makes system able to boot in such case. Suggested-by: Alexander Gordeev <agordeev@linux.ibm.com> Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/diag: make __diag8c_tmp_amode31 staticHeiko Carstens
Get rid of this sparse warning: arch/s390/kernel/diag.c:69:29: warning: symbol '__diag8c_tmp_amode31' was not declared. Should it be static? Fixes: fbaee7464fbb ("s390/tty3270: add support for diag 8c") Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/rethook: add local rethook header fileHeiko Carstens
Compiling the kernel with CONFIG_KPROBES disabled, but CONFIG_RETHOOK enabled, results in this sparse warning: arch/s390/kernel/rethook.c:26:15: warning: no previous prototype for 'arch_rethook_trampoline_callback' [-Wmissing-prototypes] 26 | unsigned long arch_rethook_trampoline_callback(struct pt_regs *regs) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Add a local rethook header file similar to riscv to address this. Reported-by: kernel test robot <lkp@intel.com> Fixes: 1a280f48c0e4 ("s390/kprobes: replace kretprobe with rethook") Link: https://lore.kernel.org/all/202302030102.69dZIuJk-lkp@intel.com Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/vmem: remove unnecessary KASAN checksVasily Gorbik
Kasan shadow memory area has been moved to the end of kernel address space since commit 9a39abb7c9aa ("s390/boot: simplify and fix kernel memory layout setup"), therefore skipping any memory ranges above VMALLOC_START in empty page tables cleanup code already handles KASAN shadow memory intersection case and explicit checks could be removed. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/vmem: fix empty page tables cleanup under KASANVasily Gorbik
Commit b9ff81003cf1 ("s390/vmem: cleanup empty page tables") introduced empty page tables cleanup in vmem code, but when the kernel is built with KASAN enabled the code has no effect due to wrong KASAN shadow memory intersection condition, which effectively ignores any memory range below KASAN shadow. Fix intersection condition to make code work as anticipated. Fixes: b9ff81003cf1 ("s390/vmem: cleanup empty page tables") Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/kasan: update kasan memory layout noteVasily Gorbik
Kasan shadow memory area has been moved to the end of kernel address space since commit 9a39abb7c9aa ("s390/boot: simplify and fix kernel memory layout setup"). Change kasan memory layout note accordingly. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/mem_detect: fix detect_memory() error handlingVasily Gorbik
Currently if for some reason sclp_early_read_info() fails, sclp_early_get_memsize() will not set max_physmem_end and it will stay uninitialized. Any garbage value other than 0 will lead to detect_memory() taking wrong path or returning a garbage value as max_physmem_end. To avoid that simply initialize max_physmem_end. Fixes: 73045a08cf55 ("s390: unify identity mapping limits handling") Reported-by: Alexander Gordeev <agordeev@linux.ibm.com> Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/ipl: add loadparm parameter to eckd ipl/reipl dataSven Schnelle
commit 87fd22e0ae92 ("s390/ipl: add eckd support") missed to add the loadparm attribute to the new eckd ipl/reipl data. Fixes: 87fd22e0ae92 ("s390/ipl: add eckd support") Cc: <stable@vger.kernel.org> Signed-off-by: Sven Schnelle <svens@linux.ibm.com> Reviewed-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-06s390/ipl: add DEFINE_GENERIC_LOADPARM()Sven Schnelle
In the current code each reipl type implements its own pair of loadparm show/store functions. Add a macro to deduplicate the code a bit. Reviewed-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Sven Schnelle <svens@linux.ibm.com> Fixes: 87fd22e0ae92 ("s390/ipl: add eckd support") Cc: <stable@vger.kernel.org> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-02-02mm: add vma_alloc_zeroed_movable_folio()Matthew Wilcox (Oracle)
Replace alloc_zeroed_user_highpage_movable(). The main difference is returning a folio containing a single page instead of returning the page, but take the opportunity to rename the function to match other allocation functions a little better and rewrite the documentation to place more emphasis on the zeroing rather than the highmem aspect. Link: https://lkml.kernel.org/r/20230116191813.2145215-2-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-02-02mm: remove __HAVE_ARCH_PTE_SWP_EXCLUSIVEDavid Hildenbrand
__HAVE_ARCH_PTE_SWP_EXCLUSIVE is now supported by all architectures that support swp PTEs, so let's drop it. Link: https://lkml.kernel.org/r/20230113171026.582290-27-david@redhat.com Signed-off-by: David Hildenbrand <david@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-01-31s390/mem_detect: do not update output parameters on failureAlexander Gordeev
Function __get_mem_detect_block() resets start and end output parameters in case of invalid mem_detect array index is provided. That violates the rule of sparing the output on fail path and leads e.g to a below anomaly: for_each_mem_detect_block(i, &start, &end) continue; One would expect start and end contain addresses of the last memory block (if available), but in fact the two will be reset to zeroes. That is not how an iterator is expected to work. Reviewed-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-31s390/cio: introduce locking for register/unregister functionsVineeth Vijayan
Unbinding an I/O subchannel with a child-CCW device in disconnected state sometimes causes a kernel-panic. The race condition was seen mostly during testing, when setting all the CHPIDs of a device to offline and at the same time, the unbinding the I/O subchannel driver. The kernel-panic occurs because of double delete, the I/O subchannel driver calls device_del on the CCW device while another device_del invocation for the same device is in-flight. For instance, disabling all the CHPIDs will trigger the ccw_device_remove function, which will call a ccw_device_unregister(), which ends up calling the device_del() which is asynchronous via cdev's todo workqueue. And unbinding the I/O subchannel driver calls io_subchannel_remove() function which calls the ccw_device_unregister() and device_del(). This double delete can be prevented by serializing all CCW device registration/unregistration calls into the driver core. This patch introduces a mutex which will be used for this purpose. Signed-off-by: Vineeth Vijayan <vneethv@linux.ibm.com> Reported-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-31s390/mm,ptdump: avoid Kasan vs Memcpy Real markers swappingVasily Gorbik
---[ Real Memory Copy Area Start ]--- 0x001bfffffffff000-0x001c000000000000 4K PTE I ---[ Kasan Shadow Start ]--- ---[ Real Memory Copy Area End ]--- 0x001c000000000000-0x001c000200000000 8G PMD RW NX ... ---[ Kasan Shadow End ]--- ptdump does a stable sort of markers. Move kasan markers after memcpy real to avoid swapping. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-31s390/boot: remove pgtable_populate_endVasily Gorbik
setup_vmem() already calls populate for all online memory regions. pgtable_populate_end() could be removed. Also rename pgtable_populate_begin() to pgtable_populate_init(). Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-31s390/boot: avoid mapping standby memoryVasily Gorbik
Commit bb1520d581a3 ("s390/mm: start kernel with DAT enabled") doesn't consider online memory holes due to potential memory offlining and erroneously creates pgtables for stand-by memory, which bear RW+X attribute and trigger a warning: RANGE SIZE STATE REMOVABLE BLOCK 0x0000000000000000-0x0000000c3fffffff 49G online yes 0-48 0x0000000c40000000-0x0000000c7fffffff 1G offline 49 0x0000000c80000000-0x0000000fffffffff 14G online yes 50-63 0x0000001000000000-0x00000013ffffffff 16G offline 64-79 s390/mm: Found insecure W+X mapping at address 0xc40000000 WARNING: CPU: 14 PID: 1 at arch/s390/mm/dump_pagetables.c:142 note_page+0x2cc/0x2d8 Map only online memory ranges which fit within identity mapping limit. Fixes: bb1520d581a3 ("s390/mm: start kernel with DAT enabled") Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-31s390/decompressor: specify __decompress() buf len to avoid overflowVasily Gorbik
Historically calls to __decompress() didn't specify "out_len" parameter on many architectures including s390, expecting that no writes beyond uncompressed kernel image are performed. This has changed since commit 2aa14b1ab2c4 ("zstd: import usptream v1.5.2") which includes zstd library commit 6a7ede3dfccb ("Reduce size of dctx by reutilizing dst buffer (#2751)"). Now zstd decompression code might store literal buffer in the unwritten portion of the destination buffer. Since "out_len" is not set, it is considered to be unlimited and hence free to use for optimization needs. On s390 this might corrupt initrd or ipl report which are often placed right after the decompressor buffer. Luckily the size of uncompressed kernel image is already known to the decompressor, so to avoid the problem simply specify it in the "out_len" parameter. Link: https://github.com/facebook/zstd/commit/6a7ede3dfccb Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Tested-by: Alexander Egorenkov <egorenar@linux.ibm.com> Link: https://lore.kernel.org/r/patch-1.thread-41c676.git-41c676c2d153.your-ad-here.call-01675030179-ext-9637@work.hours Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-31Merge tag 'v6.2-rc6' into sched/core, to pick up fixesIngo Molnar
Pick up fixes before merging another batch of cpuidle updates. Signed-off-by: Ingo Molnar <mingo@kernel.org>
2023-01-30Merge branch 'iommu-memory-accounting' of ↵Jason Gunthorpe
ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/joro/iommu intoiommufd/for-next Jason Gunthorpe says: ==================== iommufd follows the same design as KVM and uses memory cgroups to limit the amount of kernel memory a iommufd file descriptor can pin down. The various internal data structures already use GFP_KERNEL_ACCOUNT to charge its own memory. However, one of the biggest consumers of kernel memory is the IOPTEs stored under the iommu_domain and these allocations are not tracked. This series is the first step in fixing it. The iommu driver contract already includes a 'gfp' argument to the map_pages op, allowing iommufd to specify GFP_KERNEL_ACCOUNT and then having the driver allocate the IOPTE tables with that flag will capture a significant amount of the allocations. Update the iommu_map() API to pass in the GFP argument, and fix all call sites. Replace iommu_map_atomic(). Audit the "enterprise" iommu drivers to make sure they do the right thing. Intel and S390 ignore the GFP argument and always use GFP_ATOMIC. This is problematic for iommufd anyhow, so fix it. AMD and ARM SMMUv2/3 are already correct. A follow up series will be needed to capture the allocations made when the iommu_domain itself is allocated, which will complete the job. ==================== * 'iommu-memory-accounting' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/joro/iommu: iommu/s390: Use GFP_KERNEL in sleepable contexts iommu/s390: Push the gfp parameter to the kmem_cache_alloc()'s iommu/intel: Use GFP_KERNEL in sleepable contexts iommu/intel: Support the gfp argument to the map_pages op iommu/intel: Add a gfp parameter to alloc_pgtable_page() iommufd: Use GFP_KERNEL_ACCOUNT for iommu_map() iommu/dma: Use the gfp parameter in __iommu_dma_alloc_noncontiguous() iommu: Add a gfp parameter to iommu_map_sg() iommu: Remove iommu_map_atomic() iommu: Add a gfp parameter to iommu_map() Link: https://lore.kernel.org/linux-iommu/0-v3-76b587fe28df+6e3-iommu_map_gfp_jgg@nvidia.com Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2023-01-30vfio-mdev: add back CONFIG_VFIO dependencyArnd Bergmann
CONFIG_VFIO_MDEV cannot be selected when VFIO itself is disabled, otherwise we get a link failure: WARNING: unmet direct dependencies detected for VFIO_MDEV Depends on [n]: VFIO [=n] Selected by [y]: - SAMPLE_VFIO_MDEV_MTTY [=y] && SAMPLES [=y] - SAMPLE_VFIO_MDEV_MDPY [=y] && SAMPLES [=y] - SAMPLE_VFIO_MDEV_MBOCHS [=y] && SAMPLES [=y] /home/arnd/cross/arm64/gcc-13.0.1-nolibc/x86_64-linux/bin/x86_64-linux-ld: samples/vfio-mdev/mdpy.o: in function `mdpy_remove': mdpy.c:(.text+0x1e1): undefined reference to `vfio_unregister_group_dev' /home/arnd/cross/arm64/gcc-13.0.1-nolibc/x86_64-linux/bin/x86_64-linux-ld: samples/vfio-mdev/mdpy.o: in function `mdpy_probe': mdpy.c:(.text+0x149e): undefined reference to `_vfio_alloc_device' Fixes: 8bf8c5ee1f38 ("vfio-mdev: turn VFIO_MDEV into a selectable symbol") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Link: https://lore.kernel.org/r/20230126211211.1762319-1-arnd@kernel.org Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2023-01-29s390/bpf: Implement bpf_jit_supports_kfunc_call()Ilya Leoshkevich
Implement calling kernel functions from eBPF. In general, the eBPF ABI is fairly close to that of s390x, with one important difference: on s390x callers should sign-extend signed arguments. Handle that by using information returned by bpf_jit_find_kfunc_model(). Here is an example of how sign extensions works. Suppose we need to call the following function from BPF: ; long noinline bpf_kfunc_call_test4(signed char a, short b, int c, long d) 0000000000936a78 <bpf_kfunc_call_test4>: 936a78: c0 04 00 00 00 00 jgnop bpf_kfunc_call_test4 ; return (long)a + (long)b + (long)c + d; 936a7e: b9 08 00 45 agr %r4,%r5 936a82: b9 08 00 43 agr %r4,%r3 936a86: b9 08 00 24 agr %r2,%r4 936a8a: c0 f4 00 1e 3b 27 jg <__s390_indirect_jump_r14> As per the s390x ABI, bpf_kfunc_call_test4() has the right to assume that a, b and c are sign-extended by the caller, which results in using 64-bit additions (agr) without any additional conversions. Without sign extension we would have the following on the JITed code side: ; tmp = bpf_kfunc_call_test4(-3, -30, -200, -1000); ; 5: b4 10 00 00 ff ff ff fd w1 = -3 0x3ff7fdcdad4: llilf %r2,0xfffffffd ; 6: b4 20 00 00 ff ff ff e2 w2 = -30 0x3ff7fdcdada: llilf %r3,0xffffffe2 ; 7: b4 30 00 00 ff ff ff 38 w3 = -200 0x3ff7fdcdae0: llilf %r4,0xffffff38 ; 8: b7 40 00 00 ff ff fc 18 r4 = -1000 0x3ff7fdcdae6: lgfi %r5,-1000 0x3ff7fdcdaec: mvc 64(4,%r15),160(%r15) 0x3ff7fdcdaf2: lgrl %r1,bpf_kfunc_call_test4@GOT 0x3ff7fdcdaf8: brasl %r14,__s390_indirect_jump_r1 This first 3 llilfs are 32-bit loads, that need to be sign-extended to 64 bits. Note: at the moment bpf_jit_find_kfunc_model() does not seem to play nicely with XDP metadata functions: add_kfunc_call() adds an "abstract" bpf_*() version to kfunc_btf_tab, but then fixup_kfunc_call() puts the concrete version into insn->imm, which bpf_jit_find_kfunc_model() cannot find. But this seems to be a common code problem. Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20230129190501.1624747-7-iii@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-29s390/bpf: Implement bpf_jit_supports_subprog_tailcalls()Ilya Leoshkevich
Allow mixing subprogs and tail calls by passing the current tail call count to subprogs. Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20230129190501.1624747-6-iii@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-29s390/bpf: Implement arch_prepare_bpf_trampoline()Ilya Leoshkevich
arch_prepare_bpf_trampoline() is used for direct attachment of eBPF programs to various places, bypassing kprobes. It's responsible for calling a number of eBPF programs before, instead and/or after whatever they are attached to. Add a s390x implementation, paying attention to the following: - Reuse the existing JIT infrastructure, where possible. - Like the existing JIT, prefer making multiple passes instead of backpatching. Currently 2 passes is enough. If literal pool is introduced, this needs to be raised to 3. However, at the moment adding literal pool only makes the code larger. If branch shortening is introduced, the number of passes needs to be increased even further. - Support both regular and ftrace calling conventions, depending on the trampoline flags. - Use expolines for indirect calls. - Handle the mismatch between the eBPF and the s390x ABIs. - Sign-extend fmod_ret return values. invoke_bpf_prog() produces about 120 bytes; it might be possible to slightly optimize this, but reaching 50 bytes, like on x86_64, looks unrealistic: just loading cookie, __bpf_prog_enter, bpf_func, insnsi and __bpf_prog_exit as literals already takes at least 5 * 12 = 60 bytes, and we can't use relative addressing for most of them. Therefore, lower BPF_MAX_TRAMP_LINKS on s390x. Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20230129190501.1624747-5-iii@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-29s390/bpf: Implement bpf_arch_text_poke()Ilya Leoshkevich
bpf_arch_text_poke() is used to hotpatch eBPF programs and trampolines. s390x has a very strict hotpatching restriction: the only thing that is allowed to be hotpatched is conditional branch mask. Take the same approach as commit de5012b41e5c ("s390/ftrace: implement hotpatching"): create a conditional jump to a "plt", which loads the target address from memory and jumps to it; then first patch this address, and then the mask. Trampolines (introduced in the next patch) respect the ftrace calling convention: the return address is in %r0, and %r1 is clobbered. With that in mind, bpf_arch_text_poke() does not differentiate between jumps and calls. However, there is a simple optimization for jumps (for the epilogue_ip case): if a jump already points to the destination, then there is no "plt" and we can just flip the mask. For simplicity, the "plt" template is defined in assembly, and its size is used to define C arrays. There doesn't seem to be a way to convey this size to C as a constant, so it's hardcoded and double-checked during runtime. Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20230129190501.1624747-4-iii@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-29s390/bpf: Add expoline to tail callsIlya Leoshkevich
All the indirect jumps in the eBPF JIT already use expolines, except for the tail call one. Fixes: de5cb6eb514e ("s390: use expoline thunks in the BPF JIT") Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20230129190501.1624747-3-iii@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-28s390/bpf: Fix a typo in a commentIlya Leoshkevich
"desription" should be "description". Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20230128000650.1516334-27-iii@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-26s390/purgatory: Remove unused '-MD' and unnecessary '-c' flagsNathan Chancellor
When clang's -Qunused-arguments is dropped from KBUILD_CPPFLAGS, it warns while building objects in the purgatory folder: clang-16: error: argument unused during compilation: '-MD' [-Werror,-Wunused-command-line-argument] '-MMD' is always passed to the preprocessor via c_flags, even when KBUILD_CFLAGS is overridden in a folder, so clang complains the addition of '-MD' will be unused. Remove '-MD' to clear up this warning, as it is unnecessary with '-MMD'. Additionally, '-c' is also unnecessary, remove it while in the area. Signed-off-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Heiko Carstens <hca@linux.ibm.com> Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Tested-by: Linux Kernel Functional Testing <lkft@linaro.org> Tested-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-01-26s390/vdso: Drop '-shared' from KBUILD_CFLAGS_64Nathan Chancellor
When clang's -Qunused-arguments is dropped from KBUILD_CPPFLAGS, it points out that there is a linking phase flag added to CFLAGS, which will only be used for compiling clang-16: error: argument unused during compilation: '-shared' [-Werror,-Wunused-command-line-argument] '-shared' is already present in ldflags-y so it can just be dropped. Fixes: 2b2a25845d53 ("s390/vdso: Use $(LD) instead of $(CC) to link vDSO") Signed-off-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Heiko Carstens <hca@linux.ibm.com> Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Tested-by: Linux Kernel Functional Testing <lkft@linaro.org> Tested-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-01-26s390/vdso: Drop unused '-s' flag from KBUILD_AFLAGS_64Nathan Chancellor
When clang's -Qunused-arguments is dropped from KBUILD_CPPFLAGS, it warns: clang-16: error: argument unused during compilation: '-s' [-Werror,-Wunused-command-line-argument] The compiler's '-s' flag is a linking option (it is passed along to the linker directly), which means it does nothing when the linker is not invoked by the compiler. The kernel builds all .o files with '-c', which stops the compilation pipeline before linking, so '-s' can be safely dropped from KBUILD_AFLAGS_64. Signed-off-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Heiko Carstens <hca@linux.ibm.com> Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Tested-by: Linux Kernel Functional Testing <lkft@linaro.org> Tested-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-01-25s390/syscalls: get rid of system call alias functionsHeiko Carstens
bpftrace and friends only consider functions present in /sys/kernel/tracing/available_filter_functions. For system calls there is the s390 specific problem that the system call function itself is present via __se_sys##name() while the system call itself is wired up via an __s390x_sys##name() alias. The required DWARF debug information however is only available for the original function, not the alias, but within available_filter_functions only the functions with __s390x_ prefix are available. Which means the required DWARF debug information cannot be found. While this could be solved via tooling, it is easier to change the s390 specific system call wrapper handling. Therefore get rid of this alias handling and implement system call wrappers like most other architectures are doing. In result the implementation generates the following functions: long __s390x_sys##name(struct pt_regs *regs) static inline long __se_sys##name(...) static inline long __do_sys##name(...) __s390x_sys##name() is the visible system call function which is also wired up in the system call table. Its only parameter is a pt_regs variable. This function calls the corresponding __se_sys##name() function, which has as many parameters like the system call definition. This function in turn performs all zero and sign extensions of all system call parameters, taken from the pt_regs structure, and finally calls __do_sys##name(). __do_sys##name() is the actual inlined system call function implementation. For all 64 bit system calls there is a 31/32 bit system call function __s390_sys##name() generated, which handles all system call parameters correctly as required by compat handling. This function may be wired up within the compat system call table, unless there exists an explicit compat system call function, which is then used instead. Reported-by: Ilya Leoshkevich <iii@linux.ibm.com> Tested-by: Ilya Leoshkevich <iii@linux.ibm.com> Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-25s390/syscalls: remove trailing semicolonHeiko Carstens
Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-25s390/syscalls: move __S390_SYS_STUBx() macroHeiko Carstens
Move __S390_SYS_STUBx() the end of the CONFIG_COMPAT section, so both variants (compat and non-compat) are close together and can be easily compared. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-25s390/syscalls: remove __SC_COMPAT_TYPE defineHeiko Carstens
Remove __SC_COMPAT_TYPE define which is an unused leftover. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-25s390/syscalls: remove SYSCALL_METADATA() from compat syscallsHeiko Carstens
SYSCALL_METADATA() is only supposed to be used for non-compat system calls. Otherwise there would be a name clash. This also removes the inconsistency that s390 is the only architecture which uses SYSCALL_METADATA() for compat system calls, and even that only for compat system calls without parameters. Only two such compat system calls exist. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-25s390: discard .interp sectionIlya Leoshkevich
When debugging vmlinux with QEMU + GDB, the following GDB error may occur: (gdb) c Continuing. Warning: Cannot insert breakpoint -1. Cannot access memory at address 0xffffffffffff95c0 Command aborted. (gdb) The reason is that, when .interp section is present, GDB tries to locate the file specified in it in memory and put a number of breakpoints there (see enable_break() function in gdb/solib-svr4.c). Sometimes GDB finds a bogus location that matches its heuristics, fails to set a breakpoint and stops. This makes further debugging impossible. The .interp section contains misleading information anyway (vmlinux does not need ld.so), so fix by discarding it. Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Cc: <stable@vger.kernel.org> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-25s390/cpum_cf: simplify PMC_INIT and PMC_RELEASE usageThomas Richter
Simplify the use of constants PMC_INIT and PMC_RELEASE. Suggested-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-25s390/cpum_cf: merge source files for CPU Measurement counter facilityThomas Richter
With no in-kernel user, the source files can be merged. Move all functions and the variable definitions to file perf_cpum_cf.c This file now contains all the necessary functions and definitions for the CPU Measurement counter facility device driver. The files cpu_mcf.h and perf_cpum_cf_common.c are deleted. Signed-off-by: Thomas Richter <tmricht@linux.ibm.com> Acked-by: Hendrik Brueckner <brueckner@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-08-04 05:03:07 +0000