summaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
AgeCommit message (Collapse)Author
2025-06-30drm/amd: Do not include <linux/export.h> when unusedAndré Almeida
Fix the following compile time warning when building with W=1: warning: EXPORT_SYMBOL() is not used, but #include <linux/export.h> is present Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: André Almeida <andrealmeid@igalia.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2024-07-01drm/amdkfd: Use device based logging for errorsLijo Lazar
Convert some pr_* to some dev_* APIs to identify the device. Signed-off-by: Lijo Lazar <lijo.lazar@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2024-02-16drm/amdkfd: Relocate TBA/TMA to opposite side of VM holeFelix Kuehling
The TBA and TMA, along with an unused IB allocation, reside at low addresses in the VM address space. A stray VM fault which hits these pages must be serviced by making their page table entries invalid. The scheduler depends upon these pages being resident and fails, preventing a debugger from inspecting the failure state. By relocating these pages above 47 bits in the VM address space they can only be reached when bits [63:48] are set to 1. This makes it much less likely for a misbehaving program to generate accesses to them. The current placement at VA (PAGE_SIZE*2) is readily hit by a NULL access with a small offset. v2: - Move it to the reserved space to avoid concflicts with Mesa - Add macros to make reserved space management easier v3: - Move VM max PFN calculation into AMDGPU_VA_RESERVED macros Cc: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> Cc: Christian Koenig <christian.koenig@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2024-01-05Revert "drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole"Kaibo Ma
That commit causes NULL pointer dereferences in dmesgs when running applications using ROCm, including clinfo, blender, and PyTorch, since v6.6.1. Revert it to fix blender again. This reverts commit 96c211f1f9ef82183493f4ceed4e347b52849149. Closes: https://github.com/ROCm/ROCm/issues/2596 Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/2991 Reviewed-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Kaibo Ma <ent3rm4n@gmail.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2023-08-11drm/amdkfd: drop IOMMUv2 supportAlex Deucher
Now that we use the dGPU path for all APUs, drop the IOMMUv2 support. v2: drop the now unused queue manager functions for gfx7/8 APUs Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Tested-by: Mike Lothian <mike@fireburn.co.uk> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2023-08-07drm/amdkfd: Relocate TBA/TMA to opposite side of VM holeJay Cornwall
The TBA and TMA, along with an unused IB allocation, reside at low addresses in the VM address space. A stray VM fault which hits these pages must be serviced by making their page table entries invalid. The scheduler depends upon these pages being resident and fails, preventing a debugger from inspecting the failure state. By relocating these pages above 47 bits in the VM address space they can only be reached when bits [63:48] are set to 1. This makes it much less likely for a misbehaving program to generate accesses to them. The current placement at VA (PAGE_SIZE*2) is readily hit by a NULL access with a small offset. Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2023-06-09drm/amdkfd: Introduce kfd_node struct (v5)Mukul Joshi
Introduce a new structure, kfd_node, which will now represent a compute node. kfd_node is carved out of kfd_dev structure. kfd_dev struct now will become the parent of kfd_node, and will store common resources such as doorbells, GTT sub-alloctor etc. kfd_node struct will store all resources specific to a compute node, such as device queue manager, interrupt handling etc. This is the first step in adding compute partition support in KFD. v2: introduce kfd_node struct to gc v11 (Hawking) v3: make reference to kfd_dev struct through kfd_node (Morris) v4: use kfd_node instead for kfd isr/mqd functions (Morris) v5: rebase (Alex) Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Tested-by: Amber Lin <Amber.Lin@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com> Signed-off-by: Morris Zhang <Shiwu.Zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2022-02-14drm/amdkfd: Fix leftover errors and warningsRajneesh Bhardwaj
A bunch of errors and warnings are leftover KFD over the years, attempt to fix the errors and most warnings reported by checkpatch tool. Still a few warnings remain which may be false positives so ignore them for now. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2022-02-14drm/amdkfd: update SPDX license headerRajneesh Bhardwaj
Update the SPDX License header for all the KFD files. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-11-17drm/amdkfd: replace asic_family with asic_typeGraham Sider
asic_family was a duplicate of asic_type, both of type amd_asic_type. Replace all instances of device_info->asic_family with adev->asic_type and remove asic_family from device_info. Signed-off-by: Graham Sider <Graham.Sider@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-11-17drm/amdkfd: convert switches to IP version checkingGraham Sider
Converts KFD switch statements to use IP version checking instead of asic_type. Signed-off-by: Graham Sider <Graham.Sider@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-11-09drm/amdkfd: lower the VAs base offset to 8KBAlex Sierra
The low 16MB of virtual address space are currently reserved for kernel mode allocations mapped into user virtual address space. This causes conflicts with HMM/SVM mappings at low virtual addresses. We tried to move those kernel mode allocations to the upper half of the 64-bit virtual address space for GFX9, which is naturally reserved for kernel use. However, TBA (trap handler code) has problems to access addresses in the high virtual space. We have decided to set this to 8KB of the lower address space as a temporary fix, while investigate TBA address problem. It is very unlikely for user space to map memory at this low region. Signed-off-by: Alex Sierra <alex.sierra@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-07-23drm/amdkfd: enable cyan_skillfish KFDTao Zhou
Add KFD support for cyan_skillfish. v2: whitespace fixes (Alex) Signed-off-by: Tao Zhou <tao.zhou1@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-15drm/amdkfd: Disable SVM per GPU, not per processFelix Kuehling
When some GPUs don't support SVM, don't disabe it for the entire process. That would be inconsistent with the information the process got from the topology, which indicates SVM support per GPU. Instead disable SVM support only for the unsupported GPUs. This is done by checking any per-device attributes against the bitmap of supported GPUs. Also use the supported GPU bitmap to initialize access bitmaps for new SVM address ranges. Don't handle recoverable page faults from unsupported GPUs. (I don't think there will be unsupported GPUs that can generate recoverable page faults. But better safe than sorry.) Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Philip Yang <philip.yang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04drm/amdkfd: add yellow carp KFD supportAaron Liu
This patch is to add GFX10 based Yellow Carp KFD support. We will bypass IOMMU v2. Signed-off-by: Aaron Liu <aaron.liu@amd.com> Reviewed-by: Huang Rui <ray.huang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-05-19drm/amdkfd: support beige_goby KFDChengming Gui
Add KFD support for beige_goby v2: fix asic name typo v3: squash in updates (Alex) v4: squash in needs_atomics fix (Alex) Signed-off-by: Chengming Gui <Jack.Gui@amd.com> Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com> Reviewed-by: Tao Zhou <tao.zhou1@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-04-20drm/amdkfd: add svm ioctl APIPhilip Yang
Add svm (shared virtual memory) ioctl data structure and API definition. The svm ioctl API is designed to be extensible in the future. All operations are provided by a single IOCTL to preserve ioctl number space. The arguments structure ends with a variable size array of attributes that can be used to set or get one or multiple attributes. Signed-off-by: Philip Yang <Philip.Yang@amd.com> Signed-off-by: Alex Sierra <alex.sierra@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-03-10drm/amdkfd: Add Aldebaran KFD supportYong Zhao
Add initial KFD support. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2020-10-12drm/amdkfd: Support dimgrey_cavefish KFD (v2)Chengming Gui
Add KFD support for dimgrey cavefish. v2: rebase (Alex) Signed-off-by: Chengming Gui <Jack.Gui@amd.com> Reviewed-by: Tao Zhou <tao.zhou1@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2020-10-05drm/amdkfd: add Van Gogh KFD supportHuang Rui
This patch is to add GFX10 based APU Van Gogh KFD support. We will treat Van Gogh as "dgpu" (bypass IOMMU v2). Signed-off-by: Huang Rui <ray.huang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Reviewed-by: Yong Zhao <Yong.Zhao@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2020-08-26drm/amdkfd: implement the dGPU fallback path for apu (v6)Huang Rui
We still have a few iommu issues which need to address, so force raven as "dgpu" path for the moment. This is to add the fallback path to bypass IOMMU if IOMMU v2 is disabled or ACPI CRAT table not correct. v2: Use ignore_crat parameter to decide whether it will go with IOMMUv2. v3: Align with existed thunk, don't change the way of raven, only renoir will use "dgpu" path by default. v4: don't update global ignore_crat in the driver, and revise fallback function if CRAT is broken. v5: refine acpi crat good but no iommu support case, and rename the title. v6: fix the issue of dGPU initialized firstly, just modify the report value in the node_show(). Signed-off-by: Huang Rui <ray.huang@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2020-07-15drm/amdkfd: Support navy_flounder KFDChengming Gui
Add KFD support for Navy Flounder. Signed-off-by: Chengming Gui <Jack.Gui@amd.com> Reviewed-by: Tao Zhou <tao.zhou1@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2020-07-01drm/amdkfd: Support Sienna_Cichlid KFD v4Yong Zhao
v4: drop get_tile_config, comment out other callbacks Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2020-03-19drm: amd: fix spelling mistake "shoudn't" -> "shouldn't"Colin Ian King
There are spelling mistakes in pr_err messages and a comment. Fix these. Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-10-07drm/amdkfd: Check against device cgroupHarish Kasiviswanathan
Participate in device cgroup. All kfd devices are exposed via /dev/kfd. So use /dev/dri/renderN node. Before exposing the device to a task check if it has permission to access it. If the task (based on its cgroup) can access /dev/dri/renderN then expose the device via kfd node. If the task cannot access /dev/dri/renderN then process device data (pdd) is not created. This will ensure that task cannot use the device. In sysfs topology, all device nodes are visible irrespective of the task cgroup. The sysfs node directories are created at driver load time and cannot be changed dynamically. However, access to information inside nodes is controlled based on the task's cgroup permissions. Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-10-03drm/amdkfd: use navi12 specific family id for navi12 code pathshaoyunl
Keep the same use of CHIP_IDs for navi12 in kfd Signed-off-by: shaoyunl <shaoyun.liu@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-09-16drm/amdkfd: init kfd apertures v9 for renoirHuang Rui
Renoir is GMC v9, so init v9 kfd apertures. Signed-off-by: Huang Rui <ray.huang@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-09-16drm/amdkfd: Support Navi14 in KFDYong Zhao
Initial support of Navi14 in KFD. The device IDs will be added later. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-07-18amd/amdkfd: Add ASIC ARCTURUS to kfdYong Zhao
Add initial support for ARCTURUS to kfd. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Signed-off-by: Oak Zeng <Oak.Zeng@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-06-21drm/amdkfd: Add navi10 support to amdkfd. (v3)Philip Cox
KFD (kernel fusion driver) is the kernel driver for the compute backend for usermode compute stack. v2: squash in updates (Alex) v3: squash in rebase fixes (Alex) Signed-off-by: Oak Zeng <Oak.Zeng@amd.com> Signed-off-by: Philip Cox <Philip.Cox@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-05-24drm/amdkfd: Add VegaM supportKent Russell
Add the VegaM information to KFD Signed-off-by: Kent Russell <kent.russell@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-05-24drm/amdkfd: Cosmetic cleanupKent Russell
Fix some spacing issues, log output, uses of !=NULL/==NULL, unneeded extra lines and clean up a declaration from =1 to =true for clarity Signed-off-by: Kent Russell <kent.russell@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-19drm/amdkfd: Added Vega12 and Polaris12 for KFD.Gang Ba
Add Vega12 and Polaris12 device info and device IDs to KFD. Signed-off-by: Gang Ba <gaba@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-09-26drm/amdkfd: Vega20 bring up on amdkfd sideShaoyun Liu
Add Vega20 device IDs, device info and enable it in KFD. Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
2018-04-10drm/amdkfd: Support flat memory apertures for GFXv9Felix Kuehling
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-04-10drm/amdkfd: Remove limit on number of GPUs (follow-up)Felix Kuehling
This condition was missed in a previous commit with the same title. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-03-15drm/amdkfd: Aperture setup for dGPUsFelix Kuehling
Set up the GPUVM aperture for SVM (shared virtual memory) that allows sharing a part of virtual address space between GPUs and CPUs. Report the size of the GPUVM aperture that is supported by KGD accurately. The low part of the GPUVM aperture is reserved for kernel use. This is for kernel-allocated buffers that are only accessed on the GPU: - CWSR trap handler - IB for submitting commands in user-mode context from kernel mode Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-03-15drm/amdkfd: Remove limit on number of GPUsFelix Kuehling
Currently the number of GPUs is limited by aperture placement options available on GFX7 and GFX8 hardware. This limitation is not necessary. Scratch and LDS represent per-work-item and per-work-group storage respectively. Different work-items and work-groups use the same virtual address to access their own data. Work running on different GPUs is by definition in different work-groups (different dispatches, in fact). That means the same virtual addresses can be used for these apertures on different GPUs. Add a new AMDKFD_IOC_GET_PROCESS_APERTURES_NEW ioctl that removes the artificial limitation on the number of GPUs that can be supported. The new ioctl allows user mode to query the number of GPUs to allocate enough memory for all GPUs to be reported. This deprecates AMDKFD_IOC_GET_PROCESS_APERTURES. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2017-12-08drm/amdkfd: Support enumerating non-GPU devicesHarish Kasiviswanathan
Modify kfd_topology_enum_kfd_devices(..) function to support non-GPU nodes. The function returned NULL when it encountered non-GPU (say CPU) nodes. This caused kfd_ioctl_create_event and kfd_init_apertures to fail for Intel + Tonga. kfd_topology_enum_kfd_devices will now parse all the nodes and return valid kfd_dev for nodes with GPU. Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2017-08-15drm/amdkfd: Change x==NULL/false references to !xKent Russell
Upstream prefers the !x notation to x==NULL or x==false. Along those lines change the ==true or !=NULL references as well. Also make the references to !x the same, excluding () for readability. Signed-off-by: Kent Russell <kent.russell@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-08-30amdkfd: use <linux/mman.h> instead of <uapi/asm-generic/mman-common.h>Christoph Hellwig
The latter is a default version of <asm/mman.h> and not for driver use. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-01-09drm/amdkfd: Process-device data creation and lookup splitAlexey Skidanov
This patch splits the current kfd_get_process_device_data() to two functions, one that specifically creates a pdd and another one which just do lookup. This is done to enhance the readability and maintainability of the code. Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-11-18amdkfd: init aperture once per processAlexey Skidanov
Since the user space may call open() more that once from the same process, the aperture initialization should be moved from kfd_open() Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-11-20amdkfd: Fix sparse warnings in kfd_flat_memory.cOded Gabbay
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-16amdkfd: Add basic modules to amdkfdOded Gabbay
This patch adds the process module and three helper modules: - kfd_process, which handles process which open /dev/kfd - kfd_doorbell, which provides helper functions for doorbell allocation, release and mapping to userspace - kfd_pasid, which provides helper functions for pasid allocation and release - kfd_aperture, which provides helper functions for managing the LDS, Local GPU memory and Scratch memory apertures of the process This patch only contains the basic kfd_process module, which doesn't contain the reference to the queue scheduler. This was done to allow easier code review. Also, this patch doesn't contain the calls to the IOMMU driver for binding the pasid to the device. Again, this was done to allow easier code review The kfd_process object is created when a process opens /dev/kfd and is closed when the mm_struct of that process is teared-down. v3: Removed kfd_vidmem.c file Replaced direct mmput call to mmu_notifier release Removed typedefs Moved bool field to end of the structure Added new kernel params for gart usage limitation Added initialization of sa manager Fixed debug messages Remove support for LDS in 32 bit Changed code to support mmap of doorbell pages from userspace Added documentation for apertures v4: Replaced RCU by SRCU for kfd_process list management v5: Move amdkfd from drm/radeon/ to drm/amd/ Rename kfd_aperture.c to kfd_flat_memory.c Protect against multiple init calls MQD size is H/W dependent so moved it to device info structure Rename kfd_mem_obj structure's members Use delayed function for process tear-down Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>