summaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/panfrost/panfrost_mmu.c
AgeCommit message (Collapse)Author
2019-09-19Merge tag 'drm-next-2019-09-18' of git://anongit.freedesktop.org/drm/drmLinus Torvalds
Pull drm updates from Dave Airlie: "This is the main pull request for 5.4-rc1 merge window. I don't think there is anything outstanding so next week should just be fixes, but we'll see if I missed anything. I landed some fixes earlier in the week but got delayed writing summary and sending it out, due to a mix of sick kid and jetlag! There are some fixes pending, but I'd rather get the main merge out of the way instead of delaying it longer. It's also pretty large in commit count and new amd header file size. The largest thing is four new amdgpu products (navi12/14, arcturus and renoir APU support). Otherwise it's pretty much lots of work across the board, i915 has started landing tigerlake support, lots of icelake fixes and lots of locking reworking for future gpu support, lots of header file rework (drmP.h is nearly gone), some old legacy hacks (DRM_WAIT_ON) have been put into the places they are needed. uapi: - content protection type property for HDCP core: - rework include dependencies - lots of drmP.h removals - link rate calculation robustness fix - make fb helper map only when required - add connector->DDC adapter link - DRM_WAIT_ON removed - drop DRM_AUTH usage from drivers dma-buf: - reservation object fence helper dma-fence: - shrink dma_fence struct - merge signal functions - store timestamps in dma_fence - selftests ttm: - embed drm_get_object struct into ttm_buffer_object - release_notify callback bridges: - sii902x - audio graph card support - tc358767 - aux data handling rework - ti-snd64dsi86 - debugfs support, DSI mode flags support panels: - Support for GiantPlus GPM940B0, Sharp LQ070Y3DG3B, Ortustech COM37H3M, Novatek NT39016, Sharp LS020B1DD01D, Raydium RM67191, Boe Himax8279d, Sharp LD-D5116Z01B - TI nspire, NEC NL8048HL11, LG Philips LB035Q02, Sharp LS037V7DW01, Sony ACX565AKM, Toppoly TD028TTEC1 Toppoly TD043MTEA1 i915: - Initial tigerlake platform support - Locking simplification work, general all over refactoring. - Selftests - HDCP debug info improvements - DSI properties - Icelake display PLL fixes, colorspace fixes, bandwidth fixes, DSI suspend/resume - GuC fixes - Perf fixes - ElkhartLake enablement - DP MST fixes - GVT - command parser enhancements amdgpu: - add wipe memory on release flag for buffer creation - Navi12/14 support (may be marked experimental) - Arcturus support - Renoir APU support - mclk DPM for Navi - DC display fixes - Raven scatter/gather support - RAS support for GFX - Navi12 + Arcturus power features - GPU reset for Picasso - smu11 i2c controller support amdkfd: - navi12/14 support - Arcturus support radeon: - kexec fix nouveau: - improved display color management - detect lack of GPU power cables vmwgfx: - evicition priority support - remove unused security feature msm: - msm8998 display support - better async commit support for cursor updates etnaviv: - per-process address space support - performance counter fixes - softpin support mcde: - DCS transfers fix exynos: - drmP.h cleanup lima: - reduce logging kirin: - misc clenaups komeda: - dual-link support - DT memory regions hisilicon: - misc fixes imx: - IPUv3 image converter fixes - 32-bit RGB V4L2 pixel format support ingenic: - more support for panel related cases mgag200: - cursor support fix panfrost: - export GPU features register to userspace - gpu heap allocations - per-fd address space support pl111: - CLD pads wiring support removed from DT rockchip: - rework to use DRM PSR helpers - fix bug in VOP_WIN_GET macro - DSI DT binding rework sun4i: - improve support for color encoding and range - DDC enabled GPIO tinydrm: - rework SPI support - improve MIPI-DBI support - moved to drm/tiny vkms: - rework CRC tracking dw-hdmi: - get_eld and i2s improvements gm12u320: - misc fixes meson: - global code cleanup - vpu feature detect omap: - alpha/pixel blend mode properties rcar-du: - misc fixes" * tag 'drm-next-2019-09-18' of git://anongit.freedesktop.org/drm/drm: (2112 commits) drm/nouveau/bar/gm20b: Avoid BAR1 teardown during init drm/nouveau: Fix ordering between TTM and GEM release drm/nouveau/prime: Extend DMA reservation object lock drm/nouveau: Fix fallout from reservation object rework drm/nouveau/kms/nv50-: Don't create MSTMs for eDP connectors drm/i915: Use NOEVICT for first pass on attemping to pin a GGTT mmap drm/i915: to make vgpu ppgtt notificaiton as atomic operation drm/i915: Flush the existing fence before GGTT read/write drm/i915: Hold irq-off for the entire fake lock period drm/i915/gvt: update RING_START reg of vGPU when the context is submitted to i915 drm/i915/gvt: update vgpu workload head pointer correctly drm/mcde: Fix DSI transfers drm/msm: Use the correct dma_sync calls harder drm/msm: remove unlikely() from WARN_ON() conditions drm/msm/dsi: Fix return value check for clk_get_parent drm/msm: add atomic traces drm/msm/dpu: async commit support drm/msm: async commit support drm/msm: split power control from prepare/complete_commit drm/msm: add kms->flush_commit() ...
2019-09-19drm/panfrost: Prevent race when handling page faultSteven Price
When handling a GPU page fault addr_to_drm_mm_node() is used to translate the GPU address to a buffer object. However it is possible for the buffer object to be freed after the function has returned resulting in a use-after-free of the BO. Change addr_to_drm_mm_node to return the panfrost_gem_object with an extra reference on it, preventing the BO from being freed until after the page fault has been handled. Signed-off-by: Steven Price <steven.price@arm.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190913160310.50444-1-steven.price@arm.com
2019-08-30drm/panfrost: Flush and disable address space when freeing page tablesRob Herring
Currently, page tables are freed without disabling the address space first. This probably is fine as we'll switch to new page tables when the address space is allocated again and runtime PM suspend will reset the GPU clearing the registers. However, it's better to clean up after ourselves. There is also a problem that we could be accessing the h/w in tlb_inv_context() when suspended. Rework the disable code to make sure we flush caches/TLBs and disable the address space before freeing the page tables if we are not suspended. As the tlb_inv_context() hook is only called when freeing the page tables and we do a flush before disabling the AS, lets remove the flush from tlb_inv_context and avoid any runtime PM issues. Fixes: 7282f7645d06 ("drm/panfrost: Implement per FD address spaces") Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Steven Price <steven.price@arm.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190826223317.28509-8-robh@kernel.org
2019-08-30drm/panfrost: Add cache/TLB flush before switching address spaceRob Herring
It's not entirely clear if this is required, but add a flush of GPU caches and TLBs before we change an address space to new page tables. Fixes: 7282f7645d06 ("drm/panfrost: Implement per FD address spaces") Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Steven Price <steven.price@arm.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190826223317.28509-7-robh@kernel.org
2019-08-30drm/panfrost: Split mmu_hw_do_operation into locked and unlocked versionRob Herring
In preparation to call mmu_hw_do_operation with the as_lock already held, Add a mmu_hw_do_operation_locked function. Fixes: 7282f7645d06 ("drm/panfrost: Implement per FD address spaces") Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Steven Price <steven.price@arm.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190826223317.28509-6-robh@kernel.org
2019-08-30drm/panfrost: Rework page table flushing and runtime PM interactionRob Herring
There is no point in resuming the h/w just to do flush operations and doing so takes several locks which cause lockdep issues with the shrinker. Rework the flush operations to only happen when the h/w is already awake. This avoids taking any locks associated with resuming which trigger lockdep warnings. Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support") Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Steven Price <steven.price@arm.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190826223317.28509-5-robh@kernel.org
2019-08-30drm/panfrost: Remove unnecessary mmu->lock mutexRob Herring
There's no need to serialize io-pgtable calls and the as_lock is sufficient to serialize flush operations, so we can remove the per page table lock. Fixes: 7282f7645d06 ("drm/panfrost: Implement per FD address spaces") Suggested-by: Robin Murphy <robin.murphy@arm.com> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Steven Price <steven.price@arm.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190826223317.28509-4-robh@kernel.org
2019-08-19drm/panfrost: Implement per FD address spacesRob Herring
Up until now, a single shared GPU address space was used. This is not ideal as there's no protection between processes and doesn't work for supporting the same GPU/CPU VA feature. Most importantly, this will hopefully mitigate Alyssa's fear of WebGL, whatever that is. Most of the changes here are moving struct drm_mm and struct panfrost_mmu objects from the per device struct to the per FD struct. The critical function is panfrost_mmu_as_get() which handles allocating and switching the h/w address spaces. There's 3 states an AS can be in: free, allocated, and in use. When a job runs, it requests an address space and then marks it not in use when job is complete(but stays assigned). The first time thru, we find a free AS in the alloc_mask and assign the AS to the FD. Then the next time thru, we most likely already have our AS and we just mark it in use with a ref count. We need a ref count because we have multiple job slots. If the job/FD doesn't have an AS assigned and there are no free ones, then we pick an allocated one not in use from our LRU list and switch the AS from the old FD to the new one. Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Cc: Robin Murphy <robin.murphy@arm.com> Cc: Steven Price <steven.price@arm.com> Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Steven Price <steven.price@arm.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190813150115.30338-1-robh@kernel.org
2019-08-19drm/panfrost: Fix missing unlock on error in panfrost_mmu_map_fault_addr()Wei Yongjun
Add the missing unlock before return from function panfrost_mmu_map_fault_addr() in the error handling case. Fixes: 187d2929206e ("drm/panfrost: Add support for GPU heap allocations") Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Reviewed-by: Steven Price <steven.price@arm.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190814044814.102294-1-weiyongjun1@huawei.com
2019-08-12drm/panfrost: Add support for GPU heap allocationsRob Herring
The midgard/bifrost GPUs need to allocate GPU heap memory which is allocated on GPU page faults and not pinned in memory. The vendor driver calls this functionality GROW_ON_GPF. This implementation assumes that BOs allocated with the PANFROST_BO_NOEXEC flag are never mmapped or exported. Both of those may actually work, but I'm unsure if there's some interaction there. It would cause the whole object to be pinned in memory which would defeat the point of this. On faults, we map in 2MB at a time in order to utilize huge pages (if enabled). Currently, once we've mapped pages in, they are only unmapped if the BO is freed. Once we add shrinker support, we can unmap pages with the shrinker. Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: Boris Brezillon <boris.brezillon@collabora.com> Cc: Robin Murphy <robin.murphy@arm.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Steven Price <steven.price@arm.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190808222200.13176-9-robh@kernel.org
2019-08-12drm/panfrost: Convert MMU IRQ handler to threaded handlerRob Herring
In preparation to handle mapping of page faults, we need the MMU handler to be threaded as code paths take a mutex. As the IRQ may be shared, we can't use the default handler and must disable the MMU interrupts locally. Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: Boris Brezillon <boris.brezillon@collabora.com> Cc: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Steven Price <steven.price@arm.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190808222200.13176-8-robh@kernel.org
2019-08-12drm/panfrost: Consolidate reset handlingRob Herring
Runtime PM resume and job timeouts both call the same sequence of functions, so consolidate them to a common function. This will make changing the reset related code easier. The MMU also needs some re-initialization on reset, so rework its call. In the process, we hide the address space details within the MMU code in preparation to support multiple address spaces. Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Cc: Robin Murphy <robin.murphy@arm.com> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Steven Price <steven.price@arm.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190808222200.13176-7-robh@kernel.org
2019-08-12drm/panfrost: Add a no execute flag for BO allocationsRob Herring
Executable buffers have an alignment restriction that they can't cross 16MB boundary as the GPU program counter is 24-bits. This restriction is currently not handled and we just get lucky. As current userspace assumes all BOs are executable, that has to remain the default. So add a new PANFROST_BO_NOEXEC flag to allow userspace to indicate which BOs are not executable. There is also a restriction that executable buffers cannot start or end on a 4GB boundary. This is mostly avoided as there is only 4GB of space currently and the beginning is already blocked out for NULL ptr detection. Add support to handle this restriction fully regardless of the current constraints. For existing userspace, all created BOs remain executable, but the GPU VA alignment will be increased to the size of the BO. This shouldn't matter as there is plenty of GPU VA space. Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: Boris Brezillon <boris.brezillon@collabora.com> Cc: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Steven Price <steven.price@arm.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190808222200.13176-6-robh@kernel.org
2019-08-12drm/panfrost: Split panfrost_mmu_map SG list mapping to its own functionRob Herring
In preparation to create partial GPU mappings of BOs on page faults, split out the SG list handling of panfrost_mmu_map(). Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: Boris Brezillon <boris.brezillon@collabora.com> Cc: Robin Murphy <robin.murphy@arm.com> Reviewed: Steven Price <steven.price@arm.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190808222200.13176-5-robh@kernel.org
2019-07-29iommu/io-pgtable: Pass struct iommu_iotlb_gather to ->unmap()Will Deacon
Update the io-pgtable ->unmap() function to take an iommu_iotlb_gather pointer as an argument, and update the callers as appropriate. Signed-off-by: Will Deacon <will@kernel.org>
2019-07-29iommu/io-pgtable: Remove unused ->tlb_sync() callbackWill Deacon
The ->tlb_sync() callback is no longer used, so it can be removed. Signed-off-by: Will Deacon <will@kernel.org>
2019-07-29iommu/io-pgtable: Replace ->tlb_add_flush() with ->tlb_add_page()Will Deacon
The ->tlb_add_flush() callback in the io-pgtable API now looks a bit silly: - It takes a size and a granule, which are always the same - It takes a 'bool leaf', which is always true - It only ever flushes a single page With that in mind, replace it with an optional ->tlb_add_page() callback that drops the useless parameters. Signed-off-by: Will Deacon <will@kernel.org>
2019-07-29iommu/io-pgtable: Hook up ->tlb_flush_walk() and ->tlb_flush_leaf() in driversWill Deacon
Hook up ->tlb_flush_walk() and ->tlb_flush_leaf() in drivers using the io-pgtable API so that we can start making use of them in the page-table code. For now, they can just wrap the implementations of ->tlb_add_flush and ->tlb_sync pending future optimisation in each driver. Signed-off-by: Will Deacon <will@kernel.org>
2019-07-24iommu/io-pgtable: Rename iommu_gather_ops to iommu_flush_opsWill Deacon
In preparation for TLB flush gathering in the IOMMU API, rename the iommu_gather_ops structure in io-pgtable to iommu_flush_ops, which better describes its purpose and avoids the potential for confusion between different levels of the API. $ find linux/ -type f -name '*.[ch]' | xargs sed -i 's/gather_ops/flush_ops/g' Signed-off-by: Will Deacon <will@kernel.org>
2019-06-18drm/panfrost: Make sure a BO is only unmapped when appropriateBoris Brezillon
mmu_ops->unmap() will fail when called on a BO that has not been previously mapped, and the error path in panfrost_ioctl_create_bo() can call drm_gem_object_put_unlocked() (which in turn calls panfrost_mmu_unmap()) on a BO that has not been mapped yet. Keep track of the mapped/unmapped state to avoid such issues. Fixes: f3ba91228e8e ("drm/panfrost: Add initial panfrost driver") Cc: <stable@vger.kernel.org> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190618081343.16927-1-boris.brezillon@collabora.com
2019-04-12drm/panfrost: Add support for 2MB page entriesRob Herring
Add support for 2MB sized pages. This will improve our map and unmap times and save a bit of memory by avoiding 3rd level page tables for contiguous allocations. As we use shmem for buffers and huge page allocations for shmem are off by default, there isn't an improvement out of the box and userspace must enable THP for shmem. It's not clear if the h/w can support 1GB page sizes which standard ARM long format descriptors support. In any case, it is unlikely we'll see any contiguous 1GB allocations on current h/w. Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Steven Price <steven.price@arm.com> Acked-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190411215313.1937-1-robh@kernel.org
2019-04-12drm/panfrost: Add initial panfrost driverRob Herring
This adds the initial driver for panfrost which supports Arm Mali Midgard and Bifrost family of GPUs. Currently, only the T860 and T760 Midgard GPUs have been tested. v2: - Add GPU reset on job hangs (Tomeu) - Add RuntimePM and devfreq support (Tomeu) - Fix T760 support (Tomeu) - Add a TODO file (Rob, Tomeu) - Support multiple in fences (Tomeu) - Drop support for shared fences (Tomeu) - Fill in MMU de-init (Rob) - Move register definitions back to single header (Rob) - Clean-up hardcoded job submit todos (Rob) - Implement feature setup based on features/issues (Rob) - Add remaining Midgard DT compatible strings (Rob) v3: - Add support for reset lines (Neil) - Add a MAINTAINERS entry (Rob) - Call dma_set_mask_and_coherent (Rob) - Do MMU invalidate on map and unmap. Restructure to do a single operation per map/unmap call. (Rob) - Add a missing explicit padding to struct drm_panfrost_create_bo (Rob) - Fix 0-day error: "panfrost_devfreq.c:151:9-16: ERROR: PTR_ERR applied after initialization to constant on line 150" - Drop HW_FEATURE_AARCH64_MMU conditional (Rob) - s/DRM_PANFROST_PARAM_GPU_ID/DRM_PANFROST_PARAM_GPU_PROD_ID/ (Rob) - Check drm_gem_shmem_prime_import_sg_table() error code (Rob) - Re-order power on sequence (Rob) - Move panfrost_acquire_object_fences() before scheduling job (Rob) - Add NULL checks on array pointers in job clean-up (Rob) - Rework devfreq (Tomeu) - Fix devfreq init with no regulator (Rob) - Various WS and comments clean-up (Rob) Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Maxime Ripard <maxime.ripard@bootlin.com> Cc: Sean Paul <sean@poorly.run> Cc: David Airlie <airlied@linux.ie> Cc: Daniel Vetter <daniel@ffwll.ch> Cc: Lyude Paul <lyude@redhat.com> Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Steven Price <steven.price@arm.com> Signed-off-by: Marty E. Plummer <hanetzer@startmail.com> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Signed-off-by: Neil Armstrong <narmstrong@baylibre.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20190409205427.6943-4-robh@kernel.org