summaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx9.asm
AgeCommit message (Collapse)Author
2025-01-24drm/amdkfd: Clear MODE.VSKIP in gfx9 trap handlerJay Cornwall
If user shader issues S_SETVSKIP then this state will persist when executing the trap handler, causing vector instructions to be skipped. VSKIP state is already saved/restored through the MODE register. Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Reviewed-by: Lancelot Six <lancelot.six@amd.com> Suggested-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2024-12-10drm/amdkfd: Handle save/restore of lds allocated in 1280B blocksLancelot SIX
The gfx-9 trap handler is reading LDS allocation size in 256 bytes granularity (from SQ_WAVE_LDS_ALLOC), but it using the assumption that this value is always even (i.e. the LDS allocation is really done in multiple of 512 bytes). This was true so far, but gfx-950 allocates LDS in chunks of 1280 bytes, making this assumption invalid. This can cause the trap handler to try to save / restore past the end of LDS, and past the LDS allocated slot in the save are, overriding data from the following wave. This patch updates the trap handler to support LDS allocated in 1280 bytes blocks: - During restore, copy from main memory directly to LDS in batch of 1280 bytes. - During save, continue to use 512 bytes blocks (we only have 2 VGPRs we can use to hold data), making sure to mask the upper half of the wave when handling when the LDS size is not a multiple of 512 bytes. Signed-off-by: Lancelot SIX <lancelot.six@amd.com> Co-authored-by: Alex Sierra <alex.sierra@amd.com> Reviewed-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2024-12-10drm/amdkfd: Adjust CWSR trap handler for gfx950Lancelot SIX
In gfx950, the SQ_WAVE_LDS_ALLOC.LDS_SIZE field is extended to bits 12 to 22. The LDS_SIZE granularity remains unchanged (units of 64 dwords, or 256 bytes). This patch adjusts the CWSR trap handler to read the full extent of LDS_SIZE. Signed-off-by: Lancelot SIX <lancelot.six@amd.com> Reviewed-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2024-12-10drm/amdkfd: update buffer_{store,load}_* modifiers for gfx940Lancelot SIX
Instruction modifiers of the untyped vector memory buffer instructions (MUBUF encoded) changed in gfx940. The slc, scc and glc modifiers have been replaced with sc0, sc1 and nt. The current CWSR trap handler is written using pre-gfx940 modifier names, making the source incompatible with a strict gfx940 assembler. This patch updates the cwsr_trap_handler_gfx9.s source file to be compatible with all gfx9 variants of the ISA. The binary assembled code is unchanged (so the behaviour is unchanged as well), only the source representation is updated. Signed-off-by: Lancelot SIX <lancelot.six@amd.com> Reviewed-by: Jay Cornwall <jay.cornwall@amd.com> Acked-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2024-01-31drm/amdkfd: Use S_ENDPGM_SAVED in trap handlerJay Cornwall
This instruction has no functional difference to S_ENDPGM but allows performance counters to track save events correctly. Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Reviewed-by: Laurent Morichetti <laurent.morichetti@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2023-08-07drm/amdkfd: Sign-extend TMA address in trap handlerJay Cornwall
SMEM instructions can reach addresses above 47 bits but require bit 47 to be sign-extended through bits [63:48]. This allows the TMA to be relocated in a following patch. Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2023-03-31drm/amdkfd: Trap handler changes for GC 9.4.3 v2Jay Cornwall
v1: Check new exception bits in TRAPSTS register Remove single step exception workaround, now part of exception bits v2: GC 9.4.3 uses ttmp11 to store {1’b0, dispatch index [24:0], wave_id_in_workgroup[5:0]}, so use ttmp13 instead of ttmp11 to preserve ib_sts. (Laurent) Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Laurent Morichetti <Laurent.Morichetti@amd.com> Reviewed-by: Laurent Morichetti <laurent.morichetti@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2022-05-26drm/amdkfd: port cwsr trap handler from dkms branchEric Huang
Most of changes are for debugger feature, and it is to simplify trap handler support for new asics in the future. Signed-off-by: Eric Huang <jinhuieric.huang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-03-23drm/amdkfd: Fix saving the ACC vgprs for AldebaranLaurent Morichetti
get_num_acc_vgprs does not set status.scc if the number of acc vgprs is 0, so use an and instruction to set the condition code. The Aldebaran handler binary was not based on the latest version of the sources, so this update to the binary is the minimal change only adding two instructions to set the condition code. A newer version of the handler should be generated and tested in another commit. Signed-off-by: Laurent Morichetti <laurent.morichetti@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-03-10drm/amdkfd: Add aldebaran trap handler supportJay Cornwall
Similar to arcturus, but ARCH/ACC VGPRs may now be split unevenly. A new field in SQ_WAVE_GPR_ALLOC tracks the boundary between the two sets of VGPRs. Squash below patches: drm/amdkfd: Use preprocessor for IP-specific trap handler code drm/amdkfd: Fix VGPR restore race in gfx8/gfx9 trap handler drm/amdkfd: Remove duplicated code in gfx9 trap handler drm/amdkfd: Separate ARCH/ACC VGPR restore in trap handler drm/amdkfd: Reverse order of ARCH/ACC VGPR restore in trap handler Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-07-30drm/amdkfd: Remove dead code from gfx8/gfx9 trap handlersJay Cornwall
Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-07-30drm/amdkfd: Fix lost single step exceptions in gfx9 trap handlerJay Cornwall
If the trap is entered due to MODE.DEBUG_EN=1 and SAVECTX is raised concurrently the handler cannot identify the source of the exception. This causes the debugger to lose single step exception notification when a context save request arrives at the same time. When MODE.DEBUG_EN=1 and STATUS.HALT=0 (exception not already handled) jump to the second-level trap handler upon entering the trap. The second-level trap will set STATUS.HALT=1 and return to the shader. If SAVECTX was raised then control flow will return to the trap, which will then handle the context save request. Cc: Tony Tye <tony.tye@amd.com> Cc: Laurent Morichetti <laurent.morichetti@amd.com> Cc: Qingchuan Shi <qingchuan.shi@amd.com> Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Reviewed-by: Laurent Morichetti <laurent.morichetti@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-07-30drm/amdkfd: Use SQC when TCP would fail in gfx9 context save.Jay Cornwall
When a wavefront raises TRAPSTS.XNACK_ERROR with STATUS.ALLOW_REPLAY=0 subsequent memory instructions have undefined behavior. In practice SQC stores continue to work but TCP stores do not. Context save is permitted to fail after XNACK error because the wavefront will be halted and subsequently terminated. However the debugger has an interest in retrieving the wavefront VGPR/LDS state. Detect the out-of-spec case and use SQC stores during context save in place of TCP stores. Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-07-18drm/amdkfd: Merge gfx9/arcturus trap handlers, add ACC VGPR saveJay Cornwall
ACC VGPRs are a secondary VGPR set of same size as the primary VGPRs. Save them as a block immediately following VGPRs. Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-05-24drm/amdkfd: Preserve ttmp[4:5] instead of ttmp[14:15]Jay Cornwall
ttmp[4:5] is initialized by the SPI with SPI_GDBG_TRAP_DATA* values. These values are more useful to the debugger than ttmp[14:15], which carries dispatch_scratch_base*. There are too few registers to preserve both. Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-05-24drm/amdkfd: Fix gfx9 XNACK state save/restoreJay Cornwall
SQ_WAVE_IB_STS.RCNT grew from 4 bits to 5 in gfx9. Do not truncate when saving in the high bits of TTMP1. Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-05-24drm/amdkfd: Preserve wave state after instruction fetch MEM_VIOLJay Cornwall
If instruction fetch fails the wave cannot be halted and returned to the shader without raising MEM_VIOL again. Currently the wave is terminated if this occurs, but this loses information about the cause of the fault. The debugger would prefer the faulting wave state to be context-saved. Poll inside the trap handler until TRAPSTS.SAVECTX indicates context save is ready. Exit the poll loop and complete the remainder of the exception handler, then return to the shader. The next instruction fetch will be from the trap handler and not the faulting PC. Context save will then deschedule the wave and save its state. Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-11drm/amdkfd: Fix race between scheduler and context restoreJay Cornwall
The scheduler may raise SQ_WAVE_STATUS.SPI_PRIO via SQ_CMD before context restore has completed. Restoring SPI_PRIO=0 after this point may cause context save to fail as the lower priority wavefronts are not selected for execution among spin-waiting wavefronts. Leave SPI_PRIO at its SPI-initialized or scheduler-raised value. v2: Also fix race with exception handler Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-05-01drm/amdkfd: Fix CP soft hang on APUsYong Zhao
The problem happens on Raven and Carrizo. The context save handler should not clear the high bits of PC_HI before extracting the bits of IB_STS. The bug is not relevant to VEGA10 until we enable demand paging. Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-05-01drm/amdkfd: Separate trap handler assembly code and its hex valuesYong Zhao
Since the assembly code is inside "#if 0", it is ineffective. Despite that, during debugging, we need to change the assembly code, extract it into a separate file and compile the new file into hex values using sp3. That process also requires us to remove "#if 0" and modify lines starting with "#", so that sp3 can successfully compile the new file. With this change, all the above chore is no longer needed, and cwsr_trap_handler_gfx*.asm can be directly used by sp3 to generate its hex values. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-05-01drm/amdkfd: Reduce priority of context-saving waves before spin-waitJay Cornwall
Synchronization between context-saving wavefronts is achieved by sending a SAVEWAVE message to the SPI and then spin-waiting for a response. These spin-waiting wavefronts may inhibit the progress of other wavefronts in the context save handler, leading to the synchronization condition never being achieved. Before spin-waiting reduce the priority of each wavefront to guarantee foward progress in the others. Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-04-10drm/amdkfd: Add GFXv9 CWSR trap handlerFelix Kuehling
Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-08-03 11:37:51 +0000