mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-08-07 07:25:58 +00:00
a1f96d4fc2
* ci: prepare for amd rocm ci Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: fix editorconfig-checker Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: fix device not recognised Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: rename gpu-amd to gpu-hip Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: gpu-hip to gpu-rocm haha Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * CUDA: allow integrated-GPU host output buffer in debug assert On integrated GPUs (APUs), the scheduler can legitimately place a graph node's output on the host-visible buffer, which ggml_cuda_compute_forward already handles. The debug assert in ggml_cuda_graph_evaluate_and_capture required every node output to be on the device buffer, so a debug build aborts on such a node (e.g. attn_residual ADD -> ROCm_Host on RDNA3.5). The source-tensor assert directly below already permits this via the integrated + cuda_host exception; apply the same exception to the node's own output buffer. Debug-only; no effect on release/compute. Fixes test-recurrent-state-rollback on gfx1151 (Strix Halo). * ci: enable unified memory for ROCm gfx1151 job Work around a coherence issue on integrated RDNA3.5 (gfx1151) where GPU kernels reading mmap-loaded weights can return incorrect output, which makes test-llama-archs (and real inference) intermittently wrong. GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 uses managed memory, which restores coherence. Remove once the underlying ROCm/HIP issue is fixed. * test-llama-archs: skip jamba on HIP backend jamba produces incorrect output (~0.55 NMSE vs CPU) on the HIP backend on RDNA3.5 (gfx1151); the SSM kernels need separate investigation. Skip it for now, matching the existing per-backend carve-outs (WebGPU), so the ROCm CI can run the test for the remaining architectures. * ci: use HIP_LAUNCH_BLOCKING for ROCm gfx1151 job The gfx1151 ROCm CI job produced incorrect inference output (qwen3 perplexity ~88 vs ~9.4) due to an async-execution correctness issue in the HIP path. Serializing kernel launches with HIP_LAUNCH_BLOCKING=1 restores correctness. This replaces the earlier GGML_CUDA_ENABLE_UNIFIED_MEMORY workaround, which did not fix batched inference. * test-backend-sampler: skip top-k subtests on HIP backend The ROCm backend does not support the TOP_K/ARGSORT op at vocab scale (no CUB; bitonic argsort is capped at ncols <= 1024), so top-k/top-p backend samplers cannot be offloaded. The penalties, set_sampler, mixed, and top_p subtests assert that offload happened, so they fail on HIP. Skip them until TOP_K is supported on the ROCm backend. * Update tests/test-backend-sampler.cpp Co-authored-by: Aaron Teo <taronaeo@gmail.com> * Update tests/test-backend-sampler.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> Co-authored-by: Aaron Teo <aaron.teo1@ibm.com> Co-authored-by: Jim Wu <ywu@xilinx.com> Co-authored-by: Aaron Teo <taronaeo@gmail.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> |
||
|---|---|---|
| .. | ||
| actions | ||
| ISSUE_TEMPLATE | ||
| workflows | ||
| labeler.yml | ||
| pull_request_template.md | ||