mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-08-07 23:45:30 +00:00
a1f96d4fc2
* ci: prepare for amd rocm ci Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: fix editorconfig-checker Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: fix device not recognised Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: rename gpu-amd to gpu-hip Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: gpu-hip to gpu-rocm haha Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * CUDA: allow integrated-GPU host output buffer in debug assert On integrated GPUs (APUs), the scheduler can legitimately place a graph node's output on the host-visible buffer, which ggml_cuda_compute_forward already handles. The debug assert in ggml_cuda_graph_evaluate_and_capture required every node output to be on the device buffer, so a debug build aborts on such a node (e.g. attn_residual ADD -> ROCm_Host on RDNA3.5). The source-tensor assert directly below already permits this via the integrated + cuda_host exception; apply the same exception to the node's own output buffer. Debug-only; no effect on release/compute. Fixes test-recurrent-state-rollback on gfx1151 (Strix Halo). * ci: enable unified memory for ROCm gfx1151 job Work around a coherence issue on integrated RDNA3.5 (gfx1151) where GPU kernels reading mmap-loaded weights can return incorrect output, which makes test-llama-archs (and real inference) intermittently wrong. GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 uses managed memory, which restores coherence. Remove once the underlying ROCm/HIP issue is fixed. * test-llama-archs: skip jamba on HIP backend jamba produces incorrect output (~0.55 NMSE vs CPU) on the HIP backend on RDNA3.5 (gfx1151); the SSM kernels need separate investigation. Skip it for now, matching the existing per-backend carve-outs (WebGPU), so the ROCm CI can run the test for the remaining architectures. * ci: use HIP_LAUNCH_BLOCKING for ROCm gfx1151 job The gfx1151 ROCm CI job produced incorrect inference output (qwen3 perplexity ~88 vs ~9.4) due to an async-execution correctness issue in the HIP path. Serializing kernel launches with HIP_LAUNCH_BLOCKING=1 restores correctness. This replaces the earlier GGML_CUDA_ENABLE_UNIFIED_MEMORY workaround, which did not fix batched inference. * test-backend-sampler: skip top-k subtests on HIP backend The ROCm backend does not support the TOP_K/ARGSORT op at vocab scale (no CUB; bitonic argsort is capped at ncols <= 1024), so top-k/top-p backend samplers cannot be offloaded. The penalties, set_sampler, mixed, and top_p subtests assert that offload happened, so they fail on HIP. Skip them until TOP_K is supported on the ROCm backend. * Update tests/test-backend-sampler.cpp Co-authored-by: Aaron Teo <taronaeo@gmail.com> * Update tests/test-backend-sampler.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> Co-authored-by: Aaron Teo <aaron.teo1@ibm.com> Co-authored-by: Jim Wu <ywu@xilinx.com> Co-authored-by: Aaron Teo <taronaeo@gmail.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> |
||
|---|---|---|
| .. | ||
| ai-issues.yml | ||
| bench.yml.disabled | ||
| build-3rd-party.yml | ||
| build-and-test-snapdragon.yml | ||
| build-android.yml | ||
| build-apple.yml | ||
| build-cache.yml | ||
| build-cann.yml | ||
| build-cmake-pkg.yml | ||
| build-cpu.yml | ||
| build-cross.yml | ||
| build-cuda-ubuntu.yml | ||
| build-cuda-windows.yml | ||
| build-ibm.yml | ||
| build-msys.yml | ||
| build-opencl.yml | ||
| build-openvino.yml | ||
| build-riscv.yml | ||
| build-rpc.yml | ||
| build-sanitize.yml | ||
| build-self-hosted.yml | ||
| build-sycl.yml | ||
| build-virtgpu.yml | ||
| build-vulkan.yml | ||
| build-wasm.yml | ||
| build-webgpu.yml | ||
| check-vendor.yml | ||
| close-issue.yml | ||
| code-style.yml | ||
| copilot-setup-steps.yml | ||
| docker.yml | ||
| editorconfig.yml | ||
| gguf-publish.yml | ||
| hip-quality-check.yml | ||
| labeler.yml | ||
| pre-tokenizer-hashes.yml | ||
| python-check-requirements.yml | ||
| python-lint.yml | ||
| python-type-check.yml | ||
| release.yml | ||
| server-sanitize.yml | ||
| server-self-hosted.yml | ||
| server.yml | ||
| ui-build-self-hosted.yml | ||
| ui-build.yml | ||
| ui-publish.yml | ||
| ui-self-hosted.yml | ||
| ui.yml | ||
| update-ops-docs.yml | ||
| winget.yml | ||