mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2025-09-13 02:19:41 +00:00
a0535ffa0d
* implement unary REGLU/GEGLU/SWIGLU cpu ops * relax constraints * duplicate shape of source * fix ggml_vec_geglu_f16 * special case gated ops * implement unary REGLU/GEGLU/SWIGLU cuda ops * tighten constraints again * refactor into GGML_GLU_OP * metal : add glu kernels ggml-ci * add CUDA_GLU_BLOCK_SIZE [no ci] * more constraints and use 64bit ints ggml-ci * 64bit multiplication [no ci] * implement swapped variants (cpu/cuda) * update comment [no ci] ggml-ci * Vulkan: Add GLU ops and shaders * SYCL: Implement fused kernel GEGLU, SWIGLU and REGLU for single up+gate * ggml : implement GLU for split up/gate (#14181) * implement GLU for split up/gate * add tests for ggml_glu_split * Vulkan: Implement glu_split logic and shader support * add split to logging [no ci] * SYCL: refactor element_size ops and add split up and gate support to gated kernels * SYCL: switch GEGLU to use tanh approximation --------- Co-authored-by: 0cc4m <picard12@live.de> Co-authored-by: Akarshan <akarshan@menlo.ai> * GGML: increase OP count in assertion * Refactor: Optimize SYCL element-wise operations with unary function inlining This commit refactors the SYCL element-wise operations to improve performance by: - Inlining unary operations (sgn, abs, elu, gelu, silu, etc.) to reduce kernel launch overhead. - Introducing helper functions `op_xxx` for each unary operation to encapsulate the logic. - Replacing direct kernel calls with calls to these inlined functions. - Using `__dpct_inline__` to encourage compiler inlining. - Minor code cleanup and consistency improvements. The changes aim to reduce kernel launch overhead and improve the overall efficiency of element-wise operations on SYCL devices. * vulkan: Increase workgroup size for GLU, for performance (#14345) * vulkan: Increase workgroup size for GLU, for performance * vulkan: change GLU shaders to do one element per invocation rather than one row per workgroup * merge fix * metal : add support for split and swap ggml-ci --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> Co-authored-by: 0cc4m <picard12@live.de> Co-authored-by: Akarshan <akarshan@menlo.ai> Co-authored-by: Jeff Bolz <jbolz@nvidia.com> |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| CMakeLists.txt | ||
| get-model.cpp | ||
| get-model.h | ||
| run-json-schema-to-grammar.mjs | ||
| test-arg-parser.cpp | ||
| test-autorelease.cpp | ||
| test-backend-ops.cpp | ||
| test-barrier.cpp | ||
| test-c.c | ||
| test-chat-parser.cpp | ||
| test-chat-template.cpp | ||
| test-chat.cpp | ||
| test-double-float.cpp | ||
| test-gbnf-validator.cpp | ||
| test-gguf.cpp | ||
| test-grammar-integration.cpp | ||
| test-grammar-llguidance.cpp | ||
| test-grammar-parser.cpp | ||
| test-json-partial.cpp | ||
| test-json-schema-to-grammar.cpp | ||
| test-llama-grammar.cpp | ||
| test-log.cpp | ||
| test-lora-conversion-inference.sh | ||
| test-model-load-cancel.cpp | ||
| test-mtmd-c-api.c | ||
| test-opt.cpp | ||
| test-quantize-fns.cpp | ||
| test-quantize-perf.cpp | ||
| test-quantize-stats.cpp | ||
| test-regex-partial.cpp | ||
| test-rope.cpp | ||
| test-sampling.cpp | ||
| test-thread-safety.cpp | ||
| test-tokenizer-0.cpp | ||
| test-tokenizer-0.py | ||
| test-tokenizer-0.sh | ||
| test-tokenizer-1-bpe.cpp | ||
| test-tokenizer-1-spm.cpp | ||
| test-tokenizer-random.py | ||
| test-tokenizers-repo.sh | ||