mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-10-03 19:45:41 +00:00
3bcfeb700f
* scripts : add initial profiling script (wip) * src : add precompile headers (PCH) for models.h * common : add common.h as PCH * ggml : add PCH for ggml-impl.h * mtmd : use PCH for models.h * scripts : add script to build with Server/Tools/Tests * server : add PCH for common.h * docs: add profiling progress notes (wip) * ggml : add exclude for GCC + SVE on ARM Refs: https://github.com/ggml-org/llama.cpp/actions/runs/33393906061/job/99493756214?pr=28091 * ggml : attempt to fix use of std::hardware_destructive_inference_size Refs: https://github.com/ggml-org/llama.cpp/actions/runs/33396221677/job/99501265689?pr=28091 * squash! ggml : attempt to fix use of std::hardware_destructive_inference_size Add a version check for GCC 12 to conditionally apply the `-Winterference-size` pragma. * editorconfig : exclude profiling reports dir This directory will not be included in the merge later and this commit can be ignore at that point. Just fixing to keep CI happy. * ggml : skip PCH for gcc on non-x86 architectures * tests : add PCH for peg-parser/tests.h There are 7 peg-parser tests that can share one PCH instead of then each parsing the full tests.h. * common : add PCH for chat.h * docs : update linux build profiling full results Just updating after a number of PCH additions. These are not exact figures and will vary a bit from run to run, but they give a general idea of the performance impact of PCH. * cmake : introduce unity build for models This commit introduces a unity build for the models to improve compilation time. The improvements were roughly the following: ```console +------------------------+-----+------------+------------+------------+ | Build | TUs | Frontend | Backend | Total | +------------------------+-----+------------+------------+------------+ | Full, master | 396 | 811.0 s | 692.2 s | 1,503.2 s | | Full, with PCH | 405 | 380.0 s | 664.7 s | 1,044.7 s | | Full, with PCH + UB | 264 | 357.7 s | 635.7 s | 993.4 s | +------------------------+-----+------------+------------+------------+ TU = Translation Unit. Full = includes Server, Tools, and Tests. PCH = precompiled headers. UB = unity build for models. ``` * docs : update linux profiling table with unitiy build results * docs : update mac profiling results to include unity build [no ci] * docs: remove profiling reports * scripts : merge build profile scripts into one script I was lazy before and just copied the first script to enable Tests, Server, and Tools. This now merges them into a single script. * Revert "editorconfig : exclude profiling reports dir" [no ci] This reverts commit 2922a12118a0730d2f7632bcba265b44a0856c59. * src : rename ggml_view_2d_slice to gemma3n_view_2d_slice This is to be consistent with the rename in gemma4.cpp which was required to avoid a name clash. * cmake : add build profile script for windows [no ci] This commit adds a port of the scripts/build-profile.sh script to windows powershell. This was developed on Windows on ARM but should work on X64 as well but needs to be tested there as well. |
||
|---|---|---|
| .. | ||
| apple | ||
| hip | ||
| jinja | ||
| snapdragon | ||
| bench-models.sh | ||
| build-info.sh | ||
| build-profile.ps1 | ||
| build-profile.sh | ||
| ccache-clear.sh | ||
| check-requirements.sh | ||
| compare-commits.sh | ||
| compare-llama-bench.py | ||
| compare-logprobs.py | ||
| create_ops_docs.py | ||
| debug-test.sh | ||
| gen-authors.sh | ||
| gen-unicode-data.py | ||
| get-flags.mk | ||
| get-hellaswag.sh | ||
| get-pg.sh | ||
| get-wikitext-2.sh | ||
| get-winogrande.sh | ||
| get_chat_template.py | ||
| git-bisect-run.sh | ||
| git-bisect.sh | ||
| hf.sh | ||
| install-oneapi.bat | ||
| make-release-checks.sh | ||
| make-release-desc.sh | ||
| make-release-summary.txt | ||
| pr2wt.sh | ||
| release.sh | ||
| serve-static.js | ||
| server-bench.py | ||
| server-test-function-call.py | ||
| server-test-model.py | ||
| server-test-parallel-tc.py | ||
| server-test-structured.py | ||
| sync-ggml-am.sh | ||
| sync-ggml.last | ||
| sync-ggml.sh | ||
| sync_vendor.py | ||
| tool_bench.py | ||
| tool_bench.sh | ||
| ui-assets.cmake | ||
| verify-checksum-models.py | ||
| wc2wt.sh | ||