mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2025-09-07 07:39:13 +00:00
ef0144c087
* model: Add GLM 4.5 (#14921) Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Merge in PR suggestions Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * model: Add GLM 4.5 family of models (#14921) 1. Updated tensor_mapping.py with NextN tensor mappings - Added proper tensor mappings for all NextN/MTP tensors in /Users/samm/git/llama.cpp/gguf-py/gguf/tensor_mapping.py - Added mappings for: eh_proj, embed_tokens, enorm, hnorm, shared_head.head, shared_head.norm 2. Added num_nextn_predict_layers configuration - Added LLM_KV_NUM_NEXTN_PREDICT_LAYERS constant to llama-arch.h and llama-arch.cpp - Added num_nextn_predict_layers field to llama_hparams struct - Updated GLM4_MOE parameter loading in llama-model.cpp to read this parameter - Modified tensor loading logic to conditionally load NextN tensors based on num_nextn_predict_layers - Added GGUF writer support in gguf_writer.py with add_num_nextn_predict_layers() method - Updated conversion script to extract and write this parameter from HuggingFace config 3. Added FIM tokens for GLM4_MOE - Added GLM-4.5's FIM tokens to llama-vocab.cpp: - <|code_prefix|> for FIM_PRE - <|code_suffix|> for FIM_SUF - <|code_middle|> for FIM_MID 4. Removed manual NextN tensor handling - Removed the special-case handling in convert_hf_to_gguf.py that manually mapped NextN tensors - NextN tensors are now handled automatically through the proper tensor mapping system * glm 4.5 update tensors names * model: glm 4.5 apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * model: glm 4.5 apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * model: glm 4.5 apply suggestions from code review * Apply suggestions from code review * patch broken chat template * typings fix * add TENSOR_SKIP flag Co-authored-by: Diego Devesa <slarengh@gmail.com> * Update src/llama-model-loader.h Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> Co-authored-by: Diego Devesa <slarengh@gmail.com> |
||
|---|---|---|
| .. | ||
| templates | ||
| .editorconfig | ||
| ggml-vocab-aquila.gguf | ||
| ggml-vocab-baichuan.gguf | ||
| ggml-vocab-bert-bge.gguf | ||
| ggml-vocab-bert-bge.gguf.inp | ||
| ggml-vocab-bert-bge.gguf.out | ||
| ggml-vocab-command-r.gguf | ||
| ggml-vocab-command-r.gguf.inp | ||
| ggml-vocab-command-r.gguf.out | ||
| ggml-vocab-deepseek-coder.gguf | ||
| ggml-vocab-deepseek-coder.gguf.inp | ||
| ggml-vocab-deepseek-coder.gguf.out | ||
| ggml-vocab-deepseek-llm.gguf | ||
| ggml-vocab-deepseek-llm.gguf.inp | ||
| ggml-vocab-deepseek-llm.gguf.out | ||
| ggml-vocab-falcon.gguf | ||
| ggml-vocab-falcon.gguf.inp | ||
| ggml-vocab-falcon.gguf.out | ||
| ggml-vocab-gpt-2.gguf | ||
| ggml-vocab-gpt-2.gguf.inp | ||
| ggml-vocab-gpt-2.gguf.out | ||
| ggml-vocab-gpt-neox.gguf | ||
| ggml-vocab-llama-bpe.gguf | ||
| ggml-vocab-llama-bpe.gguf.inp | ||
| ggml-vocab-llama-bpe.gguf.out | ||
| ggml-vocab-llama-spm.gguf | ||
| ggml-vocab-llama-spm.gguf.inp | ||
| ggml-vocab-llama-spm.gguf.out | ||
| ggml-vocab-mpt.gguf | ||
| ggml-vocab-mpt.gguf.inp | ||
| ggml-vocab-mpt.gguf.out | ||
| ggml-vocab-nomic-bert-moe.gguf | ||
| ggml-vocab-phi-3.gguf | ||
| ggml-vocab-phi-3.gguf.inp | ||
| ggml-vocab-phi-3.gguf.out | ||
| ggml-vocab-qwen2.gguf | ||
| ggml-vocab-qwen2.gguf.inp | ||
| ggml-vocab-qwen2.gguf.out | ||
| ggml-vocab-refact.gguf | ||
| ggml-vocab-refact.gguf.inp | ||
| ggml-vocab-refact.gguf.out | ||
| ggml-vocab-starcoder.gguf | ||
| ggml-vocab-starcoder.gguf.inp | ||
| ggml-vocab-starcoder.gguf.out | ||