mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2025-09-13 10:29:43 +00:00
docs : update HOWTO‑add‑model.md for ModelBase and new model classes (#14874)
This patch updates the example in docs/development/HOWTO-add-model.md to reflect recent changes after `TextModel` and `MmprojModel` were introduced. It replaces the outdated `Model` base class with `TextModel` or `MmprojModel` and updates the registration example accordingly. Signed-off-by: Wook Song <wook16.song@samsung.com>
This commit is contained in:
parent
e2b7621e7c
commit
e7fecba934
1 changed files with 15 additions and 6 deletions
|
@ -23,11 +23,19 @@ The convert script reads the model configuration, tokenizer, tensor names+data a
|
||||||
|
|
||||||
The required steps to implement for an HF model are:
|
The required steps to implement for an HF model are:
|
||||||
|
|
||||||
1. Define the model `Model.register` annotation in a new `Model` subclass, example:
|
1. Define the model `ModelBase.register` annotation in a new `TextModel` or `MmprojModel` subclass, example:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@Model.register("MyModelForCausalLM")
|
@ModelBase.register("MyModelForCausalLM")
|
||||||
class MyModel(Model):
|
class MyModel(TextModel):
|
||||||
|
model_arch = gguf.MODEL_ARCH.MYMODEL
|
||||||
|
```
|
||||||
|
|
||||||
|
or
|
||||||
|
|
||||||
|
```python
|
||||||
|
@ModelBase.register("MyModelForConditionalGeneration")
|
||||||
|
class MyModel(MmprojModel):
|
||||||
model_arch = gguf.MODEL_ARCH.MYMODEL
|
model_arch = gguf.MODEL_ARCH.MYMODEL
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -75,9 +83,10 @@ block_mappings_cfg: dict[MODEL_TENSOR, tuple[str, ...]] = {
|
||||||
`transformer.blocks.{bid}.norm_1` will be mapped to `blk.{bid}.attn_norm` in GGUF.
|
`transformer.blocks.{bid}.norm_1` will be mapped to `blk.{bid}.attn_norm` in GGUF.
|
||||||
|
|
||||||
Depending on the model configuration, tokenizer, code and tensors layout, you will have to override:
|
Depending on the model configuration, tokenizer, code and tensors layout, you will have to override:
|
||||||
- `Model#set_gguf_parameters`
|
- `TextModel#set_gguf_parameters`
|
||||||
- `Model#set_vocab`
|
- `MmprojModel#set_gguf_parameters`
|
||||||
- `Model#write_tensors`
|
- `ModelBase#set_vocab`
|
||||||
|
- `ModelBase#modify_tensors`
|
||||||
|
|
||||||
NOTE: Tensor names must end with `.weight` or `.bias` suffixes, that is the convention and several tools like `quantize` expect this to proceed the weights.
|
NOTE: Tensor names must end with `.weight` or `.bias` suffixes, that is the convention and several tools like `quantize` expect this to proceed the weights.
|
||||||
|
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue