Philipp Emanuel Weidmann
d9f2b0407a
build: bump version to 1.1.0
CI / Check and build (Python 3.10) (push) Has been cancelled
CI / Check and build (Python 3.11) (push) Has been cancelled
CI / Check and build (Python 3.12) (push) Has been cancelled
CI / Check and build (Python 3.13) (push) Has been cancelled
2025-12-10 16:54:03 +05:30
Philipp Emanuel Weidmann
ca783db6c9
docs: update README
2025-12-10 16:30:35 +05:30
Philipp Emanuel Weidmann
6acccac994
feat: add progress bars for plotting operations
2025-12-10 13:07:34 +05:30
Philipp Emanuel Weidmann
ac154a55a0
fix: suppress CoT output for thinking models
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
Ref #75
2025-12-09 11:54:08 +05:30
Philipp Emanuel Weidmann
15781a8a0c
fix: skip common response prefix for thinking models
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
Ref #75
2025-12-09 08:25:10 +05:30
Philipp Emanuel Weidmann
24c3aeb442
feat: turn boolean settings into CLI flags
CI / Check and build (Python 3.10) (push) Has been cancelled
CI / Check and build (Python 3.11) (push) Has been cancelled
CI / Check and build (Python 3.12) (push) Has been cancelled
CI / Check and build (Python 3.13) (push) Has been cancelled
2025-12-07 11:37:07 +05:30
Philipp Emanuel Weidmann
ffbde3ac2a
fix: follow up after recent PRs
2025-12-07 10:26:16 +05:30
Philipp Emanuel Weidmann
932d737edf
feat: add silhouette coefficient to residual geometry output
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
2025-12-07 08:48:38 +05:30
Philipp Emanuel Weidmann
1f5e977f4f
Revert "perf: optimize abliteration matrix op ( #46 )" ( #74 )
...
This reverts commit 60bd531fde
2025-12-07 06:30:37 +05:30
Philipp Emanuel Weidmann
da27ba8054
fix: always left-pad inputs, and avoid optimizing for empty responses
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
Fixes #70
Co-authored-by: arnomatic <acc@eml.cc>
2025-12-06 06:31:09 +05:30
Philipp Emanuel Weidmann
baf5b0b0d1
feat: add geometric median to residual geometry output
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
2025-12-05 20:15:50 +05:30
Philipp Emanuel Weidmann
eeb28b28c1
feat: add option to plot residual vectors
CI / Check and build (Python 3.10) (push) Has been cancelled
CI / Check and build (Python 3.11) (push) Has been cancelled
CI / Check and build (Python 3.12) (push) Has been cancelled
CI / Check and build (Python 3.13) (push) Has been cancelled
2025-12-04 14:22:29 +05:30
red40maxxer
d836fb2da9
ci: add PR title lint ( #66 )
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
* ci: add PR title lint
* style: ending newline
---------
Co-authored-by: mad-cat-lon <113548315+mad-cat-lon@users.noreply.github.com>
2025-12-03 09:25:48 +05:30
red40maxxer
60bd531fde
perf: optimize abliteration matrix op ( #46 )
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
* perf: optimize abliteration matrix op
* refactor: comments and var names correspond with arditi
* refactor: fix comments and improve var notation
* fix: accidental line change and improve comments
---------
Co-authored-by: mad-cat-lon <113548315+mad-cat-lon@users.noreply.github.com>
2025-12-02 08:13:43 +05:30
Spiky Moth
1f74ac2888
Guard against refusals in broken English ( #45 )
...
CI / Check and build (Python 3.10) (push) Has been cancelled
CI / Check and build (Python 3.11) (push) Has been cancelled
CI / Check and build (Python 3.12) (push) Has been cancelled
CI / Check and build (Python 3.13) (push) Has been cancelled
* Guard against refusals in broken English
* Normalize whitespace between words
2025-11-26 11:29:08 +05:30
_Vinayyyy_
63fc0e7d5a
feat: Add bfloat16 to default dtypes list ( #44 )
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
Co-authored-by: Vinay Umrethe <vinayumrethe99@gmail.com>
2025-11-25 12:22:52 +05:30
_Vinayyyy_
1efc4ee9e1
Featuring Notebook (Colab/Kaggle) Compatibility ( #42 )
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
* feat: Add hybrid UI for notebook compatibility
* Restore notebook detection logic
* fix: Improve notebook detection with env vars
* chore: cleanup
* chore: cleanup
* correct ruff format
* refactor: Address code review feedback
- Move password handling to prompt_password
- Use only_directories=True for save path prompt
- Simplify prompt_text arguments
---------
Co-authored-by: Vinay Umrethe <vinayumrethe99@gmail.com>
2025-11-24 19:46:39 +05:30
Nikolai Kolodziej
452b35e7b7
Add trust_remote_code configuration option ( #31 )
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
* Add `trust_remote_code` configuration option and apply it when loading models and tokenizers
* Default `trust_remote_code` to `None` and set it to `True` if previously `None` so the user wouldn't be asked multiple times
* Consistently access `trust_remote_code` from `self.settings` instead of the global `settings` object.
* Introduce `trusted_models` dictionary to manage and confirm `trust_remote_code` settings during model loading
* Assign `trust_remote_code` to `evaluate_model` in `trusted_models` instead of `model`
2025-11-24 06:27:44 +05:30
Spiky Moth
b79b8b1475
Improve support for loading local datasets ( #33 )
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
* Handle loading local datasets
* Reorder branches to avoid chain of negatives
2025-11-23 11:15:34 +05:30
Philipp Emanuel Weidmann
83cbf0612a
Add option to print refusal geometry
CI / Check and build (Python 3.10) (push) Has been cancelled
CI / Check and build (Python 3.11) (push) Has been cancelled
CI / Check and build (Python 3.12) (push) Has been cancelled
CI / Check and build (Python 3.13) (push) Has been cancelled
2025-11-22 13:18:54 +05:30
Philipp Emanuel Weidmann
c35f3031f8
Allow stopping the optimization process early with Ctrl+C
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
2025-11-21 10:11:00 +05:30
Nikolai Kolodziej
2e1bb4b655
Use PYTORCH_ALLOC_CONF instead of deprecated PYTORCH_CUDA_ALLOC_CONF ( #32 )
...
* Use `PYTORCH_ALLOC_CONF` instead of deprecated `PYTORCH_CUDA_ALLOC_CONF`
* style: reformat environment variable check
2025-11-21 07:27:28 +05:30
Anthony Eufemio
af02bc6ece
Fix support for MXFP4 quantized models with Triton tensors ( #28 )
...
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
When loading models with MXFP4 quantization (e.g., openai/gpt-oss-20b),
the transformers library uses Triton tensors to wrap the quantized weights.
These Triton tensors have a .data attribute containing the underlying
PyTorch tensor, but torch.is_tensor() returns False for them.
This caused a KeyError: 'mlp.down_proj' when trying to load such models,
as the try_add() function would fail the assertion check before adding
the down projection matrices.
The fix extracts the underlying PyTorch tensor via the .data attribute
when encountering Triton tensors, allowing heretic to work with MXFP4
quantized models while maintaining full compatibility with standard models.
Tested with openai/gpt-oss-20b on PyTorch 2.9.1+cu130, transformers 4.57.1,
triton 3.5.1, and kernels 0.11.0.
2025-11-20 13:43:06 +05:30
Philipp Emanuel Weidmann
22a4a5b5b5
Add citation information to README
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
2025-11-19 12:14:17 +05:30
Philipp Emanuel Weidmann
694edf18d3
Follow up after recent PRs
CI / Check and build (Python 3.10) (push) Waiting to run
CI / Check and build (Python 3.11) (push) Waiting to run
CI / Check and build (Python 3.12) (push) Waiting to run
CI / Check and build (Python 3.13) (push) Waiting to run
2025-11-19 11:19:47 +05:30
Philipp Emanuel Weidmann
c9c022a143
Fix linting issues
2025-11-19 10:16:58 +05:30
Philipp Emanuel Weidmann
9905d9517f
Fix formatting issues
2025-11-19 10:04:43 +05:30
Philipp Emanuel Weidmann
f06e939791
Add Ruff as a dev dependency
2025-11-19 09:59:18 +05:30
Philipp Emanuel Weidmann
f3b9826ca4
Add CI workflow
2025-11-19 09:45:54 +05:30
Richard Young, PhD
13bb7b24d6
Fix KeyError when HuggingFace user profile fields are missing ( #20 )
...
Handle optional fullname and email fields in user profile gracefully
using .get() method with fallback values to prevent KeyError when
uploading models to HuggingFace.
This fixes an issue where users without a public email or fullname
set in their HuggingFace profile would encounter an error during
the upload process.
Co-authored-by: ricyoung <riyoung@gmail.com>
2025-11-19 05:32:50 +05:30
Nikolai Kolodziej
c8b6663b93
Fix multi-GPU support and memory management ( #17 )
...
* Ensure projector is on the same device as the matrix for multi-GPU support
* Optimize memory management for loaded model weights
* Refactor memory management by removing unnecessary gc.collect() calls
* Optimize memory usage (#1 )
* Improve memory management by explicitly deleting model layers and optimizing projector usage
* Optimize memory management by explicitly deleting the model and forcing garbage collection
* Add back deleted `empty_cache` call
* Fix broken file
* Remove unnecessary deletions
* Remove unnecessary empty_cache() calls
* Remove unused import of gc
* Duplicate `gc.collect` call in `empty_cache()`
* Move additional `gc.collect` call in front of `torch.x.empty_cache`
2025-11-19 05:09:12 +05:30
Ooze
61fdf72b42
Add support for Granite MoE Hybrid in model.py by including down projections for shared MLP and MoE experts ( #14 )
2025-11-18 08:32:58 +05:30
red40maxxer
7bad84b4f1
perf: clear residuals after computing direction ( #15 )
...
Co-authored-by: mad-cat-lon <113548315+mad-cat-lon@users.noreply.github.com>
2025-11-17 22:18:22 +05:30
Matt Barnson
09730bad70
MPS support ( #5 )
...
* MPS support
* oops, added issue tracker.
* Delete .beads/issues.jsonl
2025-11-17 18:42:01 +05:30
Philipp Emanuel Weidmann
b3545e4b1e
Fix retrieving package version
2025-11-16 17:35:13 +05:30
Philipp Emanuel Weidmann
3f346b6150
Change package name
2025-11-16 17:01:50 +05:30
Philipp Emanuel Weidmann
1a59d226c1
Fix spacing after images in README
2025-11-16 16:06:08 +05:30
Philipp Emanuel Weidmann
12ecf50033
Add README
2025-11-16 15:19:27 +05:30
Philipp Emanuel Weidmann
ea699dce46
Improve appearance of selection menus
2025-11-16 11:32:58 +05:30
Philipp Emanuel Weidmann
8a1aceff11
Switch to multi-objective optimization
2025-11-14 18:04:23 +05:30
Philipp Emanuel Weidmann
0bae27f359
Fix some of the problems with Falcon-E-3B
2025-11-13 11:39:08 +05:30
Philipp Emanuel Weidmann
e24080db64
Add metadata to pyproject.toml
2025-11-02 10:06:15 +05:30
Philipp Emanuel Weidmann
fae39ffb89
Move default configuration to Python
2025-11-02 09:29:55 +05:30
Philipp Emanuel Weidmann
850c21b534
Make multivariate TPE work properly
2025-11-01 16:57:12 +05:30
Philipp Emanuel Weidmann
a24e6eba96
Improve optimization
2025-10-31 16:04:28 +05:30
Philipp Emanuel Weidmann
a9655c8d31
Perform calculations involving residual vectors in float32
...
Credit to Jim Lai for pointing out potential numerical problems in https://huggingface.co/blog/grimjim/projected-abliteration
2025-10-31 13:47:24 +05:30
Philipp Emanuel Weidmann
1496e0a04c
Dynamically choose between global and per-layer refusal directions
2025-10-31 13:04:45 +05:30
Philipp Emanuel Weidmann
c638d3d012
Adjust score parameters
2025-10-25 13:15:31 +05:30
Philipp Emanuel Weidmann
47e855d5d8
Guard against missing model card data
2025-10-25 13:12:18 +05:30
Philipp Emanuel Weidmann
e2419de016
Add "abliterated" to model tags
2025-10-25 09:59:44 +05:30
