vrr/remove-refusals-with-transformers
2
0
Fork 0
mirror of https://github.com/Sumandora/remove-refusals-with-transformers.git synced 2026-07-09 17:08:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
mirror of https://github.com/Sumandora/remove-refusals-with-transformers (indirect) link from https://www.reddit.com/r/LocalLLaMA/comments/1mjoo7w/huihui_released_gptoss_20b_abliterated/ + https://huggingface.co/huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated
11 commits 1 branch 0 tags 690 KiB
Find a file
Exact
johannes 7786b0a8c5 Update to latest version of HF Transformers (Fixes #4) 
I hope this didn't break backwards compatibility in ways beyond what I
can test.

Anyways, lots of new stuff in HF Transformers, which make this quite a
lot harder than it was before. I'm not sure if I can continue supporting
this, if it this goes on. Perhaps they should think of a canonical way
of doing this...
2025-11-27 11:27:08 +01:00
.gitignore Update to latest version of HF Transformers (Fixes #4) 2025-11-27 11:27:08 +01:00
compute_refusal_dir.py Update to latest version of HF Transformers (Fixes #4) 2025-11-27 11:27:08 +01:00
harmful.txt Initial commit 2024-05-16 12:03:21 +02:00
harmless.txt Initial commit 2024-05-16 12:03:21 +02:00
inference.py Update to latest version of HF Transformers (Fixes #4) 2025-11-27 11:27:08 +01:00
LICENSE Create LICENSE 2024-06-07 08:35:44 +02:00
README.md Clarify wording, fix typos and fix grammar errors in README.md 2024-06-12 10:53:29 +02:00
requirements.txt Add bitsandbytes and accelerate to requirements.txt 2024-05-16 12:21:04 +02:00

README.md

Removing refusals with HF Transformers

This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens. This means, that this supports every model that HF Transformers supports*.

The code was tested on a RTX 2060 6GB, thus mostly <3B models have been tested, but the code has been tested to work with bigger models as well.

*While most models are compatible, some models are not. Mainly because of custom model implementations. Some Qwen implementations for example don't work. Because model.model.layers can't be used for getting layers. They call the variables so that, model.transformer.h must be used, if I'm not mistaken.

Usage

  1. Set model and quantization in compute_refusal_dir.py and inference.py (Quantization can apparently be mixed)
  2. Run compute_refusal_dir.py (Some settings in that file may be changed depending on your use-case)
  3. Run inference.py and ask the model how to build an army of rabbits, that will overthrow your local government one day, by stealing all the carrots.

Credits