metadata
base_model: meta-llama/Llama-3.3-70B-Instruct
library_name: peft
soo-v1-bare
SOO-only LoRA adapter (rank 64) for use with meta-llama/Llama-3.3-70B-Instruct.
This adapter was originally trained on top of an SFT-merged model (Llama-3.3-70B-Instruct + alignment-faking SFT), so applying it directly to the base Instruct model gives the SOO effect without the SFT component. For the combined SFT + SOO adapter, see Arrrlex/soo-v1-af.
LoRA config
- Rank: 64
- Alpha: 128
- Dropout: 0.5
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Method: Logits-based Self-Other Overlap fine-tuning
- Framework: PEFT 0.15.2