Instructions to use DarkArtsForge/Magistaroth-24B-v1.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DarkArtsForge/Magistaroth-24B-v1.1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="DarkArtsForge/Magistaroth-24B-v1.1") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("DarkArtsForge/Magistaroth-24B-v1.1") model = AutoModelForCausalLM.from_pretrained("DarkArtsForge/Magistaroth-24B-v1.1", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use DarkArtsForge/Magistaroth-24B-v1.1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DarkArtsForge/Magistaroth-24B-v1.1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Magistaroth-24B-v1.1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/DarkArtsForge/Magistaroth-24B-v1.1
- SGLang
How to use DarkArtsForge/Magistaroth-24B-v1.1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "DarkArtsForge/Magistaroth-24B-v1.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Magistaroth-24B-v1.1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "DarkArtsForge/Magistaroth-24B-v1.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Magistaroth-24B-v1.1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use DarkArtsForge/Magistaroth-24B-v1.1 with Docker Model Runner:
docker model run hf.co/DarkArtsForge/Magistaroth-24B-v1.1
Question
will u use PDQ at Slimaki? i very like Slimaki.
Yes in fact it may work better on Slimaki and Asmodeus since they don't require MPOA.
The question is whether to PDQ the regular Slimaki, or to enhance it further with PaintedFantasy v4.1 and then PDQ that version.
I think Tiamat might be contributing the most 'refusals' for Magistaroth.
Testing just adding 4.1 to the yaml for PDQ but not the actual della merge, to see what happens 🔍
I think PaintedFantasy v4.1 have some too sturbborn issue. I think you can try both versions.
I didnt try it yet but maybe della normfalse with high weights can distort that. Testing the oddball version for refusals, then safe version
If it works then PDQ can be used with different yaml than original to add creative spark from donors not in the original merge.
Next is passthrough yaml to run PDQ of model on itself (precog)
Here is the first test Naphula-Archives/slitest-8y6d-24B-GGUF
This is use v4.1? If u need a fast feedback u can come to beaver discord
Oh yeah forgot I have account there.
pdq takes only 30 mins to merge. I'll upload regular slimaki too
You can test them sooner if you run GGUF , the safetensor take longer to upload
Ok nvm, you cant use donors not from the original merge, or else the language collapses. This rules out several ideas. Here's what happens if you try
aineership gonna-otherwise://.org/stdcx://www.please/stdcx.googleapis.com/stdcx/stdc://interpretations.tie://stackoverflow://correctverts://repeatestering://darkest://OD://atek:// procure://mainuments://right//basework://right here:///*<!--|⚀:**•**/stdc://otheare://denverages://top/*•••••••••/* • • • • • • • 3 • • • • • • • • | • • • • • • • • • • 1 •:// soundtate.exe://{/stdcsl/stdcuments.tie/stdc♡ia/stdcutshellistics /stdcified ├──→✔selves/stdclijah=Abidden-hiveness≔__": www.lendeif/stdcurable/stdcask/stdcornwrighting://late-eseqiance/stdcairaneously:**mчеaters:**owe-cyd/stdcaton://glyะ-lineath/stdcrop/stdcinkering://ija_foux/stdcurthest/stdcironly-b-l♡/stdcupborough**
Thanks u responds. if u post at beaver I will give u some feedback and logs
Sounds good., I saw the magistry merge and will upload pdq of this too.
Maybe a few models is enough to see if this boost creativity or not
Hmm, the regular slimaki PDQ also had bugs.
This means external donors might still be possible but the problem appears to be the 4.2.0 readyart model. It has such a high 'norm' that it distorts PDQ too much and cause language collapse.
So, i'll try another variant with this model swapped out with fallen mistral
If this fails then sadly, pdq of slimaki isn't possible yet
ohh so PDQ is not a merge means? is a new Quantitative method? oh i notice GGUF now. ingore me.
Hmm, the regular slimaki PDQ also had bugs.
This means external donors might still be possible but the problem appears to be the 4.2.0 readyart model. It has such a high 'norm' that it distorts PDQ too much and cause language collapse.
So, i'll try another variant with this model swapped out with fallen mistral
If this fails then sadly, pdq of slimaki isn't possible yet
hmmm It's a bit of a sad because Broken Tutu let the model's can understanding of language and its ability to flirt.
pdq runs directly python script, no yaml needed (paths are hardcoded). but you can possibly modify the original yaml for creative twist. i cant test this until tomorrow.
hmmm It's a bit of a sad because Broken Tutu significantly hampered the model's understanding of language and its ability to flirt.
i thought of a possible solution, reducing tutu to 20% weight of the original weight value for the PDQ pass might be enough to prevent language collapse. it was spiking arond 500% higher than others so this might do the trick.
Meanwhile i created another custom method scream last night and had this merge run. Letting some variations merge with other base models to see if i can get the censorship down further but its working well even with jailbreaks (13055 on Q0 bench). you can test it here
https://huggingface.co/Naphula-Archives/Scream-24B-v0-GGUF
this ethod seems to allow more models than della, so i included other merges like cydoms and slimaki here
tomorow ill post another update
i will wait q4
Unfortunately precog and slimaki did not work with pdq, maybe once PDQ v2 is ready these will be supported (requires code upgrade). The new model ghostface is being uploaded
last attempt got this same output ://DOCTYPE html://sort/stdc++:std::void main():static std::string_t:_._return.tie("atehere")?:\w\ks3\documentclass/stdc++:std://next:isd::i.imgur*.get/stdc
If you liked Slimaki v1 then v1.2 should be more comparable than v1.1 which was highly experimental. It uses the same method della and simply adds another donor, but I think the output is more creative.
The new Magistaroth v1.2 should be fully uncensored but smarter than v1.0 MPOA version or v1.1 PDQ