Instructions to use molbal/horrorllama3-8b-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use molbal/horrorllama3-8b-v1.0 with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use molbal/horrorllama3-8b-v1.0 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf molbal/horrorllama3-8b-v1.0:Q4_K_M # Run inference directly in the terminal: llama cli -hf molbal/horrorllama3-8b-v1.0:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf molbal/horrorllama3-8b-v1.0:Q4_K_M # Run inference directly in the terminal: llama cli -hf molbal/horrorllama3-8b-v1.0:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf molbal/horrorllama3-8b-v1.0:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf molbal/horrorllama3-8b-v1.0:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf molbal/horrorllama3-8b-v1.0:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf molbal/horrorllama3-8b-v1.0:Q4_K_M
Use Docker
docker model run hf.co/molbal/horrorllama3-8b-v1.0:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use molbal/horrorllama3-8b-v1.0 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "molbal/horrorllama3-8b-v1.0" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "molbal/horrorllama3-8b-v1.0", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/molbal/horrorllama3-8b-v1.0:Q4_K_M
- Ollama
How to use molbal/horrorllama3-8b-v1.0 with Ollama:
ollama run hf.co/molbal/horrorllama3-8b-v1.0:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use molbal/horrorllama3-8b-v1.0 with Docker Model Runner:
docker model run hf.co/molbal/horrorllama3-8b-v1.0:Q4_K_M
- Lemonade
How to use molbal/horrorllama3-8b-v1.0 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull molbal/horrorllama3-8b-v1.0:Q4_K_M
Run and chat with the model
lemonade run user.horrorllama3-8b-v1.0-Q4_K_M
List all available models
lemonade list
- Atomic Chat
molbal/horrorllama3-8b-v1.0 model card
This model is a fine-tuned variant of the llama3-8b. It was specifically trained on a dataset of horror novels obtained from Project Gutenberg, a public domain digital library platform. It is the result of following a guide: https://github.com/molbal/llm-text-completion-finetune
Model Details
Training
The model was fine-tuned following the pipeline guide and the horror novels dataset created from public domain books available on Project Gutenberg with the topic marked as "horror". Training and dataset acquisition scripts are available at https://github.com/molbal/llm-text-completion-finetune
Intended Use
This model is an educational/practice finetune, due to lack of proper data cleaning, it is not recommended for production use.
Limitations
This model is a text completion model, meaning it will generate textual content that organically continues the given prompt. It does not respond to instructions or answer questions, as it is not an instruct/chat model type like ChatGPT. While the model is fine-tuned for generating horror-themed content, the generated text's relevance and quality can still depend on the provided prompt. This model does not have the ability to verify facts or provide accurate information. Inference times and resource usage may vary depending on the infrastructure where the model is deployed.
- Downloads last month
- 28
