maywell/kiqu_samples
Viewer β’ Updated β’ 24.9k β’ 49 β’ 4
How to use kurugai/Kurugai-EEVE-v1.0 with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="kurugai/Kurugai-EEVE-v1.0")
messages = [
{"role": "user", "content": "Who are you?"},
]
pipe(messages) # pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("kurugai/Kurugai-EEVE-v1.0")
model = AutoModelForCausalLM.from_pretrained("kurugai/Kurugai-EEVE-v1.0", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))How to use kurugai/Kurugai-EEVE-v1.0 with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kurugai/Kurugai-EEVE-v1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "kurugai/Kurugai-EEVE-v1.0",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker model run hf.co/kurugai/Kurugai-EEVE-v1.0
How to use kurugai/Kurugai-EEVE-v1.0 with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "kurugai/Kurugai-EEVE-v1.0" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "kurugai/Kurugai-EEVE-v1.0",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "kurugai/Kurugai-EEVE-v1.0" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "kurugai/Kurugai-EEVE-v1.0",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'How to use kurugai/Kurugai-EEVE-v1.0 with Docker Model Runner:
docker model run hf.co/kurugai/Kurugai-EEVE-v1.0
kurugai/Kurugai-EEVE-v1.0λ yanolja/EEVE-Korean-10.8B-v1.0λ₯Ό λ² μ΄μ€λͺ¨λΈλ‘ ν΄μ maywell/kiqu_samples λ°μ΄ν°μ μΌλ‘ νμ΅λ λͺ¨λΈμ λλ€.
RTX 8000 GPU 1EAλ‘ 16μκ° νμ΅νμμ΅λλ€.
| Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
|---|---|---|---|---|---|
| 54.72 | 49.66 | 58.91 | 52.17 | 50.87 | 61.98 |
| μΆλ‘ | μν | κΈμ°κΈ° | μ½λ© | μ΄ν΄ | λ¬Έλ² | μ±κΈν΄ | λ©ν°ν΄ | μ΄μ |
|---|---|---|---|---|---|---|---|---|
| 3.00 | 2.71 | 3.92 | 4.21 | 6.35 | 4.14 | 4.59 | 3.52 | 4.05 |
λ³Έ λͺ¨λΈμ μνμΉ΄ ν¬λ§·μ λ°λ¦ λλ€.
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:
{output}
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
μ½λ‘λ19μ κ±Έλ Έλλ° μ΄λ»κ² λμ²ν΄μΌν΄?
### Response:
μ½λ‘λ19μ 걸릴 κ²½μ° κ°μ₯ μ€μν κ²μ μμ μ κ±΄κ° μνλ₯Ό μ λͺ¨λν°λ§νκ³ μ μ ν μλ£ μ‘°μΉλ₯Ό μ·¨νλ κ²μ΄μμ. μ¬κΈ° λͺ κ°μ§ κΈ°λ³Έμ μΈ λμ λ°©λ²μ μκ°ν κ²μ:
1. **μ¦μ κ΄λ¦¬**: λ°μ΄, κΈ°μΉ¨, νΈν‘κ³€λ λ±μ μ¦μμ΄ λνλ μ μμ΄μ. μ΄λ¬ν μ¦μλ€μ΄ μ¬ν΄μ§κ±°λ μ
νλ κ²½μ° μ¦μ μμ¬μ μλ΄νκ±°λ λ³μμ λ°©λ¬ΈνμΈμ. λν μΆ©λΆν μλΆ μμ·¨μ μμ 보좩μ ν΅ν΄ λͺΈμ ν볡μ λμμ£ΌμΈμ.
2. **μ μ΄μ μΆμ λ° κ²©λ¦¬**: κ°μΌ μ¬μ€μ μκ² λλ©΄ μ μ΄μλ€κ³Όμ μν΅μ μμνκ³ νμν κ²½μ° μκ°κ²©λ¦¬λ₯Ό μμνμ
μΌ ν΄μ. μ΄λ λ€λ₯Έ μ¬λλ€μκ² λ°μ΄λ¬μ€λ₯Ό μ ννμ§ μλλ‘ μλ°©νκΈ° μν μ‘°μΉμμ.
3. **μλ£ μλΉμ€ νμ©**: μ¦μμ΄ μ¬κ°ν΄μ§ κ²½μ° μ¦κ°μ μΈ μλ£ λμμ λ°μμΌ ν΄μ. λ³μμ κ°μ μ§λ£λ₯Ό λ°κ±°λ, νμνλ€λ©΄ μ
μμ κ³ λ €ν΄λ³΄μΈμ. λν μ²λ°©λ μ½λ¬Όμ΄λ μΉλ£ κ³νμ λ°λ₯΄λ κ²λ μ€μν΄μ.
4. **μ μ μ μ§μ λ°κΈ°**: κ°μΌλ³μΌλ‘ μΈν μ€νΈλ μ€λ λΆμμ μμ°μ€λ¬μ΄ λ°μμ΄μμ. λ§μ½ νμνλ©΄ μ μ κ±΄κ° μ λ¬Έκ°μ μλ΄μ λ°μ 보μλ κ²μ κΆμ₯λλ €μ. μ¨λΌμΈ μλ΄λ κ°λ₯νλ μ°Έκ³ ν΄ μ£ΌμΈμ.
5. **μλ°© μ‘°μΉ μ μ§νκΈ°**: λ°±μ μ μ’
, λ§μ€ν¬ μ°©μ©, μ μ»κΈ° λ± μ½λ‘λ19 μλ°©μ μν κΈ°λ³Έ κ·μΉμ κ³μ μ μ§ν΄μ£ΌμΈμ. μ΄λ₯Ό ν΅ν΄ μμ κ³Ό μ£Όλ³ μ¬λλ€μ 건κ°μ 보νΈν μ μμ΄μ.
6. **μ§μμ μΈ μ 보 μ
λ°μ΄νΈ**: μ§λ³κ΄λ¦¬μ²μ΄λ κ΄λ ¨ μ λΆ κΈ°κ΄μμ μ 곡νλ μ΅μ μ 보λ₯Ό μμλ‘ νμΈν΄ 보μΈμ. μ΄λ κ² νλ©΄ μμ μκ² λ§λ λμμ ν μ μλ μ 보λ₯Ό μ»μ μ μμ΄μ.
μ½λ‘λ19λ κ°μΈλ§λ€ λ€λ₯΄κ² μν₯μ λ―ΈμΉ μ μμΌλ, κ°μΈμ μν©μ λ§κ² μ μ ν λμν νμκ° μμ΄μ. νμ μ λ¬Έκ°μμ μμλ₯Ό ν΅ν μ κ·Όμ΄ μ΅μ μ΄λΌκ³ ν μ μκ² μ£ . λ κΆκΈν μ μ΄ μμΌλ©΄ μΈμ λ μ§ λ¬Όμ΄λ΄ μ£ΌμΈμ!
μ΄ λͺ¨λΈμ μλ΄μ μ§μμ λ°μ μ μλμμ΅λλ€. μλ΄μκ² κ°μ¬μ λ§μ μ ν©λλ€.
Base model
upstage/SOLAR-10.7B-v1.0