Instructions to use DaehanKim/korjev-26b4a-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DaehanKim/korjev-26b4a-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DaehanKim/korjev-26b4a-v1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("DaehanKim/korjev-26b4a-v1")
model = AutoModelForMultimodalLM.from_pretrained("DaehanKim/korjev-26b4a-v1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DaehanKim/korjev-26b4a-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DaehanKim/korjev-26b4a-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaehanKim/korjev-26b4a-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DaehanKim/korjev-26b4a-v1

SGLang

How to use DaehanKim/korjev-26b4a-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DaehanKim/korjev-26b4a-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaehanKim/korjev-26b4a-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DaehanKim/korjev-26b4a-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaehanKim/korjev-26b4a-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DaehanKim/korjev-26b4a-v1 with Docker Model Runner:
```
docker model run hf.co/DaehanKim/korjev-26b4a-v1
```

korjev-26b4a-v1 / release_validation.json

DaehanKim

Release merged korjev-26b4a-v1 FP8 weights, evaluation and serving instructions

8041088 verified 20 days ago

Raw History Blame Contribute Delete

4.93 kB

{
  "smoke": {
    "items": 96,
    "prediction_agreement": 0.9791666666666666,
    "max_probability_delta": 0.2923086923258452,
    "mean_probability_delta": 0.009561798528451486,
    "reference": "Previous full matched FP8 benchmark; same source revisions and export recipe",
    "concurrency": 64
  },
  "full_benchmark": {
    "metrics": {
      "korean": {
        "items": 25418,
        "views": 25418,
        "accuracy": 0.9215516563065544,
        "answer_disagreement": 0.0,
        "order_agreement": 1.0,
        "js": 0.0,
        "both_wrong_items": 1994,
        "concurrency": 128,
        "seconds": 136.16059323586524,
        "decisions_per_second": 186.67662497599048,
        "service_p50_ms": 125.52499771118164,
        "service_p95_ms": 182.70281553268424,
        "http_p50_ms": 685.1250806357712,
        "scheduler_queue_p50_ms": 484.1857466381043,
        "latency_metric_excludes_initial_server_queue": true,
        "warmup_included": false,
        "failures": 0
      },
      "english": {
        "items": 12032,
        "views": 12032,
        "accuracy": 0.6565824468085106,
        "answer_disagreement": 0.0,
        "order_agreement": 1.0,
        "js": 0.0,
        "both_wrong_items": 4132,
        "concurrency": 128,
        "seconds": 78.06261895876378,
        "decisions_per_second": 154.13267144362462,
        "service_p50_ms": 121.21415138244629,
        "service_p95_ms": 180.46717643737773,
        "http_p50_ms": 819.7654015384614,
        "scheduler_queue_p50_ms": 633.3689622115344,
        "latency_metric_excludes_initial_server_queue": true,
        "warmup_included": false,
        "failures": 0
      },
      "validation": {
        "items": 1259,
        "views": 2518,
        "accuracy": 0.8030182684670373,
        "answer_disagreement": 0.11358220810166798,
        "order_agreement": 0.886417791898332,
        "js": 0.023084942771858654,
        "both_wrong_items": 196,
        "concurrency": 16,
        "seconds": 55.61233388166875,
        "decisions_per_second": 45.27772571742395,
        "service_p50_ms": 123.31819534301758,
        "service_p95_ms": 195.64812183380127,
        "http_p50_ms": 423.17572492174804,
        "scheduler_queue_p50_ms": 188.92357125878334,
        "latency_metric_excludes_initial_server_queue": true,
        "warmup_included": false,
        "failures": 0
      }
    },
    "concurrency_search": {
      "selected": {
        "concurrency": 128,
        "runs": 2,
        "median_decisions_per_second": 190.1530961114044,
        "median_service_p50_ms": 121.40101194381714
      },
      "aggregates": [
        {
          "concurrency": 1,
          "runs": 1,
          "median_decisions_per_second": 16.194536650870774,
          "median_service_p50_ms": 50.647735595703125
        },
        {
          "concurrency": 8,
          "runs": 1,
          "median_decisions_per_second": 45.94295835264903,
          "median_service_p50_ms": 111.7018461227417
        },
        {
          "concurrency": 32,
          "runs": 1,
          "median_decisions_per_second": 158.20845752359367,
          "median_service_p50_ms": 117.02775955200195
        },
        {
          "concurrency": 64,
          "runs": 1,
          "median_decisions_per_second": 185.02005123015408,
          "median_service_p50_ms": 126.47759914398193
        },
        {
          "concurrency": 128,
          "runs": 2,
          "median_decisions_per_second": 190.1530961114044,
          "median_service_p50_ms": 121.40101194381714
        },
        {
          "concurrency": 256,
          "runs": 2,
          "median_decisions_per_second": 189.78770785828365,
          "median_service_p50_ms": 121.05685472488403
        },
        {
          "concurrency": 512,
          "runs": 1,
          "median_decisions_per_second": 187.9789270075793,
          "median_service_p50_ms": 121.22511863708496
        }
      ],
      "uses_gold": false,
      "workload_items": 2048,
      "claim": "Best observed tested concurrency; top two repeated. Service p50 from same trials, initial server queue excluded."
    },
    "model": {
      "key": "korjev-26b4a-v1",
      "label": "korjev-26b4a-v1",
      "model": "google/gemma-4-26B-A4B-it",
      "revision": "4d7ae4984b7db7de8f8457170b3f1a419ee76d52",
      "precision": "FP8",
      "export": true,
      "adapter": {
        "repo": "DaehanKim/korean-jev-gemma4-26b-v7-ce-js-kl",
        "revision": "488603ea86c59531b0870c8aee50ee43d0002af4",
        "subfolder": "epoch-1.0"
      },
      "checkpoint": "KL epoch 1.0",
      "active_parameters_billion": 3.8,
      "parameter_source": "https://huggingface.co/google/gemma-4-26B-A4B-it"
    },
    "weights_or_recipe_selected_using_tests": false,
    "training": false
  },
  "tensor_count": 24198,
  "tensor_dtype_counts": {
    "BF16": 12473,
    "F8_E4M3": 11725
  },
  "example_passed": true,
  "sglang_version": "0.5.21"
}

	{
	"smoke": {
	"items": 96,
	"prediction_agreement": 0.9791666666666666,
	"max_probability_delta": 0.2923086923258452,
	"mean_probability_delta": 0.009561798528451486,
	"reference": "Previous full matched FP8 benchmark; same source revisions and export recipe",
	"concurrency": 64
	},
	"full_benchmark": {
	"metrics": {
	"korean": {
	"items": 25418,
	"views": 25418,
	"accuracy": 0.9215516563065544,
	"answer_disagreement": 0.0,
	"order_agreement": 1.0,
	"js": 0.0,
	"both_wrong_items": 1994,
	"concurrency": 128,
	"seconds": 136.16059323586524,
	"decisions_per_second": 186.67662497599048,
	"service_p50_ms": 125.52499771118164,
	"service_p95_ms": 182.70281553268424,
	"http_p50_ms": 685.1250806357712,
	"scheduler_queue_p50_ms": 484.1857466381043,
	"latency_metric_excludes_initial_server_queue": true,
	"warmup_included": false,
	"failures": 0
	},
	"english": {
	"items": 12032,
	"views": 12032,
	"accuracy": 0.6565824468085106,
	"answer_disagreement": 0.0,
	"order_agreement": 1.0,
	"js": 0.0,
	"both_wrong_items": 4132,
	"concurrency": 128,
	"seconds": 78.06261895876378,
	"decisions_per_second": 154.13267144362462,
	"service_p50_ms": 121.21415138244629,
	"service_p95_ms": 180.46717643737773,
	"http_p50_ms": 819.7654015384614,
	"scheduler_queue_p50_ms": 633.3689622115344,
	"latency_metric_excludes_initial_server_queue": true,
	"warmup_included": false,
	"failures": 0
	},
	"validation": {
	"items": 1259,
	"views": 2518,
	"accuracy": 0.8030182684670373,
	"answer_disagreement": 0.11358220810166798,
	"order_agreement": 0.886417791898332,
	"js": 0.023084942771858654,
	"both_wrong_items": 196,
	"concurrency": 16,
	"seconds": 55.61233388166875,
	"decisions_per_second": 45.27772571742395,
	"service_p50_ms": 123.31819534301758,
	"service_p95_ms": 195.64812183380127,
	"http_p50_ms": 423.17572492174804,
	"scheduler_queue_p50_ms": 188.92357125878334,
	"latency_metric_excludes_initial_server_queue": true,
	"warmup_included": false,
	"failures": 0
	}
	},
	"concurrency_search": {
	"selected": {
	"concurrency": 128,
	"runs": 2,
	"median_decisions_per_second": 190.1530961114044,
	"median_service_p50_ms": 121.40101194381714
	},
	"aggregates": [
	{
	"concurrency": 1,
	"runs": 1,
	"median_decisions_per_second": 16.194536650870774,
	"median_service_p50_ms": 50.647735595703125
	},
	{
	"concurrency": 8,
	"runs": 1,
	"median_decisions_per_second": 45.94295835264903,
	"median_service_p50_ms": 111.7018461227417
	},
	{
	"concurrency": 32,
	"runs": 1,
	"median_decisions_per_second": 158.20845752359367,
	"median_service_p50_ms": 117.02775955200195
	},
	{
	"concurrency": 64,
	"runs": 1,
	"median_decisions_per_second": 185.02005123015408,
	"median_service_p50_ms": 126.47759914398193
	},
	{
	"concurrency": 128,
	"runs": 2,
	"median_decisions_per_second": 190.1530961114044,
	"median_service_p50_ms": 121.40101194381714
	},
	{
	"concurrency": 256,
	"runs": 2,
	"median_decisions_per_second": 189.78770785828365,
	"median_service_p50_ms": 121.05685472488403
	},
	{
	"concurrency": 512,
	"runs": 1,
	"median_decisions_per_second": 187.9789270075793,
	"median_service_p50_ms": 121.22511863708496
	}
	],
	"uses_gold": false,
	"workload_items": 2048,
	"claim": "Best observed tested concurrency; top two repeated. Service p50 from same trials, initial server queue excluded."
	},
	"model": {
	"key": "korjev-26b4a-v1",
	"label": "korjev-26b4a-v1",
	"model": "google/gemma-4-26B-A4B-it",
	"revision": "4d7ae4984b7db7de8f8457170b3f1a419ee76d52",
	"precision": "FP8",
	"export": true,
	"adapter": {
	"repo": "DaehanKim/korean-jev-gemma4-26b-v7-ce-js-kl",
	"revision": "488603ea86c59531b0870c8aee50ee43d0002af4",
	"subfolder": "epoch-1.0"
	},
	"checkpoint": "KL epoch 1.0",
	"active_parameters_billion": 3.8,
	"parameter_source": "https://huggingface.co/google/gemma-4-26B-A4B-it"
	},
	"weights_or_recipe_selected_using_tests": false,
	"training": false
	},
	"tensor_count": 24198,
	"tensor_dtype_counts": {
	"BF16": 12473,
	"F8_E4M3": 11725
	},
	"example_passed": true,
	"sglang_version": "0.5.21"
	}