LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-BarExam-MCQ-1-14-Epoch2

KoHRM-Text-1.4B FullSFT BarExam MCQ 1-14 Epoch2

Full fine-tune of LLM-OS-Models/KoHRM-Text-1.4B on rounds 1-14 of the Korean Bar Exam (MOJ) multiple-choice corpus.

Base Model

• Base: LLM-OS-Models/KoHRM-Text-1.4B (KoHRM-Text-1.4B stage4d PT)
• Relation: full fine-tune
• Runtime: local KoHRM/HRM-Text PrefixLM runtime
• Export format: single-file model.safetensors plus tokenizer/config

Training

• Dataset: gyung/korean-bar-exam-moj-multiple-choice data/questions.csv, rounds 1-14
• Rows kept: 2,027 (single-answer rows; 73 composite/missing rows dropped)
• Tokens: 778,769 (avg sample 384, max 1,160)
• Epochs: 2
• Global batch size: 4,096 tokens
• Learning rate: 3.0e-5, cosine, warmup 20 steps
• Single H200 (CUDA index 7), torchrun nproc_per_node=1
• Train loss (final): 0.324
• Run time: ~5 minutes

Instruction/response template:

instruction = (
    "다음 변호사시험 선택형 문제를 읽고, 가장 옳은 정답 번호만 답하시오. "
    "출력은 '정답: <번호>' 형식 한 줄로만 하시오.\n\n"
    "[회차] 제{round}회\n"
    "[과목] {subject}\n"
    "[문항번호] {question_no}\n"
    "[문제]\n{question_text}"
)
response = "정답: {answer}"

Evaluation (round 15)

Round 15 of the same dataset is held out. 145 single-answer questions.

run	condition	accuracy	parse rate
base (no SFT)	direct	13.1 % (19/145)	61.4 %
this checkpoint	direct	26.9 % (39/145)	100 %

By subject (this checkpoint):

subject	accuracy
공법	30.8 % (12/39)
민사법	22.4 % (15/67)
형사법	30.8 % (12/39)

Random baseline (single-answer 5-way) = 20 %.

Usage

This is not a standard Hugging Face AutoModelForCausalLM chat-model export. It uses the KoHRM/HRM-Text PrefixLM runtime. Tokenizer special tokens (no chat_template):

<|im_start|>          boq (id 2)
<|im_end|>            eoq (id 3)
<|box_end|>           eoa (id 35, eos)
<|object_ref_start|>  direct condition (id 32)
<|object_ref_end|>    cot condition    (id 33)

Prompt is tokenized as <|im_start|><|object_ref_start|>{instruction}<|im_end|>, generation stops at <|box_end|>. See simple_inference_engine.py in the source repo.

Source

• Dataset: gyung/korean-bar-exam-moj-multiple-choice
• Source license: Korea Open Government License Type 1 (KOGL Type 1)
• Base model card: LLM-OS-Models/KoHRM-Text-1.4B