Instructions to use Ba2han/l-cpt-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ba2han/l-cpt-1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Ba2han/l-cpt-1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Ba2han/l-cpt-1")
model = AutoModelForImageTextToText.from_pretrained("Ba2han/l-cpt-1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ba2han/l-cpt-1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ba2han/l-cpt-1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/l-cpt-1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Ba2han/l-cpt-1

SGLang

How to use Ba2han/l-cpt-1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ba2han/l-cpt-1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/l-cpt-1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ba2han/l-cpt-1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/l-cpt-1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Studio

How to use Ba2han/l-cpt-1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/l-cpt-1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/l-cpt-1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ba2han/l-cpt-1 to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Ba2han/l-cpt-1",
    max_seq_length=2048,
)

Docker Model Runner
How to use Ba2han/l-cpt-1 with Docker Model Runner:
```
docker model run hf.co/Ba2han/l-cpt-1
```

Ba2han commited on Apr 14

Commit

effb705

verified ·

1 Parent(s): 75ed97d

Training in progress, step 11000, checkpoint

Browse files

Files changed (4) hide show

last-checkpoint/model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +3503 -3

last-checkpoint/model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:24cb31581e991ddea63485383cc6778a05d68b190bbf64ce7aec78a64bd678fc
 size 3193334216

 version https://git-lfs.github.com/spec/v1
+oid sha256:cf476373257c3648ba4ab1aca756d1238ee8cdb9e819383637f2adb5e8fc9874
 size 3193334216

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ace2b3bb98e7019030d5298efa8ac8494ab7325f2434eb68fe22be2e1ac7d7df
 size 3464388157

 version https://git-lfs.github.com/spec/v1
+oid sha256:8fb747f5bc3af38f67c6efd46e9f31d77e68f221662910efc6fcd741dcbe0bd6
 size 3464388157

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:22fb23d3f4925cb42be7387d3457dbc5736bb88b7f540d4c811555cf32f648fb
 size 1465

 version https://git-lfs.github.com/spec/v1
+oid sha256:e1b7a8930db3356c00cbcb04b613e6045c1d0fef4335e71a16302360492013c2
 size 1465

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,9 +2,9 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.9150688139918379,
   "eval_steps": 500,
-  "global_step": 10500,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -73508,6 +73508,3506 @@
       "learning_rate": 0.0005964929933549236,
       "loss": 1.3805874586105347,
       "step": 10500
     }
   ],
   "logging_steps": 1,
@@ -73527,7 +77027,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 3.8824008192103173e+18,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 0.9586435194200207,
   "eval_steps": 500,
+  "global_step": 11000,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "learning_rate": 0.0005964929933549236,
       "loss": 1.3805874586105347,
       "step": 10500
+    },
+    {
+      "epoch": 0.9151559634026942,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0005958120595250271,
+      "loss": 1.3996551036834717,
+      "step": 10501
+    },
+    {
+      "epoch": 0.9152431128135506,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.0005951313502807226,
+      "loss": 1.3089829683303833,
+      "step": 10502
+    },
+    {
+      "epoch": 0.915330262224407,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0005944508659997757,
+      "loss": 1.3411579132080078,
+      "step": 10503
+    },
+    {
+      "epoch": 0.9154174116352634,
+      "grad_norm": 0.515625,
+      "learning_rate": 0.0005937706070598278,
+      "loss": 1.4157911539077759,
+      "step": 10504
+    },
+    {
+      "epoch": 0.9155045610461198,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0005930905738383965,
+      "loss": 1.318338394165039,
+      "step": 10505
+    },
+    {
+      "epoch": 0.9155917104569761,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005924107667128726,
+      "loss": 1.3024048805236816,
+      "step": 10506
+    },
+    {
+      "epoch": 0.9156788598678325,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0005917311860605213,
+      "loss": 1.3165955543518066,
+      "step": 10507
+    },
+    {
+      "epoch": 0.9157660092786888,
+      "grad_norm": 0.64453125,
+      "learning_rate": 0.0005910518322584838,
+      "loss": 1.3904566764831543,
+      "step": 10508
+    },
+    {
+      "epoch": 0.9158531586895452,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005903727056837727,
+      "loss": 1.4618806838989258,
+      "step": 10509
+    },
+    {
+      "epoch": 0.9159403081004016,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.0005896938067132759,
+      "loss": 1.2961794137954712,
+      "step": 10510
+    },
+    {
+      "epoch": 0.9160274575112579,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005890151357237569,
+      "loss": 1.3977863788604736,
+      "step": 10511
+    },
+    {
+      "epoch": 0.9161146069221143,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.00058833669309185,
+      "loss": 1.3360893726348877,
+      "step": 10512
+    },
+    {
+      "epoch": 0.9162017563329706,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.0005876584791940637,
+      "loss": 1.3887076377868652,
+      "step": 10513
+    },
+    {
+      "epoch": 0.916288905743827,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0005869804944067799,
+      "loss": 1.6108146905899048,
+      "step": 10514
+    },
+    {
+      "epoch": 0.9163760551546833,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.000586302739106251,
+      "loss": 1.4226261377334595,
+      "step": 10515
+    },
+    {
+      "epoch": 0.9164632045655398,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005856252136686069,
+      "loss": 1.4893242120742798,
+      "step": 10516
+    },
+    {
+      "epoch": 0.9165503539763962,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0005849479184698455,
+      "loss": 1.3504881858825684,
+      "step": 10517
+    },
+    {
+      "epoch": 0.9166375033872525,
+      "grad_norm": 0.7265625,
+      "learning_rate": 0.0005842708538858389,
+      "loss": 1.3939337730407715,
+      "step": 10518
+    },
+    {
+      "epoch": 0.9167246527981089,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005835940202923306,
+      "loss": 1.3288497924804688,
+      "step": 10519
+    },
+    {
+      "epoch": 0.9168118022089652,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0005829174180649356,
+      "loss": 1.4457199573516846,
+      "step": 10520
+    },
+    {
+      "epoch": 0.9168989516198216,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.0005822410475791423,
+      "loss": 1.4103813171386719,
+      "step": 10521
+    },
+    {
+      "epoch": 0.9169861010306779,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005815649092103085,
+      "loss": 1.4175074100494385,
+      "step": 10522
+    },
+    {
+      "epoch": 0.9170732504415343,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.0005808890033336642,
+      "loss": 1.3463209867477417,
+      "step": 10523
+    },
+    {
+      "epoch": 0.9171603998523907,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0005802133303243096,
+      "loss": 1.4614325761795044,
+      "step": 10524
+    },
+    {
+      "epoch": 0.917247549263247,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0005795378905572166,
+      "loss": 1.3758034706115723,
+      "step": 10525
+    },
+    {
+      "epoch": 0.9173346986741034,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.0005788626844072265,
+      "loss": 1.3858534097671509,
+      "step": 10526
+    },
+    {
+      "epoch": 0.9174218480849597,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0005781877122490526,
+      "loss": 1.4781090021133423,
+      "step": 10527
+    },
+    {
+      "epoch": 0.9175089974958162,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005775129744572769,
+      "loss": 1.4332325458526611,
+      "step": 10528
+    },
+    {
+      "epoch": 0.9175961469066725,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.000576838471406352,
+      "loss": 1.375857949256897,
+      "step": 10529
+    },
+    {
+      "epoch": 0.9176832963175289,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0005761642034705997,
+      "loss": 1.4588314294815063,
+      "step": 10530
+    },
+    {
+      "epoch": 0.9177704457283853,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0005754901710242113,
+      "loss": 1.3294475078582764,
+      "step": 10531
+    },
+    {
+      "epoch": 0.9178575951392416,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.0005748163744412494,
+      "loss": 1.5025684833526611,
+      "step": 10532
+    },
+    {
+      "epoch": 0.917944744550098,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.000574142814095643,
+      "loss": 1.3892470598220825,
+      "step": 10533
+    },
+    {
+      "epoch": 0.9180318939609543,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0005734694903611913,
+      "loss": 1.4492141008377075,
+      "step": 10534
+    },
+    {
+      "epoch": 0.9181190433718107,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.000572796403611562,
+      "loss": 1.2923322916030884,
+      "step": 10535
+    },
+    {
+      "epoch": 0.918206192782667,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0005721235542202904,
+      "loss": 1.5104238986968994,
+      "step": 10536
+    },
+    {
+      "epoch": 0.9182933421935234,
+      "grad_norm": 0.671875,
+      "learning_rate": 0.0005714509425607825,
+      "loss": 1.4007127285003662,
+      "step": 10537
+    },
+    {
+      "epoch": 0.9183804916043798,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0005707785690063103,
+      "loss": 1.3198069334030151,
+      "step": 10538
+    },
+    {
+      "epoch": 0.9184676410152361,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0005701064339300137,
+      "loss": 1.190271258354187,
+      "step": 10539
+    },
+    {
+      "epoch": 0.9185547904260926,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0005694345377049013,
+      "loss": 1.3525913953781128,
+      "step": 10540
+    },
+    {
+      "epoch": 0.9186419398369489,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.0005687628807038482,
+      "loss": 1.1677380800247192,
+      "step": 10541
+    },
+    {
+      "epoch": 0.9187290892478053,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0005680914632995964,
+      "loss": 1.4719948768615723,
+      "step": 10542
+    },
+    {
+      "epoch": 0.9188162386586616,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.0005674202858647577,
+      "loss": 1.3417930603027344,
+      "step": 10543
+    },
+    {
+      "epoch": 0.918903388069518,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0005667493487718071,
+      "loss": 1.3469139337539673,
+      "step": 10544
+    },
+    {
+      "epoch": 0.9189905374803744,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0005660786523930881,
+      "loss": 1.4537931680679321,
+      "step": 10545
+    },
+    {
+      "epoch": 0.9190776868912307,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005654081971008105,
+      "loss": 1.2847542762756348,
+      "step": 10546
+    },
+    {
+      "epoch": 0.9191648363020871,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0005647379832670486,
+      "loss": 1.3840348720550537,
+      "step": 10547
+    },
+    {
+      "epoch": 0.9192519857129434,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.000564068011263747,
+      "loss": 1.2110737562179565,
+      "step": 10548
+    },
+    {
+      "epoch": 0.9193391351237998,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005633982814627125,
+      "loss": 1.4413111209869385,
+      "step": 10549
+    },
+    {
+      "epoch": 0.9194262845346561,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0005627287942356171,
+      "loss": 1.4561347961425781,
+      "step": 10550
+    },
+    {
+      "epoch": 0.9195134339455125,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.000562059549954,
+      "loss": 1.3817965984344482,
+      "step": 10551
+    },
+    {
+      "epoch": 0.919600583356369,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0005613905489892645,
+      "loss": 1.3496308326721191,
+      "step": 10552
+    },
+    {
+      "epoch": 0.9196877327672253,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0005607217917126804,
+      "loss": 1.4852162599563599,
+      "step": 10553
+    },
+    {
+      "epoch": 0.9197748821780817,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0005600532784953802,
+      "loss": 1.295851230621338,
+      "step": 10554
+    },
+    {
+      "epoch": 0.919862031588938,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0005593850097083633,
+      "loss": 1.326664924621582,
+      "step": 10555
+    },
+    {
+      "epoch": 0.9199491809997944,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0005587169857224907,
+      "loss": 1.4255481958389282,
+      "step": 10556
+    },
+    {
+      "epoch": 0.9200363304106507,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.0005580492069084894,
+      "loss": 1.3229376077651978,
+      "step": 10557
+    },
+    {
+      "epoch": 0.9201234798215071,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0005573816736369492,
+      "loss": 1.453386902809143,
+      "step": 10558
+    },
+    {
+      "epoch": 0.9202106292323635,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005567143862783259,
+      "loss": 1.2901363372802734,
+      "step": 10559
+    },
+    {
+      "epoch": 0.9202977786432198,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0005560473452029361,
+      "loss": 1.40326726436615,
+      "step": 10560
+    },
+    {
+      "epoch": 0.9203849280540762,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0005553805507809616,
+      "loss": 1.39547860622406,
+      "step": 10561
+    },
+    {
+      "epoch": 0.9204720774649325,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005547140033824458,
+      "loss": 1.3772691488265991,
+      "step": 10562
+    },
+    {
+      "epoch": 0.920559226875789,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.000554047703377295,
+      "loss": 1.2665115594863892,
+      "step": 10563
+    },
+    {
+      "epoch": 0.9206463762866453,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005533816511352811,
+      "loss": 1.4510654211044312,
+      "step": 10564
+    },
+    {
+      "epoch": 0.9207335256975017,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0005527158470260357,
+      "loss": 1.5079319477081299,
+      "step": 10565
+    },
+    {
+      "epoch": 0.9208206751083581,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.000552050291419053,
+      "loss": 1.4127393960952759,
+      "step": 10566
+    },
+    {
+      "epoch": 0.9209078245192144,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0005513849846836902,
+      "loss": 1.3956263065338135,
+      "step": 10567
+    },
+    {
+      "epoch": 0.9209949739300708,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.0005507199271891644,
+      "loss": 1.4360769987106323,
+      "step": 10568
+    },
+    {
+      "epoch": 0.9210821233409271,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.0005500551193045577,
+      "loss": 1.4644925594329834,
+      "step": 10569
+    },
+    {
+      "epoch": 0.9211692727517835,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0005493905613988112,
+      "loss": 1.3501391410827637,
+      "step": 10570
+    },
+    {
+      "epoch": 0.9212564221626398,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005487262538407272,
+      "loss": 1.3831145763397217,
+      "step": 10571
+    },
+    {
+      "epoch": 0.9213435715734962,
+      "grad_norm": 0.66015625,
+      "learning_rate": 0.000548062196998971,
+      "loss": 1.3456957340240479,
+      "step": 10572
+    },
+    {
+      "epoch": 0.9214307209843526,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.0005473983912420673,
+      "loss": 1.2185256481170654,
+      "step": 10573
+    },
+    {
+      "epoch": 0.9215178703952089,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0005467348369384002,
+      "loss": 1.3591208457946777,
+      "step": 10574
+    },
+    {
+      "epoch": 0.9216050198060654,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0005460715344562175,
+      "loss": 1.4143370389938354,
+      "step": 10575
+    },
+    {
+      "epoch": 0.9216921692169217,
+      "grad_norm": 0.671875,
+      "learning_rate": 0.0005454084841636253,
+      "loss": 1.4549078941345215,
+      "step": 10576
+    },
+    {
+      "epoch": 0.9217793186277781,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0005447456864285891,
+      "loss": 1.3903030157089233,
+      "step": 10577
+    },
+    {
+      "epoch": 0.9218664680386344,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005440831416189357,
+      "loss": 1.5252403020858765,
+      "step": 10578
+    },
+    {
+      "epoch": 0.9219536174494908,
+      "grad_norm": 0.6640625,
+      "learning_rate": 0.0005434208501023498,
+      "loss": 1.3850946426391602,
+      "step": 10579
+    },
+    {
+      "epoch": 0.9220407668603472,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0005427588122463785,
+      "loss": 1.3343126773834229,
+      "step": 10580
+    },
+    {
+      "epoch": 0.9221279162712035,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0005420970284184253,
+      "loss": 1.3930814266204834,
+      "step": 10581
+    },
+    {
+      "epoch": 0.9222150656820599,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.000541435498985754,
+      "loss": 1.4217712879180908,
+      "step": 10582
+    },
+    {
+      "epoch": 0.9223022150929162,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0005407742243154865,
+      "loss": 1.3846927881240845,
+      "step": 10583
+    },
+    {
+      "epoch": 0.9223893645037726,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0005401132047746036,
+      "loss": 1.378669023513794,
+      "step": 10584
+    },
+    {
+      "epoch": 0.922476513914629,
+      "grad_norm": 0.70703125,
+      "learning_rate": 0.0005394524407299454,
+      "loss": 1.3566769361495972,
+      "step": 10585
+    },
+    {
+      "epoch": 0.9225636633254853,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0005387919325482091,
+      "loss": 1.3824747800827026,
+      "step": 10586
+    },
+    {
+      "epoch": 0.9226508127363418,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0005381316805959499,
+      "loss": 1.5102699995040894,
+      "step": 10587
+    },
+    {
+      "epoch": 0.922737962147198,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.0005374716852395819,
+      "loss": 1.500016212463379,
+      "step": 10588
+    },
+    {
+      "epoch": 0.9228251115580545,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.0005368119468453758,
+      "loss": 1.219261646270752,
+      "step": 10589
+    },
+    {
+      "epoch": 0.9229122609689108,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0005361524657794594,
+      "loss": 1.3092141151428223,
+      "step": 10590
+    },
+    {
+      "epoch": 0.9229994103797672,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0005354932424078192,
+      "loss": 1.3061190843582153,
+      "step": 10591
+    },
+    {
+      "epoch": 0.9230865597906236,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.0005348342770962973,
+      "loss": 1.3815596103668213,
+      "step": 10592
+    },
+    {
+      "epoch": 0.9231737092014799,
+      "grad_norm": 0.69921875,
+      "learning_rate": 0.0005341755702105931,
+      "loss": 1.4937219619750977,
+      "step": 10593
+    },
+    {
+      "epoch": 0.9232608586123363,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005335171221162624,
+      "loss": 1.3349745273590088,
+      "step": 10594
+    },
+    {
+      "epoch": 0.9233480080231926,
+      "grad_norm": 0.70703125,
+      "learning_rate": 0.0005328589331787166,
+      "loss": 1.5849881172180176,
+      "step": 10595
+    },
+    {
+      "epoch": 0.923435157434049,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0005322010037632264,
+      "loss": 1.3104616403579712,
+      "step": 10596
+    },
+    {
+      "epoch": 0.9235223068449053,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0005315433342349149,
+      "loss": 1.2885944843292236,
+      "step": 10597
+    },
+    {
+      "epoch": 0.9236094562557617,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0005308859249587625,
+      "loss": 1.2921130657196045,
+      "step": 10598
+    },
+    {
+      "epoch": 0.9236966056666182,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.0005302287762996049,
+      "loss": 1.429068684577942,
+      "step": 10599
+    },
+    {
+      "epoch": 0.9237837550774745,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0005295718886221327,
+      "loss": 1.4047555923461914,
+      "step": 10600
+    },
+    {
+      "epoch": 0.9238709044883309,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0005289152622908937,
+      "loss": 1.5161041021347046,
+      "step": 10601
+    },
+    {
+      "epoch": 0.9239580538991872,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005282588976702887,
+      "loss": 1.2470475435256958,
+      "step": 10602
+    },
+    {
+      "epoch": 0.9240452033100436,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0005276027951245727,
+      "loss": 1.4848660230636597,
+      "step": 10603
+    },
+    {
+      "epoch": 0.9241323527208999,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0005269469550178576,
+      "loss": 1.2991611957550049,
+      "step": 10604
+    },
+    {
+      "epoch": 0.9242195021317563,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.0005262913777141077,
+      "loss": 1.3719066381454468,
+      "step": 10605
+    },
+    {
+      "epoch": 0.9243066515426127,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0005256360635771416,
+      "loss": 1.3915817737579346,
+      "step": 10606
+    },
+    {
+      "epoch": 0.924393800953469,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.000524981012970634,
+      "loss": 1.4314830303192139,
+      "step": 10607
+    },
+    {
+      "epoch": 0.9244809503643254,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005243262262581111,
+      "loss": 1.243580937385559,
+      "step": 10608
+    },
+    {
+      "epoch": 0.9245680997751817,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.0005236717038029527,
+      "loss": 1.568758249282837,
+      "step": 10609
+    },
+    {
+      "epoch": 0.9246552491860381,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0005230174459683929,
+      "loss": 1.4314721822738647,
+      "step": 10610
+    },
+    {
+      "epoch": 0.9247423985968944,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0005223634531175173,
+      "loss": 1.3028546571731567,
+      "step": 10611
+    },
+    {
+      "epoch": 0.9248295480077509,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005217097256132682,
+      "loss": 1.4798426628112793,
+      "step": 10612
+    },
+    {
+      "epoch": 0.9249166974186073,
+      "grad_norm": 0.67578125,
+      "learning_rate": 0.0005210562638184369,
+      "loss": 1.4144682884216309,
+      "step": 10613
+    },
+    {
+      "epoch": 0.9250038468294636,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005204030680956691,
+      "loss": 1.3268251419067383,
+      "step": 10614
+    },
+    {
+      "epoch": 0.92509099624032,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.0005197501388074612,
+      "loss": 1.3798325061798096,
+      "step": 10615
+    },
+    {
+      "epoch": 0.9251781456511763,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.0005190974763161626,
+      "loss": 1.4024105072021484,
+      "step": 10616
+    },
+    {
+      "epoch": 0.9252652950620327,
+      "grad_norm": 0.6640625,
+      "learning_rate": 0.0005184450809839761,
+      "loss": 1.3854591846466064,
+      "step": 10617
+    },
+    {
+      "epoch": 0.925352444472889,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005177929531729546,
+      "loss": 1.463329553604126,
+      "step": 10618
+    },
+    {
+      "epoch": 0.9254395938837454,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.0005171410932450028,
+      "loss": 1.2485421895980835,
+      "step": 10619
+    },
+    {
+      "epoch": 0.9255267432946018,
+      "grad_norm": 0.50390625,
+      "learning_rate": 0.0005164895015618766,
+      "loss": 1.3857693672180176,
+      "step": 10620
+    },
+    {
+      "epoch": 0.9256138927054581,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005158381784851836,
+      "loss": 1.2931714057922363,
+      "step": 10621
+    },
+    {
+      "epoch": 0.9257010421163145,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.0005151871243763809,
+      "loss": 1.4780501127243042,
+      "step": 10622
+    },
+    {
+      "epoch": 0.9257881915271708,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005145363395967793,
+      "loss": 1.1755802631378174,
+      "step": 10623
+    },
+    {
+      "epoch": 0.9258753409380273,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0005138858245075372,
+      "loss": 1.4387574195861816,
+      "step": 10624
+    },
+    {
+      "epoch": 0.9259624903488836,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.0005132355794696646,
+      "loss": 1.3815428018569946,
+      "step": 10625
+    },
+    {
+      "epoch": 0.92604963975974,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.000512585604844022,
+      "loss": 1.5630664825439453,
+      "step": 10626
+    },
+    {
+      "epoch": 0.9261367891705964,
+      "grad_norm": 0.53515625,
+      "learning_rate": 0.000511935900991317,
+      "loss": 1.2940813302993774,
+      "step": 10627
+    },
+    {
+      "epoch": 0.9262239385814527,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0005112864682721121,
+      "loss": 1.386154294013977,
+      "step": 10628
+    },
+    {
+      "epoch": 0.9263110879923091,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.0005106373070468151,
+      "loss": 1.374330997467041,
+      "step": 10629
+    },
+    {
+      "epoch": 0.9263982374031654,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.0005099884176756846,
+      "loss": 1.3532607555389404,
+      "step": 10630
+    },
+    {
+      "epoch": 0.9264853868140218,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0005093398005188283,
+      "loss": 1.4316142797470093,
+      "step": 10631
+    },
+    {
+      "epoch": 0.9265725362248781,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0005086914559362023,
+      "loss": 1.2776588201522827,
+      "step": 10632
+    },
+    {
+      "epoch": 0.9266596856357345,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0005080433842876125,
+      "loss": 1.395501732826233,
+      "step": 10633
+    },
+    {
+      "epoch": 0.9267468350465909,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005073955859327124,
+      "loss": 1.4338113069534302,
+      "step": 10634
+    },
+    {
+      "epoch": 0.9268339844574472,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.0005067480612310041,
+      "loss": 1.4566060304641724,
+      "step": 10635
+    },
+    {
+      "epoch": 0.9269211338683037,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.000506100810541838,
+      "loss": 1.3558721542358398,
+      "step": 10636
+    },
+    {
+      "epoch": 0.92700828327916,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0005054538342244121,
+      "loss": 1.5648887157440186,
+      "step": 10637
+    },
+    {
+      "epoch": 0.9270954326900164,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0005048071326377716,
+      "loss": 1.4658679962158203,
+      "step": 10638
+    },
+    {
+      "epoch": 0.9271825821008727,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0005041607061408118,
+      "loss": 1.4238897562026978,
+      "step": 10639
+    },
+    {
+      "epoch": 0.9272697315117291,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0005035145550922722,
+      "loss": 1.4082732200622559,
+      "step": 10640
+    },
+    {
+      "epoch": 0.9273568809225855,
+      "grad_norm": 0.75390625,
+      "learning_rate": 0.0005028686798507411,
+      "loss": 1.4516377449035645,
+      "step": 10641
+    },
+    {
+      "epoch": 0.9274440303334418,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0005022230807746533,
+      "loss": 1.372707724571228,
+      "step": 10642
+    },
+    {
+      "epoch": 0.9275311797442982,
+      "grad_norm": 0.64453125,
+      "learning_rate": 0.0005015777582222899,
+      "loss": 1.4883637428283691,
+      "step": 10643
+    },
+    {
+      "epoch": 0.9276183291551545,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0005009327125517801,
+      "loss": 1.3155707120895386,
+      "step": 10644
+    },
+    {
+      "epoch": 0.9277054785660109,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0005002879441210981,
+      "loss": 1.371456503868103,
+      "step": 10645
+    },
+    {
+      "epoch": 0.9277926279768672,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0004996434532880643,
+      "loss": 1.4533182382583618,
+      "step": 10646
+    },
+    {
+      "epoch": 0.9278797773877236,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0004989992404103452,
+      "loss": 1.382372260093689,
+      "step": 10647
+    },
+    {
+      "epoch": 0.9279669267985801,
+      "grad_norm": 0.69140625,
+      "learning_rate": 0.0004983553058454533,
+      "loss": 1.3707820177078247,
+      "step": 10648
+    },
+    {
+      "epoch": 0.9280540762094364,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.0004977116499507462,
+      "loss": 1.3942557573318481,
+      "step": 10649
+    },
+    {
+      "epoch": 0.9281412256202928,
+      "grad_norm": 0.6875,
+      "learning_rate": 0.0004970682730834275,
+      "loss": 1.5383590459823608,
+      "step": 10650
+    },
+    {
+      "epoch": 0.9282283750311491,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.0004964251756005457,
+      "loss": 1.2526518106460571,
+      "step": 10651
+    },
+    {
+      "epoch": 0.9283155244420055,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0004957823578589939,
+      "loss": 1.3667023181915283,
+      "step": 10652
+    },
+    {
+      "epoch": 0.9284026738528618,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.00049513982021551,
+      "loss": 1.3441028594970703,
+      "step": 10653
+    },
+    {
+      "epoch": 0.9284898232637182,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0004944975630266757,
+      "loss": 1.3417932987213135,
+      "step": 10654
+    },
+    {
+      "epoch": 0.9285769726745746,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0004938555866489208,
+      "loss": 1.3898224830627441,
+      "step": 10655
+    },
+    {
+      "epoch": 0.9286641220854309,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0004932138914385143,
+      "loss": 1.3794687986373901,
+      "step": 10656
+    },
+    {
+      "epoch": 0.9287512714962873,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.000492572477751572,
+      "loss": 1.456784963607788,
+      "step": 10657
+    },
+    {
+      "epoch": 0.9288384209071436,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.0004919313459440525,
+      "loss": 1.3471949100494385,
+      "step": 10658
+    },
+    {
+      "epoch": 0.928925570318,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.0004912904963717576,
+      "loss": 1.3238189220428467,
+      "step": 10659
+    },
+    {
+      "epoch": 0.9290127197288564,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0004906499293903348,
+      "loss": 1.4607516527175903,
+      "step": 10660
+    },
+    {
+      "epoch": 0.9290998691397128,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.0004900096453552722,
+      "loss": 1.3629095554351807,
+      "step": 10661
+    },
+    {
+      "epoch": 0.9291870185505692,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.0004893696446219017,
+      "loss": 1.4566161632537842,
+      "step": 10662
+    },
+    {
+      "epoch": 0.9292741679614255,
+      "grad_norm": 0.6875,
+      "learning_rate": 0.0004887299275453983,
+      "loss": 1.304275631904602,
+      "step": 10663
+    },
+    {
+      "epoch": 0.9293613173722819,
+      "grad_norm": 0.52734375,
+      "learning_rate": 0.0004880904944807791,
+      "loss": 1.2976535558700562,
+      "step": 10664
+    },
+    {
+      "epoch": 0.9294484667831382,
+      "grad_norm": 0.53515625,
+      "learning_rate": 0.00048745134578290325,
+      "loss": 1.3486498594284058,
+      "step": 10665
+    },
+    {
+      "epoch": 0.9295356161939946,
+      "grad_norm": 0.55078125,
+      "learning_rate": 0.00048681248180647426,
+      "loss": 1.172863483428955,
+      "step": 10666
+    },
+    {
+      "epoch": 0.9296227656048509,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.00048617390290603543,
+      "loss": 1.3933426141738892,
+      "step": 10667
+    },
+    {
+      "epoch": 0.9297099150157073,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.00048553560943597173,
+      "loss": 1.4262231588363647,
+      "step": 10668
+    },
+    {
+      "epoch": 0.9297970644265637,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.00048489760175051105,
+      "loss": 1.3945653438568115,
+      "step": 10669
+    },
+    {
+      "epoch": 0.92988421383742,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0004842598802037208,
+      "loss": 1.4212998151779175,
+      "step": 10670
+    },
+    {
+      "epoch": 0.9299713632482765,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0004836224451495131,
+      "loss": 1.2879077196121216,
+      "step": 10671
+    },
+    {
+      "epoch": 0.9300585126591328,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00048298529694163785,
+      "loss": 1.3234881162643433,
+      "step": 10672
+    },
+    {
+      "epoch": 0.9301456620699892,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00048234843593368716,
+      "loss": 1.2626367807388306,
+      "step": 10673
+    },
+    {
+      "epoch": 0.9302328114808455,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.0004817118624790925,
+      "loss": 1.3626574277877808,
+      "step": 10674
+    },
+    {
+      "epoch": 0.9303199608917019,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.00048107557693112627,
+      "loss": 1.475213646888733,
+      "step": 10675
+    },
+    {
+      "epoch": 0.9304071103025583,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0004804395796429031,
+      "loss": 1.4251682758331299,
+      "step": 10676
+    },
+    {
+      "epoch": 0.9304942597134146,
+      "grad_norm": 0.625,
+      "learning_rate": 0.00047980387096737515,
+      "loss": 1.5054700374603271,
+      "step": 10677
+    },
+    {
+      "epoch": 0.930581409124271,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.00047916845125733524,
+      "loss": 1.3805677890777588,
+      "step": 10678
+    },
+    {
+      "epoch": 0.9306685585351273,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0004785333208654159,
+      "loss": 1.4884077310562134,
+      "step": 10679
+    },
+    {
+      "epoch": 0.9307557079459837,
+      "grad_norm": 0.625,
+      "learning_rate": 0.000477898480144089,
+      "loss": 1.3699536323547363,
+      "step": 10680
+    },
+    {
+      "epoch": 0.93084285735684,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0004772639294456651,
+      "loss": 1.4125971794128418,
+      "step": 10681
+    },
+    {
+      "epoch": 0.9309300067676964,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00047662966912229566,
+      "loss": 1.4447113275527954,
+      "step": 10682
+    },
+    {
+      "epoch": 0.9310171561785529,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00047599569952596916,
+      "loss": 1.4526288509368896,
+      "step": 10683
+    },
+    {
+      "epoch": 0.9311043055894092,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0004753620210085134,
+      "loss": 1.3986235857009888,
+      "step": 10684
+    },
+    {
+      "epoch": 0.9311914550002656,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.00047472863392159474,
+      "loss": 1.1874592304229736,
+      "step": 10685
+    },
+    {
+      "epoch": 0.9312786044111219,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.0004740955386167158,
+      "loss": 1.3817155361175537,
+      "step": 10686
+    },
+    {
+      "epoch": 0.9313657538219783,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0004734627354452219,
+      "loss": 1.4633171558380127,
+      "step": 10687
+    },
+    {
+      "epoch": 0.9314529032328346,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.00047283022475829215,
+      "loss": 1.349452018737793,
+      "step": 10688
+    },
+    {
+      "epoch": 0.931540052643691,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0004721980069069447,
+      "loss": 1.320845603942871,
+      "step": 10689
+    },
+    {
+      "epoch": 0.9316272020545474,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0004715660822420351,
+      "loss": 1.4221367835998535,
+      "step": 10690
+    },
+    {
+      "epoch": 0.9317143514654037,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0004709344511142559,
+      "loss": 1.2934486865997314,
+      "step": 10691
+    },
+    {
+      "epoch": 0.9318015008762601,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.00047030311387413775,
+      "loss": 1.4287840127944946,
+      "step": 10692
+    },
+    {
+      "epoch": 0.9318886502871164,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.0004696720708720472,
+      "loss": 1.3304390907287598,
+      "step": 10693
+    },
+    {
+      "epoch": 0.9319757996979728,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0004690413224581879,
+      "loss": 1.5018701553344727,
+      "step": 10694
+    },
+    {
+      "epoch": 0.9320629491088291,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.0004684108689825998,
+      "loss": 1.5002820491790771,
+      "step": 10695
+    },
+    {
+      "epoch": 0.9321500985196856,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0004677807107951593,
+      "loss": 1.5061651468276978,
+      "step": 10696
+    },
+    {
+      "epoch": 0.932237247930542,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.00046715084824557827,
+      "loss": 1.3442021608352661,
+      "step": 10697
+    },
+    {
+      "epoch": 0.9323243973413983,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.00046652128168340676,
+      "loss": 1.4294358491897583,
+      "step": 10698
+    },
+    {
+      "epoch": 0.9324115467522547,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00046589201145802813,
+      "loss": 1.3213213682174683,
+      "step": 10699
+    },
+    {
+      "epoch": 0.932498696163111,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0004652630379186621,
+      "loss": 1.3561373949050903,
+      "step": 10700
+    },
+    {
+      "epoch": 0.9325858455739674,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00046463436141436394,
+      "loss": 1.4425005912780762,
+      "step": 10701
+    },
+    {
+      "epoch": 0.9326729949848237,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.00046400598229402306,
+      "loss": 1.3377797603607178,
+      "step": 10702
+    },
+    {
+      "epoch": 0.9327601443956801,
+      "grad_norm": 0.55078125,
+      "learning_rate": 0.0004633779009063662,
+      "loss": 1.4136755466461182,
+      "step": 10703
+    },
+    {
+      "epoch": 0.9328472938065365,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00046275011759995267,
+      "loss": 1.4134703874588013,
+      "step": 10704
+    },
+    {
+      "epoch": 0.9329344432173928,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00046212263272317705,
+      "loss": 1.5474461317062378,
+      "step": 10705
+    },
+    {
+      "epoch": 0.9330215926282492,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.00046149544662426825,
+      "loss": 1.4391179084777832,
+      "step": 10706
+    },
+    {
+      "epoch": 0.9331087420391055,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0004608685596512886,
+      "loss": 1.2693322896957397,
+      "step": 10707
+    },
+    {
+      "epoch": 0.933195891449962,
+      "grad_norm": 0.5390625,
+      "learning_rate": 0.0004602419721521373,
+      "loss": 1.3577617406845093,
+      "step": 10708
+    },
+    {
+      "epoch": 0.9332830408608183,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0004596156844745437,
+      "loss": 1.2863800525665283,
+      "step": 10709
+    },
+    {
+      "epoch": 0.9333701902716747,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.0004589896969660728,
+      "loss": 1.260008692741394,
+      "step": 10710
+    },
+    {
+      "epoch": 0.9334573396825311,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0004583640099741224,
+      "loss": 1.4299962520599365,
+      "step": 10711
+    },
+    {
+      "epoch": 0.9335444890933874,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.000457738623845924,
+      "loss": 1.306565761566162,
+      "step": 10712
+    },
+    {
+      "epoch": 0.9336316385042438,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00045711353892854056,
+      "loss": 1.3982917070388794,
+      "step": 10713
+    },
+    {
+      "epoch": 0.9337187879151001,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.00045648875556887175,
+      "loss": 1.2360150814056396,
+      "step": 10714
+    },
+    {
+      "epoch": 0.9338059373259565,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0004558642741136455,
+      "loss": 1.466596245765686,
+      "step": 10715
+    },
+    {
+      "epoch": 0.9338930867368128,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.00045524009490942443,
+      "loss": 1.269097924232483,
+      "step": 10716
+    },
+    {
+      "epoch": 0.9339802361476692,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00045461621830260285,
+      "loss": 1.3210543394088745,
+      "step": 10717
+    },
+    {
+      "epoch": 0.9340673855585256,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0004539926446394066,
+      "loss": 1.4368460178375244,
+      "step": 10718
+    },
+    {
+      "epoch": 0.9341545349693819,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0004533693742658955,
+      "loss": 1.3695037364959717,
+      "step": 10719
+    },
+    {
+      "epoch": 0.9342416843802384,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.00045274640752795904,
+      "loss": 1.4335851669311523,
+      "step": 10720
+    },
+    {
+      "epoch": 0.9343288337910947,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0004521237447713187,
+      "loss": 1.3742443323135376,
+      "step": 10721
+    },
+    {
+      "epoch": 0.9344159832019511,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0004515013863415273,
+      "loss": 1.4045860767364502,
+      "step": 10722
+    },
+    {
+      "epoch": 0.9345031326128074,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00045087933258396824,
+      "loss": 1.4241158962249756,
+      "step": 10723
+    },
+    {
+      "epoch": 0.9345902820236638,
+      "grad_norm": 0.625,
+      "learning_rate": 0.00045025758384385783,
+      "loss": 1.3815301656723022,
+      "step": 10724
+    },
+    {
+      "epoch": 0.9346774314345202,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.00044963614046624086,
+      "loss": 1.3415958881378174,
+      "step": 10725
+    },
+    {
+      "epoch": 0.9347645808453765,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0004490150027959941,
+      "loss": 1.476638674736023,
+      "step": 10726
+    },
+    {
+      "epoch": 0.9348517302562329,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0004483941711778232,
+      "loss": 1.2964787483215332,
+      "step": 10727
+    },
+    {
+      "epoch": 0.9349388796670892,
+      "grad_norm": 0.6875,
+      "learning_rate": 0.0004477736459562652,
+      "loss": 1.4656143188476562,
+      "step": 10728
+    },
+    {
+      "epoch": 0.9350260290779456,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00044715342747568554,
+      "loss": 1.3738725185394287,
+      "step": 10729
+    },
+    {
+      "epoch": 0.9351131784888019,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.00044653351608028304,
+      "loss": 1.4470515251159668,
+      "step": 10730
+    },
+    {
+      "epoch": 0.9352003278996583,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00044591391211408254,
+      "loss": 1.4029713869094849,
+      "step": 10731
+    },
+    {
+      "epoch": 0.9352874773105148,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0004452946159209396,
+      "loss": 1.4336940050125122,
+      "step": 10732
+    },
+    {
+      "epoch": 0.9353746267213711,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.00044467562784453825,
+      "loss": 1.4127177000045776,
+      "step": 10733
+    },
+    {
+      "epoch": 0.9354617761322275,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0004440569482283916,
+      "loss": 1.3769465684890747,
+      "step": 10734
+    },
+    {
+      "epoch": 0.9355489255430838,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.00044343857741584354,
+      "loss": 1.4565773010253906,
+      "step": 10735
+    },
+    {
+      "epoch": 0.9356360749539402,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0004428205157500644,
+      "loss": 1.4249544143676758,
+      "step": 10736
+    },
+    {
+      "epoch": 0.9357232243647965,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0004422027635740537,
+      "loss": 1.4344592094421387,
+      "step": 10737
+    },
+    {
+      "epoch": 0.9358103737756529,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.0004415853212306389,
+      "loss": 1.4506667852401733,
+      "step": 10738
+    },
+    {
+      "epoch": 0.9358975231865093,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0004409681890624753,
+      "loss": 1.373611569404602,
+      "step": 10739
+    },
+    {
+      "epoch": 0.9359846725973656,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.00044035136741204774,
+      "loss": 1.3849220275878906,
+      "step": 10740
+    },
+    {
+      "epoch": 0.936071822008222,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.00043973485662166705,
+      "loss": 1.4501248598098755,
+      "step": 10741
+    },
+    {
+      "epoch": 0.9361589714190783,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0004391186570334718,
+      "loss": 1.188583493232727,
+      "step": 10742
+    },
+    {
+      "epoch": 0.9362461208299347,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00043850276898942824,
+      "loss": 1.5099093914031982,
+      "step": 10743
+    },
+    {
+      "epoch": 0.936333270240791,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0004378871928313297,
+      "loss": 1.4234578609466553,
+      "step": 10744
+    },
+    {
+      "epoch": 0.9364204196516475,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.00043727192890079464,
+      "loss": 1.3401721715927124,
+      "step": 10745
+    },
+    {
+      "epoch": 0.9365075690625039,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.0004366569775392725,
+      "loss": 1.4871634244918823,
+      "step": 10746
+    },
+    {
+      "epoch": 0.9365947184733602,
+      "grad_norm": 0.6640625,
+      "learning_rate": 0.0004360423390880354,
+      "loss": 1.386480450630188,
+      "step": 10747
+    },
+    {
+      "epoch": 0.9366818678842166,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0004354280138881831,
+      "loss": 1.4693528413772583,
+      "step": 10748
+    },
+    {
+      "epoch": 0.9367690172950729,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00043481400228064144,
+      "loss": 1.3771522045135498,
+      "step": 10749
+    },
+    {
+      "epoch": 0.9368561667059293,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.00043420030460616205,
+      "loss": 1.2081589698791504,
+      "step": 10750
+    },
+    {
+      "epoch": 0.9369433161167856,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.000433586921205323,
+      "loss": 1.2728263139724731,
+      "step": 10751
+    },
+    {
+      "epoch": 0.937030465527642,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0004329738524185274,
+      "loss": 1.3980687856674194,
+      "step": 10752
+    },
+    {
+      "epoch": 0.9371176149384984,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0004323610985860037,
+      "loss": 1.2047626972198486,
+      "step": 10753
+    },
+    {
+      "epoch": 0.9372047643493547,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.00043174866004780583,
+      "loss": 1.4221163988113403,
+      "step": 10754
+    },
+    {
+      "epoch": 0.9372919137602111,
+      "grad_norm": 0.55078125,
+      "learning_rate": 0.0004311365371438119,
+      "loss": 1.3433525562286377,
+      "step": 10755
+    },
+    {
+      "epoch": 0.9373790631710675,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00043052473021372694,
+      "loss": 1.3914741277694702,
+      "step": 10756
+    },
+    {
+      "epoch": 0.9374662125819239,
+      "grad_norm": 0.75390625,
+      "learning_rate": 0.00042991323959707864,
+      "loss": 1.4577502012252808,
+      "step": 10757
+    },
+    {
+      "epoch": 0.9375533619927802,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00042930206563321973,
+      "loss": 1.304754614830017,
+      "step": 10758
+    },
+    {
+      "epoch": 0.9376405114036366,
+      "grad_norm": 0.64453125,
+      "learning_rate": 0.00042869120866132734,
+      "loss": 1.356920599937439,
+      "step": 10759
+    },
+    {
+      "epoch": 0.937727660814493,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.00042808066902040267,
+      "loss": 1.5599650144577026,
+      "step": 10760
+    },
+    {
+      "epoch": 0.9378148102253493,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00042747044704926994,
+      "loss": 1.2664642333984375,
+      "step": 10761
+    },
+    {
+      "epoch": 0.9379019596362057,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0004268605430865794,
+      "loss": 1.4127326011657715,
+      "step": 10762
+    },
+    {
+      "epoch": 0.937989109047062,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.00042625095747080233,
+      "loss": 1.3692606687545776,
+      "step": 10763
+    },
+    {
+      "epoch": 0.9380762584579184,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00042564169054023473,
+      "loss": 1.3256995677947998,
+      "step": 10764
+    },
+    {
+      "epoch": 0.9381634078687747,
+      "grad_norm": 0.64453125,
+      "learning_rate": 0.0004250327426329951,
+      "loss": 1.527947187423706,
+      "step": 10765
+    },
+    {
+      "epoch": 0.9382505572796311,
+      "grad_norm": 0.69140625,
+      "learning_rate": 0.0004244241140870244,
+      "loss": 1.5718119144439697,
+      "step": 10766
+    },
+    {
+      "epoch": 0.9383377066904875,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00042381580524008893,
+      "loss": 1.3383100032806396,
+      "step": 10767
+    },
+    {
+      "epoch": 0.9384248561013439,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.00042320781642977436,
+      "loss": 1.3786511421203613,
+      "step": 10768
+    },
+    {
+      "epoch": 0.9385120055122003,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.0004226001479934904,
+      "loss": 1.4253110885620117,
+      "step": 10769
+    },
+    {
+      "epoch": 0.9385991549230566,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00042199280026846873,
+      "loss": 1.3506003618240356,
+      "step": 10770
+    },
+    {
+      "epoch": 0.938686304333913,
+      "grad_norm": 0.66015625,
+      "learning_rate": 0.0004213857735917617,
+      "loss": 1.3676663637161255,
+      "step": 10771
+    },
+    {
+      "epoch": 0.9387734537447693,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.0004207790683002473,
+      "loss": 1.4172930717468262,
+      "step": 10772
+    },
+    {
+      "epoch": 0.9388606031556257,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.00042017268473062166,
+      "loss": 1.3800415992736816,
+      "step": 10773
+    },
+    {
+      "epoch": 0.9389477525664821,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.0004195666232194029,
+      "loss": 1.3398977518081665,
+      "step": 10774
+    },
+    {
+      "epoch": 0.9390349019773384,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.000418960884102931,
+      "loss": 1.2846004962921143,
+      "step": 10775
+    },
+    {
+      "epoch": 0.9391220513881948,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.000418355467717367,
+      "loss": 1.348699688911438,
+      "step": 10776
+    },
+    {
+      "epoch": 0.9392092007990511,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.00041775037439869167,
+      "loss": 1.4276093244552612,
+      "step": 10777
+    },
+    {
+      "epoch": 0.9392963502099075,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.0004171456044827097,
+      "loss": 1.4915839433670044,
+      "step": 10778
+    },
+    {
+      "epoch": 0.9393834996207638,
+      "grad_norm": 0.73046875,
+      "learning_rate": 0.0004165411583050434,
+      "loss": 1.3433510065078735,
+      "step": 10779
+    },
+    {
+      "epoch": 0.9394706490316203,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.0004159370362011356,
+      "loss": 1.340151309967041,
+      "step": 10780
+    },
+    {
+      "epoch": 0.9395577984424767,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00041533323850625,
+      "loss": 1.4006013870239258,
+      "step": 10781
+    },
+    {
+      "epoch": 0.939644947853333,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00041472976555546957,
+      "loss": 1.4418479204177856,
+      "step": 10782
+    },
+    {
+      "epoch": 0.9397320972641894,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.000414126617683699,
+      "loss": 1.4882111549377441,
+      "step": 10783
+    },
+    {
+      "epoch": 0.9398192466750457,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.00041352379522566045,
+      "loss": 1.3524689674377441,
+      "step": 10784
+    },
+    {
+      "epoch": 0.9399063960859021,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.0004129212985158967,
+      "loss": 1.5079108476638794,
+      "step": 10785
+    },
+    {
+      "epoch": 0.9399935454967584,
+      "grad_norm": 0.69140625,
+      "learning_rate": 0.0004123191278887684,
+      "loss": 1.4028526544570923,
+      "step": 10786
+    },
+    {
+      "epoch": 0.9400806949076148,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.00041171728367845554,
+      "loss": 1.3713098764419556,
+      "step": 10787
+    },
+    {
+      "epoch": 0.9401678443184712,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0004111157662189597,
+      "loss": 1.4671008586883545,
+      "step": 10788
+    },
+    {
+      "epoch": 0.9402549937293275,
+      "grad_norm": 0.53515625,
+      "learning_rate": 0.00041051457584409767,
+      "loss": 1.1971697807312012,
+      "step": 10789
+    },
+    {
+      "epoch": 0.9403421431401839,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0004099137128875063,
+      "loss": 1.4762513637542725,
+      "step": 10790
+    },
+    {
+      "epoch": 0.9404292925510402,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.00040931317768264075,
+      "loss": 1.3404229879379272,
+      "step": 10791
+    },
+    {
+      "epoch": 0.9405164419618967,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0004087129705627731,
+      "loss": 1.441551923751831,
+      "step": 10792
+    },
+    {
+      "epoch": 0.940603591372753,
+      "grad_norm": 0.64453125,
+      "learning_rate": 0.00040811309186099373,
+      "loss": 1.3485558032989502,
+      "step": 10793
+    },
+    {
+      "epoch": 0.9406907407836094,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.00040751354191021334,
+      "loss": 1.330207347869873,
+      "step": 10794
+    },
+    {
+      "epoch": 0.9407778901944658,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00040691432104315715,
+      "loss": 1.345668911933899,
+      "step": 10795
+    },
+    {
+      "epoch": 0.9408650396053221,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0004063154295923686,
+      "loss": 1.6343204975128174,
+      "step": 10796
+    },
+    {
+      "epoch": 0.9409521890161785,
+      "grad_norm": 0.625,
+      "learning_rate": 0.00040571686789020884,
+      "loss": 1.3894102573394775,
+      "step": 10797
+    },
+    {
+      "epoch": 0.9410393384270348,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.00040511863626885433,
+      "loss": 1.4507195949554443,
+      "step": 10798
+    },
+    {
+      "epoch": 0.9411264878378912,
+      "grad_norm": 0.64453125,
+      "learning_rate": 0.00040452073506030116,
+      "loss": 1.2610223293304443,
+      "step": 10799
+    },
+    {
+      "epoch": 0.9412136372487475,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.00040392316459636013,
+      "loss": 1.2862776517868042,
+      "step": 10800
+    },
+    {
+      "epoch": 0.9413007866596039,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0004033259252086588,
+      "loss": 1.297368049621582,
+      "step": 10801
+    },
+    {
+      "epoch": 0.9413879360704603,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0004027290172286411,
+      "loss": 1.4660052061080933,
+      "step": 10802
+    },
+    {
+      "epoch": 0.9414750854813166,
+      "grad_norm": 0.67578125,
+      "learning_rate": 0.00040213244098756616,
+      "loss": 1.566052794456482,
+      "step": 10803
+    },
+    {
+      "epoch": 0.9415622348921731,
+      "grad_norm": 0.53515625,
+      "learning_rate": 0.0004015361968165116,
+      "loss": 1.2307544946670532,
+      "step": 10804
+    },
+    {
+      "epoch": 0.9416493843030294,
+      "grad_norm": 0.67578125,
+      "learning_rate": 0.0004009402850463682,
+      "loss": 1.464665174484253,
+      "step": 10805
+    },
+    {
+      "epoch": 0.9417365337138858,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.000400344706007843,
+      "loss": 1.415521502494812,
+      "step": 10806
+    },
+    {
+      "epoch": 0.9418236831247421,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0003997494600314587,
+      "loss": 1.4694634675979614,
+      "step": 10807
+    },
+    {
+      "epoch": 0.9419108325355985,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.00039915454744755276,
+      "loss": 1.4592629671096802,
+      "step": 10808
+    },
+    {
+      "epoch": 0.9419979819464549,
+      "grad_norm": 0.5234375,
+      "learning_rate": 0.00039855996858627736,
+      "loss": 1.32602858543396,
+      "step": 10809
+    },
+    {
+      "epoch": 0.9420851313573112,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.00039796572377760124,
+      "loss": 1.364354133605957,
+      "step": 10810
+    },
+    {
+      "epoch": 0.9421722807681676,
+      "grad_norm": 0.490234375,
+      "learning_rate": 0.00039737181335130567,
+      "loss": 1.228593111038208,
+      "step": 10811
+    },
+    {
+      "epoch": 0.9422594301790239,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0003967782376369872,
+      "loss": 1.4669352769851685,
+      "step": 10812
+    },
+    {
+      "epoch": 0.9423465795898803,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.00039618499696405667,
+      "loss": 1.3811349868774414,
+      "step": 10813
+    },
+    {
+      "epoch": 0.9424337290007366,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.0003955920916617385,
+      "loss": 1.3459770679473877,
+      "step": 10814
+    },
+    {
+      "epoch": 0.942520878411593,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0003949995220590719,
+      "loss": 1.3409168720245361,
+      "step": 10815
+    },
+    {
+      "epoch": 0.9426080278224495,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.000394407288484909,
+      "loss": 1.4049094915390015,
+      "step": 10816
+    },
+    {
+      "epoch": 0.9426951772333058,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.00039381539126791574,
+      "loss": 1.405827283859253,
+      "step": 10817
+    },
+    {
+      "epoch": 0.9427823266441622,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.00039322383073657144,
+      "loss": 1.349738597869873,
+      "step": 10818
+    },
+    {
+      "epoch": 0.9428694760550185,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0003926326072191674,
+      "loss": 1.3446109294891357,
+      "step": 10819
+    },
+    {
+      "epoch": 0.9429566254658749,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00039204172104381134,
+      "loss": 1.1995892524719238,
+      "step": 10820
+    },
+    {
+      "epoch": 0.9430437748767312,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.00039145117253841955,
+      "loss": 1.3476479053497314,
+      "step": 10821
+    },
+    {
+      "epoch": 0.9431309242875876,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.00039086096203072346,
+      "loss": 1.3362534046173096,
+      "step": 10822
+    },
+    {
+      "epoch": 0.943218073698444,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.0003902710898482662,
+      "loss": 1.3388662338256836,
+      "step": 10823
+    },
+    {
+      "epoch": 0.9433052231093003,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.00038968155631840276,
+      "loss": 1.3150140047073364,
+      "step": 10824
+    },
+    {
+      "epoch": 0.9433923725201567,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00038909236176830044,
+      "loss": 1.4378329515457153,
+      "step": 10825
+    },
+    {
+      "epoch": 0.943479521931013,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.00038850350652493983,
+      "loss": 1.2804172039031982,
+      "step": 10826
+    },
+    {
+      "epoch": 0.9435666713418694,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.00038791499091511125,
+      "loss": 1.577147126197815,
+      "step": 10827
+    },
+    {
+      "epoch": 0.9436538207527257,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.00038732681526541736,
+      "loss": 1.384181022644043,
+      "step": 10828
+    },
+    {
+      "epoch": 0.9437409701635822,
+      "grad_norm": 0.67578125,
+      "learning_rate": 0.00038673897990227203,
+      "loss": 1.451761245727539,
+      "step": 10829
+    },
+    {
+      "epoch": 0.9438281195744386,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0003861514851518996,
+      "loss": 1.3510509729385376,
+      "step": 10830
+    },
+    {
+      "epoch": 0.9439152689852949,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00038556433134033757,
+      "loss": 1.2669039964675903,
+      "step": 10831
+    },
+    {
+      "epoch": 0.9440024183961513,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0003849775187934324,
+      "loss": 1.3124220371246338,
+      "step": 10832
+    },
+    {
+      "epoch": 0.9440895678070076,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.00038439104783684087,
+      "loss": 1.3466203212738037,
+      "step": 10833
+    },
+    {
+      "epoch": 0.944176717217864,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0003838049187960312,
+      "loss": 1.2957828044891357,
+      "step": 10834
+    },
+    {
+      "epoch": 0.9442638666287203,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.0003832191319962804,
+      "loss": 1.3593542575836182,
+      "step": 10835
+    },
+    {
+      "epoch": 0.9443510160395767,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.0003826336877626789,
+      "loss": 1.2994202375411987,
+      "step": 10836
+    },
+    {
+      "epoch": 0.9444381654504331,
+      "grad_norm": 0.703125,
+      "learning_rate": 0.0003820485864201238,
+      "loss": 1.4568206071853638,
+      "step": 10837
+    },
+    {
+      "epoch": 0.9445253148612894,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.00038146382829332294,
+      "loss": 1.4523032903671265,
+      "step": 10838
+    },
+    {
+      "epoch": 0.9446124642721458,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.00038087941370679325,
+      "loss": 1.2172088623046875,
+      "step": 10839
+    },
+    {
+      "epoch": 0.9446996136830021,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0003802953429848616,
+      "loss": 1.3055803775787354,
+      "step": 10840
+    },
+    {
+      "epoch": 0.9447867630938586,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.00037971161645166323,
+      "loss": 1.4756884574890137,
+      "step": 10841
+    },
+    {
+      "epoch": 0.944873912504715,
+      "grad_norm": 0.67578125,
+      "learning_rate": 0.0003791282344311443,
+      "loss": 1.4392821788787842,
+      "step": 10842
+    },
+    {
+      "epoch": 0.9449610619155713,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.0003785451972470578,
+      "loss": 1.3131775856018066,
+      "step": 10843
+    },
+    {
+      "epoch": 0.9450482113264277,
+      "grad_norm": 0.625,
+      "learning_rate": 0.000377962505222966,
+      "loss": 1.3837449550628662,
+      "step": 10844
+    },
+    {
+      "epoch": 0.945135360737284,
+      "grad_norm": 0.53125,
+      "learning_rate": 0.0003773801586822397,
+      "loss": 1.3735487461090088,
+      "step": 10845
+    },
+    {
+      "epoch": 0.9452225101481404,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.00037679815794805705,
+      "loss": 1.4723949432373047,
+      "step": 10846
+    },
+    {
+      "epoch": 0.9453096595589967,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0003762165033434066,
+      "loss": 1.3084514141082764,
+      "step": 10847
+    },
+    {
+      "epoch": 0.9453968089698531,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0003756351951910827,
+      "loss": 1.396877408027649,
+      "step": 10848
+    },
+    {
+      "epoch": 0.9454839583807095,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00037505423381368797,
+      "loss": 1.3752583265304565,
+      "step": 10849
+    },
+    {
+      "epoch": 0.9455711077915658,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.00037447361953363317,
+      "loss": 1.2257283926010132,
+      "step": 10850
+    },
+    {
+      "epoch": 0.9456582572024222,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.00037389335267313417,
+      "loss": 1.363704800605774,
+      "step": 10851
+    },
+    {
+      "epoch": 0.9457454066132785,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.00037331343355421825,
+      "loss": 1.3016622066497803,
+      "step": 10852
+    },
+    {
+      "epoch": 0.945832556024135,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0003727338624987162,
+      "loss": 1.2846336364746094,
+      "step": 10853
+    },
+    {
+      "epoch": 0.9459197054349913,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0003721546398282669,
+      "loss": 1.4935826063156128,
+      "step": 10854
+    },
+    {
+      "epoch": 0.9460068548458477,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.00037157576586431545,
+      "loss": 1.3338427543640137,
+      "step": 10855
+    },
+    {
+      "epoch": 0.9460940042567041,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.00037099724092811434,
+      "loss": 1.359696865081787,
+      "step": 10856
+    },
+    {
+      "epoch": 0.9461811536675604,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.00037041906534072017,
+      "loss": 1.196385383605957,
+      "step": 10857
+    },
+    {
+      "epoch": 0.9462683030784168,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.000369841239422999,
+      "loss": 1.3450770378112793,
+      "step": 10858
+    },
+    {
+      "epoch": 0.9463554524892731,
+      "grad_norm": 0.51953125,
+      "learning_rate": 0.00036926376349562034,
+      "loss": 1.392338514328003,
+      "step": 10859
+    },
+    {
+      "epoch": 0.9464426019001295,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0003686866378790603,
+      "loss": 1.3446500301361084,
+      "step": 10860
+    },
+    {
+      "epoch": 0.9465297513109858,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00036810986289360025,
+      "loss": 1.4945433139801025,
+      "step": 10861
+    },
+    {
+      "epoch": 0.9466169007218422,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0003675334388593267,
+      "loss": 1.2035317420959473,
+      "step": 10862
+    },
+    {
+      "epoch": 0.9467040501326986,
+      "grad_norm": 0.53515625,
+      "learning_rate": 0.00036695736609613323,
+      "loss": 1.3052306175231934,
+      "step": 10863
+    },
+    {
+      "epoch": 0.946791199543555,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.00036638164492371647,
+      "loss": 1.3907655477523804,
+      "step": 10864
+    },
+    {
+      "epoch": 0.9468783489544114,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0003658062756615786,
+      "loss": 1.3140581846237183,
+      "step": 10865
+    },
+    {
+      "epoch": 0.9469654983652677,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00036523125862902644,
+      "loss": 1.3715146780014038,
+      "step": 10866
+    },
+    {
+      "epoch": 0.9470526477761241,
+      "grad_norm": 0.68359375,
+      "learning_rate": 0.00036465659414517066,
+      "loss": 1.2546745538711548,
+      "step": 10867
+    },
+    {
+      "epoch": 0.9471397971869804,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0003640822825289286,
+      "loss": 1.4042918682098389,
+      "step": 10868
+    },
+    {
+      "epoch": 0.9472269465978368,
+      "grad_norm": 0.6640625,
+      "learning_rate": 0.00036350832409901934,
+      "loss": 1.4622188806533813,
+      "step": 10869
+    },
+    {
+      "epoch": 0.9473140960086932,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00036293471917396694,
+      "loss": 1.4352420568466187,
+      "step": 10870
+    },
+    {
+      "epoch": 0.9474012454195495,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0003623614680720991,
+      "loss": 1.425258994102478,
+      "step": 10871
+    },
+    {
+      "epoch": 0.9474883948304059,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0003617885711115471,
+      "loss": 1.524860143661499,
+      "step": 10872
+    },
+    {
+      "epoch": 0.9475755442412622,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.0003612160286102456,
+      "loss": 1.398781180381775,
+      "step": 10873
+    },
+    {
+      "epoch": 0.9476626936521186,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0003606438408859333,
+      "loss": 1.2950276136398315,
+      "step": 10874
+    },
+    {
+      "epoch": 0.9477498430629749,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00036007200825615126,
+      "loss": 1.3413006067276,
+      "step": 10875
+    },
+    {
+      "epoch": 0.9478369924738314,
+      "grad_norm": 0.62109375,
+      "learning_rate": 0.0003595005310382437,
+      "loss": 1.5986438989639282,
+      "step": 10876
+    },
+    {
+      "epoch": 0.9479241418846878,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.00035892940954935734,
+      "loss": 1.3787851333618164,
+      "step": 10877
+    },
+    {
+      "epoch": 0.9480112912955441,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00035835864410644095,
+      "loss": 1.3755840063095093,
+      "step": 10878
+    },
+    {
+      "epoch": 0.9480984407064005,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.0003577882350262488,
+      "loss": 1.5901002883911133,
+      "step": 10879
+    },
+    {
+      "epoch": 0.9481855901172568,
+      "grad_norm": 0.66015625,
+      "learning_rate": 0.0003572181826253332,
+      "loss": 1.480640172958374,
+      "step": 10880
+    },
+    {
+      "epoch": 0.9482727395281132,
+      "grad_norm": 0.5234375,
+      "learning_rate": 0.0003566484872200509,
+      "loss": 1.4293866157531738,
+      "step": 10881
+    },
+    {
+      "epoch": 0.9483598889389695,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00035607914912656014,
+      "loss": 1.3817474842071533,
+      "step": 10882
+    },
+    {
+      "epoch": 0.9484470383498259,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0003555101686608205,
+      "loss": 1.5607452392578125,
+      "step": 10883
+    },
+    {
+      "epoch": 0.9485341877606823,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.0003549415461385927,
+      "loss": 1.4303282499313354,
+      "step": 10884
+    },
+    {
+      "epoch": 0.9486213371715386,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.000354373281875441,
+      "loss": 1.373576283454895,
+      "step": 10885
+    },
+    {
+      "epoch": 0.948708486582395,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.0003538053761867284,
+      "loss": 1.4354960918426514,
+      "step": 10886
+    },
+    {
+      "epoch": 0.9487956359932513,
+      "grad_norm": 0.73828125,
+      "learning_rate": 0.00035323782938762,
+      "loss": 1.5275927782058716,
+      "step": 10887
+    },
+    {
+      "epoch": 0.9488827854041078,
+      "grad_norm": 0.68359375,
+      "learning_rate": 0.0003526706417930814,
+      "loss": 1.4088103771209717,
+      "step": 10888
+    },
+    {
+      "epoch": 0.9489699348149641,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.0003521038137178784,
+      "loss": 1.4045789241790771,
+      "step": 10889
+    },
+    {
+      "epoch": 0.9490570842258205,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.00035153734547657895,
+      "loss": 1.3758931159973145,
+      "step": 10890
+    },
+    {
+      "epoch": 0.9491442336366769,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.0003509712373835503,
+      "loss": 1.4020464420318604,
+      "step": 10891
+    },
+    {
+      "epoch": 0.9492313830475332,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.00035040548975295887,
+      "loss": 1.4836962223052979,
+      "step": 10892
+    },
+    {
+      "epoch": 0.9493185324583896,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.0003498401028987722,
+      "loss": 1.1181589365005493,
+      "step": 10893
+    },
+    {
+      "epoch": 0.9494056818692459,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.00034927507713475645,
+      "loss": 1.2913410663604736,
+      "step": 10894
+    },
+    {
+      "epoch": 0.9494928312801023,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.0003487104127744803,
+      "loss": 1.4168663024902344,
+      "step": 10895
+    },
+    {
+      "epoch": 0.9495799806909586,
+      "grad_norm": 0.625,
+      "learning_rate": 0.0003481461101313087,
+      "loss": 1.3656679391860962,
+      "step": 10896
+    },
+    {
+      "epoch": 0.949667130101815,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0003475821695184075,
+      "loss": 1.3501274585723877,
+      "step": 10897
+    },
+    {
+      "epoch": 0.9497542795126714,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0003470185912487404,
+      "loss": 1.3929165601730347,
+      "step": 10898
+    },
+    {
+      "epoch": 0.9498414289235277,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.0003464553756350711,
+      "loss": 1.3516440391540527,
+      "step": 10899
+    },
+    {
+      "epoch": 0.9499285783343842,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00034589252298996106,
+      "loss": 1.3244688510894775,
+      "step": 10900
+    },
+    {
+      "epoch": 0.9500157277452405,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0003453300336257722,
+      "loss": 1.443703055381775,
+      "step": 10901
+    },
+    {
+      "epoch": 0.9501028771560969,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.000344767907854663,
+      "loss": 1.4516948461532593,
+      "step": 10902
+    },
+    {
+      "epoch": 0.9501900265669532,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0003442061459885907,
+      "loss": 1.304955005645752,
+      "step": 10903
+    },
+    {
+      "epoch": 0.9502771759778096,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.00034364474833931065,
+      "loss": 1.4244271516799927,
+      "step": 10904
+    },
+    {
+      "epoch": 0.950364325388666,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.00034308371521837535,
+      "loss": 1.331715703010559,
+      "step": 10905
+    },
+    {
+      "epoch": 0.9504514747995223,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.00034252304693713697,
+      "loss": 1.3917310237884521,
+      "step": 10906
+    },
+    {
+      "epoch": 0.9505386242103787,
+      "grad_norm": 0.6328125,
+      "learning_rate": 0.0003419627438067435,
+      "loss": 1.3722879886627197,
+      "step": 10907
+    },
+    {
+      "epoch": 0.950625773621235,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.00034140280613814054,
+      "loss": 1.4091254472732544,
+      "step": 10908
+    },
+    {
+      "epoch": 0.9507129230320914,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0003408432342420714,
+      "loss": 1.4888665676116943,
+      "step": 10909
+    },
+    {
+      "epoch": 0.9508000724429477,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00034028402842907436,
+      "loss": 1.4189926385879517,
+      "step": 10910
+    },
+    {
+      "epoch": 0.9508872218538041,
+      "grad_norm": 0.55078125,
+      "learning_rate": 0.00033972518900948876,
+      "loss": 1.3841023445129395,
+      "step": 10911
+    },
+    {
+      "epoch": 0.9509743712646606,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.00033916671629344693,
+      "loss": 1.188202142715454,
+      "step": 10912
+    },
+    {
+      "epoch": 0.9510615206755169,
+      "grad_norm": 0.68359375,
+      "learning_rate": 0.00033860861059087897,
+      "loss": 1.5412949323654175,
+      "step": 10913
+    },
+    {
+      "epoch": 0.9511486700863733,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.000338050872211511,
+      "loss": 1.497607946395874,
+      "step": 10914
+    },
+    {
+      "epoch": 0.9512358194972296,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0003374935014648659,
+      "loss": 1.2207014560699463,
+      "step": 10915
+    },
+    {
+      "epoch": 0.951322968908086,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0003369364986602608,
+      "loss": 1.3597630262374878,
+      "step": 10916
+    },
+    {
+      "epoch": 0.9514101183189423,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0003363798641068116,
+      "loss": 1.4100755453109741,
+      "step": 10917
+    },
+    {
+      "epoch": 0.9514972677297987,
+      "grad_norm": 0.5390625,
+      "learning_rate": 0.00033582359811342745,
+      "loss": 1.373608946800232,
+      "step": 10918
+    },
+    {
+      "epoch": 0.9515844171406551,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00033526770098881375,
+      "loss": 1.3255009651184082,
+      "step": 10919
+    },
+    {
+      "epoch": 0.9516715665515114,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.00033471217304147116,
+      "loss": 1.368410348892212,
+      "step": 10920
+    },
+    {
+      "epoch": 0.9517587159623678,
+      "grad_norm": 0.5625,
+      "learning_rate": 0.0003341570145796947,
+      "loss": 1.2507421970367432,
+      "step": 10921
+    },
+    {
+      "epoch": 0.9518458653732241,
+      "grad_norm": 0.6875,
+      "learning_rate": 0.00033360222591157635,
+      "loss": 1.4534509181976318,
+      "step": 10922
+    },
+    {
+      "epoch": 0.9519330147840805,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.000333047807345001,
+      "loss": 1.3151963949203491,
+      "step": 10923
+    },
+    {
+      "epoch": 0.9520201641949368,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.0003324937591876487,
+      "loss": 1.3997173309326172,
+      "step": 10924
+    },
+    {
+      "epoch": 0.9521073136057933,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00033194008174699407,
+      "loss": 1.302254557609558,
+      "step": 10925
+    },
+    {
+      "epoch": 0.9521944630166497,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.0003313867753303052,
+      "loss": 1.2490872144699097,
+      "step": 10926
+    },
+    {
+      "epoch": 0.952281612427506,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.00033083384024464615,
+      "loss": 1.436303734779358,
+      "step": 10927
+    },
+    {
+      "epoch": 0.9523687618383624,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.00033028127679687343,
+      "loss": 1.3244483470916748,
+      "step": 10928
+    },
+    {
+      "epoch": 0.9524559112492187,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.00032972908529363724,
+      "loss": 1.3695107698440552,
+      "step": 10929
+    },
+    {
+      "epoch": 0.9525430606600751,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0003291772660413819,
+      "loss": 1.2084953784942627,
+      "step": 10930
+    },
+    {
+      "epoch": 0.9526302100709314,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.000328625819346345,
+      "loss": 1.2497600317001343,
+      "step": 10931
+    },
+    {
+      "epoch": 0.9527173594817878,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.000328074745514557,
+      "loss": 1.3972818851470947,
+      "step": 10932
+    },
+    {
+      "epoch": 0.9528045088926442,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.0003275240448518425,
+      "loss": 1.4092837572097778,
+      "step": 10933
+    },
+    {
+      "epoch": 0.9528916583035005,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00032697371766381785,
+      "loss": 1.4201757907867432,
+      "step": 10934
+    },
+    {
+      "epoch": 0.952978807714357,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.00032642376425589273,
+      "loss": 1.4522488117218018,
+      "step": 10935
+    },
+    {
+      "epoch": 0.9530659571252132,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0003258741849332692,
+      "loss": 1.4301923513412476,
+      "step": 10936
+    },
+    {
+      "epoch": 0.9531531065360697,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0003253249800009408,
+      "loss": 1.2767215967178345,
+      "step": 10937
+    },
+    {
+      "epoch": 0.953240255946926,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.00032477614976369643,
+      "loss": 1.2616369724273682,
+      "step": 10938
+    },
+    {
+      "epoch": 0.9533274053577824,
+      "grad_norm": 0.68359375,
+      "learning_rate": 0.00032422769452611344,
+      "loss": 1.4377288818359375,
+      "step": 10939
+    },
+    {
+      "epoch": 0.9534145547686388,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0003236796145925626,
+      "loss": 1.4595770835876465,
+      "step": 10940
+    },
+    {
+      "epoch": 0.9535017041794951,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.0003231319102672066,
+      "loss": 1.4700055122375488,
+      "step": 10941
+    },
+    {
+      "epoch": 0.9535888535903515,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00032258458185399847,
+      "loss": 1.3998115062713623,
+      "step": 10942
+    },
+    {
+      "epoch": 0.9536760030012078,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.000322037629656685,
+      "loss": 1.4860610961914062,
+      "step": 10943
+    },
+    {
+      "epoch": 0.9537631524120642,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00032149105397880214,
+      "loss": 1.2388815879821777,
+      "step": 10944
+    },
+    {
+      "epoch": 0.9538503018229205,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.0003209448551236773,
+      "loss": 1.4937905073165894,
+      "step": 10945
+    },
+    {
+      "epoch": 0.9539374512337769,
+      "grad_norm": 0.6796875,
+      "learning_rate": 0.0003203990333944291,
+      "loss": 1.3029348850250244,
+      "step": 10946
+    },
+    {
+      "epoch": 0.9540246006446333,
+      "grad_norm": 0.55078125,
+      "learning_rate": 0.0003198535890939668,
+      "loss": 1.4672210216522217,
+      "step": 10947
+    },
+    {
+      "epoch": 0.9541117500554896,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00031930852252498934,
+      "loss": 1.2499611377716064,
+      "step": 10948
+    },
+    {
+      "epoch": 0.9541988994663461,
+      "grad_norm": 0.5546875,
+      "learning_rate": 0.00031876383398998814,
+      "loss": 1.3740532398223877,
+      "step": 10949
+    },
+    {
+      "epoch": 0.9542860488772024,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.00031821952379124325,
+      "loss": 1.4285303354263306,
+      "step": 10950
+    },
+    {
+      "epoch": 0.9543731982880588,
+      "grad_norm": 0.625,
+      "learning_rate": 0.00031767559223082434,
+      "loss": 1.3995391130447388,
+      "step": 10951
+    },
+    {
+      "epoch": 0.9544603476989151,
+      "grad_norm": 0.64453125,
+      "learning_rate": 0.0003171320396105919,
+      "loss": 1.4862656593322754,
+      "step": 10952
+    },
+    {
+      "epoch": 0.9545474971097715,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00031658886623219494,
+      "loss": 1.448495626449585,
+      "step": 10953
+    },
+    {
+      "epoch": 0.9546346465206279,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00031604607239707494,
+      "loss": 1.4339920282363892,
+      "step": 10954
+    },
+    {
+      "epoch": 0.9547217959314842,
+      "grad_norm": 0.65625,
+      "learning_rate": 0.00031550365840645963,
+      "loss": 1.4500888586044312,
+      "step": 10955
+    },
+    {
+      "epoch": 0.9548089453423406,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.00031496162456136746,
+      "loss": 1.4123953580856323,
+      "step": 10956
+    },
+    {
+      "epoch": 0.9548960947531969,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0003144199711626048,
+      "loss": 1.453245997428894,
+      "step": 10957
+    },
+    {
+      "epoch": 0.9549832441640533,
+      "grad_norm": 0.546875,
+      "learning_rate": 0.00031387869851076726,
+      "loss": 1.387932300567627,
+      "step": 10958
+    },
+    {
+      "epoch": 0.9550703935749096,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0003133378069062407,
+      "loss": 1.4692347049713135,
+      "step": 10959
+    },
+    {
+      "epoch": 0.955157542985766,
+      "grad_norm": 0.55859375,
+      "learning_rate": 0.0003127972966491974,
+      "loss": 1.3816195726394653,
+      "step": 10960
+    },
+    {
+      "epoch": 0.9552446923966225,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.0003122571680395997,
+      "loss": 1.3278354406356812,
+      "step": 10961
+    },
+    {
+      "epoch": 0.9553318418074788,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.00031171742137719695,
+      "loss": 1.4659311771392822,
+      "step": 10962
+    },
+    {
+      "epoch": 0.9554189912183352,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.00031117805696152665,
+      "loss": 1.4021224975585938,
+      "step": 10963
+    },
+    {
+      "epoch": 0.9555061406291915,
+      "grad_norm": 0.66015625,
+      "learning_rate": 0.00031063907509191384,
+      "loss": 1.3670710325241089,
+      "step": 10964
+    },
+    {
+      "epoch": 0.9555932900400479,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.00031010047606747337,
+      "loss": 1.3548510074615479,
+      "step": 10965
+    },
+    {
+      "epoch": 0.9556804394509042,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.0003095622601871053,
+      "loss": 1.4097130298614502,
+      "step": 10966
+    },
+    {
+      "epoch": 0.9557675888617606,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.0003090244277494979,
+      "loss": 1.4018460512161255,
+      "step": 10967
+    },
+    {
+      "epoch": 0.955854738272617,
+      "grad_norm": 0.640625,
+      "learning_rate": 0.0003084869790531271,
+      "loss": 1.3154217004776,
+      "step": 10968
+    },
+    {
+      "epoch": 0.9559418876834733,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0003079499143962536,
+      "loss": 1.2899959087371826,
+      "step": 10969
+    },
+    {
+      "epoch": 0.9560290370943297,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.00030741323407692926,
+      "loss": 1.2671692371368408,
+      "step": 10970
+    },
+    {
+      "epoch": 0.956116186505186,
+      "grad_norm": 0.6640625,
+      "learning_rate": 0.00030687693839298884,
+      "loss": 1.4018245935440063,
+      "step": 10971
+    },
+    {
+      "epoch": 0.9562033359160425,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00030634102764205516,
+      "loss": 1.451886773109436,
+      "step": 10972
+    },
+    {
+      "epoch": 0.9562904853268988,
+      "grad_norm": 0.5703125,
+      "learning_rate": 0.00030580550212153725,
+      "loss": 1.4861435890197754,
+      "step": 10973
+    },
+    {
+      "epoch": 0.9563776347377552,
+      "grad_norm": 0.60546875,
+      "learning_rate": 0.00030527036212863,
+      "loss": 1.2825517654418945,
+      "step": 10974
+    },
+    {
+      "epoch": 0.9564647841486116,
+      "grad_norm": 0.59375,
+      "learning_rate": 0.0003047356079603151,
+      "loss": 1.3305561542510986,
+      "step": 10975
+    },
+    {
+      "epoch": 0.9565519335594679,
+      "grad_norm": 0.54296875,
+      "learning_rate": 0.0003042012399133593,
+      "loss": 1.2840900421142578,
+      "step": 10976
+    },
+    {
+      "epoch": 0.9566390829703243,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.0003036672582843159,
+      "loss": 1.4011597633361816,
+      "step": 10977
+    },
+    {
+      "epoch": 0.9567262323811806,
+      "grad_norm": 0.66015625,
+      "learning_rate": 0.00030313366336952313,
+      "loss": 1.424378514289856,
+      "step": 10978
+    },
+    {
+      "epoch": 0.956813381792037,
+      "grad_norm": 0.57421875,
+      "learning_rate": 0.0003026004554651047,
+      "loss": 1.375205159187317,
+      "step": 10979
+    },
+    {
+      "epoch": 0.9569005312028933,
+      "grad_norm": 0.7109375,
+      "learning_rate": 0.0003020676348669693,
+      "loss": 1.6048765182495117,
+      "step": 10980
+    },
+    {
+      "epoch": 0.9569876806137497,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.0003015352018708118,
+      "loss": 1.4007132053375244,
+      "step": 10981
+    },
+    {
+      "epoch": 0.9570748300246061,
+      "grad_norm": 0.61328125,
+      "learning_rate": 0.0003010031567721107,
+      "loss": 1.3569376468658447,
+      "step": 10982
+    },
+    {
+      "epoch": 0.9571619794354624,
+      "grad_norm": 0.65234375,
+      "learning_rate": 0.00030047149986612984,
+      "loss": 1.3657957315444946,
+      "step": 10983
+    },
+    {
+      "epoch": 0.9572491288463189,
+      "grad_norm": 0.609375,
+      "learning_rate": 0.0002999402314479173,
+      "loss": 1.2507611513137817,
+      "step": 10984
+    },
+    {
+      "epoch": 0.9573362782571752,
+      "grad_norm": 0.66015625,
+      "learning_rate": 0.00029940935181230535,
+      "loss": 1.4524526596069336,
+      "step": 10985
+    },
+    {
+      "epoch": 0.9574234276680316,
+      "grad_norm": 0.67578125,
+      "learning_rate": 0.000298878861253912,
+      "loss": 1.3916499614715576,
+      "step": 10986
+    },
+    {
+      "epoch": 0.9575105770788879,
+      "grad_norm": 0.6484375,
+      "learning_rate": 0.0002983487600671378,
+      "loss": 1.4126540422439575,
+      "step": 10987
+    },
+    {
+      "epoch": 0.9575977264897443,
+      "grad_norm": 0.62890625,
+      "learning_rate": 0.00029781904854616747,
+      "loss": 1.3251489400863647,
+      "step": 10988
+    },
+    {
+      "epoch": 0.9576848759006007,
+      "grad_norm": 0.578125,
+      "learning_rate": 0.00029728972698496995,
+      "loss": 1.2860187292099,
+      "step": 10989
+    },
+    {
+      "epoch": 0.957772025311457,
+      "grad_norm": 0.58203125,
+      "learning_rate": 0.0002967607956772966,
+      "loss": 1.2345279455184937,
+      "step": 10990
+    },
+    {
+      "epoch": 0.9578591747223134,
+      "grad_norm": 0.63671875,
+      "learning_rate": 0.00029623225491668477,
+      "loss": 1.4058338403701782,
+      "step": 10991
+    },
+    {
+      "epoch": 0.9579463241331697,
+      "grad_norm": 0.58984375,
+      "learning_rate": 0.0002957041049964522,
+      "loss": 1.3918859958648682,
+      "step": 10992
+    },
+    {
+      "epoch": 0.9580334735440261,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00029517634620970085,
+      "loss": 1.2986831665039062,
+      "step": 10993
+    },
+    {
+      "epoch": 0.9581206229548824,
+      "grad_norm": 0.66796875,
+      "learning_rate": 0.0002946489788493164,
+      "loss": 1.532016396522522,
+      "step": 10994
+    },
+    {
+      "epoch": 0.9582077723657388,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00029412200320796624,
+      "loss": 1.5022664070129395,
+      "step": 10995
+    },
+    {
+      "epoch": 0.9582949217765953,
+      "grad_norm": 0.6015625,
+      "learning_rate": 0.00029359541957810014,
+      "loss": 1.206302285194397,
+      "step": 10996
+    },
+    {
+      "epoch": 0.9583820711874516,
+      "grad_norm": 0.59765625,
+      "learning_rate": 0.0002930692282519525,
+      "loss": 1.3716161251068115,
+      "step": 10997
+    },
+    {
+      "epoch": 0.958469220598308,
+      "grad_norm": 0.56640625,
+      "learning_rate": 0.000292543429521537,
+      "loss": 1.2581470012664795,
+      "step": 10998
+    },
+    {
+      "epoch": 0.9585563700091643,
+      "grad_norm": 0.6171875,
+      "learning_rate": 0.0002920180236786515,
+      "loss": 1.293797254562378,
+      "step": 10999
+    },
+    {
+      "epoch": 0.9586435194200207,
+      "grad_norm": 0.5859375,
+      "learning_rate": 0.0002914930110148749,
+      "loss": 1.3799270391464233,
+      "step": 11000
     }
   ],
   "logging_steps": 1,
       "attributes": {}
     }
   },
+  "total_flos": 4.067990294700232e+18,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null