Instructions to use deepseek-ai/DeepSeek-V4.1-Flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V4.1-Flash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="deepseek-ai/DeepSeek-V4.1-Flash")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4.1-Flash", device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V4.1-Flash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V4.1-Flash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash

SGLang

How to use deepseek-ai/DeepSeek-V4.1-Flash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V4.1-Flash with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash
```

Running on 4x RTX PRO 6000 with NVMe offload for ngram

#28

by 0xSero - opened Sep 10

Discussion

0xSero

Sep 10

Hey, here’s a working GitHub, enjoy:

https://github.com/0xSero/deepseek-v4.1-flash-4x-rtx-pro-6000

dejan

Sep 10

Any idea what's the speed difference vs RAM offload? RDIMM is pricey, have just 128GB.

WiSFoR

Sep 11

•

edited Sep 11

kept failing on tool caliing benchmarks. disabling dspark solved it.
thanks for the super fast implementation 😃

aaron-newsome

Sep 11

i'm currently running glm-5.3 flash on the 4x RTX Pro 6000 rig and before that ...

GLM-5.2-NVFP4 hybrid and before that ...
deepseek v4 flash preview and before that ...
MiniMax M3 NVFP4 and before that ...
MiniMax 2.7 ??

at each point in the rig evolution for 1 to 2, to 3 and finally 4 GPU, I've always run the best model that will fit. glm-5.3-flash doesn't feel better than 5.2 to me, but damn it's so much faster, multi-modal and can handle way more concurrent agents, it's hard to go back to 5.2.

i say this because THIS model, the Deepseek v4.1 Flash beats GLM-5.3-Flash on ALL but two benchmarks. According to the benchmarks, this model should CRUSH GLM 5.3 Flash. I'm about to find out if that's true or not this weekend. I have a fresh SPEC and full 20 phase implementation plan written for an iPad app I was about to let GLM 5.3 Flash slay, but now it's going to be THIS model putting in work.

all of the talk of benchmaxxing is being put to the test on a real project and not a flappy bird clone or some lame one-shot game.

On a side note, why is every single AI YouTuber obsessed with how models perform in a string of one-shot tests. No one develops software as one-shot stunts.

aaron-newsome

Sep 12

•

edited Sep 12

I ran it to handle a few dozen tickets and I'll sum up the experience by saying I've gone back to GLM-5.3-Flash.

Thanks to the Deepseek team for releasing the weights. Super interesting architecture and holy crap is this thing fast. I ran two configs, one with Engram on NVMe and one with Engram pinned in system DRAM. I honestly expected the Engram from RAM to be much faster but it was only slightly faster, most noticeable the more simultaneous sessions there are.

I got caught up in the hype of the awesome benchmarks, but I should have listened to the people I trust who said this thing is benchmaxxed. I'll have to agree with them.

FredDumartin

Sep 12

•

edited Sep 12

So they put the "benchmaxx recipes" in the Engram weights… lame. 😅

I'll compare it against our production setup (v4-flash-vision-exp on 2x H200) for our agents, workflows, and tools. We're really happy with it — it's smart enough, and we get 400 tok/s (c1) down to 175 tok/s (c16) on the H200s.

We also have a 2x 6000 Pro box running various smaller models for the rest of the stack (Qwen-3.8-27b, embed+retrieval, OCR, STT+TTS, …). I was thinking of upgrading it with an additional 2x 6000 to run v4.1 or 5.3-flash.

Aaron, did you try Qwen-3.8-flash-next?

aaron-newsome

Sep 12

I've not yet tried the Qwen-3.8-flash-next but I hear great things about it. I did try the previous deepseek-v4, both the preview and the 0731 or whatever the number was. I thought they were both fantastic and got work done, but I still preferred the GLM 5.2 and then 5.3 flash for multi-modal and I'm still on 5.3 flash for now. When things quiet down long enough, I might give Qwen-3.8-flash-next a spin for a few days.

I've been thinking this for a while, but this V4.1 Flash benchmark has proven it ... benchmarks are useless these days. Labs (especially Chinese labs) are focused entirely on benchmark scores and don't care about the usability in real world usage.

FredDumartin

Sep 12

True! The benchmarking obsession is partly a cultural thing — there's a strong culture of exam and grade performance in East Asian education, and it clearly carries over into how these labs chase leaderboards.
Also, Artificial Analysis keeps changing their benchmarks every week now, so it's hard to have accurate references…

I'll follow your recommendation on this one — I'm going to add 2x 6000 Pro to our existing pair and try 5.3-flash.

It's good that Chinese labs keep improving their models and open-sourcing them. Honestly, it would be very beneficial if OpenAI and Anthropic released flash-sized open-source models too — it would only boost their own position in the market.

Thanks for sharing your experience and feedback!

laterbreh

Sep 13

i'm currently running glm-5.3 flash on the 4x RTX Pro 6000 rig and before that ...

GLM-5.2-NVFP4 hybrid and before that ...

deepseek v4 flash preview and before that ...

MiniMax M3 NVFP4 and before that ...

MiniMax 2.7 ??

at each point in the rig evolution for 1 to 2, to 3 and finally 4 GPU, I've always run the best model that will fit. glm-5.3-flash doesn't feel better than 5.2 to me, but damn it's so much faster, multi-modal and can handle way more concurrent agents, it's hard to go back to 5.2.

i say this because THIS model, the Deepseek v4.1 Flash beats GLM-5.3-Flash on ALL but two benchmarks. According to the benchmarks, this model should CRUSH GLM 5.3 Flash. I'm about to find out if that's true or not this weekend. I have a fresh SPEC and full 20 phase implementation plan written for an iPad app I was about to let GLM 5.3 Flash slay, but now it's going to be THIS model putting in work.

all of the talk of benchmaxxing is being put to the test on a real project and not a flappy bird clone or some lame one-shot game.

On a side note, why is every single AI YouTuber obsessed with how models perform in a string of one-shot tests. No one develops software as one-shot stunts.

My literal progression as well. I use sequential gated/staged agents for agentic code development with separating auditing and clearance pipelines for code iteration and development. I was an obsessed user with DS4 0731 for a while, I didnt even think of anything else, then GLM5.3 flash came along with multi-modal and it did feel stronger than 0731 in some ways otherwise quality was good and exceeded my own personal benchmarks with 5.2.

I am currently working on patching VLLM to run DS4.1 without vision on my quad rtx 6k system, ive got it without vision working with dspark at ~170tps im doing more kernal work but its very hacky right now. I look forward to hearing your results as it sounds like we operate similarly.

I dont rely on one shot benchmarks, as thats nonsense not real workloads. I rely on iteration and speed with strict clearance checkpoints.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment