Instructions to use microsoft/OmniParser with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/OmniParser with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="microsoft/OmniParser")# Load model directly from transformers import AutoProcessor, AutoModelForVisualQuestionAnswering processor = AutoProcessor.from_pretrained("microsoft/OmniParser") model = AutoModelForVisualQuestionAnswering.from_pretrained("microsoft/OmniParser") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use microsoft/OmniParser with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "microsoft/OmniParser" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/OmniParser", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/microsoft/OmniParser
- SGLang
How to use microsoft/OmniParser with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "microsoft/OmniParser" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/OmniParser", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "microsoft/OmniParser" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/OmniParser", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use microsoft/OmniParser with Docker Model Runner:
docker model run hf.co/microsoft/OmniParser
Not able to run the application
File "D:\practise\agents\OmniParser\gradio_demo.py", line 3, in
import gradio as gr
File "C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio_init.py", line 3, in
import gradio.simple_templates
File "C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio_simple_templates_init.py", line 1, in
from .simpledropdown import SimpleDropdown
File "C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio_simple_templates\simpledropdown.py", line 7, in
from gradio.components.base import Component, FormComponent
File "C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio\components_init.py", line 1, in
from gradio.components.annotated_image import AnnotatedImage
File "C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio\components\annotated_image.py", line 15, in
from gradio.components.base import Component
File "C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio\components\base.py", line 21, in
from gradio.blocks import Block, BlockContext
File "C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio\blocks.py", line 33, in
from gradio import (
ImportError: cannot import name 'themes' from partially initialized module 'gradio' (most likely due to a circular import) (C:\Users\Karthik.Pamidimarri\AppData\Roaming\Python\Python312\site-packages\gradio_init_.py)
It seems like you are missing some libs like gradio. Check if you have already installed the requirement, follow the guidance on the github page.