Hugging Face's logo

bharatgenai
/
patram-7b-instruct

Image-Text-to-Text
Transformers
Safetensors
English
patram1_1
text-generation
patram
visual-document-understanding
visual-question-answering
indian-documents
key-value-extraction
multi-turn-chat
chain-of-thought-reasoning
ocr
table-understanding
chart-understanding
conversational
custom_code
Model card Files
xet
 Community
6

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

BharatGen logo

Patram-7B-Instruct

Patram by BharatGen is a family of 7B parameter vision-language models trained from scratch for visual document understanding. This README covers Patram-7B-Instruct (v1.0), Patram1.1-7B-Instruct, and Patram1.2-7B-Instruct, released as revisions of the same bharatgenai/patram-7b-instruct repository — pick a revision below and run any of them.

News 📢

  • [2026-02-17] 🚀 Released Patram1.2-7B-Instruct — adds Chain-of-thought VQA, Text Extraction, Chart Understanding, Table Understanding, and Flowchart Understanding on top of v1.1, trained on BharatDocs-v1.2. Evaluated across Text-Rich Document VQA, Visually-Rich Document VQA, TableVQA, and Indian benchmarks.
  • [2025-08-08] 🚀 Released Patram1.1-7B-Instruct — adds Multi-turn VQA and Key-value extraction on top of v1.0, trained on BharatDocs-v1.1. Evaluated across Text-Rich Document VQA, Visually-Rich Document VQA, and Indian benchmarks.
  • [2025-06-02] 🚀 Released Patram-7B-Instruct — India's first document foundation model for visual document understanding, trained on BharatDocs-v1.

Index

  1. News
  2. Introduction
  3. Comparison
  4. Benchmarks
  5. Inference
  6. Examples
  7. Authors
  8. Citation
  9. Contact

Introduction

Patram-7B-Instruct by BharatGen is a 7B parameter vision-language model trained from scratch for visual document understanding. As India's first document foundation model, it is built to tackle complex document analysis. Each release is trained on a carefully curated instruction-tuned dataset, combining diverse public and custom synthetic data designed to support a broad spectrum of document understanding tasks.

  • Patram-7B-Instruct (v1.0) is the original release.
  • Patram1.1-7B-Instruct builds on it with a richer training set (BharatDocs-v1.1) and adds multi-turn and key-value-extraction capabilities.
  • Patram1.2-7B-Instruct builds further (BharatDocs-v1.2), adding chain-of-thought reasoning, text extraction, and chart/table/flowchart understanding.

Comparison

Category Item Patram 1.0 Patram 1.1 Patram 1.2
Capabilities Extractive VQA ✅ ✅ ✅
Abstractive VQA ✅ ✅ ✅
Multi-turn VQA ✅ ✅
Key-value extraction ✅ ✅
Chain-of-thought VQA ✅
Text Extraction ✅
Chart Understanding ✅
Table Understanding ✅
Flowchart Understanding ✅
Architecture Vision Encoder Vision Transformer (ViT) CLIP-style Vision Transformer CLIP-style Vision Transformer
Projector MLP projector Attention-pooling + gated-MLP projector Attention-pooling + gated-MLP projector
LLM OLMo-7B OLMo-7B OLMo-7B
Training Data Dataset BharatDocs-v1 + open-source document datasets Improved BharatDocs-v1.1 + open-source document datasets BharatDocs-v1.2 + open-source document datasets
I/O Supported formats English instructions + image input; text output English instructions + image input; text output English instructions + image input; text output
Language Supported languages English English (Indian language support upcoming) English (Indian language support upcoming)
License License Apache 2.0 Apache 2.0 Apache 2.0

Benchmarks

Patram1.2 — Text-Rich Document VQA

Metric: G-Eval (LLM-as-a-judge)

Rank Model DocVQA WTQ VisualMRC Rico ScreenQA Weighted Avg.
🥇 1 bharatgenai/patram-7b-instruct 91.8 91.6 94.6 89.6 91.16
🥈 2 Qwen/Qwen3-VL-8B-Instruct 96.4 64.8 95.9 92.0 89.79
🥉 3 OpenGVLab/InternVL3-8B 92.7 61.1 96.0 89.6 87.27
4 google/gemma-3-12b-it 85.7 58.6 95.2 87.9 84.13
5 allenai/Molmo-7B-D-0924 39.3 13.1 54.4 61.3 47.33
6 allenai/Molmo-7B-O-0924 36.6 12.0 47.4 58.2 44.08

Patram1.2 — Visually-Rich Document VQA
Rank Model ChartQA InfoVQA FlowVQA ChartBench CharXiv Avg.
🥇 1 Qwen/Qwen3-VL-8B-Instruct 81.5 79.9 93.6 79.3 74.3 81.72
🥈 2 bharatgenai/patram-7b-instruct 81.9 87.8 92.6 77.5 66.1 81.18
🥉 3 OpenGVLab/InternVL3-8B 86.3 71.8 92.1 76.0 68.0 78.84
4 google/gemma-3-12b-it 76.0 54.2 88.9 70.8 59.3 69.84
5 allenai/Molmo-7B-D-0924 30.9 26.3 43.3 57.9 29.1 37.50
6 allenai/Molmo-7B-O-0924 29.8 27.0 41.1 53.7 27.7 35.86

Patram1.2 — TableVQA
Rank Model TabFact TableVQA Weighted Avg.
🥇 1 Qwen/Qwen3-VL-8B-Instruct 82.8 68.3 81.28
🥈 2 bharatgenai/patram-7b-instruct 80.8 83.8 81.12
🥉 3 OpenGVLab/InternVL3-8B 79.9 66.2 78.46
4 google/gemma-3-12b-it 73.6 62.8 72.47
5 allenai/Molmo-7B-O-0924 50.8 24.0 47.98
6 allenai/Molmo-7B-D-0924 39.7 27.1 38.40

Patram1.2 — Indian Benchmarks & Overall Average
Rank Model Patram-Bench-VQA-en Overall Avg.
🥇 1 Qwen/Qwen3-VL-8B-Instruct 94.4 86.80
🥈 2 bharatgenai/patram-7b-instruct 92.1 86.39
🥉 3 OpenGVLab/InternVL3-8B 79.7 81.07
4 google/gemma-3-12b-it 73.2 74.91
5 allenai/Molmo-7B-O-0924 75.1 50.76
6 allenai/Molmo-7B-D-0924 79.2 50.61

Overall Avg. is the average of the Text-Rich Document VQA, Visually-Rich Document VQA, TableVQA, and Patram-Bench-VQA-en bucket averages above.

Radar chart comparing Patram1.2 against Qwen3-VL-8B-Instruct, InternVL3-8B, gemma-3-12b-it, and both Molmo-7B variants across DocVQA, WTQ, VisualMRC, Rico ScreenQA, ChartQA, InfoVQA, FlowVQA, ChartBench, CharXiv, TabFact, TableVQA, and Patram-Bench-VQA-en, alongside overall average scores

Inference

All three versions live in the same Hugging Face repository, bharatgenai/patram-7b-instruct, as separate revisions — patram1.0, patram1.1, and patram1.2. Set REVISION accordingly in the snippet below.

Environment

All three revisions use the same environment

torch==2.7.0
torchvision==0.22.0
transformers==4.50.3
accelerate==0.26.0
einops==0.8.1

Hugging Face 
import base64
import io
import urllib.request

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForCausalLM, GenerationConfig

MODEL_ID = "bharatgenai/patram-7b-instruct"
REVISION = "patram1.1"  # "patram1.0", "patram1.1", or "patram1.2"
HF_TOKEN = "hf_........"  # Insert HF token here
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

IMAGE_URL = "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQzcRxT-eX_C8Kahj9-DvCF1t1gzHctM7SIFOasQkCboQ&s=10"  # Insert image URL here
IMAGE_B64 = ""                                  # ...or a base64-encoded image instead


def load_image(url=None, b64=None):
    """Load a document image from a URL or from a base64 string."""
    if b64:
        # tolerates a leading "data:image/png;base64," prefix
        data = base64.b64decode(b64.split(",", 1)[-1])
    elif url:
        with urllib.request.urlopen(url) as response:
            data = response.read()
    else:
        raise ValueError("Provide either url= or b64=")
    return Image.open(io.BytesIO(data)).convert("RGB")


image = load_image(url=IMAGE_URL)
# image = load_image(b64=IMAGE_B64)

processor = AutoProcessor.from_pretrained(
    MODEL_ID, revision=REVISION, trust_remote_code=True, token=HF_TOKEN
)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    token=HF_TOKEN,
).to(DEVICE)
model.eval()

prompt = "Question: What is this document about? Answer based on the image."
inputs = processor.process(images=[image], text=prompt)
# float tensors (the image patches) must match the model dtype; integer ids must not be cast
inputs = {
    k: (v.to(DEVICE).to(model.dtype) if torch.is_floating_point(v) else v.to(DEVICE)).unsqueeze(0)
    for k, v in inputs.items()
}

with torch.autocast(device_type=DEVICE, enabled=True, dtype=torch.bfloat16):
    output = model.generate_from_batch(
        inputs,
        GenerationConfig(max_new_tokens=256, do_sample=False, stop_strings="<|endoftext|>"),
        tokenizer=processor.tokenizer,
    )

generated = output[0, inputs["input_ids"].size(1):]
print(processor.tokenizer.decode(generated, skip_special_tokens=True))

Patram1.2 only — supports two prompting modes via a MODE flag, direct and cot:

MODE = "cot"  # "direct" or "cot"
PROMPT_TEMPLATES = {
    "direct": "Question: {question} Answer based on the image.",
    "cot": "Question: {question} Think step by step, then answer based on the image.",
}
MAX_NEW_TOKENS = {"direct": 2048, "cot": 4096}

question = "What is the total amount on this invoice?"
prompt = PROMPT_TEMPLATES[MODE].format(question=question)
inputs = processor.process(images=[image], text=prompt)  # image = PIL.Image.open(...).convert("RGB")

Note: If you're trying this on an Apple Silicon (M1/M2/M3/M4/...) chip, please follow the official documentation by PyTorch and Hugging Face for installing dependencies:

Footnote — Patram1.2 prompting modes

Patram1.2 supports two prompting modes (direct and cot — see Hugging Face above for how to run each). Benchmark scores above were evaluated with the following mode per benchmark:

Benchmark Mode
DocVQA direct
WTQ direct
VisualMRC cot
Rico ScreenQA direct
ChartQA cot
InfoVQA cot
FlowVQA cot
ChartBench cot
CharXiv cot
TabFact cot
TableVQA cot
Patram-Bench-VQA-en cot

Examples

Examples below are from Patram1.1-7B-Instruct.

Example: asking Patram1.1 about a rescheduling notice, with document on the left and question/answer on the right

Example: asking Patram1.1 for structured JSON extraction from a lab report, with document on the left and question/answer on the right

Example: asking Patram1.1 about a passenger self-reporting form, with document on the left and question/answer on the right

Examples below are from Patram1.2-7B-Instruct.

Example: Patram 1.2 understanding a Government of Assam rescheduling notice and reasoning over the previous and revised skill test and document verification dates

Example: Patram 1.2 understanding and reasoning over a complex research paper table containing ADE and FDE benchmark results across multiple datasets and models

Example: Patram 1.2 using chain-of-thought reasoning to understand a Janata Personal Accident Insurance claim form and reason over policyholder, insured-person, and claim details

Authors

  • Principal Investigators: Prof. Ravi Kiran Sarvadevabhatla, Prof. Ganesh Ramakrishnan
  • Contributors: BharatGen Team

Citation 

@misc{bharatgen_patram_2025,
  title        = {Patram: India's Pioneering Vision-Language Foundation Model for Document Intelligence},
  author       = {{BharatGen Team}},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/bharatgenai/patram-7b-instruct}},
  note         = {Accessed: 2025-06-02}
}

@misc{bharatgen_patram1_1_2025,
  title        = {Patram1.1: India's Pioneering Vision-Language Foundation Model for Document Intelligence},
  author       = {{BharatGen Team}},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/bharatgenai/patram-7b-instruct}},
  note         = {Accessed: 2025-08-08}
}

@misc{bharatgen_patram1_2_2026,
  title        = {Patram1.2: India's Pioneering Vision-Language Foundation Model for Document Intelligence},
  author       = {{BharatGen Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/bharatgenai/patram-7b-instruct}},
  note         = {Accessed: 2026-02-17}
}

Contact

Downloads last month
2,686
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 1 Ask for provider support

Model tree for bharatgenai/patram-7b-instruct

Finetunes
1 model