Dolphin takes a different approach to OCR than most models. Instead of processing an entire document in a single pass, it uses a two-stage pipeline: first analyze the document structure (identify tables, figures, text blocks, headers), then parse each element with specialized extractors. This gives it an edge on complex layouts where single-pass models struggle.
At 400M parameters, Dolphin is mid-size and focuses on layout understanding rather than raw text extraction. If your documents have tricky layouts, multi-column pages, or mixed content types, Dolphin is worth trying.
What is Dolphin?
Dolphin comes from ByteDance, the company behind TikTok and a major contributor to open-source AI. The model uses an analyze-then-parse approach that’s fundamentally different from single-pass OCR models like Surya or Baidu Unlimited-OCR.
The key insight: complex documents have structure. A page might have headers, multi-column text, embedded tables, figures with captions, and footnotes. Single-pass models often mangle this structure because they try to read everything linearly. Dolphin first understands the layout, then processes each element with the right extractor.
Key Specs
| Spec | Value |
|---|---|
| Parameters | ~400M (estimated) |
| License | Apache 2.0 |
| Languages | 50+ |
| Multi-page | No (page-by-page) |
| Table detection | Yes (cell-level) |
| Layout recognition | Yes (two-stage) |
| Equation support | Limited |
| Min hardware | 4 GB VRAM |
What makes it different
Two-stage pipeline: Dolphin first runs a layout analysis model that identifies content types (text, table, figure, header, footnote). Then it runs specialized extractors for each content type. This is more accurate on complex layouts than single-pass models.
Content type awareness: Dolphin doesn’t just extract text. It knows what kind of content it’s looking at. A table is processed differently than a paragraph. A figure caption is handled differently than a footnote. This context-awareness improves accuracy on mixed-content pages.
Multi-column handling: Documents with multiple columns often trip up single-pass models that read left-to-right across the entire page. Dolphin’s layout analysis identifies column boundaries and reads each column separately.
ByteDance engineering: ByteDance processes billions of documents across their products. Dolphin benefits from that engineering experience and the massive training data that comes with it.
Setup
Installation
pip install dolphin-ocrOr from source:
git clone https://github.com/bytedance/dolphin.git
cd dolphin
pip install -e .Basic usage
from dolphin import DolphinOCR
# Initialize the model
ocr = DolphinOCR()
# Process a document
result = ocr.process("complex_layout.png")
# Access different content types
for block in result.blocks:
if block.type == "text":
print(f"Text: {block.text}")
elif block.type == "table":
print(f"Table: {block.to_html()}")
elif block.type == "figure":
print(f"Figure at {block.bbox}")
elif block.type == "caption":
print(f"Caption: {block.text}")Processing multi-column documents
from dolphin import DolphinOCR
ocr = DolphinOCR()
# Process a multi-column document
result = ocr.process("two_column_paper.png")
# Access columns separately
for column in result.columns:
print(f"--- Column {column.index} ---")
for block in column.blocks:
print(block.text)Table extraction
from dolphin import DolphinOCR
ocr = DolphinOCR()
# Process document with tables
result = ocr.process("invoice.png")
for table in result.tables:
print("Table found:")
print(table.to_html()) # HTML representation
print(table.to_csv()) # CSV representation
# Access individual cells
for row in table.rows:
for cell in row.cells:
print(f" Cell: {cell.text} at {cell.bbox}")When to use Dolphin
- You process documents with complex layouts (multi-column, mixed content)
- Your documents have tables embedded in text
- You need content type awareness (text vs table vs figure)
- You process invoices, forms, or structured documents
- You need accurate extraction from tricky layouts
When to use something else
- Simple single-column documents: Surya (faster, simpler)
- Multi-page PDFs: Baidu Unlimited-OCR (single-pass multi-page)
- Multilingual scripts: dots.ocr (script-agnostic)
- Ultra-lightweight: PaddleOCR-VL (34.5M params)
- Academic papers with equations: GOT-OCR 2.0 (better equation handling)
My take
Dolphin’s two-stage approach is genuinely better for complex layouts. If you process invoices, forms, academic papers with figures, or any document with mixed content types, Dolphin’s layout analysis gives it an edge over single-pass models.
The tradeoff is complexity and speed. Dolphin is slower than Surya because it runs two models instead of one. And the two-stage pipeline is more complex to set up and debug.
If your documents are simple (single-column text, no tables, no figures), use Surya instead. It’s faster and simpler. But if your documents have complex layouts, Dolphin’s extra complexity is worth it.
The ByteDance backing is a good sign for long-term development. ByteDance has the resources and motivation to keep improving this model.
FAQ
Is Dolphin better than Surya?
For complex layouts (multi-column, mixed content, tables in text), yes. For simple documents, no. Surya is faster and simpler. Dolphin’s advantage shows up on tricky layouts that trip up single-pass models.
How fast is Dolphin?
Slower than Surya because it runs two models (layout analysis + text extraction). On an RTX 3060, expect 15-30 pages per minute. On CPU, 2-5 pages per minute.
Can Dolphin process invoices?
Yes, that’s one of its strengths. Dolphin’s layout analysis identifies invoice fields (header, line items, totals) and extracts them separately. This is more accurate than single-pass models that treat invoices as plain text.
What languages does Dolphin support?
50+ languages, which is good but not as broad as Surya (90+) or PaddleOCR-VL (100+). For Latin and CJK languages, Dolphin is strong. For less-common scripts, dots.ocr is a better choice.
Can I use Dolphin commercially?
Yes. Dolphin uses the Apache 2.0 license, which allows commercial use, modification, and redistribution without restrictions.
How do I choose between Dolphin and Baidu Unlimited-OCR?
If you need multi-page PDF support, use Baidu Unlimited-OCR. If you need better layout analysis on single pages, use Dolphin. For most documents, Baidu Unlimited-OCR is more versatile. For complex layouts, Dolphin is more accurate.