ANKIT YADAV

02 · Document AI

Live in production

AI Document Intelligence and OCR

A FastAPI service that turns 30+ document types, including Emirates ID, driving licences, and vehicle documents, into structured JSON. Reported extraction accuracy is 97 percent.

Role
Designed the service
Scope
Detection, OCR, extraction, validation, auth
Stack
Python, FastAPI, LangGraph

Client details are confidential. No client data is shown.

AI Document Intelligence and OCR02 · SYNTHETIC EXAMPLESAMPLEName / اسمSample Namecard 0.98language: ar, envalid: true
Document path: upload, OpenCV preprocessing, YOLO crop, classification, OCR, structured extraction, validation, review or API.UploadOpenCVYOLO cropClassifyOCRJSON extractValidateReview or API
Document path: upload, OpenCV preprocessing, YOLO crop, classification, OCR, structured extraction, validation, review or API.

Overview

Vionex is a document API I designed for insurance files. A scan or PDF goes in. Structured JSON comes out. The set covers more than 30 document types, including Emirates ID, driving licences, passports, police reports, and vehicle documents. Google Cloud Vision reads the page. OpenAI, Claude, and Groq structure the fields. A YOLO model I fine-tuned finds the document before that read.

The published accuracy of the extraction service is 97 percent.

The problem

Manual data entry does not scale across dozens of document types, and it fails in specific ways on these files. Arabic and English sit on the same card. A PDF may contain several pages that are not the same document. Chassis and VIN strings are easy for OCR to confuse, and a wrong character is not a cosmetic error.

What I built

  • The FastAPI service, including SHA-256 and salt API keys and a sliding-window rate limit.
  • The labeling, training, validation, and production inference path for a YOLO document detector.
  • The LangGraph pipeline that sends a page through Cloud Vision and then through a model for structured extraction.
  • A PDF splitter that classifies each page.
  • Bilingual Arabic and English post-processing.
  • Chassis and VIN correction using ISO 3779 and an OCR confusion matrix.
  • Field validation before the JSON leaves the service.

Architecture

Upload, then split a PDF into pages and classify each page. The detector crops the document. Cloud Vision reads the crop. A model turns that text into a JSON object for the document type. Validation and the VIN correction run before the response. API-key auth sits on the outside of that pipeline.

Diagram

Key engineering decisions

Cloud Vision is the reader. The language models do not replace OCR. They structure what the reader returned, which keeps the extraction step from having to rediscover the pixels.

YOLO is in the path because the document is not the whole page. I labeled a detection set, trained it, checked it, and only then put the weights in inference. ONNX is part of the serving toolchain around that model so the detector can run beside the API without a training stack in production.

More than one model sits behind extraction. If a provider fails, another can structure the same OCR text. The contract out of the service stays JSON.

Hard problems and how I solved them

Arabic and English on one card. Post-processing keeps both scripts instead of collapsing the page to a single language.

VIN and chassis strings. Correction follows ISO 3779 and a matrix of characters that OCR commonly swaps, instead of trusting the first read.

Mixed PDFs. The splitter classifies page by page so a police report and an identity card in one file do not become one JSON object.

Reliability and failure handling

Auth and the rate limit protect the API. Validation runs before a caller treats the payload as final. A second model is available when the first extraction call fails. Low-confidence review rules beyond that gate are part of the workflow that consumes the JSON.

Outcome

The service is live. It replaced manual entry for 30+ document types with structured JSON, at 97 percent extraction accuracy.

Stack

Python, FastAPI, LangGraph, Ultralytics YOLO, OpenCV, PyMuPDF, Google Cloud Vision, OpenAI, Claude, Groq. SHA-256 API keys.

What I would improve next

A public error breakdown by document type, so the 97 percent figure can be read as a per-class number rather than a single rate.

Client details are confidential. No client data is shown.