Section 08

Document Parsing

Turning ~70 courts' cause list PDFs — many scanned, none consistent, all subject to change — into structured entries. This is the hardest problem in the product.

Why this is hard

A cause list is conceptually a table: item number, case number, parties, court number, judge, time. In practice, across ~70 courts, it arrives as:

The failure that matters is not a crash. It is a cause list that parses successfully into plausible-looking entries that are subtly wrong — a shifted column, an entry attributed to the wrong item number. That reaches a user as a confident, incorrect statement about their day in court.

Pipeline

flowchart TB
    A[Cause list PDF] --> B{Text layer present?}
    B -->|yes| C[Extract words
with x,y positions]
    B -->|no| D[Render pages to image]
    D --> E[OCR with layout
bounding boxes]
    E --> C
    C --> F[Strip repeated
headers and footers]
    F --> G[Detect column bands
from x-position clustering]
    G --> H[Group words into rows
by y-proximity]
    H --> I[Merge wrapped
continuation lines]
    I --> J{Court template
in registry?}
    J -->|yes| K[Apply template:
column map + regexes]
    J -->|no| L[Infer layout,
flag as novel]
    K --> M[Validate entries]
    L --> M
    M --> N{Confidence}
    N -->|high| O[Emit CAUSELIST_ENTRY]
    N -->|low| P[LLM fallback
extraction]
    P --> Q[Validate again]
    Q -->|passes| O
    Q -->|fails| R[Human review queue]
    R --> S[Corrected entries
+ template update]
    S --> O
      

Fig 8.1 — Cause list extraction pipeline

Court templates

A template is data, not code. This is what makes a court fixable in minutes without a deploy.

causelist_template:
  court_id: patna-hc
  version: 4
  detect:
    header_contains: ["HIGH COURT OF JUDICATURE AT PATNA", "CAUSE LIST"]
    page_width_pt: 595            # A4 portrait
  strip:
    repeated_top_pt: 84           # header band
    repeated_bottom_pt: 48        # footer band
  columns:                        # x-ranges in points
    item_no:   [ 28,  70]
    case_no:   [ 70, 210]
    parties:   [210, 400]
    advocate:  [400, 500]
    court_no:  [500, 567]
  rules:
    row_gap_pt: 4                 # below this, lines belong to one row
    continuation: "parties column non-empty, item_no empty"
    section_heading: "^[A-Z][A-Z ]{6,}$"
  validate:
    case_no_pattern: "^(WRITA|WRITC|CRIMISC)/\\d+/\\d{4}$"
    item_no_monotonic: true
    min_entries_per_page: 5

Validation is where correctness comes from

The validate block is the most important part of the template, because it is what distinguishes "parsed" from "parsed correctly".

• Case number pattern — entries failing the court's own known format are almost always column drift.
• Monotonic item numbers — item numbers that jump backwards mean rows were merged or split wrongly.
• Minimum entries per page — a page yielding two entries where it normally yields forty indicates the layout changed.
• Page-count sanity — total entries far outside the historical range for that court and weekday.

Any validation failure routes the page to the LLM fallback rather than emitting suspect entries. Emitting nothing is always better than emitting something wrong, because a missing cause list is visible and a wrong one is not.

Technology

PaddleOCR is already used for captcha, so the dependency, the deployment story, and the team's familiarity are all shared. That is worth more than a marginal accuracy difference from a second OCR engine.

Handling format changes

Courts redesign their cause lists without warning. The system should notice before users do.

flowchart LR
    A[Nightly parse] --> B[Compare metrics
to 30-day baseline]
    B --> C{Deviation?}
    C -->|entries per page down| D[Flag template
as suspect]
    C -->|validation failures up| D
    C -->|OCR confidence down| D
    C -->|normal| E[Proceed]
    D --> F[Alert + route pages
to LLM fallback]
    F --> G[LLM extracts
+ describes layout]
    G --> H[Propose template v+1]
    H --> I[Diff against live entries
on same document]
    I --> J{Human approves?}
    J -->|yes| K[Promote template]
    J -->|no| L[Manual authoring]
    K --> M[Old version retained
for rollback]
      

Fig 8.2 — Template drift detection and repair

Because templates are versioned data rows, promoting a fix is a database write with an audit trail — not a code change, a review, a merge, and a deploy. A court can be repaired in minutes by someone who is not a Python engineer, which matters when ~70 courts can each break independently.

Order and judgment PDFs

A related but easier problem, and it should not be conflated with cause lists.

Orders are the natural home for LLM work — low volume, prose input, tolerant of imperfection, high perceived value. Cause lists are the opposite on every axis. Treating them as one problem leads either to unaffordable cause list parsing or to uselessly shallow order summaries. See Core Flows Fig 6.5, and note the standing rule there: a model-derived date is never authoritative.

Shared infrastructure with captcha

Both hard problems have the same shape — a cheap deterministic path, a failure signal, an expensive AI research path, and a verified artifact promoted back into the cheap path. Build the surrounding machinery once: