---
title: "PDF to Markdown with Python: A Resumable OCR Pipeline for RAG"
url: https://apyhub.com/blog/pdf-to-markdown-python-ocr-rag
author: Callable Labs
published: 2026-10-25T08:51:32.140418Z
tags: [python, ocr, pdf, markdown, rag]
---

# PDF to Markdown with Python: A Resumable OCR Pipeline for RAG

# PDF to Markdown with Python: A Resumable OCR Pipeline for RAG

## Introduction

A search assistant finds the right paragraph, gives a plausible answer and then cannot tell the reader which page supports it. The problem started before retrieval: the extraction step discarded the page boundary.

For a useful PDF ingestion pipeline, keep the source reference alongside the text from the beginning. Save every completed page, too. If page 37 fails, restarting should not require processing pages 1 through 36 again.

In this tutorial, we'll use the [Callable Labs OCR & PDF to Markdown API on ApyHub](https://apyhub.com/callable-labs/service/ocr-pdf-to-markdown) to build that extraction step. We build this API at Callable Labs. The example produces Markdown and a page manifest; your existing chunker and search index can use those files next.

## Start with one PDF page

Use a PDF you are authorized to process. This example reads a local `report.pdf`, so you do not need to host it publicly. Create an [ApyHub API key](https://apyhub.com/), keep it on your server and install the Python dependency:

```bash
python -m pip install requests
export APY_TOKEN='your-apyhub-api-key'
```

The document endpoint accepts a PDF source and a 1-based page number. Choose JSON output so you receive both `markdown` and `page.count`. Reading the entire PDF takes one request per page; a single request does not convert the whole document.

## Make extraction resumable

Save this as `extract_pdf.py` next to `report.pdf`:

```python
import base64
import hashlib
import json
import os
import time
from pathlib import Path

import requests

URL = "https://api.eu.apyhub.com/callable-labs/ocr-pdf-to-markdown/v1/ocr/document"
source = Path("report.pdf")
pdf = source.read_bytes()
options = {"format": "json", "mode": "auto", "dpi": 200, "include_blocks": False}
identity = pdf + json.dumps(options, sort_keys=True).encode("utf-8")
folder = Path("extracted") / hashlib.sha256(identity).hexdigest()
folder.mkdir(parents=True, exist_ok=True)
encoded = base64.b64encode(pdf).decode("ascii")
interval = float(os.environ.get("MIN_INTERVAL_SECONDS", "10"))

with requests.Session() as client:
    client.headers["apy-token"] = os.environ["APY_TOKEN"]

    def read_page(number):
        checkpoint = folder / f"page-{number:04d}.json"
        if checkpoint.exists():
            return json.loads(checkpoint.read_text(encoding="utf-8"))
        time.sleep(interval)
        response = client.post(
            URL,
            json={"document": {"base64": encoded}, "page": number, **options},
            timeout=90,
        )
        response.raise_for_status()
        result = response.json()
        if result["page"]["index"] != number or not isinstance(result["markdown"], str):
            raise ValueError("Unexpected page response")
        temporary = checkpoint.with_suffix(".tmp")
        temporary.write_text(json.dumps(result, ensure_ascii=False), encoding="utf-8")
        temporary.replace(checkpoint)
        return result

    first = read_page(1)
    total = first["page"]["count"]
    manifest = []
    markdown = []
    for number in range(1, total + 1):
        result = first if number == 1 else read_page(number)
        markdown.append(f"<!-- source: {source.name}; page: {number} -->\n\n{result['markdown']}")
        manifest.append({"source": source.name, "page": number, "file": f"page-{number:04d}.json"})

(folder / "report.md").write_text("\n\n---\n\n".join(markdown), encoding="utf-8")
(folder / "manifest.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8")
print(f"Saved {total} pages to {folder}")
```

Run `python extract_pdf.py`. The output directory is keyed by the PDF contents and extraction settings. Changing the input or DPI creates a different directory, which prevents an old checkpoint from silently supplying text for a new job. The script sends requests sequentially; adjust the interval to the limits in your ApyHub workspace.

If a request fails, the script stops and keeps completed pages. After resolving the failure, rerun it to resume. Check the atom balance (atoms are ApyHub's usage unit) before retrying a quota error. After a timeout, processing may already have completed, so another submission can consume additional atoms.

## Carry the page metadata into retrieval

Use the page JSON files as your chunker's inputs. Give every chunk a document ID and page number in its metadata, rather than depending on the Markdown comment surviving later transformations. A citation can then point back to the source page even when the text has been split into smaller passages.

Headings and tables can make chunking more useful, but inspect the extracted structure first. A table split across pages may need application-specific handling. OCR may misread figures or merge columns, so compare representative pages with the source before indexing a large collection. If your workflow needs table cells or bounding boxes, keep `include_blocks` enabled and use the structured output.

The source PDF limit is 20 MiB, with one page processed per request. Base64 adds roughly a third to upload size, and ApyHub's transport limit can be lower. For a larger permitted upload, use an authorized public URL that remains valid throughout extraction. Review the service's Privacy Notice before processing sensitive documents.

## Let an agent read PDFs as a tool

Through [ApyHub MCP](https://apyhub.com/mcp), an agent in Claude, ChatGPT, Cursor or any MCP client can call this OCR API directly, like every endpoint in the [ApyHub catalog](https://apyhub.com/catalog), without a wrapper or tool definition. Your agent can turn a PDF page into Markdown in one call instead of writing its own parsing code, with consistent output and fewer tokens.

[**Connect ApyHub MCP to your agent →**](https://apyhub.com/mcp)

## Conclusion

Start with a short PDF containing both prose and a table. Compare its extracted pages with the original, then connect the saved page records to your retrieval pipeline. The [ApyHub listing](https://apyhub.com/callable-labs/service/ocr-pdf-to-markdown) has the request reference and playground for the first page.

[**Try the PDF to Markdown API for free →**](https://apyhub.com/callable-labs/service/ocr-pdf-to-markdown)

## FAQ

**How do I convert a PDF to Markdown with Python?** Send each page to the [OCR & PDF to Markdown API](https://apyhub.com/callable-labs/service/ocr-pdf-to-markdown) with `"format": "json"` and save the `markdown` field. The `extract_pdf.py` script above does this for every page and writes one combined `report.md` plus a `manifest.json`.

**Does it work on scanned PDFs?** Yes. It runs OCR on scanned and digital PDFs, and also reads single images such as photos and screenshots.

**Can I convert a whole PDF in one request?** No. Each request reads one page, and every response includes the total page count. PDFs up to 20 MiB and 1,000 pages are supported.

**Does it extract tables?** Yes. Tables come back as HTML, Markdown and cell rows. Bar, line and pie charts can be turned into data tables too.

**How do I keep page numbers for RAG citations?** Store the document ID and page number in each chunk's metadata, taken from the page JSON files. Don't rely on Markdown comments, since chunkers often strip them.

**How do I test the PDF to Markdown API before writing code?** Use the playground on the [API page](https://apyhub.com/callable-labs/service/ocr-pdf-to-markdown), run the request with curl, or save it in [Voiden](https://voiden.md/), an open-source API client that stores requests as Markdown files next to your code.

**Is there a free plan?** Yes. The ApyHub Starter plan includes 5 API calls a day and 3,000 atoms a month, with no credit card. That covers testing a few pages.

## About Callable Labs

[Callable Labs](https://callablelabs.com) builds APIs for machine-readable workflows, designed for automation and AI agents. Besides OCR and PDF to Markdown, their APIs on ApyHub include:

* [Invoice](https://apyhub.com/callable-labs/service/generate-invoice-document-from-json-api), [report](https://apyhub.com/callable-labs/service/generate-report-document-from-json-api) and [Word template](https://apyhub.com/callable-labs/service/validate-generate-documents-from-templates) generation
* [Diagram, math and code rendering](https://apyhub.com/callable-labs/service/diagram-math-code-render-api)
* [Image background removal](https://apyhub.com/callable-labs/service/background-removal) and [QR and barcode decoding](https://apyhub.com/callable-labs/service/read-qr-barcode-image)
* [Speech to text and subtitles](https://apyhub.com/callable-labs/service/convert-speech-to-text-subtitles)

See all 20 on the [Callable Labs provider page](https://apyhub.com/providers/callable-labs).

## About ApyHub

[ApyHub](https://apyhub.com/) is a curated API catalog and trusted operational layer for developers and AI agents. The [catalog](https://apyhub.com/catalog) has over 1,500 endpoints and capabilities and keeps growing, all under one subscription billed in atoms. Every service carries machine-readable certification (GDPR, SOC 2, ISO 27001), is MCP-ready by default and runs in dedicated EU and US regions. More than 65,000 developer workspaces use ApyHub every month, and the free tier needs no credit card. Have an API of your own? [Publish it on ApyHub](https://apyhub.com/api-provider).