xpark.dataset.PDFToImage#

class xpark.dataset.PDFToImage(*, dpi: int = 400, max_pages: int = 0, max_pixels: int | None = 10000000)#

Render PDF pages to images.

Renders every page of each PDF and optionally downscales each page to fit a pixel budget. The output is one list<image> per input PDF row containing all produced images in reading order.

Each row’s pdf_path may be a local path or a remote URI supported by fsspec (e.g. cos://bucket/prefix/doc.pdf); remote documents are downloaded once per row into memory before rendering.

Pages are rendered, downscaled and encoded one at a time, so only a single decoded page is resident per in-flight row. Peak memory is therefore bounded by batch_size × max_pixels (plus the encoded payloads that make up the returned column). batch_size defaults to 1; raise it only when the documents are known to be small. For long documents, cap the work with max_pages.

Parameters:

  • dpi – Rendering resolution in dots-per-inch. Defaults to 400.

  • max_pages – Maximum pages to render per PDF (0 = unlimited). Defaults to 0.

  • max_pixels – Maximum pixel count (width × height) per output image. Pages exceeding this budget are proportionally downscaled. None disables resize. Defaults to 10,000,000.

Examples

from xpark.dataset import from_items
from xpark.dataset.expressions import col
from xpark.dataset.processors.pdf_to_image import PDFToImage

ds = from_items([{"pdf_path": "cos://bucket/pdfs/doc.pdf"}])
ds = ds.with_column(
    "pages",
    PDFToImage(dpi=400, max_pixels=12_000_000).with_column(col("pdf_path")),
)

Methods

__call__(pdfs)

Call self as a function.

options(**kwargs)

with_column(pdfs)

__call__(pdfs: pa.ChunkedArray) → pa.Array#

Call self as a function.

options(**kwargs: Unpack[ExprUDFOptions]) → Self#
with_column(pdfs: pa.ChunkedArray) → pa.Array#