xpark.dataset.VideoTextAreaRatio#

class xpark.dataset.VideoTextAreaRatio(*, frame_sample_num: int | None = None, fps: int | None = None, keyframes_only: bool = False, start_time: float = 0.0, end_time: float | None = None, mode: Literal['accurate', 'fast'] = 'accurate', extract_text: bool = False, ratio_fn: Callable[[VideoTextLayout], float] | None = None, engine: Literal['paddleocr', 'rapidocr'] = 'paddleocr', ocr_batch: int = 4, **ocr_kwargs)#

Score a video from its on-screen text layout, a float32 in [0, 1].

Each input row is one video link; frames are sampled (a fixed count via frame_sample_num or proportionally to duration via fps) then reduced to a single score by ratio_fn. Following xpark’s ray-data filter convention the operator only measures — keep/drop is left to Dataset.filter():

ds = ds.with_column("text_area_ratio", VideoTextAreaRatio(...).with_column(col("video")))
ds = ds.filter(expr=col("text_area_ratio") <= 0.3)

Input accepts a local path, a remote URI (cos:// / s3:// / http(s)://), or raw video bytes.

ratio_fn can customize how per-frame text layouts are reduced to a score. Built-in scorers: default_area_ratio() (default), subtitle_area_ratio(), and watermark_area_ratio(). Use functools.partial() to bind scorer parameters:

  • subtitle_area_ratio(region=...): score boxes whose centers fall inside normalized (top, bottom, left, right) bounds.

  • watermark_area_ratio(min_frame_ratio=..., pos_tol=..., area_tol=...): score static text regions that recur across sampled frames; min_frame_ratio is required frame coverage, while pos_tol and area_tol control matching tolerance for position and size.

Parameters:

  • frame_sample_num – Number of frames sampled uniformly across the (trimmed) timeline. Mutually exclusive with fps; defaults to 3 when neither is given.

  • fps – Sample at this many frames per second instead of a fixed count, so longer videos yield proportionally more frames. Mutually exclusive with frame_sample_num.

  • keyframes_only – Sample only from keyframes (faster decode, coarser temporal coverage).

  • start_time – Start of the sampled time range in seconds. Defaults to 0.0.

  • end_time – End of the sampled time range in seconds. Defaults to the full duration.

  • mode – "accurate" (default, best accuracy) or "fast" (faster, lower resource usage).

  • extract_text – When True, run text recognition so TextLayoutBox.text is populated and low-confidence detection boxes are filtered out via text_confidence. When False (default), use detection-only OCR (faster, but the detector alone tends to over-fire on textureless regions like sky/sand/water — the no text case — since there is no recognition step to reject those false positives; text fields are empty).

  • ratio_fn – Optional (VideoTextLayout) -> float returning a score in [0, 1]. Defaults to default_area_ratio() (mean per-frame text area ratio).

  • engine – OCR backend, "paddleocr" (default) or "rapidocr". Forwarded to ImageExtractTextLayout.

  • ocr_batch – Max number of frames fed to the OCR backend in a single call. Frame extraction runs concurrently across videos in the incoming UDF batch, but the resulting frames are then flushed through OCR sequentially in slices of at most ocr_batch frames per video to keep CPU/memory bounded (a single ONNX Runtime session is CPU-bound and does not benefit from concurrent callers). Defaults to 4: benchmarks show 1 → 4 recovers the per-call overhead gap while 4 → 16 is essentially flat, so 4 is the memory-friendly sweet spot. Set higher if you have spare memory; set to 1 to disable batching entirely.

  • **ocr_kwargs – Extra params forwarded to the OCR engine via ImageExtractTextLayout. See that class for supported kwargs per engine.

Examples

from functools import partial

from xpark.dataset import VideoTextAreaRatio, from_items
from xpark.dataset.expressions import col
from xpark.dataset.processors.video_text_area_ratio import watermark_area_ratio

ds = from_items([{"video": "example.mp4"}])
ds = ds.with_column(
    "text_area_ratio",
    VideoTextAreaRatio(frame_sample_num=3)
    .options(num_workers={"CPU": 1}, batch_size=1)
    .with_column(col("video")),
)
ds = ds.filter(expr=col("text_area_ratio") <= 0.3)

# Filter videos with heavy static overlay text, such as watermarks or logos.
ds = ds.with_column(
    "watermark_ratio",
    VideoTextAreaRatio(ratio_fn=partial(watermark_area_ratio, min_frame_ratio=0.8))
    .options(num_workers={"CPU": 1}, batch_size=1)
    .with_column(col("video")),
)
ds = ds.filter(expr=col("watermark_ratio") <= 0.1)

Methods

__call__(videos)

Call self as a function.

options(**kwargs)

with_column(videos)

__call__(videos: pa.ChunkedArray) → pa.Array#

Call self as a function.

options(**kwargs: Unpack[ExprUDFOptions]) → Self#
with_column(videos: pa.ChunkedArray) → pa.Array#