xpark.dataset.checkpoint.build_checkpoint_filesystem

xpark.dataset.checkpoint.build_checkpoint_filesystem(path: str, storage_options: dict[str, Any] | None = None) → FileSystem | None

Build a PyArrow filesystem for checkpoint I/O from Xpark storage options.

s3:// / cos:// and tclake:// always return a PyFileSystem wrapping the fsspec layer; other schemes are built from explicit storage_options entries. Returns None when Ray/PyArrow can infer the filesystem alone (local paths, unknown schemes without custom options).

Parameters:

• path – Checkpoint path (local or URI such as s3://... / cos://...).

• storage_options – Xpark DatasetContext.storage_options mapping (keyed by protocol, e.g. {"cos": {...}}).

Returns:

A PyArrow filesystem for CheckpointConfig.override_filesystem, or None to let CheckpointConfig infer the filesystem.