xpark.dataset.connectors.DeleteVectorDB#

class xpark.dataset.connectors.DeleteVectorDB(connection_factory: Callable[[], Any], collection_name: str, **kwargs: Any)[source]#

Operator for batch-deleting records from a vector database by ID.

Supported backends: [‘elasticsearch’, ‘milvus’].

Uses the backend’s native delete API. Deleting a non-existent ID is a no-op (idempotent delete semantics).

The operator is a pure side-effect: it returns a boolean column and does not produce meaningful output data.

Parameters:

  • connection_factory – A callable that returns a vector database client (e.g. lambda: pymilvus.MilvusClient(uri="...")). The backend is auto-detected from the returned client type.

  • collection_name – Name of the target collection.

  • **connector_kwargs – Extra keyword arguments. The connector-init reserved keys max_retries, retry_delay, retry_backoff are consumed by the connector itself; all remaining kwargs are forwarded to the underlying delete call (e.g. timeout).

Examples

import pymilvus
from xpark.dataset import from_items
from xpark.dataset.connectors import DeleteVectorDB
from xpark.dataset.expressions import col

ds = from_items([{"id": "1"}, {"id": "2"}, {"id": "3"}])
ds = ds.with_column(
    "result",
    DeleteVectorDB(
        connection_factory=lambda: pymilvus.MilvusClient(uri="http://localhost:19530"),
        collection_name="my_col",
    )
    .options(num_workers={"IO": 1})
    .with_column(col("id")),
)
print(ds.take_all())

Methods

__call__(ids)

Execute batch delete.

options(**kwargs)

with_column(ids)

Execute batch delete.
__call__(ids: pa.ChunkedArray) → pa.Array#

Execute batch delete.

Parameters:

ids – A ChunkedArray of IDs to delete.

Returns:

A boolean array (one False per row). The operator is used for its side-effects only.

options(**kwargs: Unpack[ExprUDFOptions]) → Self#
with_column(ids: pa.ChunkedArray) → pa.Array#

Execute batch delete.

Parameters:

ids – A ChunkedArray of IDs to delete.

Returns:

A boolean array (one False per row). The operator is used for its side-effects only.