xpark.dataset.LLMReranker#

class xpark.dataset.LLMReranker(*, base_url: str, model: str, api_key: str = 'NOT_SET', mode: str = 'pointwise', window_size: int = 20, stride: int = 10, max_qps: int | None = None, max_concurrency: int | None = None, max_retries: int = 0, hint: str | list[str] | None = None, fallback_score: float = 0.5, **kwargs: Any)#

LLM-based reranker; supports pointwise and listwise modes.

Pointwise (default): one LLM call per (query, candidate) pair, each returns a decimal in [0, 1].

Listwise: an LLM call ranks a window of candidates for one query (RankLLM-style [2] > [1] > [3] output). When the candidate list is longer than window_size, a sliding-window strategy is used following the RankLLM convention: starting from the tail (least-relevant end), each window is reranked in place and then the window slides up by stride candidates with overlap, propagating high-relevance items toward the head. After all windows complete, the final candidate order is mapped back to per-candidate scores via 1 - rank / candidate_count.

Parameters:

  • base_url – LLM server base URL.

  • model – LLM model name.

  • api_key – LLM API key.

  • mode – "pointwise" (default) or "listwise".

  • max_qps – Max queries per second.

  • max_concurrency – Max in-flight LLM requests.

  • max_retries – Per-request retry budget.

  • hint – Optional extra instructions appended to the LLMReranker.

  • fallback_score – Score used when the LLM response is unparseable and no signal can be salvaged (pointwise: per-call; listwise: per-row, when the transport layer returns no content at all).

  • window_size – Listwise specific - sliding-window width.

  • stride – Listwise specific - how many positions the window advances between consecutive LLM calls. Must satisfy 0 < stride <= window_size. A smaller stride gives more overlap (and therefore more LLM calls but higher quality).

  • **kwargs – Forwarded to LLMChatCompletions (e.g. temperature).

Examples

LLMReranker(
    model="deepseek-v3-0324",
    base_url=os.getenv("LLM_ENDPOINT"),
    api_key=os.getenv("LLM_API_KEY"),
    mode="listwise",
    window_size=20,
    stride=10,
    temperature=0.0,
)

Methods

__call__(query, candidates)

Call self as a function.

options(**kwargs)

with_column(query, candidates)

__call__(query: pa.ChunkedArray, candidates: pa.ChunkedArray) → pa.Array#

Call self as a function.

options(**kwargs: Unpack[ExprUDFOptions]) → Self#
with_column(query: pa.ChunkedArray, candidates: pa.ChunkedArray) → pa.Array#