Python API¶

The public API is load, OpenDecider.system_one, OpenDecider.system_one_batch, the question helpers Choice, Score and Noul, the answer fields below and, from 0.4.0, the agent-facing surfaces: the MCP server's tool names, arguments and answers, opendecider.tools and the public names in opendecider.integrations. Patch releases never change it; see the versioning policy.

load¶

load(name_or_path="manjunathshiva/opendecider-nano", device=None, revision=None, dtype=None, base_url=None,
     api_key=None, timeout=None) -> OpenDecider

Which package extra a model needs: nano none; small, small-td, medium-td and large-td opendecider[small]; the MLX builds opendecider[mlx]; served models none. A missing one raises ImportError with the install line. With opendecider-client (no PyTorch), only served models load: a Hub name or local folder raises ImportError saying to install opendecider instead.

OpenDecider.system_one¶

model.system_one(state, questions) -> dict

• state: a string, or any JSON-serialisable object (serialised to JSON text before the model reads it).
• questions: a non-empty dict of named questions, each a dict or a Choice / Score / Noul.

Returns:

{"model": "opendecider-nano",
 "answers": {"department": {...}, "urgency": {...}},
 "usage": {"input_tokens": 106, "output_tokens": 0},
 "latency_ms": 17.4}

plus "warnings": [...] when a state was truncated to fit the model's input length. Invalid questions raise ValueError with a message naming the question.

OpenDecider.system_one_batch¶

model.system_one_batch(states, questions) -> list[dict]

The same questions about many states, one result per state (same shape as system_one, without latency_ms). opendecider-nano runs them as one padded batch.

Questions¶

Choice(instructions, criteria)        # criteria: {"label": "description" or None, ...} or ["label", ...]; at least 2
Score(instructions, criteria)         # criteria: ordered levels, lowest first; at least 2
Noul(instructions, criteria={})       # yes / no; optional {"true": "...", "false": "..."} descriptions

As dicts: {"type": "choice" | "score" | "noul", "instructions": "...", "criteria": ...}.

Answers¶

An answer also carries "truncated": true when its state was cut to fit the model's input length.

Agent tools and integrations¶

opendecider.tools is the core behind the MCP server and the agent framework integrations:

from opendecider.tools import Decider, choose, decide, score, yes_no

d = Decider("manjunathshiva/opendecider-nano")   # or tools.shared(name): one copy per model name
choose(d, state, "Which team?", {"billing": "charges", "tech": "bugs"})
# {'choice': 'billing', 'probabilities': {...}, 'confidence': ...}
yes_no(d, state, "Is this spam?")   # {'answer': 'yes' or 'no', 'probability_yes': ..., 'confidence': ...}
score(d, state, "How urgent?", ["low", "medium", "high"])   # {'level', 'label', 'expected_level', ...}

The integrations need opendecider 0.4.0 or later, and 0.5.0 for the guard hooks, decision_runnable(), decision_evaluator() and DecisionMultiSelector (see Agent frameworks).

tools.Router(routes, instructions, *, model=..., fallback=None, min_confidence=0.0, on_error="raise", on_decision=None) is the router every integration builds on. Calling it with a state returns a route name (the fallback when the top route's probability is below min_confidence); .decide(state) returns the full tools.Decision (route, reason, choice, confidence, probabilities, model, latency, truncation, error), which on_decision also receives and .last keeps. An empty state takes the fallback without a decision (or raises ValueError without one). on_error="fallback" takes the fallback when the decision fails. See Production.

tools.decide_batch(decider, states, questions) answers the same questions about up to 256 states in one batch (1,024 questions in all, states times questions); tools.status(decider) reports the model, whether it is loaded, the version and the limits, without loading it.

Invalid input raises ValueError, a model that cannot load raises tools.ModelError, and a model server that cannot answer raises remote.ServerError, each naming the problem. tools.as_result(fn, *args) returns those as {"error": "..."} instead, for frameworks that hide a tool's exception text from the model.

Prompt guard¶

from opendecider.guard import Guard, GuardrailError

guard = Guard(checks=None, model="manjunathshiva/opendecider-small-td", threshold=None, on_error="block",
              on_decision=None, window_chars=4000)
guard.check(text)          # GuardResult (also guard(text)); guard.acheck(text) in async code
guard.check_many(texts)    # one GuardResult per text, short texts in shared batches
guard.enforce(text)        # the GuardResult, or raises GuardrailError (a ValueError) when the text is blocked

• checks: {"name": "question"}, yes/no questions that call the text `prompt`; the default is opendecider.guard.ATTACK_CHECKS (jailbreak, prompt_injection).
• threshold: a probability for every check, or {"check": probability}; by default the model's measured threshold for the default checks (opendecider.guard.THRESHOLDS: per model, and per published GGUF and MLX build), else 0.5. A check is flagged at or above its threshold.
• on_error: "block" blocks text that could not be checked, "allow" lets it through; either way reason="error".
• on_decision: a function or list, called with every GuardResult; a hook that raises is logged and ignored.
• GuardResult: passed, reason (passed, flagged, empty_input, error), violations, probabilities, thresholds, model, latency_ms, windows, truncated, error, and to_dict().

The framework hooks take a Guard or its settings as keywords; see Agent guardrails.

Environment variables for served models¶