If It Can Be Computed, Never Call a Model: The Deterministic Fast Path Runtime of resolve-harness

resolve-harness is a deliberately minimal agent harness, built to be "decomposable and replaceable", that pairs a LangGraph tool loop and LiteLLM multi-model routing, tiered memory, sandboxed tools, and a Planner → Specialist → Evaluator → Reporter orchestration (PSE). Its core claim is "if it can be computed, never call a model"—ahead of the Specialist execution layer sits a three-tier Fast Path: built-in matchers answer directly with zero model calls; on a miss, the model writes its own detector, validated in an AST whitelist sandbox and persisted for reuse; stable detectors are promoted into the source tree by humans. This article walks from the harness overview to the design trade-offs and security boundaries of this deterministic pipeline.

Why "If It Can Be Computed, Never Call a Model"

Most "tasks" are actually deterministic: arithmetic, base conversion, leap year checks, date calculations, unit conversion. Having an LLM handle them repeatedly carries three costs:

• Slow: a model call takes seconds; a Python snippet takes milliseconds;
• Expensive: every call costs tokens;
• Unstable: the same question might be answered correctly today and incorrectly tomorrow.

So the design principle is simple: ask whether code can compute it first, then ask the model. Queries that hit the deterministic path complete with zero model calls—and the entire process is transparent to the UI, memory, and orchestration loop. The task tree renders as usual, just with a zero-model label attached.

But the real point of this section isn't to save a few tokens. It cleanly splits the system into two layers: controllable determinism (handed to code) and intelligence beyond control (handed to the model). The cleaner the split, the bolder you can be about piling on imagination—because the "uncontrolled" part is locked firmly inside the model call, while the "reliable" part can be copied, accelerated, and reused without limit. Every trick that follows builds on this split.

Harness Overview: Four Pillars

To understand where the Fast Path fits, you need to see the skeleton it lives in. The README includes a self-positioning table: this project's goal isn't "yet another framework," but to break down the key agent mechanisms clearly, with every piece replaceable:

Pillar	Responsibility
Loop	LangGraph StateGraph: agent → tools → agent… loop, max_steps prevents runaway (src/resolve_harness/graph/)
Harness	Single assembly point: config, model routing, tool registry, memory, loop (src/resolve_harness/harness.py)
Memory	Short-term (session transcript) + long-term (SQLite facts, read/written via tools) (src/resolve_harness/memory/)
Fast Path	Deterministic queries answered directly in pure code; on a miss, generate a detector and persist it for reuse (src/resolve_harness/fastpath.py · codegen.py)

Model routing is based on LiteLLM: a single openai/<model> string works across providers, with support for per-role overrides—Planner uses a cheap model, Specialist uses a strong one, each independently configured.

These four pillars matter because they decouple "intelligence" from "infrastructure." Want to swap models? Change one routing line. Add memory? Touch the Memory pillar. Squeeze in new capability? Hang it on the tool registry. The Fast Path is just one of the four—but it's the one that turns the whole harness from "can chat" into "can work."

PSE Orchestration: Four Roles Turn Goals into Deliverables

Beyond chat mode, task mode is a multi-agent pipeline:

• Planner: decomposes a single goal into an ordered list of subtasks (structured JSON);
• Specialist: each subtask is handed to an expert Agent, which enters its own agent → tools → agent tool loop (read/write sandbox files, fetch, run scripts), with parallel fan-out support;
• Evaluator: checks each result against the goal, returning {passed, score, feedback}; failure triggers re-planning;
• Reporter: aggregates each subtask's outputs and artifact paths, producing the final deliverable.

The subtasks the Planner produces look like this (from the real runtime structure):

{
  "subtasks": [
    {"index": 0, "title": "Fetch data source", "instruction": "fetch the official public endpoint and persist to disk", "artifacts": []},
    {"index": 1, "title": "Structured parsing", "instruction": "organize raw data into a table", "artifacts": ["data/parsed.csv"]},
    {"index": 2, "title": "Generate report", "instruction": "write the deliverable doc from the table", "artifacts": ["report.md"]}
  ]
}

After checking each subtask, the Evaluator returns a verdict:

{  "passed": true,  "score": 92,  "feedback": "Data complete and consistent with the goal"}

When passed is false and the max_replan_rounds limit isn't reached, the system feeds feedback back to the Planner for re-decomposition—meaning task mode corrects itself rather than handing you a wrong answer once. Parallelism is controlled by the parallel parameter (default 4, range 1+); multiple Specialists each run in isolated sandboxes and cannot see each other's files.

The key design is delegated autonomy: task mode does not require step-by-step human approval (all tools are sandboxed), while the human gate remains in interactive chat (file-writing tools require confirmation). The Fast Path is embedded at the Specialist's entry point—each subtask first asks "can this be computed without a model?" Subtasks that can be computed are completed with zero model calls and appear normally in the task tree, just labeled zero-model. The orchestration layer is completely unaware of this, which is exactly what "transparent to the loop" means.

Three-Tier Architecture: Built-in Matchers → Codegen → Promote

The example below is deliberately simple—just base conversion. The point isn't the example itself, but the mechanism: any deterministic logic can be hung on the same hook.

Tier 1: Built-in Matchers—Hit Means Zero Model

Take the built-in base conversion matcher as an example (fastpath.py). A regex recognizes colloquial phrasing, then pure Python computes the result:

# The shipped detector matches Chinese queries (the project's primary audience);
# this English edition illustrates the same mechanism for an English-speaking demo.
_BASE_RE = re.compile(
    r"(\d+)\s*(?:in|to|of)?\s*(binary|octal|hex(?:adecimal)?)"
    r"|(hex(?:adecimal)?|octal|binary)\s*(?:of|for)?\s*(\d+)", re.I)
_BASE_MAP = {"binary": 2, "octal": 8, "hexadecimal": 16, "hex": 16}

def _try_base_convert(text: str) -> FastAnswer | None:
    m = _BASE_RE.search(text)
    if not m:
        return None
    num_str, base_name = (m.group(1), m.group(2)) if m.group(1) else (m.group(4), m.group(3))
    base = _BASE_MAP.get(base_name.lower())
    if base is None:
        return None
    value = int(num_str)
    if base == 2:
        result = bin(value)[2:]
    elif base == 8:
        result = oct(value)[2:]
    else:
        result = hex(value)[2:]
    return FastAnswer(text, "base_convert", f"{num_str} in {base_name} is {result}.", result)

In practice: for a question like "what is 255 in hexadecimal," the time from regex match to answer is under a millisecond—no network, zero tokens, and the answer format is always identical. Similar matchers include safe arithmetic evaluation (AST whitelist validates each node before eval), date calculations, unit conversion, and a dozen more built-in matchers. Their existence isn't to show off these few features, but to give you a template: those "always-the-same-answer" jobs in your domain should look exactly like this.

Tier 2: Codegen—Let the Model Write Its Own Accelerator

What about the long tail that built-in matchers can't cover? It falls through to the Codegen tier: the model writes a pure function that satisfies a fixed contract—

def detect(text: str) -> str | None:
    """Return the answer string on a hit; return None on a miss."""

The generated function is validated in an AST whitelist sandbox, then written to data/fastpath_plugins/ and reused immediately in the current session; the next time a similar question arrives, it hits directly without even calling the model. This is the most interesting part of the Fast Path: the first time runs the full pipeline (model call + code generation + validation); after that, the accelerator is one the model built for itself.

To be honest about the boundary here: codegen.py's validation only checks safety (AST whitelist: banned from importing modules other than re/math, banned dunder access, executed in a sanitized builtins namespace, no I/O and no eval)—it does not verify correctness. A semantically wrong detector still passes the sandbox, gets persisted, and gets called zero-model over and over—and because it runs so fast, it looks "utterly correct." So the real safety net for this tier is the third gate.

Tier 3: Promote—Elevating Temporary Artifacts to First-Class Citizens

Detectors in data/fastpath_plugins/ are the "candidate zone." On the Plugins page, you can review each detector's full source code; stable ones can be promoted with one click: they're merged into src/resolve_harness/generated_detectors.py and distributed with the source, while the runtime candidate copy is deleted. Promotion preserves existing records (locked in by the regression test test_promote_preserves_existing_detectors) and never overwrites them. This step requires human confirmation—it's both a quality gate and the critical leap from "temporary artifact" to "project asset." Safety rests on the sandbox; correctness rests on human eyes—this is the unchanging dual safeguard of the Fast Path.

End-to-End Walkthrough: Three Queries, Three Fates

Mechanisms are easy to abstract away; let's trace three queries that genuinely occur in practice, and see which tier each lands in, and what the task tree and token usage look like.

Query A: "What is 255 in hexadecimal?" → Hits the built-in base_convert matcher. A zero-model node appears in the task tree, this call's token usage is exactly 0, and ff is returned in milliseconds. The model is never woken up.

Query B: "Group these orders by amount bracket" (assuming no ready-made built-in matcher) → The built-in layer returns None, so it falls through to Codegen. The model generates a detect(text) detector → it passes the AST whitelist → it's written to data/fastpath_plugins/ → executed immediately and returns the result. The next query of the same kind hits the plugin directly, zero model. This is the classic "slow the first time, free forever after" path.

Query C: "Analyze why last quarter's revenue dropped and write a retro for the boss" → This is an open-ended goal; the entire Fast Path steps aside. It enters PSE orchestration: the Planner splits it into three subtasks—"pull data / attribution analysis / write the doc"—the Specialists run in parallel (each in its own sandbox), the Evaluator accepts, and the Reporter aggregates everything into report.md. All three nodes in the task tree carry model usage and bear no zero-model label—the deterministic parts (if any step involves computation) may still be intercepted by the Fast Path, while the open-ended parts honestly go through the model.

The three fates correspond to three cost structures: A is free and instantaneous, B costs once then is free forever, C pays the price of intelligence every time. What the system does is push queries as far left as possible.

From Examples to Your Imagination

By now you should see it: base conversion, arithmetic, dates—they're all just seeds. What this harness really sells you is a skeleton you can grow your own deterministic logic and domain workflows on. Here are a few brainstorms you can pick up immediately—each is a direct combination of the harness's capabilities, not some fictional "someone-else's-case":

• Crystallize repetitive information into detectors: say you pull down a publicly disclosed dataset and compute a few metrics every week. Let Codegen write the detector the first time; afterwards it returns zero-model in milliseconds. Once stable, promote it into the source so the whole team shares it.
• Orchestrate multi-step workflows with PSE: one sentence—"research X and produce a comparison report"—and the Planner splits, Specialists fetch + analyze in parallel, the Evaluator gates quality, and the Reporter emits the doc. You only supply the goal; splitting and execution are left to the orchestration.
• Compose local mini-tools from the sandbox toolchain: fetch + write-file + run_script combine into a fully local pipeline where data never leaves the machine—ideal for scenarios where "you don't want to feed data to a third party."
• Let human promotion crystallize team assets: common logic goes from "a one-off session plugin" to "a function everyone can use in the repo"; knowledge no longer scatters across chat logs.
• Hybrid orchestration: let the Fast Path swallow every deterministic step in the flow, leaving only the parts that truly need judgment to the model—your Agent becomes fast, cheap, and more stable.

Where your imagination lands depends on your domain. Finance, documents, data cleaning, research reports, internal tooling… whenever there's a step where "the same input should always yield the same output," it's worth hanging a Fast Path; whenever there's a goal that is "multi-step, parallelizable, and needs acceptance," it's worth handing to PSE. The examples are just a doorstop; behind the door is your own play.

Security Boundaries: Generated Code Can't Run Naked

Having the model write code and execute it in-process demands a direct answer on security. resolve-harness uses an AST whitelist as a dual safeguard:

_FORBIDDEN_ATTRS = {"eval", "exec", "format", "format_map",
                    "globals", "locals", "mro", "subclasses", "init"}

_ALLOWED_NODES = (
    ast.Module, ast.FunctionDef, ast.arguments, ast.arg,
    ast.Return, ast.Expr, ast.Assign, ast.AnnAssign,
    ast.Call, ast.Name,
)

Two rules: node whitelist—only the listed node types may appear in the syntax tree; attribute blacklist—known escape hatches are banned.

There's a real attack-and-defense story here: early versions blocked eval/exec/__class__ traversal, but f-string's format_map was its twin escape hatch—a dunder traversal chain hidden inside an ordinary string literal could bypass the attribute check. After adding the format_map block, the regression test test_format_map_dunder_bypass_rejected permanently locked down that path.

The companion run_script tool uses the same defense-in-depth approach: script whitelist + forced sandbox file writes + subprocess environment sanitization (sensitive variables like API keys are not passed through).

The point of security boundaries isn't only to "stop bad actors"—it's to let you experiment boldly. Precisely because generated code can't be written to disk unless it passes the sandbox (save_plugin forces validate_ast before persistence, rejecting any unsafe code), you can boldly let the model build detectors during development without worrying it's quietly doing something evil.

Observability and Quality Gates

Fast must be understandably fast:

• Task tree labeling: subtasks hit by the Fast Path carry a zero-model tag, distinguishable at a glance;
• Verifiable zero usage: the zero-model path's usage is exactly 0 tokens, locked in by a regression test;
• Event stream: the Planner's plan, the Evaluator's evaluation, and each subtask's execution events are reported throughout, so you can watch your orchestration "grow" in real time;
• Reviewable source: the Plugins page lets you view and delete any detector.

Quality uses human-machine collaboration: the AST whitelist locks down security, while correctness is gated by human review and promotion confirmation. The roadmap's next step is automating this—running a set of boundary self-tests with known answers against detectors before persistence, forming a "machine fallback + human review" dual safeguard.

Retrospective: A Reusable Design Principle

The entire pipeline distills to one principle: the degree of automation should be proportional to the strength of validation.

• Fully automated built-in matchers: logic written by humans, covered by tests, trustworthy;
• Model-generated Codegen: only allowed to execute after passing the security sandbox;
• Promote for long-term persistence: must pass human eyes.

Any team looking to accelerate engineering workflows with AI can apply this ladder as a self-check: which step is pure code? Which step introduces generated artifacts? Does the validation strength of the generated artifact match its degree of automation?

Viewed across the whole harness, it's the same story: the intelligence of Planner/Specialist handles the uncertain parts, while Evaluator and the quality gates hold the deterministic parts—each layer doing its job, making the system both flexible and reliable. And for you, this principle is also a user manual: every detector and every piece of orchestration you hang on should be automated only as far as it is trustworthy. Imagination can be boundless; validation cannot be lazy.

Source Navigation

Full code: https://github.com/erishen/resolve-harness

• src/resolve_harness/graph/loop.py
• src/resolve_harness/harness.py
• src/resolve_harness/tasks.py
• src/resolve_harness/fastpath.py
• src/resolve_harness/codegen.py
• src/resolve_harness/generated_detectors.py
• src/resolve_harness/memory/long_term.py
• tests/test_tasks.py · tests/test_fastpath.py · tests/test_codegen.py