Introduction: The Trust Paradox of AI Agents
AI agents have a fundamental problem: the LLM generates the output and then judges whether it's "done" — like a student writing and grading their own exam.
The mainstream desktop AI agent approach: let the same Agent handle decomposition, execution, and completion judgment, popping up a confirmation for the user to approve or reject before critical actions. This model is clean, intuitive, and effective — for tasks like document drafting, schedule management, or email replies, "looks right" is usually good enough.
But when tasks involve code generation, builds, and tests, there can be a vast gap between "looks done" and "actually works." The Agent runs ls and says "files created," but the directory path is wrong. The Agent runs cat and says "code is correct," but a critical import is missing. Even when the user reviews, they're only seeing the Agent's own summary — if the self-report is unreliable, the user's approval is built on sand.
I'm developing a desktop AI workbench (ai-workbench, open source), built on Electron + React, local-first, supporting any OpenAI-compatible model. For the verification problem, I took a different path: splitting planning, execution, and verification into three independent roles, where the verifier explicitly does not trust the executor's self-report, and has its own independent eyes to gather evidence.
This article breaks down the architecture's concrete design across four dimensions — trust, verification, safety, and auditability — and how these designs manifest in code.
Chapter 1: PSE Three-Role Separation
The workflow engine (main/workflow.js) splits the task lifecycle into three roles, each with an independent system prompt, and even configurable to use different model configurations:
// main/workflow.js — role credential resolution
const roleCreds = (role) => {
const id = models && typeof models === 'object' ? models[role] : undefined;
const configs = (settings && Array.isArray(settings.modelConfigs) && settings.modelConfigs) || [];
if (id) {
const cfg = configs.find((c) => c.id === id);
if (cfg) return { baseURL: cfg.baseURL, apiKey: cfg.apiKey, model: cfg.model };
}
// fallback: primary model
return { baseURL: settings.baseURL, apiKey: settings.apiKey, model: settings.model };
};
The three roles' responsibilities are strictly defined in main/prompts.js:
- Planner: decomposition only, outputs a JSON plan with acceptance criteria (AC), executes no commands
- Specialist: executes only the current step, produces deliverables, may run shell commands but results are returned as evidence
- Evaluator: independent verification, explicitly instructed not to trust the executor's self-report
The Evaluator's system prompt states it plainly:
// main/prompts.js
const EVALUATOR_SYS = `You are an independent review engineer (Evaluator), responsible for verifying whether the Specialist's deliverable meets the step's acceptance criteria (ac).
Key principle: evidence-driven, do not trust the executor's self-report.
- You must cite evidence from the deliverable text (excerpt key sentences/fragments) to prove whether each ac is satisfied, rather than trusting the Specialist's "completed" claim.
- If this step provides a [Command Execution Results] section, prioritize real output as evidence: acs depending on run results can be judged PASS/FAIL directly based on exit code and stdout/stderr.
...`;
"Do not trust the executor's self-report" — this single sentence is the watershed of the entire architecture. In a single-agent model, the Agent says "done" and enters the user approval phase; here, the Evaluator first asks: where is the evidence?
The significance of role separation goes beyond "having an extra checker." More critically: different roles use different system prompts, and can even use different models — Planner benefits from a strong reasoning model for decomposition, Specialist can use a fast model for execution, and Evaluator needs a "nitpicker" mindset prompt for rigorous review. This separation ensures each role focuses only on its responsibility, unaffected by the confirmation bias of "I just wrote this, it should be fine."
If the Evaluator judges PARTIAL (core done but with gaps), it injects feedback into the next Specialist retry. If it judges FAIL or BLOCKED, it triggers fail-fast to skip dependent downstream steps, preventing cascading no-ops:
// main/workflow.js — fail-fast mechanism
if (blockedBy.length) {
const verdict = {
verdict: 'SKIPPED',
feedback: `Skipped via fail-fast: prerequisite steps ${blockedBy.map((b) => `"${b.title}"(${b.verdict})`).join(', ')} have failed/blocked. This step cannot succeed and will not be executed to save time.`,
};
// ...skip this step
}
Chapter 2: Giving the Verifier Independent Eyes
Evaluator's Independent Evidence Gathering
This is ai-workbench's most unique design. The Evaluator doesn't just review the Specialist's deliverable text — it can request the framework to execute a batch of read-only commands to confirm whether artifacts actually exist.
In EVALUATOR_SYS's output format, there's a verify field:
// main/prompts.js — Evaluator output format
{
"verdict": "PASS" | "PARTIAL" | "FAIL" | "BLOCKED",
"acResults": [
{ "ac": "acceptance criteria text", "status": "PASS" | "PARTIAL" | "FAIL", "evidence": "evidence excerpted from deliverable or command execution results" }
],
"feedback": "if not fully passed, describe what's missing; empty string if passed",
"verify": ["optional: read-only evidence-gathering commands you want the framework to execute, e.g. find/ls/cat/grep"]
}
When the workflow engine receives the verify field, it actually executes these read-only commands and feeds the real output back to the Evaluator for a second-round judgment:
// main/workflow.js — two-phase independent evidence gathering
if (Array.isArray(parsed.value.verify) && parsed.value.verify.length) {
on({ type: 'phase', phase: roleKey, message: `Independent evidence gathering (${parsed.value.verify.length} read-only commands)` });
for (const vc of parsed.value.verify.slice(0, 8)) {
const vr = await runReadOnlyCommand(String(vc || ''), projectDir, { timeoutMs: 30000, signal });
verifications.push({ command: vr.command, exitCode: vr.exitCode, stdout: vr.stdout, stderr: vr.stderr, blocked: vr.exitCode === 'FORBIDDEN' });
}
// second-round judgment with evidence
const second = await chatStream({ ...base, messages: [
{ role: 'system', content: sysPrompt },
{ role: 'user', content: buildEvaluatorUser(task, step, specialistText, projCtx, executions, verifications, ...) },
]});
}
This means the Evaluator isn't just "reading" the Specialist's self-report — it's "investigating." The Specialist says "created src/App.js," the Evaluator can submit find . -name App.js to verify. The Specialist says "build succeeded," the Evaluator can submit ls dist/ to confirm the artifacts actually exist.
The two-phase judgment design lets the Evaluator first make a preliminary assessment based on the deliverable text, and if it finds "claimed but cannot be confirmed from text," it proactively proposes evidence-gathering commands, then makes a final judgment with real output. This is far more reliable than purely reviewing text — the file system doesn't lie, and exit codes don't sugarcoat.
The Security Contract of Read-Only Evidence
The Evaluator's evidence-gathering commands aren't handed directly to the shell — they go through a security channel stricter than the Specialist's controlled execution. runReadOnlyCommand in main/executor.js has its own independent whitelist and prohibitions:
// main/executor.js — read-only evidence whitelist
const READONLY_ALLOWED = [
'ls', 'find', 'cat', 'grep', 'rg', 'head', 'tail', 'wc', 'stat', 'test', 'file',
'readlink', 'pwd', 'echo', 'printenv', 'realpath', 'du', 'tree',
'sort', 'uniq', 'cut', 'tr', 'basename', 'dirname',
// framework read-only health check commands
'cd', 'php', 'composer', 'mvn', 'gradle', 'ng', 'node', 'java', 'python', 'python3',
'go', 'cargo', 'ruby', 'bundle', 'dotnet', 'rails', 'pytest', 'tsc', 'jest', 'yarn', 'pnpm',
];
const READONLY_DESTRUCTIVE = [
'rm', 'mv', 'cp', 'dd', 'mkfs', 'chmod', 'chown', 'chgrp', 'touch', 'ln', 'mkdir',
'rmdir', 'git', 'npm', 'npx', 'curl', 'wget', 'kill', 'pkill', 'tee', 'shutdown',
'reboot', 'eval', 'exec', 'source', 'xargs', 'sudo', 'su', 'passwd',
];
const READONLY_FORBIDDEN_RE = /(>>?|\$\(|`)/; // no redirects, command substitution, or backticks
The readOnlyReason function checks each item: whether the first word is in the whitelist, whether the full command contains destructive tokens (covering piped and chained commands), and whether it contains redirects or command substitution. Any violation returns exitCode='FORBIDDEN' with a human-readable reason — never executed.
The core idea: read-only commands can't cause damage, so they don't need user confirmation — but only if the framework can reliably determine a command is truly "read-only." The triple check of whitelist + destructive token blacklist + redirect/substitution prohibition ensures the Evaluator's evidence-gathering commands are safe and controlled.
Chapter 3: Layered Execution Safety: From Binary to a Four-Tier Model
Most AI agents' execution safety is binary: pop up a confirmation before important actions, user approves or rejects. Simple and clear, but ls and rm -rf / get the same confirmation intensity — wasting user attention while providing no special protection for truly dangerous commands.
ai-workbench splits execution safety into four tiers, implemented in main/executor.js:
Tier 1: FORBIDDEN — direct rejection, no confirmation
// main/executor.js
const FORBIDDEN_PATTERNS = [
{ re: /\bsudo\s+[\w-]/, why: 'sudo requires interactive password input in terminal (UI cannot receive the prompt, will hang until timeout)...' },
{ re: /\bsu\s+(-|\w)/, why: 'su user switching is interactive privilege escalation, prohibited.' },
{ re: /\bpasswd\b/, why: 'passwd is an interactive command, prohibited.' },
];
Commands like sudo/su/passwd — interactive privilege escalation — are rejected outright without a confirmation dialog. In an Electron UI environment there's no controlling terminal, so password prompts never appear and would hang until timeout.
Tier 2: DANGEROUS — always强制 individual confirmation
const DANGEROUS_PATTERNS = [
/\brm\s+-rf?\s+\//, // rm -rf /
/\brm\s+-rf?\s+~/, // rm -rf ~
/\bmkfs\b/,
/\bdd\b\s+if=/,
/:\s*\(\)\s*\{/, // fork bomb
/\bchmod\s+-R?\s+0/, // chmod -R 0...
/\bshutdown\b/,
/\breboot\b/,
/>\s*\/dev\/(sd|hd|nvme)/, // direct write to block device
/\bcurl\b[^\n|]*\|\s*(sh|bash)/, // curl ... | sh
];
Commands matching dangerous patterns always require individual confirmation — even if the user previously checked "remember this project," there's no exemption.
Tier 3: normal — first confirmation, then "remember"
The first execution within a project directory during a run requires user confirmation; once the user checks "remember," subsequent non-dangerous commands for this project skip confirmation:
let allowed = isDirApproved(projectDir) && !dangerous;
if (!allowed) {
const approved = await requestApproval({ emit: on, dir: projectDir, command: cmd, dangerous });
if (!approved) { executions.push({ command: cmd, denied: true }); continue; }
}
Tier 4: read-only whitelist — no user confirmation needed
The Evaluator's verify evidence commands execute through runReadOnlyCommand, passing the read-only whitelist check — no user confirmation needed but no write operations allowed.
The core idea of this four-tier model: confirmation cost should be proportional to risk. ls doesn't need confirmation, rm -rf / always does, and sudo shouldn't appear in auto-generated commands at all. This is more granular than binary approval, and reduces the risk of users blindly clicking approve due to confirmation fatigue.
Chapter 4: Environment Awareness: Prevention Over Correction
Without toolchain pre-checks, an Agent might plan a task requiring PHP, but the machine doesn't have PHP installed — only discovering this at execution time, wasting planning and execution effort.
ai-workbench performs a one-time environment probe at workflow startup. main/capabilities.js uses command -v (read-only, no side effects) to detect common toolchains:
// main/capabilities.js
const TOOLS = [
'node', 'npm', 'npx', 'python3', 'python', 'uv',
'php', 'composer', 'laravel', 'java', 'mvn', 'gradle',
'ng', 'go', 'cargo', 'ruby', 'docker', 'git',
];
function detectCapabilities() {
const caps = {};
for (const t of TOOLS) caps[t] = has(t);
caps._stacks = {
phpLaravel: !!(caps.php && caps.composer && caps.laravel),
nodeFrontend: !!(caps.node && caps.npm),
angularCli: !!(caps.node && caps.npm && caps.ng),
javaSpring: !!(caps.java && (caps.mvn || caps.gradle)),
pythonFastapi: !!(caps.python3 && caps.uv),
};
return caps;
}
Detection results are injected into the Planner's prompt, explicitly informing which toolchains are available and which are missing:
// main/workflow.js — injected into Planner
s += `\n[Runtime Environment Toolchain Detection Results (actual machine state, strictly follow)]
Available toolchains (command exists): ${avail || '(none)'}
Missing toolchains: ${missing || '(none)'}
Stack readiness:
${stackLines}
Requirements:
- Only plan tasks that [available toolchains] can actually execute; do not assume missing tools exist.`;
If the Planner finds required toolchains missing, it either switches to an available alternative stack or schedules a "install dependencies" step first. This is far more efficient than letting the Specialist execute halfway only to hit command not found.
Chapter 5: Path Consistency: Preventing LLM Rename Drift
This is a real problem discovered in practice. In multi-step tasks, if the LLM creates a demo-spring-boot directory in step 1, by step 4's retry it might "forget" the name used earlier and start a new spring-boot-demo. This causes verification commands to execute in the wrong directory, and the Evaluator to falsely judge FAIL.
ai-workbench uses inferProjectDir to infer the project root directory name from step 1's mkdir -p/cat > commands, then injects it into all subsequent steps and retries:
// main/workflow.js
if (i === 0) taskProjectDir = inferProjectDir(executions) || taskProjectDir;
// injected into Specialist prompt
s += `
R${ruleN}.[Mandatory reuse of project root] This task established project root "${taskProjectDir}" in step 1. You must [strictly and exclusively reuse] this root: all relative paths in commands are based on it.
- If this is a [multi-component / full-stack] task, each component must be created in [its own subdirectory] under "${taskProjectDir}", never creating sibling directories at the container root level.
Verification paths depend on this root name. Renaming or creating sibling directories will cause artifact path mismatches and FAIL from the Evaluator.`;
It also maintains a createdDirs set, accumulating actually-created directories across steps, injected into subsequent steps and Evaluator evidence commands, enforcing full paths and prohibiting bare names or abbreviations.
This problem rarely arises in document scenarios (documents don't "not exist" due to different path names), but in software engineering it's a frequent trap — directory names are the foundation of verification paths, and once drift occurs, all subsequent path-based checks fail.
Chapter 6: Evidence Archiving: Making Runs Auditable
AI agent execution is often a black box — when something goes wrong, you don't know which step failed; when it succeeds, you don't know why. ai-workbench archives structured evidence for every run, and all exit paths — success, user abort, error — are archived:
// main/runlog.js
function saveRunLog(data) {
const dir = logsDir();
const ts = started.toISOString().replace(/[:.]/g, '-');
const base = `${ts}-${slugify(data && data.task)}`;
const jsonPath = path.join(dir, base + '.json');
const mdPath = path.join(dir, base + '.md');
fs.writeFileSync(jsonPath, JSON.stringify(data, null, 2), 'utf-8');
fs.writeFileSync(mdPath, buildRunMarkdown(data), 'utf-8');
return { jsonPath, mdPath };
}
The JSON file contains the complete run record: runId, timestamps, task type, project directory, Planner plan, each step's Specialist deliverable, Evaluator judgment and evidence, command execution results, toolchain detection results, and overall conclusion. The MD file is a human-readable version you can open directly.
The workflow engine calls persistRun in three branches: final (successful completion), aborted (user abort), and error (exception exit):
// main/workflow.js
// success archive
persistRun();
// user abort
persistRun({ aborted: true });
// exception exit
persistRun({ error: msg });
This means even if a run crashes midway, you can see from the logs: what the Planner planned, what each step's Specialist produced, what the Evaluator judged, which commands were executed, and what the exit codes and output were. The evidence chain is complete and traceable.
Conclusion: From "Trusting Self" to "Not Trusting the Executor"
The single-agent self-verification model is clean and efficient — the Agent decomposes, executes, and judges completion, with the user as a safety net at key nodes. For scenarios where "looks right is enough," this is perfectly adequate.
But when tasks involve "did the build succeed," "did tests pass," "did route registration take effect" — scenarios that require running to verify — self-verification's reliability degrades. The Agent's self-report may be unreliable, the user's approval may be perfunctory, and the gap between "looks done" and "actually works" only surfaces under independent evidence gathering.
ai-workbench's design philosophy is simple: don't trust the executor. Give the verifier an independent role, an independent prompt, and even an independent evidence-gathering terminal. Let evidence come from the file system and exit codes, not from the executor's self-assessment. This mechanism is heavier and more complex, but in software engineering scenarios, real runtime evidence is more reliable than any self-report.
Source Code Navigation
main/workflow.js– PSE workflow orchestration: Planner→Specialist→Evaluator loop, two-phase independent evidence gathering, fail-fast, path consistency injectionmain/prompts.js– Three-role system prompts (Planner/Specialist/Evaluator/Reviewer), task type profiles, Specialist execution rulesmain/executor.js– Controlled command execution engine: four-tier safety model (FORBIDDEN/DANGEROUS/normal/read-only whitelist), read-only evidence executormain/capabilities.js– Runtime toolchain detection: command -v pre-check + stack readiness derivationmain/runlog.js– Run evidence archiving: structured JSON + human-readable MD dual-file persistencemain/keychain.js– macOS keychain wrapper: API key encrypted storage, multi-account supportmain/project.js– Project context collection: read-only directory tree + key file summariesmain/llm.js– LLM streaming call wrapper: unified proxy/retry/loggingelectron.js– Main process entry: IPC channel registration, workflow event forwardingpreload.js– Renderer bridge: contextBridge secure API exposure
Full source code: github.com/erishen/ai-workbench