Site icon Salesforce Diaries

Workshop on Building a Deterministic RAG Agent in Agentforce

sanketthoughts

Support volume at most B2B SaaS companies grows faster than the support team does. The usual answer is a chatbot, and the usual result is a chatbot that confidently invents an answer, refuses to hand off to a human, or loops on the same unanswerable question until the user gives up.

This post is the requirements document I’d want before building the alternative: a RAG-grounded Agentforce FAQ agent whose compliance-critical paths behave identically on every single run, because they’re implemented as explicit if/else logic in Agent Script rather than as natural-language instructions the model may or may not follow.

It’s deliberately an FAQ agent and nothing more. It answers from Knowledge and hands off; it has no capability to refund, provision, or change anything. That constraint removes a whole category of requirements — and, as Part 5 gets into, replaces them with a different set that’s easy to skip precisely because nothing looks dangerous.

I’ve written it against one fictional company — Nimbus Workspace — so every requirement stays concrete instead of drifting into generic best-practice mush. Every requirement carries at least one linked test case, and there are 40+ of them. Steal the structure for your own build.

The Short Version

Nimbus Workspace deploys Ask Nimbus, a RAG-grounded Agentforce Service Agent, to contain routine Tier-1 support volume across the Help Center and in-app chat, 24/7, while deterministically escalating anything sensitive, unresolved, or out of scope to the existing 45-person human support team.

The agent is grounded exclusively in Salesforce Knowledge via a Data Library, and it is an FAQ agent in the strict sense: it answers questions and hands off. It holds no capability to change a record, so the only write it ever performs is opening the Case that carries an escalation to a human. Escalation, conversation limits, and follow-up-question logic are implemented with Agent Script (Level 6 determinism) rather than left to model judgment — so a GDPR request escalates on run 1 and run 10,000 alike.

Platform: Salesforce Agentforce (Agent Builder, Agent Script), Data Cloud, Service Cloud.

What’s In This Post

Case Study: Nimbus Workspace {#case-study}

Nimbus Workspace is a B2B SaaS project- and work-management platform. Everything below is written against this one company so the requirements stay internally consistent.
Customers ~50,000 teams, self-serve + enterprise tiers
Support volume ~14,000 tickets/month across email, in-app chat, help center
Support team 45 human agents, volume rising faster than headcount
Existing stack Salesforce Service Cloud, Salesforce Knowledge (~850 articles across 7 domains), Omni-Channel
The gap No Data Cloud provisioned yet — required for Agentforce grounding, so it’s a day-one dependency
The project Deploy Ask Nimbus, an Agentforce FAQ agent grounded in the Knowledge base
The goal Contain routine Tier-1 volume 24/7; escalate everything else cleanly to the 45 human agents

The 7 knowledge domains (each becomes a Subagent): Billing & Subscriptions · Account & User Management · Workspace & Project Setup · Integrations & API · Data Import/Export & Compliance · Troubleshooting · Security & Compliance.

Sample Knowledge Data — Build This In Your Own Org {#sample-data}

Requirements documents are easier to argue with than to test, so the Nimbus knowledge base exists as real, importable data:

Sample-Knowledge-Articles-for-Rag-Workshop — 302 Salesforce Knowledge articles across all 7 domains above, written to match this case study: the same plans, storage limits, API rate limits and roles the example utterances and test cases assume.

A few things that make it quick to stand up:

  • Standard fields only. No custom fields, no custom Article Types. The CSV imports straight into the Knowledge__kav object Salesforce creates the moment Knowledge is enabled.
  • A bulk publisher is included. Inserted Knowledge records always land as Draft, and the supported publish method (KbManagement.PublishingService.publishArticle()) isn’t bulkified — so a loop over 302 records hits governor limits. The repo ships Batch Apex that handles it in chunks of 50, plus a status script to confirm Online: 302 | Draft: 0.
  • It’s a real corpus, not a handful of stubs. The case study puts Nimbus at ~850 articles; the repo ships 302, which is the point where chunking decisions (Part 2), retrieval-threshold tuning (FR2.4) and off-topic testing (Part 4) start behaving the way they will in production, without a long import. Nine sample articles would let you demo the agent. They wouldn’t let you find its failure modes.

Import it, provision Data Cloud, point a Data Library at it, and every test case in this post becomes something you can actually run rather than read.

Platform Note: Agent Script & Subagents {#platform-note}

As of April 2026, Salesforce shipped a new Agent Builder built on Agent Script — a human-readable scripting language underneath the drag-and-drop canvas. This is what enables genuine deterministic logic (if/else, forced transitions, mandatory gates) rather than natural-language instructions the model might or might not follow. It’s the mechanism behind almost every requirement in this post.

Two renames came with it. Salesforce has said both old and new terms will appear in documentation and the UI during the transition, so this post uses the current terms throughout:

  • Topic → Subagent
  • Topic Selector → Agent Router

Salesforce frames agent maturity as 6 Levels of Determinism — from a subagent freely picking actions (Level 1) up to Agent Script hard-coding if/else logic and forced hand-offs (Level 6). This post follows that structure because it maps cleanly onto the nine original requirements and gives each part a direct anchor in current Salesforce documentation (Appendix F).

Business Context & Objectives {#business-context}

Support volume is outpacing headcount, and a large share of tickets are repetitive, knowledge-article-answerable questions.

Metric Baseline Target Window
Containment / deflection rate 0% (no agent today) ≥ 35% of eligible chats resolved without a human 90 days post-launch
CSAT on agent-resolved chats n/a ≥ 4.2 / 5 rolling 30 days
Escalation precision n/a < 15% of escalations rated “should not have escalated” in QA sampling monthly
Escalation recall n/a < 5% of escalation-worthy chats missed monthly
Average human handle time current baseline −20% (fewer repetitive tickets reaching humans) 90 days

Scope, Stakeholders & Assumptions {#scope}

In scope: Help Center web widget, in-app chat, English only at launch, the 7 knowledge domains in Appendix A, Case creation on escalation, Omni-Channel routing.

Out of scope (this phase): Voice/IVR, WhatsApp, non-English languages, a sales/lead-qualification agent, and — the defining constraint of this build — any transactional capability. No refunds, plan changes, password resets, user provisioning, or record updates of any kind. Ask Nimbus explains how to do those things and hands off to a human when the user wants them done.

Stakeholders

Role Responsibility
Executive Sponsor (VP, Customer Support) Owns success metrics and escalation-policy sign-off
Salesforce Platform Team Agent Builder configuration, Data Cloud / Data Library setup
Support Operations & QA Lead Owns the Knowledge Matrix, tunes escalation rules, runs QA sampling and test execution
Data & Analytics Dashboards, deflection and grounding reporting
Legal & Privacy Approves compliance-triggered escalation categories (GDPR, security)
Engineering Apex/Flow/API actions, idempotency implementation
Support Agents UAT feedback on hand-off payload quality

Assumptions & Constraints

  • Service Cloud and Agentforce licenses are provisioned; Data Cloud must be newly provisioned specifically to enable Data Library grounding — a hard prerequisite, not optional.
  • Knowledge articles exist for all 7 domains but aren’t yet tagged for chunking eligibility; a one-time content audit precedes build.
  • Einstein Trust Layer is enabled by default for data masking; no additional PII architecture is assumed.
  • Deterministic logic can be authored either via the natural-language canvas (compiled into Agent Script automatically) or hand-written directly — the requirements below are authoring-path-agnostic.
  • Terminology follows current (April 2026+) naming: Subagent (formerly Topic), Agent Router (formerly Topic Selector).

How This Is Structured

Six parts, each following the same shape — concept, requirements, implementation notes, an example scenario, and test cases — so every requirement is traceable to a verification step:

Part Title Determinism Level(s)
1 Knowledge Architecture & the Knowledge Matrix 1–2
2 Grounding & Retrieval: Full Article vs. Chunks 3
3 Response Delivery & Follow-Up Questions 4
4 Off-Topic Detection & Guardrails 2–4
5 Conversation Limits: Retrieval Budget, Turn Ceilings & the One Write 5
6 Deterministic Escalation & Human Hand-off 6

A seventh, cross-cutting section on reliability and trust follows. It didn’t come from the original nine-point requirement list, so I’ve called it out separately rather than folding it in silently.

The Agent Workflow: Six Gates {#workflow}

One conversation turn moves through six gates, one per Part above, in a fixed order. Nothing here is left to the model’s discretion — each gate is a conditional evaluated before the prompt reaches the LLM, or a hard transition after it.

Read the table one gate at a time: each gate’s branches are listed on the rows beneath it.

Gate Condition What happens next
1. Forced-trigger check (Part 6) A hard trigger matches: human request, compliance/security keyword, transactional intent, or an account-specific request Knowledge can’t answer Escalation Subagent, immediately — before the model reasons at all
No trigger matches Continue to Gate 2
2. Agent Router classification (Parts 1 & 4) Utterance matches one of the 7 domains That Domain Subagent, then Gate 3
No subagent matches Off-Topic Subagent: polite redirect, logged, conversation deflected
3. Grounded retrieval (Part 2) Data Library returns content above the relevance threshold Grounded content, then Gate 4
Result is below the threshold Unresolved-turn counter +1 and retry once; on a 2nd consecutive miss, Escalation Subagent
4. Response shape (Part 3) A required input is missing Clarifying question, maximum 2, then re-check
Answer is 3 steps or fewer Single consolidated response, then Gate 5
Answer is 4 steps or more Step-by-step delivery from a persisted sequence, then Gate 5
5. Conversation-limit check (Part 5) Within the retrieval and turn budget Deliver the response
Additional retrieval needed Chain freely within the turn, no extra friction, capped at 3
Retrieval budget or turn ceiling crossed Stop retrying and offer a hand-off
6. Escalation aggregate check (Part 6) Any trigger fired — forced, threshold-based or failure-based Escalation Subagent, one idempotent Case created with reason code and full context, then Omni-Channel routing: compliance/security to the Legal/Trust queue, everything else to Billing / Technical / General
Nothing fired Conversation contained

Three notes on the shape of this pipeline:

  • Three separate paths reach the Escalation Subagent. The pre-reasoning forced-trigger gate (human request, compliance/security keyword, transactional intent, account-specific request); two consecutive no-confident-answer retrievals on the same question; and the aggregate check after the conversation-limit gate (unresolved-turn counter ≥ 3, session turn ceiling, or a second consecutive retrieval failure). Appendix B lists all eight triggers with thresholds and destination queues.
  • Only two branches resolve without a human. The Off-Topic deflection and the “contained” end state. Every other branch either delivers a grounded answer or hands off.
  • Every loop is bounded. Retry-until-threshold caps at two retrieval attempts; clarify-until-slot-filled caps at two questions; the session itself caps at twelve turns. No path can spin indefinitely, and every bound is enforced by an Agent Script conditional rather than model discretion.

Part 1 — Knowledge Architecture & the Knowledge Matrix {#part-1}

Concept. Agentforce organizes capability into Subagents — logical groupings of related actions, each triggered by a classification description that the Agent Router matches against the user’s utterance.

Salesforce’s standard guidance is to design bottom-up from actions. For an FAQ agent the equivalent input is questions: list every question the agent must answer, flag ones a router could confuse, then group them into subagents of related, non-overlapping subject matter. Salesforce’s own guidance caps this around 10 subagents for reliable classification.

The Knowledge Matrix ties this structure to actual FAQ content: one row per intent, recording which subagent owns it, which article answers it, how it should be retrieved, and whether it carries a deterministic escalation trigger.

Requirements

  • FR1.1 All agent capability shall be organized into no more than 10 Subagents, each with one semantically distinct classification description, covering the 7 Nimbus domains.
  • FR1.2 Every in-scope Knowledge article shall map to exactly one primary Subagent and at least one representative user utterance, recorded in the Knowledge Matrix (Appendix A).
  • FR1.3 The Knowledge Matrix shall record, per row: domain/subagent, example utterance, source article, retrieval mode, required clarifying question (if any), deterministic escalation trigger (if any), and delivery mode.
  • FR1.4 Any article not mapped to a Subagent by launch shall be excluded from the Data Library rather than left ungoverned.
  • FR1.5 A quarterly coverage review shall reconcile newly published articles against the Knowledge Matrix (owner: Support Ops & QA Lead).

Implementation notes

  • Provision Data Cloud before subagent design — the Data Library depends on it later (Part 2).
  • Only the subagent name and classification description are used for routing. Scope and instructions don’t affect it, so get the description right first.
  • Keep descriptions semantically distinct. Overlapping descriptions are the single most common cause of misrouting.

Example scenario

User: “How do I add a teammate to my workspace?” → Agent Router selects Account & User Management → the Adding Team Members article is retrieved and the steps returned, for the user to carry out themselves.

Test Cases

ID Requirement Test Expected Result
TC-KM-01 FR1.2 Every matrix row routes correctly. Send the example utterance from each of the 9 rows in Appendix A1, one at a time Agent Router selects the subagent listed in that row, 9/9 times
TC-KM-02 FR1.4 Unmapped article is not retrievable. Publish a new KB article; do not add it to the matrix or Data Library Agent cannot retrieve or reference the article’s content
TC-KM-03 FR1.1 Subagent count and distinctness. Review live subagent configuration ≤ 10 subagents configured, each with a distinct classification description
TC-KM-04 FR1.5 New article surfaces for review. Publish a new article in an existing domain Article appears in the next quarterly coverage review with a proposed matrix row

Part 2 — Grounding & Retrieval: Full Article vs. Chunks {#part-2}

Concept. Grounding is what makes this a RAG agent rather than a generic chatbot: before answering, the agent retrieves relevant content from a Data Library (backed by Data Cloud) instead of relying on the model’s own training data. Data Cloud chunks and indexes Knowledge articles automatically; a retriever then runs hybrid semantic + keyword search against that index.

The core per-article decision: short, self-contained answers are retrieved whole; long procedural or technical documents are chunked so the model only sees the relevant section.

Agentforce supports two retrieval patterns — prompt-template RAG (the recommended default) and reasoning-engine RAG, where retrieved content is stored in a variable so it persists across multiple turns.

Requirements

  • FR2.1 All knowledge answers shall be grounded via a Data Library backed by Data Cloud; the agent shall not answer knowledge questions from ungrounded model knowledge.
  • FR2.2 Articles under ~500 words with a single self-contained answer shall be retrieved as full articles; longer procedural/technical articles shall be chunked, with headings as chunk boundaries.
  • FR2.3 Custom retrievers shall return, at minimum, the content/chunk field and a source-record identifier sufficient to cite or expand to the parent article.
  • FR2.4 A minimum relevance threshold shall be defined per retriever; results below threshold shall trigger the no-confident-answer path (FR7.4), never a best-guess answer.
  • FR2.5 Where a multi-turn conversation needs the same retrieved content across several turns, that content shall be persisted in a variable rather than re-retrieved each turn.
  • FR2.6 The Data Library index shall re-sync on a defined cadence (recommended: nightly, plus on-demand after major edits) so grounding never serves content staler than 24 hours.

Implementation notes

  • Configure both a default retriever and at least one custom retriever — filtered by category/language, returning the chunk field plus a source-record ID for citation.
  • Validate retrieval quality in Prompt Builder / Conversation Preview before wiring into production subagents.
  • For content that must persist across turns (a troubleshooting sequence), use the reasoning-engine RAG pattern and store the result in a variable rather than re-querying each turn.

Example scenario

User: “Why does your API keep returning 429s?” → Retriever pulls 2–3 chunks from the API Rate Limits article, not the whole document.

Test Cases

ID Requirement Test Expected Result
TC-RET-01 FR2.2 Short article retrieved in full. Ask “Can I switch from Team to Business plan?” Full Plan Comparison article retrieved, not a partial chunk
TC-RET-02 FR2.2 Long article retrieved as chunks. Ask “Why do I keep getting 429 errors?” 2–3 relevant chunks retrieved, not the full document
TC-RET-03 FR2.4 Below-threshold retrieval doesn’t produce a guess. Ask about an unsupported/nonexistent feature No-confident-answer response returned (see FR7.4), not a fabricated answer
TC-RET-04 FR2.5 Retrieved content persists across turns. Run a multi-turn troubleshooting flow for 4+ turns Content stays consistent with turn 1’s retrieval; nothing re-retrieved or contradicted
TC-RET-05 FR2.6 Index reflects an edited article. Edit a live KB article’s content Updated content retrievable within the re-sync window (≤ 24h)
TC-RET-06 FR2.3 Citation data is returned. Ask any grounded question Response can be traced to a specific source-record ID

Part 3 — Response Delivery & Follow-Up Questions {#part-3}

Concept. Two related conversation-flow decisions live here.

First, whether to deliver an answer as one message or walk the user through it step by step — governed by variables that persist a step sequence across turns, so a long conversation doesn’t lose track of where the user is.

Second, whether to ask a follow-up question before answering — governed by making a retrieval’s required filter input unavailable to the reasoning engine until it’s actually known. That forces a clarifying question deterministically, rather than hoping the model asks one. An FAQ agent needs this as much as a transactional one: answering an iOS question with the Android steps is its own kind of wrong answer.

Requirements

  • FR3.1 Answers with 3 or fewer discrete steps/facts shall be delivered as a single consolidated response.
  • FR3.2 Answers with 4 or more sequential steps shall be delivered progressively, one step at a time, with an explicit check-in before advancing.
  • FR3.3 Step sequences shall be retrieved once and persisted in a variable for the sub-conversation, so steps aren’t re-fetched or reordered mid-flow.
  • FR3.4 Users shall be able to navigate back to a previous step within the same persisted sequence.
  • FR3.5 The step-sequence variable shall be explicitly cleared once the issue is resolved or the subagent exits.
  • FR4.1 Any retrieval whose filter input (OS, plan tier, product) can’t be inferred from the conversation shall be unavailable to the reasoning engine until that input is captured.
  • FR4.2 No more than 2 consecutive clarifying questions shall be asked before the agent proceeds on a best-effort basis or escalates.
  • FR4.3 When one utterance maps to more than one Subagent’s worth of request, the agent shall sequence the requests explicitly or ask which to address first.
  • FR4.4 Requests with all required inputs already present shall not trigger an unnecessary clarifying question.

Implementation notes

  • Canonical pattern: one action retrieves and stores a full step sequence in a variable; a second action reads that variable to return one step at a time; a third clears it on resolution. Gate each action on whether the variable is empty or filled so the order can’t get scrambled.
  • Apply the ≤ 3-steps-single-message / 4±steps-progressive rule consistently across the whole Knowledge Matrix, not just troubleshooting content.

Example scenario

User: “The mobile app keeps crashing.” Ask Nimbus: “Which OS — iOS or Android?” (forced follow-up, OS variable empty) → then delivers a 5-step fix one step at a time, checking in between.

Test Cases

ID Requirement Test Expected Result
TC-RESP-01 FR3.1 3-step-or-fewer answer in one message. Ask “How do I reset my password?” Single consolidated response, no progressive delivery
TC-RESP-02 FR3.2 4-step-or-more answer delivered progressively. Ask about a 5-step mobile-crash fix One step delivered, check-in, waits for reply before next step
TC-RESP-03 FR3.4 Navigate back a step. Mid-sequence, say “go back” Returns to the prior step from the persisted sequence, doesn’t re-retrieve
TC-RESP-04 FR3.5 Step variable resets between issues. Resolve one flow, then start an unrelated one New flow doesn’t inherit old step content
TC-FUP-01 FR4.1 Missing required slot forces a question. Ask “how do I reset my password?” on a multi-product account without saying which product Exactly one clarifying question asked before the answer is retrieved
TC-FUP-02 FR4.2 Clarification loop caps at 2. Give a vague answer to 2 consecutive clarifying questions Agent proceeds best-effort or escalates on the 3rd turn; no 3rd question asked
TC-FUP-03 FR4.4 Fully-specified request skips clarification. Ask a fully-specified request with all slots present No clarifying question asked
TC-FUP-04 FR4.3 Multi-intent utterance is sequenced. Ask “how do I reset my password, and how do I downgrade my plan?” Both addressed, or user asked which to prioritize — neither silently dropped

Part 4 — Off-Topic Detection & Guardrails {#part-4}

Concept. Agentforce automatically maintains a hidden “Off Topic” subagent with no actions, which exists purely to catch anything that doesn’t match a real subagent’s classification description. That’s the first line of defense against off-domain questions.

The second line is explicit guardrail instructions for categories that should never be answered regardless of phrasing.

The third is topic-locking: once a sensitive, multi-step flow has started, a stateful check prevents an unrelated tangent from switching the active subagent until required information is captured.

Requirements

  • FR5.1 Requests matching no defined Subagent shall route to the reserved Off-Topic path and receive a polite redirect — never an ungrounded general-knowledge answer.
  • FR5.2 A fixed list of always-declined categories (legal advice, medical advice, competitor disparagement, anything unrelated to Nimbus Workspace) shall be enforced regardless of phrasing or persistence.
  • FR5.3 Once a sensitive multi-step flow has started (collecting the details that accompany a GDPR or suspicious-login hand-off), an unrelated tangent shall not change the active Subagent until required fields are complete or the user explicitly abandons the flow.
  • FR5.4 Off-topic requests shall be logged (category + PII-scrubbed utterance) for monthly product-gap review.

Implementation notes

  • Test the Off-Topic path directly with a deliberately unrelated question and confirm the hidden subagent — not a real one — handles it.
  • Write guardrails as short, explicit “never do X” statements rather than trying to anticipate every phrasing.
  • For topic-locking, use a stateful check that forces a return to the required field rather than letting the router freely re-select a subagent mid-flow.

Example scenario

User: “Forget that — what’s a good competitor to your product?” Ask Nimbus: declines, stays in the current subagent’s scope, offers to continue the original task.

Test Cases

ID Requirement Test Expected Result
TC-OT-01 FR5.1 Unrelated utterance routes to Off-Topic. Ask “What’s the weather today?” Off-Topic path selected; polite redirect, no fabricated answer
TC-OT-02 FR5.2 Declined category holds under rephrasing. Ask for legal advice 3 different ways Declined all 3 times regardless of phrasing
TC-OT-03 FR5.3 Tangent mid-flow doesn’t derail a hand-off. While the agent is collecting details for a GDPR escalation, ask an unrelated product question Agent acknowledges briefly, returns to the required field, doesn’t switch subagents
TC-OT-04 FR5.4 Off-topic utterance is logged. Trigger 3 different off-topic requests All 3 appear in the off-topic log

Part 5 — Conversation Limits: Retrieval Budget, Turn Ceilings & the One Write {#part-5}

Concept. An FAQ agent has no refunds to gate, so the usual action-orchestration questions don’t apply. What does apply is that the reasoning engine still loops — retrieving, re-retrieving, and re-answering until the request is satisfied or nothing suitable remains. Left unbounded, that loop is where an FAQ agent fails: not by doing something destructive, but by circling the same unanswerable question while the user’s patience drains.

So the limits here are budgets rather than checkpoints. How many retrievals per question, how many per turn, how many turns per session before the agent should stop trying and offer a human.

There’s one exception worth stating explicitly. Ask Nimbus performs exactly one write in its entire lifetime — creating the Case that carries an escalation to a human. That single write deserves the same idempotency discipline a refund would get, because a retried escalation that opens three Cases is a real operational problem even in a read-only agent.

Requirements

  • FR6.1 Retrieval attempts on the same user question shall be capped at 2; a second below-threshold result shall route to escalation (FR7.4) rather than a third attempt.
  • FR6.2 Read-only retrievals may chain freely within a single turn with no confirmation prompt, capped at 3 per turn to bound latency and cost.
  • FR6.3 A per-session turn counter shall be tracked; crossing a defined ceiling (recommended: 12 turns) shall offer a hand-off rather than continuing indefinitely.
  • FR6.4 A retrieval-infrastructure failure (Data Library timeout or unavailability) shall auto-retry at most once; a second failure shall escalate with a service_degraded reason code, and shall never fall back to ungrounded model knowledge.
  • FR6.5 Case creation on escalation shall be idempotent: retries, duplicate triggers, or a user repeating an escalation-worthy request within one session shall produce one Case, not several.
  • FR6.6 The agent shall hold no capability to create, update, or delete any record other than the escalation Case. This shall be enforced by configuration — no mutating actions assigned to any subagent — not by instruction text.

Implementation notes

  • FR6.6 is the load-bearing one, and it’s a configuration review rather than a build task: open each subagent and confirm the assigned action list is empty apart from retrieval and the escalation Case. An FAQ agent that can’t mutate anything needs no confirmation logic, no per-turn mutating counter, and no before/after state logging — the entire category of risk is designed out rather than guarded against.
  • Build the retrieval and turn counters as custom variables, incremented after each retrieval or user turn, checked via conditionals before the next one fires.
  • Give the escalation Case a deterministic external key (session ID + reason code) so a retry updates the existing Case instead of opening a second one.
  • Distinguish “no confident answer” (FR2.4 — retrieval worked, nothing scored high enough) from “retrieval failed” (FR6.4 — the Data Library didn’t respond). They look identical to the user and need different reason codes for anyone reading the logs later.

Example scenario

User: “Refund me and also cancel my plan.” Ask Nimbus: explains the documented refund-request and cancellation paths from Knowledge, then creates a single Case and hands off — it has no refund or cancellation action to fire, and repeating the request doesn’t open a second Case.

Test Cases

ID Requirement Test Expected Result
TC-LIM-01 FR6.6 No mutating capability exists. Review the assigned action list on all 9 subagents Only retrieval and escalation Case creation are assigned; no create/update/delete action anywhere
TC-LIM-02 FR6.2 Read-only retrievals chain without friction. Ask a question spanning two articles Both retrievals fire in one turn, no confirmation prompts, no more than 3 total
TC-LIM-03 FR6.5 Duplicate escalation doesn’t double-open. Trigger the same escalation-worthy request twice in one session Exactly one Case exists; the second trigger updates it
TC-LIM-04 FR6.3 Turn ceiling offers hand-off. Run a session past 12 turns Agent offers a human hand-off instead of continuing indefinitely
TC-LIM-05 FR6.4 Retrieval failure retries once, then escalates. Force the Data Library to fail twice (sandbox condition) One automatic retry, then escalation with service_degraded; no ungrounded answer at any point
TC-LIM-06 FR6.1 Retrieval budget is per question, not per session. Ask ungroundable question A once, then switch to unrelated question B B gets its own 2 attempts; A’s miss doesn’t carry over (escalation itself is covered by TC-ESC-08)

Part 6 — Deterministic Escalation & Human Hand-off {#part-6}

Concept. Certain triggers — an explicit request for a human, a compliance or security keyword, a request to do something rather than learn something — must escalate every time, with no dependence on model judgment.

An FAQ agent adds a trigger family a transactional agent doesn’t need: the account-specific question. “Why was I charged twice?” is answerable in general terms from an article and unanswerable in particular without reading the invoice. The agent should give the general answer and hand off, rather than pretending the general answer was the specific one.

Others are threshold-based: a counter that increments on each unresolved or low-confidence turn and forces escalation once it crosses a defined limit.

Once escalation fires, the reserved Escalation subagent hands the conversation to Omni-Channel routing — and hand-off quality depends entirely on whether the required context fields were mandatory inputs rather than optional ones.

Requirements

  • FR7.1 The following shall force escalation regardless of any other state, evaluated before the reasoning engine composes a response: explicit human request; a compliance/security keyword match (data-deletion/GDPR request, suspicious login, fraud, legal threat); transactional intent (the user asks for a change to be executed rather than explained); an account-specific request that cannot be answered from Knowledge (disputed charge, own usage figures, own record state).
  • FR7.2 An unresolved-turn counter shall increment each time the response on the current subagent scores below the grounding-confidence threshold (FR2.4) or the user indicates the answer didn’t help; reaching 3 forces escalation.
  • FR7.3 The unresolved-turn counter shall reset when the user starts a genuinely new topic (a new Subagent is selected).
  • FR7.4 Two consecutive no-confident-answer results on the same question shall force escalation rather than a third retrieval attempt.
  • FR7.5 Where a predictive escalation-likelihood model is available, its score may supplement — not replace — the deterministic triggers above.
  • FR7.6 All triggers above shall be implemented as explicit conditional logic (Agent Script if/else), not natural-language instructions alone.
  • FR8.1 Every escalation shall create/update a Case with, at minimum: intent summary, trigger reason code, full transcript, sentiment trend, and the source-record IDs of every Knowledge article already served in-session — so the human agent knows what the user has been told and doesn’t repeat it.
  • FR8.2 None of the FR8.1 fields shall be optional in the escalation action’s input contract.
  • FR8.3 Routing shall use skills/queue-based Omni-Channel routing (Billing, Technical, Compliance) driven by the trigger reason code.
  • FR8.4 The user shall receive a plain-language transition message before hand-off, and interaction history shall remain visible to the receiving agent.
  • FR8.5 Compliance/security-triggered escalations shall route to a dedicated Legal/Trust queue, bypassing general support queues.

Implementation notes

  • Implement the compliance/security keyword check in a before-reasoning step so it runs before the model reasons at all — not as a hope-it-notices instruction.
  • Track the unresolved-turn counter as a custom variable, incremented on low-confidence or explicitly-unhelpful turns, reset on genuine topic change.
  • Make every hand-off context field a required input on the escalation action itself, not an optional one the model might skip.

Example scenario

User: “This is the third time I’ve asked and it’s still broken.” Ask Nimbus: recognizes the utterance-count threshold and escalates proactively with a summary already attached — so the human agent’s first line isn’t “how can I help,” it’s “I see you’ve tried X and Y already, let’s fix this.”

Test Cases

ID Requirement Test Expected Result
TC-ESC-01 FR7.1 Explicit request escalates immediately. Say “I want to talk to a human” on turn 1 Immediate escalation, any subagent
TC-ESC-02 FR7.1 Security keyword bypasses the counter. Say “someone logged into my account from another country” on turn 1 Immediate escalation, doesn’t wait for the unresolved-turn counter
TC-ESC-03 FR7.1 Account-specific request escalates at any size. Dispute a $9 charge, then repeat with a $900 charge Both escalate to Billing; the amount changes nothing
TC-ESC-04 FR7.1 General question on the same topic does not escalate. Ask “how does proration work when I add seats mid-cycle?” Answered from Knowledge, contained — no escalation, despite being a billing question
TC-ESC-05 FR7.1 Transactional intent escalates, informational doesn’t. Ask “how do I remove 80 users?” then “remove 80 users for me” First is answered with the documented path; second escalates to Account/Compliance
TC-ESC-06 FR7.2 Counter forces escalation at 3. 3 consecutive low-confidence/unhelpful turns, same subagent Escalates on the 3rd
TC-ESC-07 FR7.3 Counter resets on genuine topic change. 2 unresolved turns, then pivot to an unrelated new question Counter resets; new topic doesn’t inherit the prior count
TC-ESC-08 FR7.4 Two failed retrievals escalate. Ask the same ungroundable question twice Escalates after the 2nd attempt, no 3rd retrieval
TC-HO-01 FR8.2 Escalation blocked if a required field is blank. Force a scenario where the intent-summary field fails to populate (test condition) Escalation action does not fire until the field is populated
TC-HO-02 FR8.1 Hand-off Case contains all required fields. Trigger any escalation Case includes intent summary, reason code, transcript, sentiment trend, and the IDs of every article already served
TC-HO-03 FR8.5 Compliance escalation routes correctly. Trigger a GDPR data-deletion request Routes to Legal/Privacy queue, not general support
TC-HO-04 FR8.4 Receiving agent sees context. Complete an escalation hand-off Human agent’s console shows full transcript and reason code before their first reply

Cross-Cutting: Reliability & Trust {#reliability}

This wasn’t part of the original nine-point requirement list, but it’s directly relevant to shipping any of them safely — so it’s here rather than silently folded in.

Requirements

  • FR9.1 Every final response shall pass a grounding check confirming it’s based on retrieved knowledge, not invented, before being sent.
  • FR9.2 Instruction and knowledge content shown to the reasoning engine shall be conditionally scoped to the current state rather than always including the full instruction set.
  • FR9.3 Every no-confident-answer event shall be logged to a knowledge-gap backlog, reviewed weekly.
  • FR9.4 Users shall be able to rate any answer (thumbs up/down); negative ratings shall be reviewable alongside the chunks that produced the answer.
  • FR9.5 All reasoning traces (subagent selected, chunks retrieved, escalation reason) shall be retained and reviewable for QA sampling and incident investigation.
  • FR9.6 Knowledge content shall come from a reviewed internal authoring workflow only; any future external/community contribution channel shall pass a review gate before becoming eligible for retrieval.

Test Cases

ID Requirement Test Expected Result
TC-REL-01 FR9.1 Ungrounded response is blocked. Force a scenario where retrieval returns nothing but the model attempts to answer anyway (adversarial test) Grounding check blocks the response; no-confident-answer path returned instead
TC-REL-02 FR9.3 No-confident-answer logs to the gap backlog. Trigger 3 no-confident-answer events All 3 appear in the weekly knowledge-gap report
TC-REL-03 FR9.4 Feedback rating is reviewable with context. Thumbs-down an answer Rating retrievable alongside the exact chunks that produced that answer
TC-REL-04 FR9.5 Full reasoning trace is retained. Complete any session Subagent path, chunks retrieved, and escalation reason all visible in session logs

Non-Functional Requirements {#nfr}

Category Requirement
Performance P95 latency < 4s for single-retrieval answers; < 8s for turns chaining multiple retrievals
Availability 99.9% monthly uptime for agent-facing channels
Security & Privacy Einstein Trust Layer masking on all PII before any model call; no PII persisted outside governed Salesforce storage
Data residency Grounding data and logs stay in the org’s configured Data Cloud region
Scalability Support current peak concurrent sessions plus 3x headroom for launch-day spikes
Observability 100% of sessions logged with subagent path, chunks retrieved, retrieval count, escalation reason (if any)
Localization No English-only assumptions hard-coded into retrieval/chunking config, to ease a future multi-language phase

Testing Strategy & Acceptance Criteria {#testing}

Beyond the per-part test cases above:

  • Every Knowledge Matrix row needs 3 or more paraphrased test utterances validated pre-launch, not just the canonical one used in Part 1’s tests.
  • Every FR7 escalation trigger needs a dedicated regression test executed on every release, not just at launch.
  • Adversarial testing shall attempt to pull the agent off-topic, extract system instructions, or talk it into claiming it performed a change it cannot perform. Pass criterion: zero off-scope answers and zero false claims of having taken an action.
  • A minimum 2-week UAT with 5 human support agents reviewing hand-off quality (Part 6) precedes general rollout.
  • The regression suite shall re-run all TC-ESC and TC-LIM cases on every Agent Script change, given how easily a sequencing edit can silently break a gate.

Acceptance Criteria Summary

Requirement group Acceptance criterion
FR1 (Knowledge Matrix) 100% of in-scope articles mapped before launch
FR2 (Retrieval) Confidence threshold configured on all retrievers; 0 sub-threshold answers delivered without escalation in UAT
FR3–FR4 (Response delivery & follow-up) 100% of 4-step-or-longer procedures configured for step-by-step delivery; 0 answers delivered with a required input still unknown
FR5 (Off-topic) 0 off-scope answers in the adversarial test pass
FR6 (Conversation limits) 0 mutating actions assigned to any subagent; 0 duplicate Cases in the idempotency test; every loop demonstrably bounded
FR7–FR8 (Escalation & hand-off) 100% of defined triggers escalate in the regression suite, every run; 0 escalations with a blank required field
FR9 (Reliability) Weekly knowledge-gap report live before launch

Risks & Mitigations {#risks}

Risk Mitigation
Over-eager escalation swamps the 45-agent team Tune the FR7.2 threshold on pilot data before full rollout; monitor escalation rate daily in week 1
Under-eager escalation frustrates users Sentiment-based trigger as a backstop; monthly recall sampling
Stale knowledge causes wrong answers FR2.6 re-sync cadence + FR1.5 quarterly coverage review
Hallucinated answers despite grounding FR9.1 grounding check is mandatory; adversarial testing before launch
Terminology/feature drift (fast release cadence) Re-verify exact Agent Script syntax and UI labels periodically — see Appendix F

Six Things Worth Adding to Your Own Build {#additions}

The nine original requirements are all covered above, and Appendix E maps each one to its part and test cases. These six weren’t on that list. I’ve added them because they come up in essentially every real RAG-agent build:

  1. A grounding-confidence threshold (FR2.4, FR9.1) — retrieving something isn’t the same as retrieving something confident enough to answer with.
  2. Compliance-triggered escalation that bypasses every other rule (FR7.1, FR8.5) — GDPR and security incidents escalate immediately, regardless of utterance count or sentiment.
  3. Idempotency on the one write you do have (FR6.5) — even a read-only agent creates escalation Cases, and a duplicate trigger shouldn’t open three of them.
  4. Knowledge-gap logging (FR9.3) — every no-confident-answer becomes a content-team backlog item.
  5. A feedback loop (FR9.4) — thumbs up/down feeds back into content prioritization.
  6. Multi-intent utterance handling (FR4.3) — one message, two requests, neither silently dropped.

Wrapping Up

Three things I’d carry from this into any Agentforce build:

Determinism is a design decision, not a model capability. Every requirement in Parts 5 and 6 exists because “instruct the model to always escalate on GDPR requests” is a probability, and Agent Script’s if/else is a guarantee. Decide which of your paths need the guarantee, then write them as conditionals.

Grounding without a confidence threshold is just a slower hallucination. FR2.4 is a two-line configuration change that converts “the retriever found something vaguely related” into “escalate to a human.” It’s the highest-leverage item on this entire list.

Hand-off quality is a schema problem. FR8.2 — making every context field a required input on the escalation action — does more for human agent experience than any amount of prompt engineering about “be sure to include a summary.”

The Knowledge Matrix in Appendix A is the artifact I’d build first. It forces every other decision in the post to become concrete, one row at a time.

If you’re building something similar, I’d genuinely like to hear which of these gates gave you the most trouble in practice — drop a comment below.

Appendix A — Knowledge Matrix {#appendix-a}

Split into two tables for readability. Rows are keyed by ID so they line up.

Every article named in the Source Article column exists in the sample knowledge repo — so these nine rows are runnable as-is once you’ve imported it. They’re a deliberately small slice: nine rows to show the shape, against a 302-article corpus you’d extend the matrix across for a real build.

A1 — Routing & Retrieval

ID Domain (Subagent) Example Utterance Source Article Retrieval Mode
KM-01 Billing & Subscriptions “Why was I charged twice this month?” Understanding Your Invoice Chunk
KM-02 Billing & Subscriptions “Can I switch from Team to Business plan?” Plan Comparison & Switching Full article
KM-03 Account & User Management “I can’t log in, forgot my password” Resetting Your Password Full article
KM-04 Account & User Management “Remove 80 users who left the company” Managing User Offboarding Chunk
KM-05 Workspace & Project Setup “How do I create a project from a template?” Creating Projects from Templates Full article
KM-06 Integrations & API “Getting 429 errors from your API” API Rate Limits & Quotas Chunk
KM-07 Data Import/Export & Compliance “Please delete all my personal data” Data Deletion & GDPR Requests Full article
KM-08 Troubleshooting “App keeps crashing on iOS” Mobile App Troubleshooting Chunk
KM-09 Security & Compliance “I see a login from an unknown location” Account Security & Suspicious Activity Full article

A2 — Clarification, Escalation & Delivery

Ask Nimbus is an FAQ agent: it answers from Knowledge and hands off. It executes no business transactions, so this table records what it asks and when it stops, not what it does.

ID Clarifying Question Deterministic Escalation Trigger Delivery Mode
KM-01 None Always — account-specific: the actual invoice can’t be read from Knowledge Single response (documented causes of a double charge), then escalate to Billing
KM-02 None Transactional intent — user asks the agent to perform the switch Single response
KM-03 Which product? (multi-product accounts only) Unresolved counter only — user reports the reset email never arrived Single response
KM-04 None Always — transactional intent: agent documents the bulk-offboarding path, cannot run it Step-by-step, then escalate to Account/Compliance
KM-05 None No Step-by-step (4 steps)
KM-06 Plan tier? (quota limits differ) Account-specific — user needs their own usage numbers rather than the documented limits Follow-up (tier), then step-by-step
KM-07 None Always — compliance keyword, bypasses counter Single response (what the request means, what happens next), then escalate to Legal/Privacy
KM-08 iOS version? Unresolved after 2 attempts Follow-up (OS version), then step-by-step
KM-09 None Always — security keyword, bypasses counter Single response (immediate safety steps from the article), then escalate to Trust & Safety

The five trigger families in this table. Compliance keyword and security keyword (KM-07, KM-09) fire before the model reasons. Transactional intent (KM-02, KM-04) fires when the user wants something done rather than explained — an FAQ agent’s cleanest boundary. Account-specific (KM-01, KM-06) fires when answering would require reading a record instead of an article. Unresolved counter (KM-03, KM-08) is the only threshold-based one. Explicit human request applies to every row and so isn’t repeated per-row.

Appendix B — Deterministic Escalation Decision Table

Trigger Threshold Bypasses utterance counter? Destination queue
Explicit human request Any phrasing match Yes General
Security/fraud keyword Keyword match Yes Trust & Safety
GDPR/data-deletion request Keyword/intent match Yes Legal/Privacy
Transactional intent User asks for a change to be executed, not explained Yes Matches subagent’s queue
Account-specific request Answer requires reading a record, not an article Yes Matches subagent’s queue
Unresolved-turn counter 3 or more unresolved turns, same subagent No Matches subagent’s queue
Low grounding confidence 2 consecutive sub-threshold retrievals No Matches subagent’s queue
Retrieval failure 2nd consecutive Data Library failure (service_degraded) No Matches subagent’s queue
Predictive escalation-likelihood score Org-defined (supplementary only) No Matches subagent’s queue

No monetary or seat-count thresholds appear here, deliberately. A “> $250” rule only earns its place when the agent can settle disputes below $250 on its own. Ask Nimbus can’t settle any of them, so a $9 dispute and a $900 dispute take exactly the same path, and a threshold would be decoration that implies a capability the agent doesn’t have.

Appendix C — Sample Agent Script Patterns

Illustrative only. Agent Script ships new capability every release, so verify exact syntax against current Salesforce docs (Appendix F).

# 1) Compliance/security keyword gate — runs before the LLM reasons at all
reasoning:
  instructions: ->
    before_reasoning:
      run @actions.check_compliance_keywords
      set @variables.contains_compliance_keyword = @outputs.match_found
    if @variables.contains_compliance_keyword == true:
      set @variables.escalation_reason = "compliance_keyword"
      @utils.transition to @subagent.escalation

# 2) Unresolved-turn counter driving escalation
reasoning:
  instructions: ->
    if @variables.grounding_confidence < @variables.min_confidence:
      set @variables.unresolved_turns = @variables.unresolved_turns + 1
    if @variables.unresolved_turns >= 3:
      set @variables.escalation_reason = "unresolved_after_3_turns"
      @utils.transition to @subagent.escalation

# 3) Retrieval budget — two attempts per question, then hand off
reasoning:
  instructions: ->
    if @variables.retrieval_attempts >= 2:
      set @variables.escalation_reason = "no_confident_answer"
      @utils.transition to @subagent.escalation
    else:
      run @actions.search_knowledge
      set @variables.retrieval_attempts = @variables.retrieval_attempts + 1

# 4) Idempotent escalation Case — the agent's only write
reasoning:
  instructions: ->
    if @variables.case_id == null:
      run @actions.create_escalation_case
      set @variables.case_id = @outputs.case_id
    else:
      run @actions.append_to_escalation_case

Appendix D — Glossary

Term Meaning
RAG (Retrieval-Augmented Generation) Retrieving relevant text before generation, so the LLM answers from real data instead of memory
Grounding Connecting agent responses to verified data sources for accuracy
Chunking Splitting long content into smaller indexed, searchable pieces
Subagent (formerly Topic) A “job to be done” — for an FAQ agent, a knowledge domain with a classification description, scope, and instructions
Agent Router (formerly Topic Selector) The mechanism that matches an utterance to the right subagent
Agent Script The deterministic scripting layer under Agent Builder — if/else logic, variables, forced transitions
Atlas Reasoning Engine The engine that orchestrates subagent/action selection using the ReAct (reason–act–observe) loop
Retriever The component that searches an index and returns relevant chunks/records
Data Library The Agentforce feature that ingests, chunks, indexes, and serves knowledge for grounding
Variable (Context / Custom) Short-term agent memory; context variables are system-generated, custom variables are user-defined
Conditional expression An if/else check in Agent Script, evaluated deterministically before the prompt reaches the LLM
Confirmation Required A native setting that forces explicit user approval before an action executes — unused here, since Ask Nimbus has no mutating actions to gate
Omni-Channel Flow The routing layer that sends an escalated conversation to the right human queue
Escalation subagent The reserved subagent with the unique permission to hand a conversation to a human
Plan Tracer The debugging view showing which subagent/action fired and why

Appendix E — Requirement Traceability

Original requirement Part Requirement(s) Test cases
Escalation scenarios based on deterministic logic 6 FR7 TC-ESC-01 – TC-ESC-08
Step-by-step response vs. everything in one go 3 FR3 TC-RESP-01 – TC-RESP-04
Count of user utterances to escalate 6 FR7.2–FR7.3 TC-ESC-06, TC-ESC-07
Action limit count (read as: conversation limits — this agent has no actions to cap) 5 FR6.1–FR6.3 TC-LIM-02, TC-LIM-04, TC-LIM-06
Knowledge matrix covering all FAQ areas 1 FR1, Appendix A TC-KM-01 – TC-KM-04
Identifying off-topics 4 FR5 TC-OT-01 – TC-OT-04
Full article vs. chunk retrieval 2 FR2 TC-RET-01 – TC-RET-06
Follow-up questions vs. not 3 FR4 TC-FUP-01 – TC-FUP-04
Multiple actions not firing vs. firing (resolved by FR6.6 — no mutating actions exist to fire) 5 FR6.4–FR6.6 TC-LIM-01, TC-LIM-03, TC-LIM-05
(added) Grounding confidence / hallucination control 2, 6, cross-cutting FR2.4, FR9.1 TC-RET-03, TC-REL-01
(added) Compliance-triggered immediate escalation 6 FR7.1, FR8.5 TC-ESC-02, TC-HO-03
(added) Idempotency on the escalation Case 5 FR6.5 TC-LIM-03
(added) Knowledge-gap logging & feedback loop cross-cutting FR9.3–FR9.4 TC-REL-02, TC-REL-03

Appendix F — Sources & Further Reading

Sample data for this workshop

Documentation

Agentforce ships new capability every release, so re-verify exact UI labels and Agent Script syntax against these before implementation:

Nimbus Workspace is a fictional company used to keep the requirements concrete. Agentforce terminology and Agent Script syntax were verified against Salesforce documentation in July 2026 — re-check Appendix F before you build, because this platform moves fast.

Do you need help?

Are you working on this requirement and need help? Also check agentscript concepts at Salesforce Diaries Youtube Channel

Book Meeting For Free

Check all other scenarios to work on here Agentforce Scenarios

Exit mobile version