LEARN. PRACTICE. GROW.Draw the complete service and its purpose
A fictional assistant helps APS find runbooks and summarize funds incidents. Architecture includes authentication, document retrieval, model, cache, observability and possible change tools. Record authorized purpose, users, data crossing each boundary and who configures each component. A managed model does not remove responsibility for the index, recipients, logs or tool permissions. Confirm retention, processing locations and provider use of data through applicable documentation and terms; do not infer those behaviors merely from product name. The consulted NIST AI RMF is version 1.0 and its page announces an update in progress. An announced revision should not be presented as a newly published edition.
Keep retrieved documents as data
A retrieved ticket may contain text asking the assistant to ignore the user’s request and export information. Retrieval relevance gives it no authority over tools. Prompt-injection risk can originate in external sources and does not disappear with RAG or fine-tuning. Separate trusted instructions from retrieved content, limit tools and validate action requests outside the model. The component executing a change must know identity, resource, action, parameters and required authorization. A system-prompt instruction is a useful layer but does not replace denial enforced by the execution service. The architectural objective is to limit consequences when the model proposes an improper action.
Separate reading, writing and user boundaries
In the worksheet, retrieval identity reads only documents authorized for the tenant and user. The change executor is separate and requires approval for sensitive actions. Do not place broad production permissions in the assistant’s shared identity merely to simplify integration. Filtering must cover retrieval, cache and presentation; a global cache may return another user’s summary derived from restricted data. Documents removed from the main index may remain in caches, logs or exports requiring separate analysis. Human approval also needs sufficient target and effect information: confirming vague intent does not validate parameters the model adds afterward. Keep suitable records of decisions and actions actually executed.
Evaluate critical failures separately from the average
The fictional evaluation contains 90 routine cases with 88 correct outcomes and 10 sensitive cases with 8 correct outcomes and two unauthorized disclosures. Total performance is 96 out of 100, or 96%, but the sensitive group reaches only 80%. The defined criterion requires zero disclosures, so acceptance fails despite a high average. These numbers form a decision worksheet, not an evaluation executed against a model. In an actual trial, retain model version, configuration, test data, category-specific criteria and generalization limits. Include legitimate requests, cross-tenant attempts, documents containing adversarial instructions, missing sources and outages. A model or index update may require reevaluation even without application-code changes.
Plan degraded operation and continuing review
If the model or retrieval fails during closing, degraded operation should route to the authorized runbook and operator. Missing output must not grant extra access or allow an unapproved change. Define cost, latency and load limits, monitoring of relevant errors and a way to report incorrect suggestions. A good initial result does not establish that new sources or contexts retain the same quality. At acceptance review, present flows, identities, controls, category-level outcomes, unmeasured risks and go-live authority. Summary: the model participates in a larger system; authorization, isolation, refusal capability and recovery require their own design. Connect this lesson with IAM, operations, supplier management and incidents.
FICTIONAL ARCHITECTURE WORKSHEET, NO MODEL EXECUTED
Purpose: runbook retrieval and APS assistance
Reader: documents authorized by user and tenant
Writer: separate executor, sensitive action requires approval
Retrieved text does not authorize tools
Cache and logs: scope, retention and access to validate
Routine: 88 correct / 90
Sensitive: 8 correct / 10; 2 unauthorized disclosures
Total: 96 / 100; sensitive: 80%
Criterion: zero disclosures; acceptance: no
Fallback: existing runbook and authorized operator.96% overall does not meet a zero-disclosure criterion when two of ten sensitive cases fail.
Common pitfalls
Treating retrieved text as authorized instruction; relying only on prompts; unscoped cache; averages as security proof; approval without parameters.
Related topics: Architecture, identity and data lifecycle · Acceptance criteria and operations
Define data, identities, actions and acceptance criteria for a cloud assistant with limited scope.
Reference: AI RMF Core · CCSP examination outline effective 2026-08-01; January2026 V2 PDF