Skip to content

Control 2.6: Model Risk Management (OCC 2011-12/SR 11-7)

Control ID: 2.6
Pillar: Management
Regulatory Reference: OCC Bulletin 2011-12 (Sound Practices for Model Risk Management), Federal Reserve SR 11-7 (Supervisory Guidance on Model Risk Management), FDIC FIL-22-2017 (FDIC adoption of OCC 2011-12), FFIEC IT Examination Handbook (Management; Information Security), FINRA Rule 3110 (Supervision), FINRA Rule 4511 (Books and Records), FINRA Regulatory Notice 25-07 (AI Tools — RFC, contextual only), SEC Rules 17a-3 / 17a-4 (Recordkeeping for model artifacts and validation evidence), SOX Sections 302 / 404 (Internal Controls over Financial Reporting), GLBA 501(b) (Safeguards Rule), NYDFS 23 NYCRR 500 (where AI agents touch covered customer information)
Last UI Verified: April 2026
Governance Levels: Baseline / Recommended / Regulated

Objective

Establish governance so that AI agents used in customer-facing or decision-support roles are subject to the same rigorous governance as traditional quantitative models, including model inventory, independent validation, ongoing performance monitoring, bias detection, and documented change control. This control supports compliance with OCC Bulletin 2011-12, Federal Reserve SR 11-7, FDIC FIL-22-2017, FFIEC IT Handbook, FINRA Rule 3110, FINRA Rule 4511, SEC Rules 17a-3 / 17a-4, SOX §§ 302 / 404, GLBA 501(b), and NYDFS 23 NYCRR 500 by providing a structured framework for treating AI agents as models within the firm's existing MRM program. It does not replace, and must not be presented in the firm's MRM policy or WSPs as a substitute for, the firm's Model Risk Management Committee, independent model validation function, effective challenge process, or three-lines-of-defense governance.

Non-Substitution — Tooling Supports MRM, It Does Not Replace It

The Microsoft Purview DSPM for AI inventory, Azure AI Foundry evaluation harness, Copilot Studio analytics, Agent 365 governance console, and Microsoft Entra Agent ID identity surfaces referenced in this control are inventory, evaluation, and evidence-collection surfaces. They do not replace, and must not be presented in the firm's MRM policy, WSPs, or examiner submissions as a substitute for:

  • The firm's Model Risk Management Committee and the formal model-tiering and approval decisions reserved to it under OCC 2011-12 / SR 11-7.
  • Independent model validation by personnel organizationally and functionally separate from the model owner / developer (SR 11-7 §V). "Independent" is the regulatory test — third-party engagement is one way to achieve it but is not required and is not equivalent.
  • Effective challenge — the critical, objective review by qualified personnel not involved in model development, which is the cornerstone of SR 11-7 and cannot be performed by any platform telemetry on its own.
  • Three-lines-of-defense governance — first line (model owner / developer), second line (independent MRM / validation / compliance), third line (Internal Audit). The Microsoft surfaces produce evidence; lines-of-defense judgments are made by people.
  • Registered-principal supervisory review under FINRA Rule 3110 of any AI-agent business activity (see Control 2.12).
  • Books-and-records retention of model artifacts under SEC 17a-4 / FINRA 4511 — Copilot Studio analytics and Foundry evaluation runs are not WORM-compliant for 17a-4(f) purposes; long-term model documentation and validation evidence must land in Purview retention or an approved 17a-4(f) vendor (see Controls 1.7 and 1.9).

Sovereign Cloud Availability — GCC, GCC High, DoD, China

As of the verification date in this document, several Microsoft surfaces this control depends on have parity gaps in sovereign clouds; FSI tenants operating in GCC, GCC High, DoD, or China-operated clouds must verify availability before relying on the automated path and must implement compensating manual controls until parity is confirmed:

  • Microsoft Purview DSPM for AI — verify regional availability and connector parity for Copilot, Copilot Studio, Foundry, and Entra-registered AI apps in your sovereign cloud.
  • Azure AI Foundry evaluation harness — verify Foundry availability and evaluator parity in Azure Government / DoD; some built-in safety evaluators rely on commercial-cloud-hosted judge models.
  • Agent 365 Admin Center governance console — Microsoft has not announced parity for the Agent 365 Admin Center, governance templates, or admin-gated publish/activate workflows in GCC, GCC High, or DoD as of GA. See Control 2.25.
  • Microsoft Entra Agent ID — verify lifecycle workflow parity in sovereign clouds. See Control 2.26.
  • Anthropic Claude models in Copilot Studio — third-party model availability in sovereign clouds is independently gated; verify per region.

Compensating control until parity is confirmed: maintain a manual model inventory (SharePoint list backed by Purview retention) covering every AI agent that meets the firm's model-classification criteria; perform validation by the firm's existing MRM function using the firm's existing model risk policy; retain validation evidence under 17a-4(f)-compliant retention. Re-verify the sovereign roadmap quarterly via the Microsoft 365 Government roadmap.

Why This Matters for FSI

  • OCC 2011-12: Establishes model risk management framework requirements for banks
  • Fed SR 11-7: Requires model development, validation, and ongoing monitoring (identical to OCC 2011-12; jointly issued)
  • FINRA Rule 3110: AI supervision and oversight requirements for broker-dealers
  • SOX Sections 302/404: Internal controls must include documented model controls (see SOX AI Governance below)
  • Interagency RFI on AI (2021): Confirmed OCC 2011-12 applies to AI/ML systems

SOX AI Governance and PCAOB Standards

SOX applies to AI systems implicitly through Internal Control over Financial Reporting (ICFR). AI agents that affect financial data, reporting, or controls must be documented and tested as IT general controls.

PCAOB AI Audit Standards (In Development):

  • PCAOB AS 1105 (Audit Evidence) and AS 2301 (Audit Samples) are under review for AI-specific guidance
  • July 2024 PCAOB statement indicated AI audit standards research in progress
  • Organizations should document AI system controls with sufficient detail to support external audit

AI System Documentation for SOX Compliance:

Documentation Element Purpose Retention
Agent Card (capabilities, limitations) Control documentation Life of system + 7 years
Validation test results Control testing evidence 7 years (SOX 802)
Change log with approvals Change management evidence 7 years (SOX 802)
Performance monitoring reports Ongoing control monitoring 7 years (SOX 802)

No companion solution by design

Not all controls have a companion solution in FSI-AgentGov-Solutions; solution mapping is selective by design. This control is operated via native Microsoft admin surfaces and verified by the framework's assessment-engine collectors. See the Solutions Index for the catalog and coverage scope.

Control Description

While Copilot Studio agents may not be "models" in the traditional sense, their use in customer-facing or decision-support roles requires similar governance.

Underlying Model Availability in Copilot Studio (verify in your tenant)

Copilot Studio's underlying foundation-model lineup changes frequently and varies by tenant region, license, and Microsoft release wave. As of the verification date in this document, Copilot Studio supports multiple OpenAI GPT-family models and has announced support for additional model providers (including Anthropic Claude) in commercial cloud; availability, GA status, default-model assignment, and sovereign-cloud parity must be verified in your tenant via Copilot Studio: Choose a generative AI model before publication.

For MRM purposes the firm must:

  • Record the specific underlying model and version in use for each agent in the model inventory and Agent Card (do not record only "Copilot Studio default").
  • Treat any change in underlying model — including a Microsoft-driven default-model migration — as a model change under SR 11-7 §VI, requiring documented impact assessment and re-validation per the agent's risk tier.
  • Subscribe to the Microsoft 365 Message Center and Power Platform release plans so that vendor-driven model changes are detected before they take effect (cross-reference Control 2.3 — Change Management).
  • Treat third-party model providers (e.g., Anthropic Claude when available) as vendor models under SR 11-7 §V — see the Vendor Model Governance section below.

Infrastructure vs MRM Framework

Microsoft provides infrastructure platforms (Dataverse, SharePoint, Power Platform, Application Insights) that can support MRM processes, but organizations must design and implement their own MRM frameworks. There is no built-in "MRM solution" that automatically classifies agents, schedules validations, or generates compliance reports. The capabilities below describe governance processes that leverage Microsoft infrastructure.

Capability Description Implementation FSI Application
Model Classification Determine if agent qualifies as "model" Organization-defined process using Dataverse/SharePoint Tier-based governance
Model Inventory Catalog agents that function as models Custom SharePoint list or Dataverse table Regulatory examination readiness
Independent Validation Third-party review of agent behavior Organization-managed process Compliance with SR 11-7
Performance Monitoring Track output quality over time Copilot Studio Analytics (built-in) + custom RAI metrics Early issue detection
Agent Cards Standardized documentation of capabilities/limitations Custom template in SharePoint/Dataverse Model transparency

Agent-as-Model Classification

Criteria Model Treatment Example
Makes decisions affecting customers Tier 1 Credit recommendation agent
Provides financial calculations Tier 1/2 Investment calculator agent
Customer-facing recommendations Tier 2 Product recommendation agent
Information retrieval only Non-model FAQ/knowledge base agent

Key Configuration Points

Organization-Designed MRM Framework

  • Classify all agents using OCC 2011-12 model definition criteria (organization-defined process)
  • Maintain model inventory with tier assignments, owners, and validation status (custom Dataverse table or SharePoint list)
  • Create Agent Cards documenting capabilities, limitations, and performance benchmarks (custom template)
  • Establish an independent validation program. SR 11-7 §V defines independence functionally — validation must be performed by personnel organizationally and functionally separate from the model owner / developer, with the authority and competence to challenge the model. Internal second-line MRM staff can satisfy this; third-party engagement is one way to demonstrate independence (often used for Tier 1 / customer-facing / decision-support agents) but is not required by SR 11-7 and is not a substitute for the firm's own validation judgment.
  • Define validation schedule and tracking mechanism (custom workflow or calendar integration)

Platform-Enabled Monitoring (mapped to SR 11-7 pillars)

SR 11-7 sets three validation pillars: conceptual soundness, ongoing monitoring, and outcomes analysis. Each maps to a specific Microsoft surface that produces evidence the firm's validators consume — the surface does not perform the validation.

SR 11-7 Pillar Primary Microsoft Surface What It Produces Where Validators Use It
Conceptual soundness Microsoft Purview DSPM for AI (inventory of agents, models, knowledge sources, prompts, and interactions) + Agent 365 Admin Center (publication approvals, governance template applied) + Agent Card Documented model purpose, design, data, intended use, limitations Pre-deployment validation memo and Model Risk Committee review
Ongoing monitoring Copilot Studio Analytics (conversational and autonomous-agent metrics) + Azure AI Foundry monitoring (when underlying model is on Foundry) + Application Insights / Azure Monitor + DSPM for AI interaction reports KPI drift, exception rate, sensitive-prompt rate, sentiment, customer-comment summaries Quarterly monitoring report to MRM Committee; threshold breaches feed Control 3.4 incident workflow
Outcomes analysis Azure AI Foundry built-in evaluators (groundedness, relevance, coherence, fluency; safety: hate/unfairness, violence, protected materials; agent-specific: tool-call accuracy, task completion) + Copilot Studio test panel + custom evaluators Quantitative quality / safety / agent-behavior scores against an evaluation dataset Periodic re-validation evidence; effective-challenge inputs

In addition:

  • Manifest / solution version control — use Power Platform solution export/import and Application Lifecycle Management pipelines for point-in-time reconstruction of agent configuration. Pair with Control 2.3 (Change Management).
  • Prompt and knowledge-source change capture — document every change to system prompt, instructions, or knowledge source (RAG corpus) with impact assessment, approver, and effective date. Cross-reference Control 2.16 (RAG Source Integrity).
  • Identity attribution — every agent in the model inventory must be tied to a Microsoft Entra Agent ID for attribution of actions to a governed identity. See Control 2.26.
  • Long-term retention of validation evidence — pre-deployment validation memos, ongoing monitoring reports, outcomes-analysis runs, and Model Risk Committee minutes are records of the business; route to Purview retention or an approved 17a-4(f) vendor for 6-year WORM retention. See Controls 1.7 and 1.9.

Implementation Reference

See Portal Walkthrough for example inventory schemas and Agent Card templates that organizations can adapt.

Zone-Specific Requirements

Zone Requirement Rationale
Zone 1 (Personal) Non-model classification typical; minimal MRM governance Personal productivity, no customer impact
Zone 2 (Team) May be model; internal validation; standard documentation Shared agents may influence decisions
Zone 3 (Enterprise) Likely model; independent third-party validation; comprehensive documentation Customer-facing, regulatory examination focus

Roles & Responsibilities

Role Line of Defense Responsibility
Agent Owner / Copilot Studio Agent Author 1st line Model classification submission, Agent Card authorship, day-to-day monitoring of agent performance, response to MRM findings
AI Governance Lead 1st / 2nd line bridge Model-tiering recommendation to the MRM Committee, governance-template application at publish/activate time, coordination with Compliance
Model Risk Manager (independent validation function) 2nd line Independent validation per SR 11-7 (conceptual soundness, ongoing monitoring, outcomes analysis); effective challenge; validation report to MRM Committee
Compliance Officer 2nd line Regulatory mapping, FINRA / SEC / OCC / NYDFS examination readiness, WSP integration
AI Administrator Operational Agent 365 Admin Center publication and activation approvals, governance template enforcement, Researcher / Computer Use configuration
Power Platform Admin Operational Environment governance, solution / manifest version control, Application Insights and Copilot Studio analytics configuration
Internal Audit 3rd line Periodic independent audit of the MRM program for AI agents; assurance to the audit committee on operating effectiveness
Model Risk Committee (governance body) Governance Final tiering decisions for Tier 1 / customer-facing agents; acceptance of validation reports; approval of model retirement
Control Relationship
1.6 - Microsoft Purview DSPM for AI Authoritative AI inventory and prompt/response capture feeding the model inventory and ongoing-monitoring pillar
1.7 - Comprehensive Audit Logging 17a-4(f) WORM retention path for model artifacts, validation evidence, and MRM Committee minutes
1.9 - Data Retention and Deletion Retention schedule for validation evidence (6 years)
1.10 - Communication Compliance Supervisory review of customer-facing agent communications feeding outcomes analysis
2.1 - Managed Environments Platform substrate that hosts governed agents in scope for MRM
2.3 - Change Management Captures vendor-driven model changes (e.g., Microsoft default-model migration) and prompt / knowledge-source changes as MRM-relevant model changes
2.5 - Testing and Validation Pre-deployment validation evidence
2.7 - Vendor and Third-Party Risk Management Vendor-model governance under SR 11-7 §V (Azure OpenAI, Anthropic Claude, partner models)
2.8 - Access Control and Segregation of Duties Independence of validation function from model owner / developer
2.11 - Bias Testing Fairness component of outcomes analysis
2.12 - Supervision (FINRA Rule 3110) Registered-principal supervisory layer over AI-enabled business activity
2.14 - Training and Awareness Competence requirement for validators and effective-challenge participants
2.16 - RAG Source Integrity Validation Knowledge-source integrity feeds conceptual-soundness validation
2.25 - Agent 365 Admin Center Governance Console Publication / activation approval surface; governance templates; MRM-relevant lifecycle gating
2.26 - Entra Agent ID Identity Governance Identity attribution of agent actions to a governed Entra Agent ID
3.1 - Agent Inventory and Metadata Management Authoritative agent registry feeding the model inventory
3.2 - Usage Analytics and Activity Monitoring Usage data feeding ongoing-monitoring pillar
3.3 - Compliance and Regulatory Reporting MRM reporting to regulators and the audit committee
3.4 - Incident Reporting and Root Cause Analysis Threshold breaches and outcomes-analysis failures escalate into the IR / RCA workflow; RCA feedback returns to MRM
3.6 - Orphaned Agent Detection Ownerless / orphaned agents are MRM-actionable; an unowned model has no accountable owner under SR 11-7
3.9 - Microsoft Sentinel Integration Cross-source telemetry for ongoing monitoring and incident correlation

Implementation Playbooks

Step-by-Step Implementation

This control has detailed playbooks for implementation, automation, testing, and troubleshooting:

Vendor Model Governance (SR 11-7 Section V)

Vendor Models Require Equal Rigor

Federal Reserve SR 11-7 Section V explicitly requires that vendor-provided models be validated with the same rigor as internally-developed models. For AI agents using Microsoft Azure OpenAI, OpenAI APIs, or other third-party model providers, organizations must:

  1. Obtain sufficient documentation from the vendor to understand model behavior and limitations
  2. Conduct independent validation appropriate to the model's risk tier
  3. Monitor ongoing model performance including tracking vendor model updates
  4. Assess vendor model changes before deployment to production
Vendor Model Governance Requirement SR 11-7 Reference FSI-AgentGov Implementation
Documentation from vendor Section V Request Azure OpenAI model cards, benchmark data
Validation despite vendor source Section V Independent validation per tier (see Agent-as-Model Classification)
Ongoing monitoring Section V Track model performance in Copilot Studio Analytics
Change assessment Section V Review Microsoft model update announcements before deployment

Cross-Reference: For broader third-party risk management including vendor due diligence and contractual requirements, see Control 2.7 - Vendor and Third-Party Risk Management. For third-party relationship supervision, see Federal Reserve SR 13-19 (Guidance on Managing Outsourcing Risk).

Third-Party Attestation

For Tier 1 agents in customer-facing or decision-support roles, consider engaging third-party assessors with expertise in financial services recordkeeping and AI governance (e.g., Cohasset Associates for SEC 17a-4, CFTC 1.31, FINRA recordkeeping compliance). See Troubleshooting for escalation guidance.

Verification Criteria

Confirm control effectiveness by verifying:

  1. Every agent in the firm's authoritative inventory (Control 3.1 / DSPM for AI) has a documented model-classification decision with rationale, classifier identity, and decision date.
  2. Model inventory records, for each in-scope agent: tier, owner, sponsor, underlying model and version, knowledge sources, validation status, last validation date, and next scheduled validation.
  3. Agent Cards exist for every Tier 1 and Tier 2 agent and document intended use, data, limitations, performance benchmarks, and known failure modes.
  4. Independent validation (conceptual soundness, ongoing monitoring, outcomes analysis) has been completed for each in-scope agent at a frequency commensurate with risk, model complexity, and change activity (SR 11-7 §V) — not on a fixed annual basis. Validation cadence is documented in the model risk policy and approved by the MRM Committee.
  5. Effective challenge of each Tier 1 validation is evidenced by MRM Committee minutes referencing the specific challenge questions raised and their disposition.
  6. Ongoing-monitoring thresholds are defined per agent, alerts route to a named owner, and threshold breaches are escalated to the IR workflow per Control 3.4.
  7. Vendor-model change notifications (Microsoft Message Center, Power Platform release plan, Foundry model deprecations) are reviewed and dispositioned as model changes per SR 11-7 §V before they take effect in production.
  8. Solution / manifest version control enables point-in-time reconstruction of agent configuration for any date in the retention window.
  9. Validation evidence, MRM Committee minutes, model retirement decisions, and vendor-model assessments are retained on 17a-4(f)-compliant media for six years (with the first two easily accessible).
  10. A documented model-retirement / sunset workflow exists and has been exercised at least once, with retirement evidence retained alongside development and validation evidence.

Additional Resources

Regulatory sources

Microsoft Learn — surfaces this control depends on (verified April 2026)

Updated: April 2026 | Version: v1.4.0 | UI Verification Status: Current