Skip to content

Control 2.6: Model Risk Management (OCC 2011-12/SR 11-7)

Control ID: 2.6 Pillar: Management Regulatory Reference: OCC 2011-12, Federal Reserve SR 11-7, FINRA Rule 3110, SOX 302/404 Last UI Verified: January 2026 Governance Levels: Baseline / Recommended / Regulated Last Verified: 2026-02-03

Objective

Ensure AI agents used in customer-facing or decision-support roles are subject to the same rigorous governance as traditional quantitative models, including model inventory, independent validation, performance monitoring, bias detection, and documented change control.

Why This Matters for FSI

  • OCC 2011-12: Establishes model risk management framework requirements for banks
  • Fed SR 11-7: Requires model development, validation, and ongoing monitoring (identical to OCC 2011-12; jointly issued)
  • FINRA Rule 3110: AI supervision and oversight requirements for broker-dealers
  • SOX 302/404: Internal controls must include documented model controls (see SOX AI Governance below)
  • Interagency RFI on AI (2021): Confirmed OCC 2011-12 applies to AI/ML systems

SOX AI Governance and PCAOB Standards

SOX applies to AI systems implicitly through Internal Control over Financial Reporting (ICFR). AI agents that affect financial data, reporting, or controls must be documented and tested as IT general controls.

PCAOB AI Audit Standards (In Development):

  • PCAOB AS 1105 (Audit Evidence) and AS 2301 (Audit Samples) are under review for AI-specific guidance
  • July 2024 PCAOB statement indicated AI audit standards research in progress
  • Organizations should document AI system controls with sufficient detail to support external audit

AI System Documentation for SOX Compliance:

Documentation Element Purpose Retention
Agent Card (capabilities, limitations) Control documentation Life of system + 7 years
Validation test results Control testing evidence 7 years (SOX 802)
Change log with approvals Change management evidence 7 years (SOX 802)
Performance monitoring reports Ongoing control monitoring 7 years (SOX 802)

Control Description

While Copilot Studio agents may not be "models" in the traditional sense, their use in customer-facing or decision-support roles requires similar governance.

Copilot Studio Model Availability (as of February 2026)

Organizations should track the underlying models available in Copilot Studio for MRM documentation:

  • GPT-5 Chat — GA as the default model for Copilot Studio agents
  • GPT-4o — Deprecated as default; agents previously using GPT-4o have been migrated to GPT-5 Chat (GPT-4o remains available in select regions and via manual model selection)
  • Anthropic Claude models — Available natively in Copilot Studio as an alternative model provider (no custom connector required)

Update model inventory and Agent Cards when underlying models change, as model transitions may affect validation baselines and performance benchmarks.

Infrastructure vs MRM Framework

Microsoft provides infrastructure platforms (Dataverse, SharePoint, Power Platform, Application Insights) that can support MRM processes, but organizations must design and implement their own MRM frameworks. There is no built-in "MRM solution" that automatically classifies agents, schedules validations, or generates compliance reports. The capabilities below describe governance processes that leverage Microsoft infrastructure.

Capability Description Implementation FSI Application
Model Classification Determine if agent qualifies as "model" Organization-defined process using Dataverse/SharePoint Tier-based governance
Model Inventory Catalog agents that function as models Custom SharePoint list or Dataverse table Regulatory examination readiness
Independent Validation Third-party review of agent behavior Organization-managed process Compliance with SR 11-7
Performance Monitoring Track output quality over time Copilot Studio Analytics (built-in) + custom RAI metrics Early issue detection
Agent Cards Standardized documentation of capabilities/limitations Custom template in SharePoint/Dataverse Model transparency

Agent-as-Model Classification

Criteria Model Treatment Example
Makes decisions affecting customers Tier 1 Credit recommendation agent
Provides financial calculations Tier 1/2 Investment calculator agent
Customer-facing recommendations Tier 2 Product recommendation agent
Information retrieval only Non-model FAQ/knowledge base agent

Key Configuration Points

Organization-Designed MRM Framework

  • Classify all agents using OCC 2011-12 model definition criteria (organization-defined process)
  • Maintain model inventory with tier assignments, owners, and validation status (custom Dataverse table or SharePoint list)
  • Create Agent Cards documenting capabilities, limitations, and performance benchmarks (custom template)
  • Establish independent validation program (third-party for Tier 1, internal for Tier 2) - organizational process
  • Define validation schedule and tracking mechanism (custom workflow or calendar integration)

Platform-Enabled Monitoring

  • Implement performance monitoring with defined thresholds using Copilot Studio Analytics (built-in) and custom RAI telemetry (see Control 2.9)
  • Implement manifest version control for point-in-time reconstruction using solution export/import
  • Document all prompt changes with impact assessment and approval (custom change management process)

Implementation Reference

See Portal Walkthrough for example inventory schemas and Agent Card templates that organizations can adapt.

Zone-Specific Requirements

Zone Requirement Rationale
Zone 1 (Personal) Non-model classification typical; minimal MRM governance Personal productivity, no customer impact
Zone 2 (Team) May be model; internal validation; standard documentation Shared agents may influence decisions
Zone 3 (Enterprise) Likely model; independent third-party validation; comprehensive documentation Customer-facing, regulatory examination focus

Roles & Responsibilities

Role Responsibility
AI Governance Lead Model classification decisions, validation program oversight
Model Risk Manager MRM framework alignment, validation scheduling
Compliance Officer Regulatory requirements, examination readiness
Power Platform Admin Performance monitoring configuration, manifest versioning
Control Relationship
2.5 - Testing and Validation Pre-deployment validation
2.11 - Bias Testing Fairness assessment
3.1 - Agent Inventory Model inventory integration
3.3 - Compliance Reporting MRM reporting

Implementation Playbooks

Step-by-Step Implementation

This control has detailed playbooks for implementation, automation, testing, and troubleshooting:

Vendor Model Governance (SR 11-7 Section V)

Vendor Models Require Equal Rigor

Federal Reserve SR 11-7 Section V explicitly requires that vendor-provided models be validated with the same rigor as internally-developed models. For AI agents using Microsoft Azure OpenAI, OpenAI APIs, or other third-party model providers, organizations must:

  1. Obtain sufficient documentation from the vendor to understand model behavior and limitations
  2. Conduct independent validation appropriate to the model's risk tier
  3. Monitor ongoing model performance including tracking vendor model updates
  4. Assess vendor model changes before deployment to production
Vendor Model Governance Requirement SR 11-7 Reference FSI-AgentGov Implementation
Documentation from vendor Section V.1 Request Azure OpenAI model cards, benchmark data
Validation despite vendor source Section V.2 Independent validation per tier (see Agent-as-Model Classification)
Ongoing monitoring Section V.3 Track model performance in Copilot Studio Analytics
Change assessment Section V.4 Review Microsoft model update announcements before deployment

Cross-Reference: For broader third-party risk management including vendor due diligence and contractual requirements, see Control 2.7 - Vendor and Third-Party Risk Management. For third-party relationship supervision, see Federal Reserve SR 13-19 (Guidance on Managing Outsourcing Risk).

Third-Party Attestation

For Tier 1 agents in customer-facing or decision-support roles, consider engaging third-party assessors with expertise in financial services recordkeeping and AI governance (e.g., Cohasset Associates for SEC 17a-4, CFTC 1.31, FINRA recordkeeping compliance). See Troubleshooting for escalation guidance.

Verification Criteria

Confirm control effectiveness by verifying:

  1. All agents reviewed for model classification with documented rationale
  2. Model inventory maintained with current tier assignments and validation dates
  3. Agent Cards created for all Tier 1/2 agents with performance benchmarks
  4. Independent validation completed per tier requirements (annual for Tier 1/2)
  5. Performance monitoring active with threshold alerts configured
  6. Manifest version control enables point-in-time reconstruction

Additional Resources

Updated: January 2026 | Version: v1.2 | UI Verification Status: Current