Skip to content

Control 2.11: Bias Testing and Fairness Assessment

Control ID: 2.11 Pillar: Management Regulatory Reference: Fed SR 11-7, SEC AI Priorities, ECOA, FINRA Rule 3110 Last UI Verified: January 2026 Governance Levels: Baseline / Recommended / Regulated Last Verified: 2026-02-03

Objective

Implement systematic bias testing and fairness assessment for AI agents to identify and remediate discriminatory outputs, supporting compliance with fair lending laws and regulatory expectations for AI fairness in financial services.

Why This Matters for FSI

  • FINRA Rule 3110: Supervision requirements apply to AI systems used for investor communications and recommendations
  • Fed SR 11-7: Model validation must include assessment of potential discriminatory outcomes
  • SEC AI Priorities: Examinations focus on AI fairness in customer-facing applications
  • ECOA: Prohibits discrimination based on protected classes in credit decisions (applicable when agents influence credit, lending, or insurance decisions)

ECOA Applicability

Equal Credit Opportunity Act (ECOA) requirements apply specifically when AI agents influence credit decisions, lending recommendations, or insurance determinations. For agents not involved in these functions, focus on FINRA Rule 3110 supervision and SR 11-7 model risk requirements.

Control Description

This control establishes bias testing through:

  1. Protected Class Identification - Define classes per ECOA (race, color, religion, national origin, sex, marital status, age, public assistance status)
  2. Fairness Metrics - Implement statistical measures (demographic parity, equalized odds, calibration)
  3. Test Dataset Creation - Build representative datasets spanning protected classes
  4. Bias Detection Procedures - Systematic testing of agent outputs across demographic groups
  5. Remediation Workflow - Process for addressing identified bias issues
  6. Documentation - Maintain testing evidence for regulatory examination

Key Configuration Points

  • Define protected classes relevant to agent use case (ECOA + state-specific)
  • Create fairness test datasets with representative demographic distribution
  • Establish baseline fairness metrics before deployment
  • Configure automated bias testing in CI/CD pipeline for Zone 3 agents
  • Set remediation SLAs: Critical bias (24h), High (7d), Medium (30d)
  • Document bias testing results with statistical analysis
  • Schedule recurring bias assessments (quarterly minimum for Zone 3)

Zone-Specific Requirements

Zone Requirement Rationale
Zone 1 (Personal) Awareness training; report suspected bias; annual review Low external impact, basic awareness needed
Zone 2 (Team) Pre-deployment bias testing; documented assessment; quarterly review Shared agents warrant structured testing
Zone 3 (Enterprise) Comprehensive fairness assessment; automated monitoring; independent validation; remediation SLAs Customer-facing requires rigorous bias controls

Roles & Responsibilities

Role Responsibility
AI Governance Lead Define testing requirements, oversee fairness program
Data Science Team Develop fairness metrics, execute statistical analysis
Compliance Officer Validate regulatory alignment, approve test methodology
Agent Owner Remediate identified bias, implement corrective actions
Control Relationship
2.6 - Model Risk Management Bias testing is component of model validation
2.5 - Testing & Validation Fairness testing integrated with QA
2.18 - Conflict of Interest Testing Complementary testing for recommendation bias (COI Testing Framework)
3.10 - Hallucination Feedback Bias reports feed quality management

Implementation Playbooks

Step-by-Step Implementation

This control has detailed playbooks for implementation, automation, testing, and troubleshooting:

Verification Criteria

Confirm control effectiveness by verifying:

  1. Protected classes documented per ECOA and applicable state law
  2. Fairness test dataset includes representative demographic distribution
  3. Baseline fairness metrics established and documented
  4. Bias testing executed before Zone 3 agent deployment
  5. Bias assessment report generated with statistical analysis and remediation recommendations

Additional Resources

Updated: January 2026 | Version: v1.2 | UI Verification Status: Current