Control 2.18: Automated Conflict of Interest Testing

Control ID: 2.18 Pillar: Management Regulatory Reference: SEC Reg BI, SEC Rule 10b-5, FINRA 2111, FINRA Rule 3110 Last UI Verified: February 2026 Governance Levels: Baseline / Recommended / Regulated Last Verified: 2026-02-03

Objective

Implement automated testing to detect potential conflicts of interest in AI agent recommendations, particularly for agents providing product recommendations, investment guidance, or financial advice, supporting compliance with Regulation Best Interest.

Why This Matters for FSI

SEC Reg BI / FINRA Rule 2111: Broker-dealers must act in best interest of retail customers; agents must not show bias. The FINRA 2026 Annual Regulatory Oversight Report emphasizes that AI agents providing investment recommendations must meet suitability and best interest standards—firms cannot outsource suitability obligations to AI systems and must validate that AI recommendations are suitable for the specific customer.
SEC 10b-5: Anti-fraud provisions prohibit self-serving recommendations
FINRA Rule 3110: AI supervision requires testing for conflicts and biases

Updated February 2026

FINRA 2026 examination priorities include validation that AI-assisted recommendations undergo the same suitability analysis as human recommendations.

Control Description

This control establishes conflict testing through:

Proprietary Bias Detection - Test for preferential recommendation of firm's own products
Commission Bias Testing - Detect bias toward higher-compensation products
Cross-Selling Analysis - Identify inappropriate bundling or upselling patterns
Competitor Fairness - Ensure competitor products aren't unfairly excluded
Prompt Audit - Review system prompts for prohibited bias instructions
Scoring Validation - Verify product scoring algorithms are unbiased

Conflict Types to Test

Conflict Type	Description	Example
Employee vs Customer	Agent recommendations favor firm over client	Recommending proprietary products over better alternatives
Customer vs Customer	Agent treats different customers unfairly	Allocating limited resources to preferred customers
Related-Party	Agent involved in transactions with affiliated entities	Recommendations involving parent/subsidiary companies
Cross-Business Unit	Information barriers not properly enforced	Research influencing investment banking recommendations

Key Configuration Points

Define conflict-of-interest test scenarios relevant to agent use case
Create test datasets with comparable proprietary vs. competitor products
Establish bias thresholds (e.g., proprietary recommendations should not exceed market share)
Configure automated testing in pre-deployment pipeline
Schedule recurring conflict testing (quarterly minimum)
Audit system prompts for prohibited bias language
Document testing methodology and results for examination

Automation Available

See COI Testing Framework in FSI-AgentGov-Solutions for automated conflict of interest testing with 10 predefined scenarios covering proprietary bias, suitability, fee transparency, and cross-selling.

Copilot Studio Evaluation Framework

Copilot Studio's built-in evaluation framework can complement automated COI testing with an 8-step methodology for evidence-based agent validation. Relevant capabilities include:

Classification grading — Supports detection of proprietary bias, commission bias, and suitability issues by classifying agent responses against expected categories
Capability verification — Helps validate that agents invoke the correct topics and tools for different recommendation scenarios
Quality assessment — Aids in evaluating whether agent responses meet appropriateness and fairness standards
Comparative monitoring — Enables sequential evaluation runs to track quality trends and detect regressions over time

The evaluation framework supports observable, repeatable, and explainable testing—key attributes for demonstrating supervisory diligence under FINRA Rule 3110. See the Verification & Testing playbook for evaluation methodology guidance.

Zone-Specific Requirements

Zone	Requirement	Rationale
Zone 1 (Personal)	Awareness only; no recommendation agents	Personal agents typically don't make recommendations
Zone 2 (Team)	Basic conflict testing for recommendation agents; documented methodology	Team recommendation agents warrant validation
Zone 3 (Enterprise)	Comprehensive Reg BI testing; automated monitoring; independent validation; quarterly review	Customer-facing recommendations require rigorous conflict controls

Roles & Responsibilities

Role	Responsibility
Compliance Officer	Define testing requirements, validate Reg BI alignment
AI Governance Lead	Configure testing, oversee methodology
Model Risk Manager	Independent validation of testing approach
Agent Owner	Remediate identified conflicts, update prompts

Control	Relationship
2.11 - Bias Testing	Complementary bias testing for fairness
2.6 - Model Risk Management	Conflict testing is MRM component
2.12 - Supervision	Supervisory review of recommendations
2.20 - Adversarial Testing	Red team testing for hidden biases

Implementation Playbooks

Step-by-Step Implementation

This control has detailed playbooks for implementation, automation, testing, and troubleshooting:

Portal Walkthrough — Step-by-step portal configuration
PowerShell Setup — Automation scripts
Verification & Testing — Test cases and evidence collection
Troubleshooting — Common issues and resolutions

Verification Criteria

Confirm control effectiveness by verifying:

Test scenarios defined for proprietary, commission, and cross-selling bias
Test datasets include comparable proprietary and competitor products
Automated conflict testing executes in pre-deployment pipeline
System prompts audited and free of prohibited bias instructions
Conflict testing report generated with statistical analysis

Regulatory Requirements

FINRA Rule 2111 (Suitability) and Regulation Best Interest

AI agents providing investment recommendations must meet suitability and best interest standards. The FINRA 2026 Annual Regulatory Oversight Report emphasizes that firms cannot outsource suitability obligations to AI systems—human supervisors must validate that AI recommendations are suitable for the specific customer.

Testing Requirements:

Validate AI recommendations against customer profiles
Test for prohibited conflicts of interest
Document suitability basis for material recommendations
Maintain evidence of supervisory review