Skip to content

Control 2.16: RAG Source Integrity Validation

Control ID: 2.16 Pillar: Management Regulatory Reference: Fed SR 11-7, OCC 2011-12, Interagency Third-Party Guidance (2023), FINRA 4511 Last UI Verified: January 2026 Governance Levels: Baseline / Recommended / Regulated Last Verified: 2026-02-03

Objective

Establish governance over knowledge sources used for Retrieval-Augmented Generation (RAG) in Copilot Studio agents, ensuring content used to ground responses is accurate, current, approved, and properly attributed.

Why This Matters for FSI

  • Fed SR 11-7: Model validation requires assessment of data inputs including knowledge sources
  • Interagency Third-Party Guidance (2023): Third-party risk management applies to external knowledge sources
  • OCC 2011-12: Data quality controls for model inputs apply to RAG sources
  • FINRA 4511: Knowledge source approvals are books and records

Control Description

This control establishes RAG governance through:

  1. Source Approval Workflow - Formal approval before content becomes agent knowledge
  2. Content Versioning - Track changes to knowledge sources over time
  3. Integrity Verification - Validate source content hasn't been tampered with
  4. Citation Logging - Track which sources agents use in responses
  5. Staleness Detection - Identify outdated content requiring refresh
  6. Content Owner Accountability - Assign owners responsible for source accuracy

Built-In vs Custom Capabilities

Capability SharePoint/Copilot Studio Implementation
Auto-sync from SharePoint Built-in Native configuration
Filter conditions (Modified on, Author) Built-in Native configuration
Permission-based access Built-in (live validation) Native configuration
Version history Built-in (SharePoint) Native configuration
Official source marking Built-in Native configuration
Approval workflows Power Automate Custom configuration required
Content checksums/hashing Not built-in Custom implementation required
Drift detection Not built-in Custom implementation required
Tamper verification Not built-in Custom implementation required
Citation accuracy validation Not built-in Custom evaluation pipeline required

Comprehensive Integrity Validation

SharePoint provides basic content governance features (versioning, permissions, filter conditions). However, comprehensive integrity validation including content checksums, drift detection, and tamper verification requires custom implementation using Power Automate, Azure Functions, or external integrity monitoring tools.

Key Configuration Points

Built-In Configuration (SharePoint/Copilot Studio)

  • Define approved knowledge source types (SharePoint, OneDrive, Dataverse, external)
  • Configure content versioning on knowledge libraries (SharePoint built-in)
  • Implement metadata schema: Owner, Approval Date, Review Date, Classification (SharePoint columns)
  • Enable Copilot Studio citation tracking (built-in when using SharePoint sources)
  • Configure filter conditions to control which content is indexed (SharePoint built-in)

Custom Implementation Required

  • Create SharePoint approval workflow for knowledge source additions (Power Automate configuration)
  • Set staleness thresholds with automated alerting (90 days recommended, 30 days for regulatory content) - requires Power Automate flow
  • Configure Power Automate alerts for expired content (custom flow)
  • Implement content integrity checksums (custom solution - not built-in)
  • Deploy drift detection monitoring (custom solution - not built-in)

Automation Available

See RAG Source Validator in FSI-AgentGov-Solutions for automated SHA-256 hash validation, schema drift detection, and freshness monitoring across SharePoint, Dataverse, and Azure Blob sources.

Zone-Specific Requirements

Zone Requirement Rationale
Zone 1 (Personal) User-managed sources; annual review; basic versioning Personal use has limited grounding needs
Zone 2 (Team) Approved sources only; content owner assigned; quarterly review Shared agents warrant source governance
Zone 3 (Enterprise) Formal approval workflow; mandatory versioning; staleness alerts; citation logging Customer-facing requires maximum source integrity

Roles & Responsibilities

Role Responsibility
AI Governance Lead Define source approval criteria, oversee integrity program
Content Owner Maintain source accuracy, execute periodic review
SharePoint Admin Configure libraries, implement versioning and workflows
Compliance Officer Approve regulatory content sources, validate retention
Control Relationship
4.1 - SharePoint IAG SharePoint-specific knowledge governance
2.6 - Model Risk Management Source validation is MRM component
3.10 - Hallucination Feedback Source issues identified through feedback
1.3 - SharePoint Content Governance Permissions control source access

Implementation Playbooks

Step-by-Step Implementation

This control has detailed playbooks for implementation, automation, testing, and troubleshooting:

Verification Criteria

Confirm control effectiveness by verifying:

Built-In Capabilities

  1. Content versioning enabled on all knowledge libraries (SharePoint setting)
  2. Metadata schema applied with Owner and Review Date fields (SharePoint columns)
  3. Citation logging captures source references in agent responses (Copilot Studio feature)
  4. Filter conditions configured to control indexed content (SharePoint built-in)

Custom Implementations (if deployed)

  1. Knowledge source approval workflow functions correctly (Power Automate flow)
  2. Staleness alert triggers for content past threshold (Power Automate flow)
  3. Content integrity checksums validate source authenticity (custom implementation)

Verification Scope

Items 5-7 require custom implementation. Verify these only if your organization has deployed custom workflows for RAG governance.

Additional Resources

Updated: January 2026 | Version: v1.2 | UI Verification Status: Current