Skip to content

Portal Walkthrough: Control 2.16 - RAG Source Integrity Validation

Last Updated: January 2026 Portals: Copilot Studio, SharePoint Admin Center Estimated Time: 1-2 hours initial setup, ongoing monitoring

Prerequisites

  • Copilot Studio Environment Admin or Maker role
  • SharePoint Admin role (for site-based knowledge sources)
  • Knowledge source inventory documented
  • Approval workflow defined for new knowledge sources
  • Content versioning strategy established

Step-by-Step Configuration

Step 1: Inventory Existing Knowledge Sources

  1. Open Copilot Studio
  2. Select the target environment
  3. Open each agent and review Knowledge settings
  4. Document all knowledge sources:
  5. SharePoint sites
  6. Dataverse tables
  7. External websites (if allowed)
  8. Uploaded files

Step 2: Configure SharePoint Knowledge Source

  1. In Copilot Studio, open the agent
  2. Navigate to Knowledge > + Add knowledge
  3. Select SharePoint
  4. Configure the connection:
  5. Enter SharePoint site URL
  6. Select specific libraries/folders (avoid entire site if possible)
  7. Review indexing scope

FSI Best Practice: Limit knowledge sources to specific document libraries rather than entire sites to reduce scope and improve accuracy.

Step 3: Establish Content Approval Workflow

  1. In SharePoint Admin Center, configure content approval:
  2. Navigate to target site > Site settings > Site content types
  3. Enable content approval for document libraries
  4. Or use Power Automate approval flow:
  5. Create flow triggered on new document upload
  6. Route to content owner for approval
  7. Only approved content is indexed

Step 4: Configure Knowledge Source Versioning

  1. Enable versioning on SharePoint document libraries:
  2. Library settings > Versioning settings
  3. Enable Create major versions
  4. Set version history limits (e.g., 50 versions)
  5. Document version control requirements:
  6. Major version = approved for agent use
  7. Minor version = draft, not indexed

Step 5: Set Up Staleness Detection

  1. Create a monitoring process for content freshness:
  2. Define maximum age thresholds (e.g., 365 days)
  3. Create Power Automate flow or scheduled report

  4. Alert content owners of stale documents

  5. Configure Copilot Studio refresh settings:

  6. Knowledge sources sync periodically
  7. Document expected refresh frequency

Step 6: Enable Citation Logging

  1. Verify agent includes citations in responses:
  2. Test agent and confirm source citations appear

  3. Document citation format for audit

  4. Configure audit logging (via Control 1.7):

  5. Ensure Copilot Studio audit events capture knowledge queries
  6. Retain logs per regulatory requirements

Configuration by Governance Level

Setting Baseline (Zone 1) Recommended (Zone 2) Regulated (Zone 3)
Knowledge Source Approval Informal Documented approval Formal workflow
Content Versioning Optional Enabled Required with audit
Staleness Threshold None 365 days 90 days
Citation Display Optional Enabled Required
Source Inventory Informal Documented Audited quarterly
External Sources Allowed Restricted Prohibited

FSI Example Configuration

Agent: Client Portfolio Advisor
Environment: FSI-Wealth-Prod

Knowledge Sources:
  1. Source: SharePoint - Investment Research Library
     URL: https://contoso.sharepoint.com/sites/research/library
     Scope: "Published Research" folder only
     Approval: Content approval workflow enabled
     Owner: Research Team Lead
     Review Frequency: Monthly

  2. Source: SharePoint - Product Documentation
     URL: https://contoso.sharepoint.com/sites/products/docs
     Scope: "Approved Documents" library
     Approval: Major versions only (minor = draft)
     Owner: Product Management
     Review Frequency: Quarterly

  3. Source: Dataverse - Client FAQs
     Table: fsi_clientfaqs
     Scope: Status = "Published"
     Approval: Via Dataverse workflow
     Owner: Client Services Lead
     Review Frequency: Monthly

Prohibited Sources:
  - External websites
  - Personal OneDrive
  - Unapproved SharePoint sites
  - Third-party knowledge bases

Staleness Policy:
  - Alert at 90 days without update
  - Review required at 180 days
  - Removal consideration at 365 days

Validation

After completing these steps, verify:

  • All knowledge sources are inventoried and documented
  • Approval workflow is functioning for new content
  • Versioning is enabled on SharePoint sources
  • Staleness monitoring is active
  • Citations appear in agent responses
  • Audit logging captures knowledge queries

Back to Control 2.16 | PowerShell Setup | Verification Testing | Troubleshooting