Skip to content

Control 3.8: Copilot Hub and Governance Dashboard - Verification & Testing

This playbook provides verification and testing procedures for Control 3.8.

Verification Steps

1. M365 Admin Center Access

  • Navigate to Copilot section
  • Verify all five navigation items accessible
  • Confirm Settings tabs load correctly

2. Agents Section Access

  • Navigate to Agents section
  • Verify Overview metrics display
  • Confirm Registry shows all agents

3. PPAC Copilot Access

  • Navigate to PPAC Copilot section
  • Verify Settings page loads
  • Confirm Copilot Studio dashboard accessible

4. Settings Configuration

  • Verify FSI-recommended settings applied
  • Confirm web search disabled
  • Check external AI providers blocked

Compliance Checklist

Item Required For Status
Copilot settings documented Audit evidence
Web search disabled FINRA 4511 compliance
External AI providers blocked Data governance
Agent approval workflow configured Risk management
Usage reports exported monthly FINRA 4511
MCP Servers reviewed Security

Test Cases

Test Case FAC-01: Admin Exclusion Group Correctly Removes Copilot Access

Objective: Verify Admin Exclusion Group correctly removes Microsoft 365 Copilot access for excluded users

Prerequisites: - Admin Exclusion Group created with name CopilotForM365AdminExclude - Test user has M365 Copilot license assigned - Test user is NOT currently in Admin Exclusion Group

Steps:

  1. Baseline verification:
  2. Sign in as test user
  3. Navigate to Microsoft Teams or Outlook
  4. Verify Copilot Chat is accessible and functional

  5. Document current access state

  6. Add user to Admin Exclusion Group:

  7. As administrator, navigate to Microsoft Entra admin center > Groups
  8. Open CopilotForM365AdminExclude group
  9. Add test user to group membership

  10. Document timestamp of addition

  11. Wait for propagation:

  12. Wait 24 hours for group membership change to propagate

  13. Note: Propagation can take up to 24 hours per Microsoft documentation

  14. Verify exclusion:

  15. Sign in as test user (force new authentication session)
  16. Navigate to Microsoft Teams > Copilot Chat
  17. Attempt to access Copilot features

  18. Document behavior (access denied, features not visible, error message)

  19. Verify license assignment unchanged:

  20. As administrator, verify test user still has M365 Copilot license assigned

  21. Confirm exclusion is behavioral (group-based), not license-based

  22. Remove from exclusion group and verify restoration:

  23. Remove test user from Admin Exclusion Group
  24. Wait 24 hours for propagation
  25. Sign in as test user and verify Copilot access restored

Expected Result: - User in Admin Exclusion Group cannot access Copilot features despite having valid license - Copilot Chat not visible in Teams/Outlook, or displays "not available" message - After removal from group (and propagation), access is restored

Evidence to Collect: - Screenshot of test user with Copilot access before exclusion - Screenshot of Admin Exclusion Group membership showing test user - Screenshot of test user without Copilot access after exclusion - Entra ID audit log entry showing group membership change - Timestamp documentation for 24-hour propagation verification

Regulatory Mapping: FINRA 3110 (supervisory restrictions), SOX 404 (IT access controls)

Test Case FAC-02: Deployment Group Limits Copilot Availability to Specified User Population

Objective: Verify Deployment Group correctly limits Copilot availability to users in approved deployment phase

Prerequisites: - Deployment group created (e.g., Copilot-Pilot-IT-Compliance) - Two test users with M365 Copilot licenses: - Test User A: Member of deployment group - Test User B: NOT member of deployment group (but has license)

Steps:

  1. Create deployment group:
  2. As administrator, create deployment group in M365 Admin Center > Copilot > Settings
  3. Add Test User A to deployment group
  4. Verify Test User B is NOT in deployment group

  5. Document group configuration

  6. Configure Copilot for deployment group only:

  7. In M365 Admin Center, configure Copilot to be available only to deployment group members

  8. Save settings and document timestamp

  9. Wait for propagation:

  10. Wait 8 hours for settings to propagate across tenant

  11. Test User A (in deployment group):

  12. Sign in as Test User A
  13. Navigate to Teams > Copilot Chat
  14. Verify Copilot features are accessible and functional

  15. Document successful access

  16. Test User B (NOT in deployment group):

  17. Sign in as Test User B
  18. Navigate to Teams > Copilot Chat
  19. Verify Copilot features are NOT accessible

  20. Document denial behavior (features hidden, error message, etc.)

  21. Verify license assignments:

  22. Confirm both Test User A and Test User B have identical M365 Copilot license assignments
  23. Verify difference in access is deployment group membership, not licensing

Expected Result: - Test User A (in deployment group): Copilot access granted - Test User B (not in deployment group): Copilot access denied despite valid license - Deployment group configuration enforces phased rollout control

Evidence to Collect: - Deployment group membership list showing Test User A included, Test User B excluded - Screenshot of Test User A successfully accessing Copilot - Screenshot of Test User B denied access to Copilot - License assignment report showing both users have M365 Copilot licenses - M365 Admin Center settings showing deployment group configuration

Regulatory Mapping: SOX 404 (documented IT controls)

Test Case FAC-03: Web Search Disabled Users Cannot Access Web-Grounded Copilot Responses

Objective: Verify web search control prevents Copilot from accessing external web data when disabled

Prerequisites: - M365 Admin Center access to Copilot > Settings > Data access - Test user with M365 Copilot access - Web search control set to "Enabled" initially (baseline)

Steps:

  1. Baseline test with web search enabled:
  2. As administrator, verify web search is enabled (M365 Admin > Copilot > Settings > Data access)
  3. Sign in as test user
  4. In Copilot Chat, ask a question that requires external web data (e.g., "What are the latest news headlines today?")
  5. Document Copilot response — should include web-grounded content or indicate web search used

  6. Sign out

  7. Disable web search:

  8. As administrator, navigate to M365 Admin Center > Copilot > Settings > Data access
  9. Set "Web search for M365 Copilot" to "Disabled"

  10. Save settings and document timestamp

  11. Wait for propagation:

  12. Wait 8 hours for setting to propagate across tenant

  13. Note: Microsoft documentation indicates up to 8 hours for Copilot settings propagation

  14. Test with web search disabled:

  15. Sign in as test user (force new session)
  16. In Copilot Chat, ask the same question requiring external web data
  17. Document Copilot response — should indicate web search not available, or limit response to organizational data only

  18. Verify no web-grounded content in response

  19. Verify organizational data still accessible:

  20. Ask Copilot a question that can be answered from organizational data (e.g., "Summarize my recent emails")
  21. Verify Copilot can still access and respond using organizational Microsoft 365 data
  22. Confirm only web search is disabled, not all Copilot functionality

Expected Result: - With web search enabled: Copilot provides web-grounded responses - With web search disabled: Copilot does NOT access external web data, limits responses to organizational data - Organizational data access remains functional when web search disabled

Evidence to Collect: - Screenshot of M365 Admin Center showing web search enabled (baseline) - Screenshot of Copilot response with web-grounded content (baseline) - Screenshot of M365 Admin Center showing web search disabled - Screenshot of Copilot response WITHOUT web content (web search disabled) - Screenshot of Copilot successfully using organizational data (web search disabled) - Timestamp documentation for 8-hour propagation verification

Regulatory Mapping: GLBA 501(b) (prevent external data leakage), FINRA (MNPI protection)

Test Case 4: Agent Access Restrictions

Objective: Verify restricted agent access prevents third-party agent discovery

Steps:

  1. Configure agent access to organizational agents only
  2. Wait for propagation
  3. Attempt to discover third-party agents
  4. Verify only organizational agents available

Expected Result: Third-party agents not discoverable

Test Case 5: AI Administrator Role Permissions

Objective: Verify AI Administrator can configure Copilot settings without Global Admin

Steps:

  1. Assign AI Administrator role to test user
  2. Sign in as AI Administrator
  3. Navigate to M365 Admin > Copilot > Settings
  4. Modify Copilot settings (User Access, Data Access, Actions)
  5. Verify settings changes applied successfully

Expected Result: Settings changes applied successfully without Global Admin

Test Case 6: Agent Approval Workflow

Objective: Verify agents require approval

Steps:

  1. Configure agent approval requirement
  2. Publish test agent
  3. Verify agent appears in Requests tab
  4. Approve agent
  5. Verify agent available

Expected Result: Agents require approval before availability

Test Case 7: MCP Server Blocking

Objective: Verify blocked servers are inaccessible

Steps:

  1. Block a test MCP Server
  2. Attempt to use blocked capability
  3. Verify capability unavailable

Expected Result: Blocked servers cannot be used

Evidence Collection

For audits, collect:

AI Feature Access Control Evidence: - Admin Exclusion Group membership list (export monthly) - Deployment group configuration and user assignments per phase - Web search control settings documentation (enabled/disabled per zone) - Agent access control settings (allowed agent types per zone) - Copilot Chat pinning configuration per department/role - Evidence of 24-hour propagation validation for exclusion groups - Evidence of 8-hour propagation validation for settings changes

General Copilot Governance Evidence: - Copilot settings configuration export (M365 Admin Center > Copilot > Settings) - Feature access control settings documentation (all four tabs: User access, Data access, Actions, Other) - Agent registry export (M365 Admin Center > Agents > All agents) - Usage reports (monthly) — Copilot Chat Active Users, Assisted Hours, Satisfaction Rate - Audit log of configuration changes (Entra ID > Audit logs, filter for Copilot-related events) - MCP Server availability list (M365 Admin Center > Agents > Tools) - AI Administrator role assignment documentation - Compliance Officer approval records for Admin Exclusion Group membership changes

Next Steps

SSPM Configuration Verification

Security Posture Assessment Test Cases

The following test cases validate configuration points flagged by security posture assessments. Each test maps to a specific setting in the Configuration Hardening Baseline.

Test ID Configuration Point Expected Result Portal Path Evidence
SSPM-3.8-01 AI Prompts toggle Disabled at tenant level PPAC > Settings > Power Platform Settings Screenshot
SSPM-3.8-02 Generative Actions toggle Disabled at tenant level PPAC > Settings > Power Platform Settings Screenshot
SSPM-3.8-03 File Analysis Models Disabled PPAC > Settings > Power Platform Settings Screenshot
SSPM-3.8-04 Model Knowledge Disabled PPAC > Settings > Power Platform Settings Screenshot
SSPM-3.8-05 Semantic Search with AI Disabled PPAC > Settings > Power Platform Settings Screenshot
SSPM-3.8-06 Move Data Across Regions Disabled PPAC > Settings > Power Platform Settings Screenshot
SSPM-3.8-07 Bing Search Disabled PPAC > Settings > Power Platform Settings Screenshot
SSPM-3.8-08 Transcript access Restricted to compliance roles M365 Admin > Copilot > Settings Screenshot
SSPM-3.8-09 DLP for publishing DLP policy enforcement active PPAC > Policies > Data policies Screenshot

Test Procedures

SSPM-3.8-01: AI Prompts Toggle

  1. Navigate to PPAC > Settings > Power Platform Settings
  2. Locate "AI Prompts" toggle
  3. Verify toggle is set to Disabled at the tenant level
  4. Pass criteria: AI Prompts toggle is off — makers cannot create AI prompt actions
  5. Evidence: Screenshot showing Power Platform Settings page with AI Prompts toggle state

SSPM-3.8-02: Generative Actions Toggle

  1. Navigate to PPAC > Settings > Power Platform Settings
  2. Locate "Generative Actions" toggle
  3. Verify toggle is set to Disabled at the tenant level
  4. Pass criteria: Generative Actions toggle is off — generative AI actions are not available to makers
  5. Evidence: Screenshot showing Power Platform Settings page with Generative Actions toggle state

SSPM-3.8-03: File Analysis Models

  1. Navigate to PPAC > Settings > Power Platform Settings
  2. Locate "File Analysis Models" toggle
  3. Verify toggle is set to Disabled
  4. Pass criteria: File Analysis Models is disabled — no automated file analysis via AI
  5. Evidence: Screenshot showing toggle state

SSPM-3.8-04: Model Knowledge

  1. Navigate to PPAC > Settings > Power Platform Settings
  2. Locate "Model Knowledge" toggle
  3. Verify toggle is set to Disabled
  4. Pass criteria: Model Knowledge is disabled — agents cannot access general model knowledge
  5. Evidence: Screenshot showing toggle state

SSPM-3.8-05: Semantic Search with AI

  1. Navigate to PPAC > Settings > Power Platform Settings
  2. Locate "Semantic Search with AI" toggle
  3. Verify toggle is set to Disabled
  4. Pass criteria: Semantic Search with AI is disabled — AI-powered search is not active
  5. Evidence: Screenshot showing toggle state

SSPM-3.8-06: Move Data Across Regions

  1. Navigate to PPAC > Settings > Power Platform Settings
  2. Locate "Move Data Across Regions" toggle
  3. Verify toggle is set to Disabled
  4. Pass criteria: Cross-region data movement is disabled — data stays within the configured region
  5. Evidence: Screenshot showing toggle state

SSPM-3.8-07: Bing Search

  1. Navigate to PPAC > Settings > Power Platform Settings
  2. Locate "Bing Search" toggle
  3. Verify toggle is set to Disabled
  4. Pass criteria: Bing Search is disabled — agents cannot query external web data via Bing
  5. Evidence: Screenshot showing toggle state

SSPM-3.8-08: Transcript Access

  1. Navigate to M365 Admin Center > Copilot > Settings
  2. Review transcript access configuration
  3. Verify transcript access is restricted to compliance roles only (not all users or all admins)
  4. Pass criteria: Only designated compliance roles can access agent interaction transcripts
  5. Evidence: Screenshot showing transcript access control settings with role assignments

SSPM-3.8-09: DLP for Publishing

  1. Navigate to PPAC > Policies > Data policies
  2. Verify at least one DLP policy is active and applies to the target environments
  3. Verify the policy blocks or restricts high-risk connectors
  4. Confirm DLP enforcement is active for agent publishing (agents cannot publish if they violate DLP)
  5. Pass criteria: DLP policy enforcement is active and applies to all governed environments
  6. Evidence: Screenshot showing DLP policy list with environment assignments and connector classifications

Updated: February 2026 | Version: v1.3 | Classification: Verification Testing