Control 3.6: Supervision and Oversight (FINRA 3110 / SEC Reg BI) — Verification & Testing

Test cases and evidence collection procedures for validating supervisory controls over Copilot-assisted activities.

Test Cases

Objective: Confirm that Copilot-assisted communications flow through the supervisory review process
Steps:
Have a test registered representative draft an investment recommendation using Copilot.
Send the recommendation to a test client account.
Verify the communication appears in the assigned supervisor's review queue.
Complete the review (approve, reject, or escalate) and confirm audit trail.
Expected Result: Communication flows through supervisory review with complete audit trail of actions taken.
Evidence: Review queue screenshot and audit log entry showing review completion.

Objective: Validate that Copilot-drafted investment recommendations are held pending supervisory approval
Steps:
Configure pre-send hold for investment recommendation communications.
Have a test user draft a recommendation via Copilot and attempt to send.
Verify the message is held and not delivered until supervisor approves.
Have the supervisor approve the message and confirm delivery.
Expected Result: Message is held, supervisor reviews and approves, and the message is then delivered with approval timestamp.
Evidence: Message trace showing hold status, approval action, and delivery confirmation.

Objective: Verify that supervisory ratios are within acceptable limits for effective oversight
Steps:
Run the supervisor-to-representative ratio script.
Verify no supervisor oversees more than 50 Copilot-enabled representatives.
Review each supervisor's review queue backlog.
Confirm all supervisors are completing reviews within the defined SLA.
Expected Result: All supervisory ratios are within policy limits and review SLAs are being met.
Evidence: Ratio report and SLA compliance metrics.

Objective: Confirm that Copilot-assisted recommendations capture required Reg BI documentation elements
Steps:
Review a sample of 10 Copilot-drafted recommendations that were supervisory approved.
Verify each recommendation includes: client suitability basis, cost disclosure, conflict of interest disclosure, and alternatives considered.
Confirm the supervisory review log captures the reviewer's assessment of each element.
Expected Result: All sampled recommendations contain required Reg BI elements and supervisory attestation.
Evidence: Sampled review records showing Reg BI element completeness.

Objective: Verify that the audit trail correctly captures agent-specific interactions for supervisory review when a Teams channel agent or declarative agent is used
Steps:
Deploy a test Teams channel agent or use an existing declarative agent in a non-production channel.
Have a test registered representative interact with the agent (e.g., ask it to summarize account information or draft a communication).
Wait 15–30 minutes for audit events to propagate to the Purview audit log.
Run Script 5 (Agent Interaction Audit) from the PowerShell setup guide to retrieve agent-specific CopilotInteraction events.
Verify the returned records contain: AgentId, AgentName, the interacting user's identity, and the interaction timestamp.
Confirm the XPIA field is present and set to false for normal interactions (no cross-prompt injection attempt detected).
Expected Result: Agent interactions appear in the audit log with correct AgentId, AgentName, user identity, and timestamp. XPIA flag is captured. The records are exportable for supervisory review evidence.
Evidence: CSV export from Script 5 showing agent interaction records; Purview audit log screenshot showing CopilotInteraction records with AgentId populated.

Objective: Confirm that the firm's written supervisory procedures (WSPs) address every currently deployed Copilot agent
Steps:
Generate a list of deployed Teams channel agents and declarative agents from the Microsoft 365 Admin Center (Admin Center > Agents > All agents / Registry).
Cross-reference each deployed agent against the agent inventory section of the firm's WSP Copilot addendum.
For any agent not listed in the WSP, flag as a gap requiring immediate documentation.
For listed agents, verify the WSP entry includes: agent scope, authorized actions, supervisory review cadence, and the person responsible for oversight.
Expected Result: All deployed agents are listed in the WSP with complete supervisory documentation. Zero undocumented agents are found.
Evidence: Agent inventory from Admin Center (screenshot or export) cross-referenced against WSP agent list; any gaps documented with remediation date.

Evidence Item	Source	Format	Retention
Supervisory review logs	Purview audit log	CSV export	7 years
Pre-send hold records	Message trace	CSV	7 years
Supervisor ratio report	PowerShell	Text export	With control documentation
Reg BI documentation samples	Review records	Redacted copies	7 years
Agent interaction audit records	Script 5 output	CSV export	7 years
WSP agent coverage gap report	Test 6 results	Document	With control documentation

Regulation	Requirement	How This Control Helps
FINRA 3110	Supervisory system and WSP requirements	Supports compliance with supervisory review obligations for AI-assisted activities
FINRA 3110(a)	Supervisory system must cover all tools used by associated persons, including agents	Agent audit trail capture and WSP coverage verification confirm agent supervision
SEC Reg BI	Care, disclosure, and conflict obligations	Helps meet best-interest documentation requirements for recommendations
FINRA 3120	Supervisory control system testing	Supports annual testing of supervisory effectiveness including agent supervision