Skip to content

Control 3.6: Supervision and Oversight (FINRA 3110 / SEC Reg BI) — Verification & Testing

Test cases and evidence collection procedures for validating supervisory controls over Copilot-assisted activities.

Regulatory applicability

Regulatory citations identify commonly referenced requirements. Applicability, record classification, and retention periods are a firm and counsel determination; this framework does not assert that any regulation applies or is satisfied. See the Regulatory Applicability Questions appendix.

Test Cases

Test 1: Supervisory Review Workflow

  • Objective: Confirm that Copilot-assisted communications flow through the supervisory review process
  • Steps:
  • Have a test registered representative draft an investment recommendation using Copilot.
  • Send the recommendation to a test client account.
  • Verify the communication appears in the assigned supervisor's review queue.
  • Complete the review (approve, reject, or escalate) and confirm audit trail.
  • Expected Result: Communication flows through supervisory review with complete audit trail of actions taken.
  • Evidence: Review queue screenshot and audit log entry showing review completion.

Test 2: Pre-Send Hold for Investment Recommendations

  • Objective: Validate that Copilot-drafted investment recommendations are held pending supervisory approval
  • Steps:
  • Configure pre-send hold for investment recommendation communications.
  • Have a test user draft a recommendation via Copilot and attempt to send.
  • Verify the message is held and not delivered until supervisor approves.
  • Have the supervisor approve the message and confirm delivery.
  • Expected Result: Message is held, supervisor reviews and approves, and the message is then delivered with approval timestamp.
  • Evidence: Message trace showing hold status, approval action, and delivery confirmation.

Test 3: Supervisor Capacity Validation

  • Objective: Verify that supervisory ratios are within acceptable limits for effective oversight
  • Steps:
  • Run the supervisor-to-representative ratio script.
  • Verify no supervisor oversees more than 50 Copilot-enabled representatives unless compliance/legal approves a firm-specific threshold.
  • Review each supervisor's review queue backlog.
  • Confirm all supervisors are completing reviews within the defined SLA.
  • Expected Result: All supervisory ratios are within policy limits and review SLAs are being met.
  • Evidence: Ratio report and SLA compliance metrics.

Test 4: Reg BI Documentation Completeness

  • Objective: Confirm that Copilot-assisted recommendations capture required Reg BI documentation elements
  • Steps:
  • Review a sample of 10 Copilot-drafted recommendations that were supervisory approved.
  • Verify each recommendation includes: client suitability basis, cost disclosure, conflict of interest disclosure, and alternatives considered.
  • Confirm the supervisory review log captures the reviewer's assessment of each element.
  • Expected Result: All sampled recommendations contain required Reg BI elements and supervisory attestation.
  • Evidence: Sampled review records showing Reg BI element completeness.

Test 5: Agent Audit Trail Capture (FINRA 3110(a) Agent Supervision)

  • Objective: Verify that the audit trail correctly captures agent-specific interactions for supervisory review when a Teams channel agent or declarative agent is used
  • Steps:
  • Deploy a test Teams channel agent or use an existing declarative agent in a non-production channel.
  • Have a test registered representative interact with the agent (e.g., ask it to summarize account information or draft a communication).
  • Wait for audit events to propagate to the Purview audit log using the tenant's documented audit-ingestion expectation; allow up to 24 hours before treating delayed events as missing.
  • Run Script 5 (Agent Interaction Audit) from the PowerShell setup guide to retrieve agent-specific CopilotInteraction events.
  • Verify the returned records contain: AgentId, AgentName, the interacting user's identity, and the interaction timestamp.
  • Confirm the AccessedResources.XPIADetected flag is absent or false for normal interactions (no cross-prompt injection attempt detected on referenced resources).
  • Expected Result: Agent interactions appear in the audit log with correct AgentId, AgentName, user identity, and timestamp. The AccessedResources.XPIADetected flag is captured when a detection occurs. The records are exportable for supervisory review evidence.
  • Evidence: CSV export from Script 5 showing agent interaction records; Purview audit log screenshot showing CopilotInteraction records with AgentId populated.

Test 6: WSP Coverage Verification for Deployed Agents

  • Objective: Confirm that the firm's written supervisory procedures (WSPs) address every currently deployed Copilot agent
  • Steps:
  • Generate a list of deployed Teams channel agents and declarative agents from the Microsoft 365 Admin Center (Admin Center > Agents > All agents / Registry).
  • Cross-reference each deployed agent against the agent inventory section of the firm's WSP Copilot addendum.
  • For any agent not listed in the WSP, flag as a gap requiring immediate documentation.
  • For listed agents, verify the WSP entry includes: agent scope, authorized actions, supervisory review cadence, and the person responsible for oversight.
  • Expected Result: All deployed agents are listed in the WSP with complete supervisory documentation. Zero undocumented agents are found.
  • Evidence: Agent inventory from Admin Center (screenshot or export) cross-referenced against WSP agent list; any gaps documented with remediation date.

Evidence Collection

Evidence Item Source Format Retention
Supervisory review logs Purview audit log CSV export 7 years; retention schedule: confirm
Pre-send hold records Message trace CSV 7 years; retention schedule: confirm
Supervisor ratio report PowerShell Text export With control documentation
Reg BI documentation samples Review records Redacted copies 7 years; retention schedule: confirm
Agent interaction audit records Script 5 output CSV export 7 years; retention schedule: confirm
WSP agent coverage gap report Test 6 results Document With control documentation

Compliance Mapping

Regulation Requirement How This Control Helps
FINRA 3110 Supervisory system and WSP citation Provides evidence for supervisory review of AI-assisted activities; applicability: confirm
FINRA 3110(a) Supervisory system citation for tools used by associated persons Agent audit trail capture and WSP coverage verification provide evidence for counsel/compliance review of agent supervision
SEC Reg BI Care, disclosure, and conflict citation Provides evidence for best-interest documentation review; applicability: confirm
FINRA 3120 Supervisory control system testing Supports annual testing of supervisory effectiveness including agent supervision

Next Steps