Control 3.7: Regulatory Reporting — Verification & Testing
Test cases and evidence collection procedures to validate regulatory reporting capabilities for Copilot governance data.
Regulatory applicability
Regulatory citations identify commonly referenced requirements. Applicability, record classification, and retention periods are a firm and counsel determination; this framework does not assert that any regulation applies or is satisfied. See the Regulatory Applicability Questions appendix.
Test Cases
Test 1: Report Data Accuracy
- Objective: Verify that regulatory report data matches source audit logs and compliance metrics
- Steps:
- Generate the quarterly governance summary report using the automation script.
- Cross-reference the reported Copilot interaction count with a manual audit log search for the same period.
- Cross-reference policy match counts with the Communication compliance dashboard.
- Calculate the variance between reported and source data.
- Expected Result: Reported data matches source data within a 1% margin (accounting for timing differences).
- Evidence: Comparison spreadsheet showing reported values versus source data with variance calculations.
Test 2: Report Completeness
- Objective: Confirm that all required report sections and data fields are populated
- Steps:
- Review each regulatory report template against the applicable regulation's requirements.
- Verify that no required sections are blank or contain placeholder data.
- Confirm that all Copilot-specific governance metrics are included.
- Validate that the reporting period matches the regulatory submission window.
- Expected Result: All report sections are complete, data fields are populated, and the reporting period is accurate.
- Evidence: Completed report with a checklist showing all required sections verified.
Test 3: Report Generation Timeliness
- Objective: Validate that reports can be generated within the required timeframe before regulatory deadlines
- Steps:
- Initiate a full report generation cycle using the automation scripts.
- Measure the time from initiation to completed report output.
- Verify the report is generated at least 10 business days before the submission deadline.
- Confirm the pre-submission review workflow completes within allocated time.
- Expected Result: Report generation completes within 2 hours and the full review cycle completes 10+ days before deadline.
- Evidence: Timestamps showing generation start, completion, and review cycle duration.
Test 4: Historical Report Reproducibility
- Objective: Confirm that historical regulatory reports can be regenerated with the same data
- Steps:
- Select a previously submitted report from the prior quarter.
- Regenerate the report using the same date range and parameters.
- Compare the regenerated report with the original submission.
- Verify data consistency between the two versions.
- Expected Result: Regenerated report matches the original within acceptable variance.
- Evidence: Side-by-side comparison of original and regenerated report data.
Evidence Collection
| Evidence Item | Source | Format | Retention |
|---|---|---|---|
| Generated regulatory reports | Automation scripts | CSV/Excel | 7 years; retention schedule: confirm |
| Data accuracy validation | Comparison analysis | Spreadsheet | With report |
| Report generation timestamps | System logs | Text | With report |
| Pre-submission review sign-off | Workflow system | PDF/Email | 7 years |
Compliance Mapping
| Regulation | Requirement | How This Control Helps |
|---|---|---|
| FINRA 3120 | Internal annual supervisory-control report to senior management, not a regulator filing | Provides evidence for data-driven supervisory-control reporting review |
| SEC Form ADV | Potential disclosure review for AI tool usage | Advisory disclosure applicability: confirm |
| OCC 2026-13 / SR 26-2 | Current model-risk-management context; generative and agentic AI are outside the revised guidance's scope | Provides risk reporting context for AI governance without treating Copilot as covered by the revised MRM guidance |
| FFIEC | IT examination data requests | Provides evidence for examiner data-production review; applicability: confirm |
Next Steps
- Review Troubleshooting for reporting issues
- Proceed to Control 3.8 for model risk management
- Back to Control 3.7