Skip to content

Control 3.13: Agent 365 Admin Center Analytics and Reporting

Control ID: 3.13 Pillar: Reporting Regulatory Reference: FINRA Rule 3110 (Supervision), FINRA Rule 4511 (General Requirements for Books and Records), SEC Rule 17a-3 (Records to Be Made by Certain Exchange Members, Brokers, and Dealers), SEC Rule 17a-4 (Records to Be Preserved by Certain Exchange Members, Brokers, and Dealers), SOX Section 302 (Corporate Responsibility for Financial Reports), SOX Section 404 (Management Assessment of Internal Controls) Last UI Verified: April 2026 Governance Levels: Baseline / Recommended / Regulated

Agent 365 Generally Available — May 1, 2026

With Agent 365 reaching GA on May 1, 2026, the full analytics surface — including hero metrics (Active Users, Total Sessions, Exception Rate, Agent Runtime) on the Overview page — becomes available to all tenants with Agent 365 or Microsoft 365 E7 licensing. The Agent Registry (agent inventory listing) has been Generally Available since Frontier and remains accessible. Financial institutions should plan to operationalize the full analytics dashboard at or after GA. Metric availability and UI layout verified against March 2026 Frontier preview; re-verify after May 1, 2026 GA release.

Objective

Establish a formal process for leveraging the Microsoft 365 Agent 365 Admin Center Analytics dashboard to maintain continuous supervisory visibility over all AI agents deployed within the Microsoft 365 tenant, in satisfaction of FINRA Rule 3110 supervisory obligations and SEC Rule 17a-3/17a-4 recordkeeping requirements.

Why This Matters for FSI

FINRA Rule 3110 requires member firms to establish and maintain a system of supervision reasonably designed to achieve compliance with applicable securities laws and regulations. As AI agents increasingly execute client-facing and operational functions — including trade research, document generation, and customer interaction — regulators expect firms to demonstrate that these automated systems are subject to the same supervisory rigor as human registered representatives.

The Agent 365 Admin Center Overview page provides the centralized supervisory dashboard that satisfies this obligation. Specifically:

  • FINRA 3110, FINRA 25-07: The pending requests queue and ownerless agent governance cards provide real-time supervisory signals that unauthorized or unattended agents are identified and remediated promptly.
  • FINRA 4511 / SEC 17a-3/4: The inventory export function generates the primary examination artifact demonstrating the firm's complete agent roster at any point in time. Quarterly or monthly exports constitute contemporaneous business records.
  • SOX 302/404: The exception rate metric and active user trends support management's assertion that AI-assisted processes are operating within expected parameters and that IT general controls over automated systems are effective.
  • Examination Readiness: FINRA examination staff increasingly request evidence of AI governance programs. A documented, recurring review process anchored in the Admin Center Analytics dashboard provides defensible evidence of supervisory adequacy.

Failure to monitor deployed agents creates regulatory exposure: an undetected ownerless agent processing client data, or an agent with a sustained high exception rate producing erroneous output, represents both a supervisory failure under FINRA 3110, FINRA 25-07 and a potential recordkeeping deficiency under SEC 17a-4.

Control Description

Capability Description
Agent Registry (GA) Comprehensive inventory of all agents deployed in the tenant — Microsoft-built, partner-built (external), and custom/line-of-business agents. Reflects the total breadth of automation deployed. Accessible at M365 Admin Center > Agents > Overview.
Hero Metric: Active Users Count of unique users who interacted with at least one agent during the last 30-day period. Measures adoption scope and identifies departments with high agent engagement. (GA at May 1, 2026)
Hero Metric: Total Sessions Count of complete agent invocations in which an agent performed a task or answered a query. Provides volumetric baseline for workload assessment and capacity planning. (GA at May 1, 2026)
Hero Metric: Exception Rate Percentage of agent sessions that completed without errors (i.e., the agent successfully finished its intended task). A declining exception rate is a leading indicator of degraded agent reliability. (GA at May 1, 2026)
Hero Metric: Agent Runtime Total agent-assisted time, computed as the sum of session durations (end time minus start time) across all agents in the measurement window. Quantifies the operational dependency on agent automation. (GA at May 1, 2026)
Agents by Publisher Breakdown of agents by creator: (a) created by your organization — further divided by agents shared by the creator versus used only by the creator, and (b) created by external partners. Supports third-party risk management oversight.
Agents by Platform Distribution of agents across deployment platforms: Copilot Studio (Full License / Lite License), Azure AI Foundry, and external partner platforms. Informs licensing compliance and platform governance scope.
Active Users Over Time 30-day trend chart of daily active user engagement. Reveals adoption momentum, usage spikes, and unexpected declines. Anomalies may indicate unauthorized usage or agent failures.
Pending Requests Governance Card Count of agent deployment requests awaiting admin approval (last 30 days). Displays three oldest pending requests and a week-over-week delta badge. "Manage requests" button navigates to Agent Registry > Requests tab for disposition.
Ownerless Agents Governance Card Count of agents without an assigned owner. Provides direct "Assign Owner" button navigating to Agent Registry > Ownerless Agents filter. Ownerless agents represent unattended automation posing supervisory and data governance risk.
Inventory Export Full agent list export from the All Agents page (CSV format). Functions as the primary examination evidence artifact for agent governance review. Captures agent name, publisher, platform, status, and owner for all deployed agents.
Researcher with Computer Use Reporting Configuration and usage reporting for the Researcher agent's Computer Use capability. Accessible at Agents > Researcher > Computer Use (Frontier only). Enables documentation of approved and excluded websites.

Key Configuration Points

  • Access Path: Microsoft 365 Admin Center > Sign in with Entra Global Admin or AI Administrator role > Left navigation "Show all" > Agents > Overview
  • Metric Scope: All hero metrics cover the trailing 30-day window. There is no currently available UI control to adjust the measurement window.
  • Supported Agent Types for Metrics: Agents built using Copilot Agent Builder, SharePoint-hosted agents, Microsoft 365 Agents Toolkit, and agents instrumented with the Agent 365 Observability SDK (see Control 3.14). Microsoft-built agents and partner agents are tracked in the Registry but may have limited metric granularity.
  • Inventory Export Location: M365 Admin Center > Agents > All Agents > Export button (top-right of agent list). Export includes all agents visible to the admin's role scope.
  • Pending Request SLA: No system-enforced SLA exists; firms must define and document internal SLA policy. Zone 3 requirement: 48-hour resolution target.
  • Role Requirements: Entra Global Admin or AI Administrator role is required for full Analytics access (per the Agent 365 GA role limitations — see role catalog). Read-only or fine-grained roles are not available at GA; organizations should use Entra Privileged Identity Management (PIM) for just-in-time elevation where supported.

Export is Point-in-Time

The inventory export captures the agent roster at the moment of export. It does not constitute a continuous audit log. Financial institutions must establish a recurring export schedule and retain exports as business records under SEC Rule 17a-4. See Zone-Specific Requirements below for retention schedules.

Observability SDK Dependency

Custom agents (non-Microsoft-built, non-Copilot-Studio) will not appear in hero metrics unless they have implemented the Agent 365 Observability SDK. See Control 3.14 for implementation requirements. Agents without SDK instrumentation will appear in the Registry (if registered) but will not contribute to session, exception rate, or runtime metrics.

Zone-Specific Requirements

Zone Requirement Rationale
Zone 1 — Baseline Monthly review of Agent Registry total count and platform breakdown. Document agent count in IT governance log. Acknowledge and resolve ownerless agent alerts within 30 days. Establishes minimum supervisory awareness of deployed automation inventory. Satisfies baseline FINRA 3110, FINRA 25-07 supervisory system requirement for firms with limited AI deployment.
Zone 1 — Baseline Enroll in Frontier program within 90 days of Frontier GA to prepare for full metric availability. Forward-looking preparation; avoids metric visibility gap when Frontier transitions to GA.
Zone 2 — Recommended Weekly review of all governance cards (Pending Requests, Ownerless Agents). Track exception rate trend in supervisory log. Document active user trend data monthly. Export full agent inventory quarterly; retain exports for 3 years. Supports active supervisory program under FINRA 3110, FINRA 25-07. Quarterly export constitutes contemporaneous business record under SEC 17a-3. Exception rate trend supports SOX 404 IT general controls assessment.
Zone 2 — Recommended Establish internal SLA for pending request disposition (recommended: 5 business days). Ensures deployment requests do not age without supervisory action; documents approval workflow for exam evidence.
Zone 3 — Regulated Daily monitoring of exception rate and pending requests count. Alert thresholds configured for exception rate drops exceeding 5 percentage points week-over-week or pending request count exceeding 10. High-volume, high-risk deployment environment requiring continuous supervisory oversight. Threshold alerting satisfies FINRA 3110, FINRA 25-07 requirement for a reasonably designed supervisory system.
Zone 3 — Regulated Export full agent inventory monthly as a dated examination artifact; retain for 6 years (FINRA 4511 minimum). Store in immutable, WORM-compliant storage consistent with SEC 17a-4(f) requirements. Monthly export cadence ensures granular point-in-time record of agent roster changes. 6-year retention satisfies FINRA 4511 and SEC 17a-4 maximum retention requirements.
Zone 3 — Regulated 48-hour SLA for pending request disposition. Documented escalation path if SLA is breached. Prevents unauthorized agent deployments from operating without supervisory approval beyond a defined window.
Zone 3 — Regulated Configure Power Automate or M365 health alert workflow to notify Compliance Officer when ownerless agent count increases or exception rate alert threshold is breached. Automated alert ensures supervisory notifications are contemporaneous with the triggering event, consistent with FINRA 3110, FINRA 25-07(b) written supervisory procedures requirements.

Roles & Responsibilities

Role Responsibility
Entra Global Admin / AI Administrator Maintain access to Agent 365 Admin Center. Perform or delegate weekly/monthly analytics reviews. Disposition pending agent deployment requests within defined SLA. Assign owners to ownerless agents. AI Administrator is the recommended least-privilege role for day-to-day analytics review; Entra Global Admin is required only where AI Administrator does not surface a needed action.
Chief Compliance Officer (CCO) Own the supervisory program covering AI agents. Approve internal SLA policy for pending requests. Review monthly or quarterly inventory export. Attest to AI agent supervisory controls in SOX 302 certifications.
IT Risk / Governance Lead Configure alert thresholds and automated notification workflows. Maintain documentation of review cadence and findings. Escalate exception rate anomalies to CCO and business line owners.
Business Line Agent Owners Acknowledge ownership of deployed agents. Respond to ownerless agent notifications within defined SLA. Investigate exception rate anomalies for agents within their line of business.
Internal Audit Validate that analytics review cadence meets Zone-specific requirements. Verify inventory exports are retained with correct timestamps. Include Agent 365 Analytics review in annual IT general controls testing.
Third-Party Risk Manager Review "Agents by Publisher" breakdown to track external partner agent count. Verify that partner agents are subject to vendor management review consistent with OCC third-party risk guidance.
Control Relationship
3.1 — Agent Inventory Foundational master inventory control. Agent 365 Admin Center Analytics dashboard operationalizes 3.1's inventory requirements by providing real-time registry visibility and export capability.
3.2 — Usage Analytics Operational usage detail complement. 3.2 covers granular per-agent usage data; 3.13 provides the tenant-level supervisory dashboard aggregating usage across all agents.
3.6 — Orphaned Agent Detection Ownerless Agents governance card in the Analytics dashboard provides the primary detection mechanism that triggers 3.6 remediation procedures.
2.25 — Agent 365 Admin Center Governance Console Governance actions companion. 2.25 covers deployment approval workflows and policy enforcement; 3.13 covers the analytics and reporting layer of the same Admin Center.
3.14 — Agent 365 Observability SDK Custom agent telemetry feeder. Agents instrumented with the 3.14 Observability SDK contribute session, exception rate, and runtime data to the 3.13 hero metrics dashboard. Without 3.14 implementation, custom agent data is absent from 3.13 metrics.

Implementation Playbooks

The following playbooks provide step-by-step implementation guidance for Control 3.13:

  1. Portal Walkthrough — Accessing and Navigating Agent 365 Analytics
  2. PowerShell Setup — Automated Inventory Export and Alert Configuration
  3. Verification Testing — Confirming Analytics Visibility and Export Integrity
  4. Troubleshooting — Resolving Common Analytics and Export Issues

Verification Criteria

  1. An Entra Global Admin or AI Administrator can successfully navigate to M365 Admin Center > Agents > Overview and view the Agent Registry count.
  2. After May 1, 2026 GA (or for Frontier-enrolled tenants pre-GA): hero metrics (Active Users, Total Sessions, Exception Rate, Agent Runtime) are visible and displaying non-zero values for tenants with deployed agents that fall within supported agent types.
  3. Pending Requests governance card is visible and reviewed on the cadence required by the firm's Zone designation.
  4. Ownerless Agents governance card is visible; count is zero or all ownerless agents have an active remediation ticket.
  5. Agent inventory export can be successfully generated from the All Agents page and produces a complete CSV listing all known deployed agents.
  6. Exported inventory files are stored in a designated records repository with date-stamped filenames consistent with SEC 17a-4 retention requirements.
  7. A documented review log (spreadsheet, ticketing system, or GRC platform entry) exists evidencing the most recent analytics review with reviewer identity and date.
  8. Alert thresholds for exception rate and pending request count are documented in written supervisory procedures (Zone 2 and Zone 3).
  9. Power Automate or equivalent automated alert workflow is operational and has been tested for Compliance Officer notification (Zone 3).
  10. All custom agents deployed in the tenant appear in the Agent Registry; any custom agents absent from hero metrics are documented as lacking Observability SDK instrumentation and are subject to a remediation timeline per Control 3.14.

Additional Resources

Documentation Currency

Agent 365 reaches general availability on May 1, 2026. UI and feature availability verified against the April 2026 Frontier preview build and current Microsoft Learn documentation. Re-verify all navigation paths and feature availability against Microsoft Learn after May 1, 2026 GA. This control was last UI-verified in April 2026.

Updated: April 2026 | Version: v1.4.0 | UI Verification Status: Current