Skip to content

Control 3.13: Agent 365 Admin Center Analytics and Reporting

Control ID: 3.13
Pillar: Reporting
Regulatory Reference: FINRA Rule 3110 (Supervision), FINRA Rule 4511 (General Requirements for Books and Records), SEC Rule 17a-3 (Records to Be Made by Certain Exchange Members, Brokers, and Dealers), SEC Rule 17a-4 (Records to Be Preserved by Certain Exchange Members, Brokers, and Dealers), SOX Section 302 (Corporate Responsibility for Financial Reports), SOX Section 404 (Management Assessment of Internal Controls)
Last UI Verified: May 2026
Governance Levels: Baseline / Recommended / Regulated

Agent 365 Generally Available — May 1, 2026

With Agent 365 generally available as of May 1, 2026, the Overview page currently surfaces Microsoft Learn hero metrics for Agent Registry, Active Users, Agent Run-time, and Registry Sync. Microsoft Learn lists Agents with exceptions as an actionable view, not as an Overview hero metric. Financial institutions should operationalize the analytics dashboard now that GA is in effect and label any internally calculated exception-rate trend as a derived metric. Metric availability and UI layout last verified May 2026 (post-GA).

Frontier program and Agent 365 coexistence

Microsoft's Frontier program (an early-access program; the Microsoft 365 E7 SKU bundles Agent 365) remains active after the general availability of Microsoft Agent 365. Frontier provides early access to scenarios like Project Opal that are gated behind the Frontier subscription, while Agent 365 covers the core agent governance and administration surface. The two coexist; this control's guidance applies under both programs. See the Frontier Suite get-started guide and the Agent 365 admin overview for current scope.

Objective

Establish a formal process for leveraging the Microsoft 365 Agent 365 Admin Center Analytics dashboard to maintain continuous supervisory visibility over all AI agents deployed within the Microsoft 365 tenant, in satisfaction of FINRA Rule 3110 supervisory obligations and SEC Rule 17a-3/17a-4 recordkeeping requirements.

Why This Matters for FSI

FINRA Rule 3110 requires member firms to establish and maintain a system of supervision reasonably designed to achieve compliance with applicable securities laws and regulations. As AI agents increasingly execute client-facing and operational functions — including trade research, document generation, and customer interaction — regulators expect firms to demonstrate that these automated systems are subject to the same supervisory rigor as human registered representatives.

The Agent 365 Admin Center Overview page provides the centralized supervisory dashboard that satisfies this obligation. Specifically:

  • FINRA RN 24-09 / Rule 3110: The pending requests queue and ownerless agent governance cards provide real-time supervisory signals that unauthorized or unattended agents are identified and remediated promptly.
  • FINRA 4511 / SEC 17a-3/4: The inventory export function generates the primary examination artifact demonstrating the firm's complete agent roster at any point in time. Quarterly or monthly exports constitute contemporaneous business records.
  • SOX 302/404: Agents-with-exceptions signals, any internally derived exception-rate trend, and active user trends support management's assertion that AI-assisted processes are operating within expected parameters and that IT general controls over automated systems are effective.
  • Examination Readiness: FINRA examination staff increasingly request evidence of AI governance programs. A documented, recurring review process anchored in the Admin Center Analytics dashboard provides defensible evidence of supervisory adequacy.

Failure to monitor deployed agents creates regulatory exposure: an undetected ownerless agent processing client data, or an agent with a sustained increase in sessions with exceptions producing erroneous output, represents both a supervisory failure under FINRA RN 24-09 / Rule 3110 and a potential recordkeeping deficiency under SEC 17a-4.

Control Description

Capability Description
Hero Metric: Agent Registry Total count of agents available in the organization's catalog, including Microsoft-built, partner-built, and custom line-of-business agents. Accessible from Microsoft 365 admin center > Agents > Overview, with detailed inventory under Explore All agents > Registry.
Hero Metric: Active Users Count of unique users who interacted with at least one agent during the trailing 30-day period by sending a prompt and receiving a response. Measures adoption scope and identifies departments with high agent engagement.
Hero Metric: Agent Run-time Total hours worked by agents during the last 30 days, calculated from when a user request begins to when it is completed and aggregated across agent activities such as tool calls and response preparation.
Hero Metric: Registry Sync External connected platforms scanned for agent discovery and monitoring. Supports governance coverage for non-Microsoft or connected agent platforms that are integrated with the agent registry.
Actionable View: Pending Requests for Agents Count of agent deployment requests awaiting admin approval. "Manage requests" navigates to the Agent Registry > Requests tab for disposition.
Actionable View: Agents without Owners Count of agents without an assigned owner and still pending owner assignment. Provides direct navigation to the Agents without owners filtered registry view.
Actionable View: Agents with Exceptions Count of agents with errors in their conversations. "View details" navigates to a filtered registry view of agents with errors for investigation and remediation.
Derived Internal Metric: Exception Rate If the firm calculates this trend from per-agent Activity data, define it as sessions with errors divided by total sessions for the selected period. Increasing values indicate reliability or quality issues. This is not a Microsoft Overview hero KPI.
Agents by Publisher Breakdown of agents by creator category, including your organization, third-party publishers, and Microsoft. Supports third-party risk management oversight.
Agents by Platform Distribution of agents across supported creation platforms such as Microsoft 365 Copilot Agent Builder, Microsoft Copilot Studio, Agents Toolkit, SharePoint, Microsoft Foundry, and connected non-Microsoft platforms. Informs licensing and platform governance scope.
Active Users Over Time 30-day trend chart of daily active user engagement. Reveals adoption momentum, usage spikes, and unexpected declines. Anomalies may indicate unauthorized usage or agent failures.
Inventory Export Full agent list export from the All Agents page (CSV format). Functions as the primary examination evidence artifact for agent governance review. Captures agent name, publisher, platform, status, and owner for all deployed agents.
Researcher with Computer Use Reporting Configuration and usage reporting for the Researcher agent's Computer Use capability. Accessible at Agents > Researcher > Computer Use. Researcher with Computer Use is available via the Microsoft Frontier program (preview) as of February 2026 for tenants with Microsoft 365 Copilot licensing — verify current status against the Microsoft Frontier program page at edit time. Enables documentation of approved and excluded websites.

Key Configuration Points

  • Overview Access Path: Microsoft 365 Admin Center > Sign in with Entra Global Admin or AI Administrator role > Left navigation "Show all" > Agents > Overview
  • Hero Metric Scope: Agent overview hero metrics cover the trailing 30-day window and currently include Agent Registry, Active Users, Agent Run-time, and Registry Sync.
  • Agent Detail Activity Metrics: The per-agent Activity tab includes Active users, Sessions, Exceptions, and Agent run-time. Treat any "Exception Rate" as an internal derived metric calculated from exception and session counts, not as a Microsoft Overview hero metric.
  • Supported Agent Types for Metrics: Overview counts and platform cards reflect supported Microsoft and connected platforms. Per-agent Activity metrics are currently documented for Microsoft 365 Copilot Agent Builder, SharePoint, and Microsoft 365 Agents Toolkit agent types; verify tenant rollout before using the data as supervisory evidence.
  • Inventory Export Location: M365 Admin Center > Agents > All Agents > Export button (top-right of agent list). Export includes all agents visible to the admin's role scope.
  • Enablement Prerequisite: At least one user must be assigned a qualifying Microsoft Agent 365 license or bundled entitlement before Agent 365 analytics can be enabled in the tenant. If that prerequisite is missing, document the gap as a provisioning issue rather than a dashboard defect.
  • Pending Request SLA: No system-enforced SLA exists; firms must define and document internal SLA policy. Zone 3 requirement: 48-hour resolution target.
  • Role Requirements: Entra Global Admin or AI Administrator is required for write actions in Analytics; AI Reader (introduced May 2026) provides read-only access to the same dashboards and is the recommended least-privilege role for compliance officers, internal auditors, and FINRA Rule 3110 supervisors who need visibility without authoring authority. See the role catalog for canonical role definitions; use Entra Privileged Identity Management (PIM) for just-in-time elevation of AI Administrator where supported.

Export is Point-in-Time

The inventory export captures the agent roster at the moment of export. It does not constitute a continuous audit log. Financial institutions must establish a recurring export schedule and retain exports as business records under SEC Rule 17a-4. See Zone-Specific Requirements below for retention schedules.

Activity Telemetry Dependency

Custom or connected agents may appear in the Registry before complete activity telemetry is available. Do not infer a zero sessions, exceptions, or run-time value as proof of no usage unless the agent platform's metric support and ingestion timing have been verified. See Control 3.14 for custom telemetry considerations.

Zone-Specific Requirements

Zone Requirement Rationale
Zone 1 — Baseline Monthly review of Agent Registry total count and platform breakdown. Document agent count in IT governance log. Acknowledge and resolve ownerless agent alerts within 30 days. Establishes minimum supervisory awareness of deployed automation inventory. Supports baseline FINRA RN 24-09 / Rule 3110 supervisory system requirements for firms with limited AI deployment.
Zone 1 — Baseline Assign the tenant's qualifying Microsoft Agent 365 license or bundled entitlement to at least one user within 90 days where full hero-metric visibility is required for supervisory cadence. Forward-looking preparation; helps make the analytics surface available before downstream supervisory dependencies are written into the firm's policy.
Zone 2 — Recommended Weekly review of all actionable views (Pending Requests, Agents without Owners, Agents with Exceptions). Track agents-with-exceptions count and any internally derived exception-rate trend in the supervisory log. Document active user trend data monthly. Export full agent inventory quarterly; retain exports for 3 years. Supports active supervisory program under FINRA RN 24-09 / Rule 3110. Quarterly export constitutes contemporaneous business record under SEC 17a-3. Exceptions trend review supports SOX 404 IT general controls assessment.
Zone 2 — Recommended Establish internal SLA for pending request disposition (recommended: 5 business days). Helps keep deployment requests from aging without supervisory action; documents approval workflow for exam evidence.
Zone 3 — Regulated Daily monitoring of agents with exceptions, any internally derived exception rate, and pending requests count. Alert thresholds configured for agents-with-exceptions count increases, derived exception-rate increases exceeding 5 percentage points week-over-week, or pending request count exceeding 10. High-volume, high-risk deployment environment requiring continuous supervisory oversight. Threshold alerting supports FINRA RN 24-09 / Rule 3110 requirements for a reasonably designed supervisory system.
Zone 3 — Regulated Export full agent inventory monthly as a dated examination artifact; retain for 6 years (FINRA 4511 minimum). Store in immutable, WORM-compliant storage consistent with SEC 17a-4(f) requirements. Monthly export cadence provides granular point-in-time record of agent roster changes. 6-year retention supports FINRA 4511 and SEC 17a-4 maximum retention requirements.
Zone 3 — Regulated 48-hour SLA for pending request disposition. Documented escalation path if SLA is breached. Reduces the time that unauthorized agent deployments may operate without supervisory approval beyond a defined window.
Zone 3 — Regulated Configure Power Automate or M365 health alert workflow to notify Compliance Officer when ownerless agent count increases, agents-with-exceptions count increases, or a derived exception-rate alert threshold is breached. Automated alerting supports supervisory notifications that are contemporaneous with the triggering event, consistent with FINRA Rule 3110(b) and RN 24-09 written supervisory procedures expectations.

Roles & Responsibilities

Role Responsibility
Entra Global Admin / AI Administrator Maintain access to Agent 365 Admin Center. Perform or delegate weekly/monthly analytics reviews. Disposition pending agent deployment requests within defined SLA. Assign owners to ownerless agents. AI Administrator is the recommended least-privilege role for day-to-day analytics review; Entra Global Admin is required only where AI Administrator does not surface a needed action.
AI Reader (May 2026) Read-only access to all Analytics dashboards, hero metrics, agent registry, and inventory exports. Recommended least-privilege role for compliance officers, internal auditors, FINRA Rule 3110 supervisors, and SOC analysts who need visibility into AI governance posture without authoring authority. Cannot disposition pending requests or assign owners — escalate to AI Administrator for write actions.
Chief Compliance Officer (CCO) Own the supervisory program covering AI agents. Approve internal SLA policy for pending requests. Review monthly or quarterly inventory export. Attest to AI agent supervisory controls in SOX 302 certifications.
IT Risk / Governance Lead Configure alert thresholds and automated notification workflows. Maintain documentation of review cadence and findings. Escalate agents-with-exceptions or internally derived exception-rate anomalies to CCO and business line owners.
Business Line Agent Owners Acknowledge ownership of deployed agents. Respond to ownerless agent notifications within defined SLA. Investigate sessions with exceptions for agents within their line of business.
Internal Audit Validate that analytics review cadence meets Zone-specific requirements. Verify inventory exports are retained with correct timestamps. Include Agent 365 Analytics review in annual IT general controls testing.
Third-Party Risk Manager Review "Agents by Publisher" breakdown to track external partner agent count. Verify that partner agents are subject to vendor management review consistent with OCC third-party risk guidance.
Control Relationship
3.1 — Agent Inventory Foundational master inventory control. Agent 365 Admin Center Analytics dashboard operationalizes 3.1's inventory requirements by providing real-time registry visibility and export capability.
3.2 — Usage Analytics Operational usage detail complement. 3.2 covers granular per-agent usage data; 3.13 provides the tenant-level supervisory dashboard aggregating usage across all agents.
3.6 — Orphaned Agent Detection Ownerless Agents governance card in the Analytics dashboard provides the primary detection mechanism that triggers 3.6 remediation procedures.
2.25 — Agent 365 Admin Center Governance Console Governance actions companion. 2.25 covers deployment approval workflows and policy enforcement; 3.13 covers the analytics and reporting layer of the same Admin Center.
3.14 — Agent 365 Observability SDK Custom agent telemetry feeder. Agents instrumented with the 3.14 Observability SDK can contribute per-agent activity telemetry such as sessions, exceptions, and run-time. Exception Rate is an internal derived metric when calculated from exception and session counts, not an Overview hero metric.

Implementation Playbooks

The following playbooks provide step-by-step implementation guidance for Control 3.13:

  1. Portal Walkthrough — Accessing and Navigating Agent 365 Analytics
  2. PowerShell Setup — Automated Inventory Export and Alert Configuration
  3. Verification Testing — Confirming Analytics Visibility and Export Integrity
  4. Troubleshooting — Resolving Common Analytics and Export Issues

Verification Criteria

  1. An Entra Global Admin or AI Administrator can successfully navigate to M365 Admin Center > Agents > Overview and view the Agent Registry count.
  2. Hero metrics (Agent Registry, Active Users, Agent Run-time, Registry Sync) are visible once the tenant's qualifying Microsoft Agent 365 license or bundled entitlement is assigned to at least one user and deployed agents fall within supported agent types.
  3. Pending Requests governance card is visible and reviewed on the cadence required by the firm's Zone designation.
  4. Agents without Owners governance card is visible; count is zero or all ownerless agents have an active remediation ticket.
  5. Agents with Exceptions view is visible where activity data is available; any agents with errors have an active investigation or remediation ticket.
  6. Agent inventory export can be successfully generated from the All Agents page and produces a complete CSV listing all known deployed agents.
  7. Exported inventory files are stored in a designated records repository with date-stamped filenames consistent with SEC 17a-4 retention requirements.
  8. A documented review log (spreadsheet, ticketing system, or GRC platform entry) exists evidencing the most recent analytics review with reviewer identity and date.
  9. Alert thresholds for agents with exceptions, any internally derived exception rate, and pending request count are documented in written supervisory procedures (Zone 2 and Zone 3).
  10. Power Automate or equivalent automated alert workflow is operational and has been tested for Compliance Officer notification (Zone 3).
  11. All custom agents deployed in the tenant appear in the Agent Registry; any custom agents absent from activity metrics are documented as lacking confirmed telemetry support and are subject to a remediation timeline per Control 3.14.

Additional Resources

Documentation Currency

Agent 365 reached general availability on May 1, 2026. Microsoft Learn currently identifies Agent Registry, Active Users, Agent Run-time, and Registry Sync as Overview hero metrics, with Agents with exceptions documented as an actionable view. Re-verify all navigation paths and feature availability against Microsoft Learn periodically as the post-GA surface continues to evolve. This control was last UI-verified in May 2026.

Updated: June 2026 | Version: v1.6.2 | UI Verification Status: Current