Control 2.7: Vendor and Third-Party Risk Management — Verification & Testing

Companion to Control 2.7. Verification confirms vendor risk management is operating as designed and produces the evidence financial-services examiners and internal audit will request.

Last Verified: 2026-11-06

Verification scope: Documentation-only; none of these tenant tests, UI checks, screenshots, or API reads were executed. Zone thresholds and contract minimums below are organizational acceptance criteria, not product guarantees.

Verification Checklist

#	Test	Expected Evidence	Pass Criteria
1	Connector inventory currency and coverage	Resource inventory CSV plus raw custom connector evidence; timestamp ≤ 7 days for Zone 2/3, ≤ 30 days for Zone 1; coverage/exception register	All Zone 2/3 environments reconciled; built-in HTTP, classic chatbot, solution-packaged custom connector (omitted by `Get-AdminPowerAppConnector`) and other inventory gaps explicitly reviewed; reviewer initials present
2	Risk-tier completeness	Tier T1–T5 populated for every row; T3/T4/T5 connectors have a documented business owner	Zero blanks in Tier or Owner columns for Zone 2/3
3	DLP enforcement of Blocked tier	Non-production maker test of blocked connector and allowed positive control; actual policy error/details and publish/runtime result	Supported connector/tool use is rejected at the documented enforcement stage; capture actual logs where available, not an assumed Purview event
4	Custom connector governance	Actual Environment Maker / Dataverse role assignments, approved producer environments, custom connector DLP classifications and Host URL rules	Creation permissions reviewed and usage restrictions tested; default-environment limitations/approved exceptions documented; sharing limits not treated as creation restrictions
5	SOC 2 / equivalent on file	Vendor file shows SOC 2 Type II within validity window OR accepted alternative + risk-acceptance memo	Coverage = 100% of Tier T2–T5 vendors
6	AI-specific contract clauses	Procurement export shows clause flags for model change notice, no-training-on-customer-data, AI incident notice, audit rights	Coverage = 100% of Tier T5 vendors
7	Transitive data-flow map	Architecture artifact lists every downstream subprocessor reached by Zone 3 multi-tool agents	One map per Zone 3 multi-tool agent; reviewed within last 12 months
8	Quarterly governance reporting	Meeting minutes / committee deck referencing the report	At least one report per quarter in last 12 months
9	Vendor incident drill	Tabletop exercise log for at least one Zone 3 critical vendor	Conducted within last 12 months; gaps captured and remediated
10	Records retention	Evidence repository governed by Purview retention aligned to FINRA 4511 / SEC 17a-4 schedule	Retention label applied; deletion holds in effect during open exams

Vendor Assessment Requirements by Zone

Assessment Area	Zone 1	Zone 2	Zone 3
Vendor vetting	Self-attestation; Microsoft / verified-publisher only	Documented questionnaire	Risk-tiered due diligence aligned to Interagency 2023 Guidance
Security documentation	None required (Microsoft baseline relied on)	SOC 2 Type II recommended	SOC 2 Type II required (SOC 1 where SOX-relevant)
Contract review	Standard terms	Legal review with security addendum	Negotiated security and AI addendum; audit rights
Monitoring frequency	Annual	Quarterly	Continuous (telemetry + advisory feeds)
Audit rights	Not required	Recommended	Required (or accepted SOC 2 in lieu)
Exit / contingency planning	Optional	Documented	Documented and tabletop-tested annually
Reporting cadence	None	Summary to AI governance	Detailed report to AI governance + risk/board committee

AI-Specific Vendor Assessment Questionnaire

Section A — Model Information

Which AI/ML models power the service? List provider, model family, and current version.
How are model updates handled? What is the customer notification process and lead time?
Is a model card or equivalent documentation available on request?
Does the vendor maintain a documented evaluation suite (accuracy, safety, bias)?

Section B — Training Data Governance

Is customer data used to train, fine-tune, or improve any vendor model? If yes, under what consent mechanism?
Are training data sources documented? Is provenance traceable?
Are bias or fairness evaluation results available?
How are data subject rights (GLBA, state privacy law) supported?

Section C — Output Quality and Safety

What output safety guardrails exist (content filters, jailbreak resistance)?
How is hallucination rate measured and reported?
What is the incident response process for AI-specific failures (e.g., harmful or non-compliant output)?
Is human-in-the-loop available for high-risk operations?

Section D — Transparency and Explainability

Are outputs explainable or auditable to the prompt and tool calls that produced them?
Is end-to-end logging available to the customer for AI interactions?
Does the vendor support content provenance signals (e.g., C2PA) for generated content?

Section E — Compliance and Regulatory Alignment

Which AI-specific certifications or attestations does the vendor hold (e.g., ISO/IEC 42001)?
Does the vendor support customer regulatory examinations (FINRA, SEC, OCC, Federal Reserve)?
Is a documented AI governance framework (NIST AI RMF or equivalent) in place?

Section F — Model Risk and Recordkeeping Integration

Does the vendor support OCC Bulletin 2026-13 (formerly OCC 2011-12) / Fed SR 26-2 (formerly SR 11-7) model risk management workflows (model inventory, validation evidence)?
Can AI-generated content be archived in a SEC 17a-4(f) / FINRA 4511-compliant manner?
Can records be distinguished as AI-generated vs human-generated?

AI Vendor Risk Scoring Rubric

Factor	Weight	Low (1)	High (5)
Model transparency	20%	Full disclosure (model card, version, eval)	Black box, undocumented
Training data governance	15%	Documented, audited, no customer training	Undisclosed, customer data used
Output quality controls	20%	Comprehensive guardrails, measured	Minimal, unmeasured
Customer data protection	15%	No training, regional residency, encryption	Unrestricted use, unclear residency
Regulatory readiness	15%	ISO 42001 / NIST AI RMF aligned, exam-ready	No attestations, no exam support
Incident response	15%	Defined AI-incident process, tested	Undefined / ad-hoc

Weighted score ≥ 4.0 → Tier T5 (high); 2.5–3.9 → enhanced monitoring; ≤ 2.4 → standard.

Dynamic Tool / Plugin / MCP Governance Checklist

Approved tools recorded per agent; supported DLP/ACP restrictions tested
MCP Allow all turned off; only approved individual tools enabled; new tools checked as disabled by default
Independent-publisher and community tools removed or blocked using the applicable supported connector/agent controls unless reviewed
Upstream changes reviewed through controlled vendor processes; no assumed universal tool auto-update-disable switch
Transitive data-flow map exists for every Zone 3 multi-tool agent
Applicable app/connector approvals and maker permissions documented; no assumed universal Copilot Studio tenant plugin-marketplace allowlist
Tool-call logging coverage tested for request/response metadata and identity; gaps/redaction/retention recorded
Quarterly review of approved tool inventory completed and signed off
Actual MCP hosting/service route verified; network controls tested only for traffic they govern, not inferred from corporate device egress

Product evidence: MCP tool selection documents the Allow all default and new-tool behavior. ACP blocks supported MCP servers, not individual MCP tools; custom/HTTP/virtual connector gaps need classic DLP. Copilot Studio data policies describe maker/publish violation details. Environment roles establish custom connector creation rights. Agent telemetry depends on configured logging and sensitive-data settings, not a universal Purview request/response feed.

For model/provider evidence, capture Overview → Model, release/retirement status, actual route-specific terms and Microsoft 365 provider access plus PPAC environment/group controls. Native availability is not proof of GA or identical coverage; see the portal model review.

Transitive Data Exposure Map (Template)

TRANSITIVE DATA EXPOSURE MAP
============================
Agent:        [Agent name]                     Zone: [1/2/3]
Owner:        [Business owner]                 Reviewed: [YYYY-MM-DD]

Tool 1: [Connector / plugin / MCP]
  Publisher: [Actual publisher organization — not the protocol name]
  Hosting operator / service route: [Actual operator and downstream services]
  Data sent: [Class — e.g., customer NPI, MNPI, supervisory log]
  Vendor:    [Name]
  Subprocessors: [List]
  Residency: [Region]
  Recordkeeping coverage: [Yes / No / Partial — reference]

Tool 2: ...

Tool N: ...

Aggregate risk:
  Number of distinct vendors reached: [N]
  Number of distinct subprocessors reached: [N]
  Customer data leaves [primary region]?: [Yes / No]
  Recordkeeping gaps identified: [List]
  Compensating controls: [List]
  Recommendation: [Allow / Restrict / Block / Re-architect]

Contract Requirements Checklist

Standard clauses

Encryption (in transit and at rest)
Incident notification SLA (24 hours for critical)
Audit rights (annual minimum) or SOC 2 acceptance
Subprocessor approval and notification
Data return / destruction at termination
Indemnification aligned to data class handled

AI-specific clauses

Material model change notice (≥ 30 days)
Training data restrictions (no training on customer data without consent)
AI incident notification (24 hours for integrity / safety incidents)
Output monitoring and audit support
Human oversight provisions where decisions affect customers
Explainability commitments

FSI-specific clauses

Books-and-records vendor: SEC 17a-4(f) / FINRA 4511 attestation
Communication archiving vendor: AI-generated content tagging
Identity verification vendor: synthetic identity / deepfake detection capability
LLM provider: data residency aligned to firm's books-and-records jurisdiction

Evidence Package for Examination

For registered Microsoft 365/Agent 365 plugins and MCP servers, capture actual publisher-category settings, plugin user availability and separate server block state. If individual-tool runtime controls are exposed, record supported server type, discovery and Tooling Gateway scope; rollout is not universal. Test dependent-plugin behavior separately from other plugins using the same server (registered tool controls).

For each examination cycle, assemble:

Latest resource inventory and raw custom connector snapshot, coverage register and SHA-256 manifest. The snapshot omits solution-packaged custom connectors (documented Get-AdminPowerAppConnector limitation), so the register must show reconciliation with solution contents
Raw DLP policies, separately retrieved custom Host URL patterns/connector configurations, ACP scope/status, and actual enforcement test results. Get-DlpPolicy output omits new default-group connectors that were never moved and can show stale connector names; record the portal reconciliation that covers them
Custom connector inventory, actual maker-role assignments and usage-restriction evidence; no invented creation-switch screenshot
Per-vendor due diligence pack (Tier T2–T5)
Quarterly vendor risk reports for the prior 12 months
Transitive data-flow maps for Zone 3 multi-tool agents
Exit-plan tabletop minutes for Zone 3 critical vendors
Records-retention configuration showing FINRA 4511 / SEC 17a-4 alignment

Updated: November 2026 | Version: v1.6.2 | UI Verification Status: Needs Review (documentation-only; no tenant tests)