AI Vendor Due Diligence Questionnaire

An AI Vendor Due Diligence Questionnaire systematically evaluates AI service providers across algorithm transparency, data handling, bias controls, and operational security. It maps vendor practices to your control framework requirements—from SOC 2 AI principles to ISO/IEC 23053 standards—while collecting evidence for model governance, training data lineage, and explainability documentation.

Key takeaways:

• Covers 8 critical domains: algorithm governance, data practices, bias/fairness, security controls, compliance alignment, performance monitoring, incident response, and contractual terms
• Maps directly to SOC 2 AI criteria, ISO 27001 Annex A controls, NIST AI RMF, and sector-specific requirements (FFIEC, HIPAA)
• Requires evidence collection beyond standard DDQs: model cards, bias testing reports, drift monitoring logs
• Risk tiers vendors based on AI model criticality, data sensitivity, and decision impact
• Integrates with existing TPRM workflows but adds AI-specific control validation steps

Your standard vendor questionnaire won't catch algorithmic bias, model drift, or training data poisoning. AI vendors introduce risks that traditional security assessments miss entirely—from unexplainable decision-making to perpetual data retention for model improvement.

An AI-specific DDQ bridges this gap. It translates emerging AI governance standards into concrete evidence requests your vendors can actually fulfill. Think of it as your regular security questionnaire plus targeted probes for algorithmic transparency, fairness testing, and model lifecycle management.

The questionnaire serves three purposes: risk discovery (what AI capabilities exist), control validation (how risks are managed), and evidence collection (proof of stated controls). Unlike generic AI questionnaires floating around, this framework ties each question to specific regulatory requirements and control objectives you're already tracking.

Core Questionnaire Sections

1. Algorithm Governance and Transparency

Start with the fundamentals: what models power the service, how decisions get made, and who oversees the AI systems.

Key evidence to collect:

• Model architecture documentation (not proprietary code, but high-level design)
• Decision logic explanations for your specific use cases
• Model governance charter showing approval processes
• Change management procedures for algorithm updates

Control mapping: SOC 2 CC7.1-7.3 (System Monitoring), ISO 27001 A.12.1.2 (Change Management)

2. Data Practices and Privacy

AI vendors consume data differently than traditional SaaS providers. They need historical data for training, ongoing data for inference, and often retain both indefinitely.

Critical questions focus on:

• Training data sources and licensing
• Data retention periods for model improvement vs. service delivery
• Cross-customer data usage and isolation controls
• Right to deletion implementation (true deletion vs. anonymization)

Regulatory alignment: GDPR Articles 5, 17, 22; CCPA Section 1798.100; HIPAA Security Rule (for healthcare data)

3. Bias Detection and Fairness

This section separates mature AI vendors from experimental ones. Look for systematic approaches, not philosophical statements.

Evidence requirements:

• Bias testing methodology and frequency
• Protected class definitions used in testing
• Actual bias test results for similar use cases
• Remediation procedures when bias is detected

Framework references: NIST AI RMF GOVERN 1.2, MAP 1.5, MEASURE 2.2

4. Security Controls for AI Systems

Standard security controls plus AI-specific threats like model inversion attacks, data poisoning, and adversarial inputs.

Additional controls beyond standard InfoSec:

• Model access logging and anomaly detection
• Training data integrity verification
• Inference API rate limiting and abuse prevention
• Model IP protection measures

5. Performance Monitoring and Drift Detection

AI models degrade. Your DDQ needs to verify ongoing performance management, not just point-in-time accuracy.

Key metrics to track:

• Baseline performance benchmarks
• Drift detection thresholds and alerting
• Retraining triggers and schedules
• Performance SLAs specific to your use case

Industry-Specific Applications

Financial Services

FFIEC guidance on model risk management (SR 11-7) requires:

• Independent model validation reports
• Ongoing monitoring procedures
• Conceptual soundness documentation
• Outcomes analysis comparing predictions to actual results

Add questions on fair lending compliance (Reg B), explainability for adverse actions, and alignment with OCC's third-party risk management guidance.

Healthcare

Beyond HIPAA, consider FDA guidance on clinical decision support software:

• Intended use statements and limitations
• Clinical validation studies
• Post-market surveillance procedures
• Physician-in-the-loop requirements

Technology and SaaS

Focus on:

• API stability during model updates
• Backwards compatibility commitments
• Multi-tenancy isolation for model training
• Customer-specific model tuning options

Implementation Best Practices

1. Risk-Tier Your AI Vendors

Not every AI component needs deep scrutiny. Create tiers based on:

Risk Factor	High Risk	Medium Risk	Low Risk
Decision Impact	Customer-facing automated decisions	Internal analytics	Content recommendations
Data Sensitivity	PII/PHI processing	Aggregated data	Public data only
Regulatory Exposure	Regulated decisions (credit, healthcare)	Compliance-adjacent	No regulatory impact
Business Criticality	Core business process	Operational efficiency	Nice-to-have features

2. Phase Your Evidence Collection

Don't demand everything upfront. Structure requests by implementation phase:

Phase 1 (Pre-contract): Basic AI capabilities, high-level architecture, key certifications Phase 2 (Onboarding): Detailed technical documentation, test results, governance procedures
Phase 3 (Ongoing): Performance reports, drift analyses, incident logs

3. Automate Where Possible

Manual AI assessments don't scale. Build automation for:

• Initial questionnaire routing based on AI capability declarations
• Evidence expiration tracking (bias tests, performance benchmarks)
• Control mapping to your existing framework requirements
• Risk scoring based on questionnaire responses

4. Create Reciprocal Transparency

Share your AI governance requirements upfront. Vendors who can't meet basic transparency standards will self-select out early, saving everyone time.

Common Implementation Mistakes

1. Treating All AI Equally

A spell-checker using NLP has different risk profiles than a credit decisioning model. Your questionnaire should branch based on AI application type.

2. Accepting Vague Responses

"We use responsible AI practices" isn't evidence. Demand specific procedures, test results, and governance artifacts. If they can't produce documentation, they don't have controls.

3. Ignoring Update Cycles

AI models change faster than traditional software. Quarterly assessments miss critical updates. Build continuous monitoring into your program from day one.

4. Focusing Only on the Model

The model is some the risk. Data pipelines, feature engineering, deployment infrastructure, and human oversight processes contain the other 80%.

5. Neglecting Contractual Protections

Technical controls mean nothing without contractual backing. Ensure your agreements include:

• Right to audit AI systems specifically
• Notification requirements for material model changes
• Performance degradation remedies
• Liability allocation for algorithmic decisions

Frequently Asked Questions

How is an AI vendor questionnaire different from our standard security questionnaire?

Standard questionnaires assess data protection and availability. AI questionnaires add algorithm transparency, bias testing, model governance, and performance monitoring—risks that don't exist in traditional software.

What if vendors claim proprietary concerns when we ask for algorithm details?

You don't need source code. Request model cards, high-level architecture, feature importance rankings, and decision tree visualizations. These provide transparency without exposing IP.

Should we require SOC 2 Type 2 reports with AI-specific controls?

Yes, but most vendors aren't there yet. Accept SOC 2 + supplementary AI governance documentation. Look for SOC 2 Section 9 criteria on AI as an interim measure.

How do we assess vendors using third-party AI models (like GPT-4oo or Claude)?

Focus on their implementation controls: prompt engineering standards, output filtering, use case restrictions, and fallback procedures. They can't control the base model but should govern its application.

What evidence actually proves "unbiased" AI?

No AI is perfectly unbiased. Look for regular bias testing reports, clear testing methodology, defined protected classes, threshold policies, and documented remediation when bias is found.

How often should we reassess AI vendors given model updates?

Annually for full assessments, quarterly for high-risk vendors. Require notification of material changes and right to reassess after major updates.

Can we use the same questionnaire for all types of AI vendors?

Use a modular approach. Core questions apply universally, then add modules for computer vision, NLP, predictive analytics, or generative AI based on vendor capabilities.