Technical Due Diligence for AI/ML Products: What to Evaluate Before Acquiring

AI products are the most opaque category to evaluate in a technical acquisition. The surface-level demo is impressive; the engineering reality underneath — model ownership, data provenance, inference economics, and benchmark integrity — is where the real value (or risk) lives.

This guide is for acquirers evaluating AI-native products, ML-enhanced software, and LLM-based applications. It covers what the demo doesn't show you.

The AI stack: what you're actually buying

Before evaluating anything else, establish what layer of the AI stack the product actually occupies:

Layer	What it means	Value and risk
Foundation model provider	OpenAI, Anthropic, Google — the model itself	Not acquirable as a product
Fine-tuned model	Pre-trained model adapted on proprietary data	Valuable if the data is owned
Retrieval-augmented application	LLM + proprietary document store (RAG)	Valuable if the data corpus is proprietary
Orchestration layer	Prompts, chains, agents built on foundation models	Valuable if the workflow design is defensible
Data collection business	Product that generates training data as a byproduct	Highest long-term defensibility

Most acquired "AI products" are in the orchestration or RAG categories. That's not inherently bad — but the valuation should reflect it.

Model ownership and IP

What do they actually own?

• Foundation model: They almost certainly don't own this. They have a licence from OpenAI, Anthropic, or similar.
• Fine-tuned weights: If they've fine-tuned on proprietary data, who owns the resulting weights? The model provider's terms vary — review them.
• Prompts: Prompts are generally not patentable and are difficult to protect as IP. Their value is operational, not legal.
• Training data: This is the actual IP question. See below.

Training data provenance

The legal risk landscape for AI training data is actively evolving:

• Scraped web data: May be subject to ongoing litigation (NYT v. OpenAI, others). If the product was trained on scraped data without explicit licences, assess legal exposure.
• User-generated data: Was user consent obtained to use interaction data for training? In EU markets, this is a GDPR issue.
• Licensed data: What are the licence terms? Can the licence be transferred in an acquisition?
• Proprietary data: The most defensible. Is it actually proprietary, or is it available elsewhere?

Ask for the training data documentation. If there isn't any, that's a red flag.

Benchmark integrity

AI product benchmarks are frequently misleading. Common problems:

Test set contamination

The test set used to report accuracy should be held out — never used to train or tune the model. If the team that built the model also built the benchmark, ask how they ensured the test data wasn't leaked into training.

Cherry-picked examples

Demo outputs are selected to be impressive. Request a random sample of 50 production outputs from the last 30 days. Evaluate those.

Metric gaming

A model can achieve high accuracy on a benchmark by exploiting statistical patterns in the test set rather than demonstrating general capability. Ask for the failure cases — what does the model get wrong, and how often?

Comparison to baseline

What is the performance improvement over a baseline (GPT-4o with a basic prompt, or a rule-based system)? A 2% improvement over a baseline that costs 10x less to operate is not a moat.

Inference economics

This is the hidden cost most acquirers miss.

Cost per inference

• What is the compute cost per API call or model inference?
• Is this using a third-party API (billed per token/call) or self-hosted inference (GPU costs)?
• What is the current gross margin after compute costs?

Scaling economics

Model inference costs as a function of user scale:

• Linear with requests (most API-based products)
• Super-linear if context length grows with usage
• Fixed if using self-hosted inference at sufficient scale

A product at ₹50L ARR with 40% gross margins might have 15% gross margins at ₹5Cr ARR if inference costs scale linearly and compute isn't optimised.

Model provider dependency

• Is the product locked to a single model provider?
• What happens if OpenAI raises prices or changes API terms?
• Is there a strategy to diversify model providers or move to open-source models?

The data flywheel: is there one?

The most valuable AI businesses have data flywheels — the product generates training data that improves the model, which improves the product, which attracts more users, which generates more data.

Evaluate:

• Does the product collect feedback on model outputs? (Thumbs up/down, corrections, implicit signals)
• Is this feedback actually used to improve the model, or just logged?
• How frequently is the model retrained or fine-tuned?
• Does the model demonstrably improve over time?

A product without a data flywheel is a point-in-time implementation. One with a working flywheel has compounding value.

MLOps and model lifecycle

Retraining pipeline

• How is the model retrained? Is this automated or manual?
• What triggers a retraining cycle? (Schedule, performance degradation, data volume threshold)
• How long does a retraining cycle take?
• Is there a process for evaluating model quality before deploying a new version?

Model versioning and rollback

• Are model versions tracked?
• Can the team roll back to a previous model version if a new one performs worse?
• How are model changes communicated to users if output behaviour changes?

Monitoring and drift detection

• Is model performance monitored in production?
• Is there alerting for performance degradation?
• Is there a process for detecting data drift (the distribution of inputs changing over time)?

Without monitoring, a model can silently degrade in production for months before anyone notices.

Hallucination and reliability

For AI products where accuracy matters (legal, medical, financial, factual Q&A):

• What is the factual accuracy rate?
• How are hallucinations detected and handled?
• Is there a citation or source-attribution mechanism to support outputs?
• Are users shown confidence levels, or is output presented as authoritative?

A product where hallucinations are business-critical risks (wrong legal advice, incorrect financial calculations) requires explicit hallucination mitigation — not just disclaimers.

Third-party AI provider risk

If the product depends on a third-party foundation model:

• What are the uptime SLA commitments from the model provider?
• What is the fallback if the provider has an outage?
• What happens if the provider discontinues a model version? (GPT-3.5 is being phased out; products built on it must migrate)
• Is there a contractual data processing agreement with the model provider for any user data sent in prompts?

Valuation considerations

| Finding | Implication | |---------|-----------|| | No proprietary training data | Value is workflow, not AI — adjust multiple | | Inference margins below 30% | Profitability risk at scale | | No data flywheel | Point-in-time moat, not compounding | | Single model provider dependency | Provider risk; factor in migration cost | | Test set contamination suspected | Discount performance claims | | No retraining pipeline | Model will stagnate; budget for MLOps buildout | | User data used for training without clear consent | Legal risk in EU/UK markets |

---

Evaluating an AI or ML product for acquisition and want to understand what's actually under the hood? Contact us — we conduct AI-specific technical due diligence covering model ownership, inference economics, data provenance, and MLOps maturity.