# Technical Due Diligence for AI/ML Products: What to Evaluate Before Acquiring — Hunchbite > How to evaluate AI and machine learning products before acquisition — model ownership, data moats, inference costs, retraining pipelines, benchmark integrity, and the difference between real AI and GPT wrappers. Source: https://hunchbite.com/guides/technical-due-diligence-ai-ml-products [Home](https://hunchbite.com/)/[Guides](https://hunchbite.com/guides)/[Rescuing Software](https://hunchbite.com/guides/topic/fix-my-software)/Technical Due Diligence for AI/ML Products: What to Evaluate Before Acquiring [Rescuing Software](https://hunchbite.com/guides/topic/fix-my-software) # Technical Due Diligence for AI/ML Products: What to Evaluate Before Acquiring How to evaluate AI and machine learning products before acquisition — model ownership, data moats, inference costs, retraining pipelines, benchmark integrity, and the difference between real AI and GPT wrappers. By HunchbiteMarch 12, 202612 min read due diligenceAImachine learning > **AI products** are the most opaque category to evaluate in a technical acquisition. The surface-level demo is impressive; the engineering reality underneath — model ownership, data provenance, inference economics, and benchmark integrity — is where the real value (or risk) lives. This guide is for acquirers evaluating AI-native products, ML-enhanced software, and LLM-based applications. It covers what the demo doesn't show you. ## The AI stack: what you're actually buying Before evaluating anything else, establish what layer of the AI stack the product actually occupies: | Layer | What it means | Value and risk | | ------------------------------- | --------------------------------------------------- | --------------------------------------------- | | Foundation model provider | OpenAI, Anthropic, Google — the model itself | Not acquirable as a product | | Fine-tuned model | Pre-trained model adapted on proprietary data | Valuable if the data is owned | | Retrieval-augmented application | LLM + proprietary document store (RAG) | Valuable if the data corpus is proprietary | | Orchestration layer | Prompts, chains, agents built on foundation models | Valuable if the workflow design is defensible | | Data collection business | Product that generates training data as a byproduct | Highest long-term defensibility | Most acquired "AI products" are in the orchestration or RAG categories. That's not inherently bad — but the valuation should reflect it. ## Model ownership and IP ### What do they actually own? * **Foundation model:** They almost certainly don't own this. They have a licence from OpenAI, Anthropic, or similar. * **Fine-tuned weights:** If they've fine-tuned on proprietary data, who owns the resulting weights? The model provider's terms vary — review them. * **Prompts:** Prompts are generally not patentable and are difficult to protect as IP. Their value is operational, not legal. * **Training data:** This is the actual IP question. See below. ### Training data provenance The legal risk landscape for AI training data is actively evolving: * **Scraped web data:** May be subject to ongoing litigation (NYT v. OpenAI, others). If the product was trained on scraped data without explicit licences, assess legal exposure. * **User-generated data:** Was user consent obtained to use interaction data for training? In EU markets, this is a GDPR issue. * **Licensed data:** What are the licence terms? Can the licence be transferred in an acquisition? * **Proprietary data:** The most defensible. Is it actually proprietary, or is it available elsewhere? Ask for the training data documentation. If there isn't any, that's a red flag. Hunchbite Service #### Technical Due Diligence Independent technical assessment for investors, acquirers, and founders — a report you can put in front of a board. [Request due diligence](https://hunchbite.com/technical-due-diligence) ## Benchmark integrity AI product benchmarks are frequently misleading. Common problems: ### Test set contamination The test set used to report accuracy should be held out — never used to train or tune the model. If the team that built the model also built the benchmark, ask how they ensured the test data wasn't leaked into training. ### Cherry-picked examples Demo outputs are selected to be impressive. Request a random sample of 50 production outputs from the last 30 days. Evaluate those. ### Metric gaming A model can achieve high accuracy on a benchmark by exploiting statistical patterns in the test set rather than demonstrating general capability. Ask for the failure cases — what does the model get wrong, and how often? ### Comparison to baseline What is the performance improvement over a baseline (GPT-4o with a basic prompt, or a rule-based system)? A 2% improvement over a baseline that costs 10x less to operate is not a moat. ## Inference economics This is the hidden cost most acquirers miss. ### Cost per inference * What is the compute cost per API call or model inference? * Is this using a third-party API (billed per token/call) or self-hosted inference (GPU costs)? * What is the current gross margin after compute costs? ### Scaling economics Model inference costs as a function of user scale: * Linear with requests (most API-based products) * Super-linear if context length grows with usage * Fixed if using self-hosted inference at sufficient scale A product at ₹50L ARR with 40% gross margins might have 15% gross margins at ₹5Cr ARR if inference costs scale linearly and compute isn't optimised. ### Model provider dependency * Is the product locked to a single model provider? * What happens if OpenAI raises prices or changes API terms? * Is there a strategy to diversify model providers or move to open-source models? Inference economics and benchmark claims are exactly the areas where a seller's numbers and the engineering reality diverge — if the deal is large enough that being wrong is expensive, [get an independent code and AI audit before you close →](https://hunchbite.com/audit) ## The data flywheel: is there one? The most valuable AI businesses have data flywheels — the product generates training data that improves the model, which improves the product, which attracts more users, which generates more data. Evaluate: * Does the product collect feedback on model outputs? (Thumbs up/down, corrections, implicit signals) * Is this feedback actually used to improve the model, or just logged? * How frequently is the model retrained or fine-tuned? * Does the model demonstrably improve over time? A product without a data flywheel is a point-in-time implementation. One with a working flywheel has compounding value. ## MLOps and model lifecycle ### Retraining pipeline * How is the model retrained? Is this automated or manual? * What triggers a retraining cycle? (Schedule, performance degradation, data volume threshold) * How long does a retraining cycle take? * Is there a process for evaluating model quality before deploying a new version? ### Model versioning and rollback * Are model versions tracked? * Can the team roll back to a previous model version if a new one performs worse? * How are model changes communicated to users if output behaviour changes? ### Monitoring and drift detection * Is model performance monitored in production? * Is there alerting for performance degradation? * Is there a process for detecting data drift (the distribution of inputs changing over time)? Without monitoring, a model can silently degrade in production for months before anyone notices. ## Hallucination and reliability For AI products where accuracy matters (legal, medical, financial, factual Q&A): * What is the factual accuracy rate? * How are hallucinations detected and handled? * Is there a citation or source-attribution mechanism to support outputs? * Are users shown confidence levels, or is output presented as authoritative? A product where hallucinations are business-critical risks (wrong legal advice, incorrect financial calculations) requires explicit hallucination mitigation — not just disclaimers. If the product applies AI to a domain where errors are financially material — automating payroll, say — pair this checklist with [technical due diligence for HR and payroll software](https://hunchbite.com/guides/technical-due-diligence-hr-payroll-software), where calculation accuracy and labour-law compliance leave no room for approximation. ## Third-party AI provider risk If the product depends on a third-party foundation model: * What are the uptime SLA commitments from the model provider? * What is the fallback if the provider has an outage? * What happens if the provider discontinues a model version? (GPT-3.5 is being phased out; products built on it must migrate) * Is there a contractual data processing agreement with the model provider for any user data sent in prompts? Hunchbite Service #### Code Audit Services An independent, written assessment of code quality, security, and architecture — with prioritised, actionable fixes. [Request a code audit](https://hunchbite.com/code-audit-services) ## Valuation considerations | Finding | Implication | |---------|-----------|| | No proprietary training data | Value is workflow, not AI — adjust multiple | | Inference margins below 30% | Profitability risk at scale | | No data flywheel | Point-in-time moat, not compounding | | Single model provider dependency | Provider risk; factor in migration cost | | Test set contamination suspected | Discount performance claims | | No retraining pipeline | Model will stagnate; budget for MLOps buildout | | User data used for training without clear consent | Legal risk in EU/UK markets | --- ## Need this done for you? AI product acquisitions fail when the buyer doesn't know what they actually own. Hunchbite conducts AI-specific technical due diligence — model ownership, inference cost, training data provenance, MLOps maturity, and a written report with risk ratings and remediation estimates before you close. **[→ Technical Due Diligence Service](https://hunchbite.com/technical-due-diligence)** Call [+91 90358 61690](tel:+919035861690) · [Book a free call](https://cal.com/hunchbite/discovery) · [Contact form](https://hunchbite.com/contact) Ask an AI assistant about this guide: [ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fhunchbite.com%2Fguides%2Ftechnical-due-diligence-ai-ml-products%20and%20help%20me%20apply%20it%20to%20my%20situation.%20Answer%20my%20questions%20about%20it.&hints=search) · [Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fhunchbite.com%2Fguides%2Ftechnical-due-diligence-ai-ml-products%20and%20help%20me%20apply%20it%20to%20my%20situation.%20Answer%20my%20questions%20about%20it.) · [Perplexity](https://www.perplexity.ai/search?q=Read%20https%3A%2F%2Fhunchbite.com%2Fguides%2Ftechnical-due-diligence-ai-ml-products%20and%20help%20me%20apply%20it%20to%20my%20situation.%20Answer%20my%20questions%20about%20it.) On this page * [The AI stack: what you're actually buying](#the-ai-stack-what-youre-actually-buying) * [Model ownership and IP](#model-ownership-and-ip) * [What do they actually own?](#what-do-they-actually-own) * [Training data provenance](#training-data-provenance) * [Benchmark integrity](#benchmark-integrity) * [Test set contamination](#test-set-contamination) * [Cherry-picked examples](#cherry-picked-examples) * [Metric gaming](#metric-gaming) * [Comparison to baseline](#comparison-to-baseline) * [Inference economics](#inference-economics) * [Cost per inference](#cost-per-inference) * [Scaling economics](#scaling-economics) * [Model provider dependency](#model-provider-dependency) * [The data flywheel: is there one?](#the-data-flywheel-is-there-one) * [MLOps and model lifecycle](#mlops-and-model-lifecycle) * [Retraining pipeline](#retraining-pipeline) * [Model versioning and rollback](#model-versioning-and-rollback) * [Monitoring and drift detection](#monitoring-and-drift-detection) * [Hallucination and reliability](#hallucination-and-reliability) * [Third-party AI provider risk](#third-party-ai-provider-risk) * [Valuation considerations](#valuation-considerations) * [Need this done for you?](#need-this-done-for-you) Ask AI about this guide * [Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fhunchbite.com%2Fguides%2Ftechnical-due-diligence-ai-ml-products%20and%20help%20me%20apply%20it%20to%20my%20situation.%20Answer%20my%20questions%20about%20it.&hints=search) * [Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fhunchbite.com%2Fguides%2Ftechnical-due-diligence-ai-ml-products%20and%20help%20me%20apply%20it%20to%20my%20situation.%20Answer%20my%20questions%20about%20it.) * [Open in Perplexity](https://www.perplexity.ai/search?q=Read%20https%3A%2F%2Fhunchbite.com%2Fguides%2Ftechnical-due-diligence-ai-ml-products%20and%20help%20me%20apply%20it%20to%20my%20situation.%20Answer%20my%20questions%20about%20it.) FAQ How do you tell if an AI product has a real moat or is just a GPT wrapper? Ask: what happens if you remove the third-party AI API and replace it with a competing one? If the answer is 'almost nothing changes,' there's no AI moat. Real AI moats come from proprietary training data, fine-tuned models, or feedback loops that improve the model over time. A wrapper product's value is the product design and distribution, not the AI — which isn't a bad business, but it's valued differently. What are the biggest hidden costs in an AI/ML product acquisition? Inference costs. A product that looks profitable at current scale may be deeply unprofitable at 5x scale if the cost per inference is high. GPU compute for inference is expensive — especially for large model calls. Request a breakdown of cost of goods sold (COGS) that includes compute and API costs, and model how margins behave as user volume grows. How do you evaluate the quality of training data in an AI product? Data quality is harder to evaluate than code quality. Focus on: Is the training data owned, licensed, or scraped? Scraped web data creates legal risk (see recent litigation). Is there a documented data pipeline — how data was collected, cleaned, and labeled? Is there a test set that's genuinely held out, or has the test set been used to tune the model (test set contamination)? A product whose benchmarks were built by the team that built the model is not a reliable benchmark. Next step ## Acquiring an AI or ML product? Hunchbite runs AI-specific technical due diligence — model ownership, training-data provenance, inference economics, benchmark integrity, and MLOps maturity — and delivers a written report with risk ratings and remediation estimates before you close. Start with a free audit and know exactly what you'd be buying. [Get a Free Audit](https://hunchbite.com/audit)[Technical Due Diligence](https://hunchbite.com/technical-due-diligence) Trusted by VMAC Industries, TKD Logistics, Astitva Jewellery & more. [See our recent work →](https://hunchbite.com/case-studies) Fixed-price, no hourly billing · No obligation · We tell you upfront if we're not a fit Continue Reading [Rescuing SoftwareThe Seller's Guide to Technical Due DiligenceHow to prepare your software for technical due diligence as a seller — what buyers will check, what kills deals, what to fix before access is granted, and how to present your technology confidently.11 min read](https://hunchbite.com/guides/sellers-guide-technical-due-diligence) [Rescuing SoftwareTechnical Due Diligence for SaaS Acquisitions: An Investor's GuideHow to evaluate the technology behind a SaaS acquisition — what to check beyond the standard code review, how findings affect ARR multiples, and what makes SaaS tech a strong or weak asset.12 min read](https://hunchbite.com/guides/tech-due-diligence-saas-acquisition) [All Guides](https://hunchbite.com/guides)