Adding GenAI to a .NET SaaS: real costs, timelines, traps

The question people are really asking

When a CTO or a founder reaches out to "add some AI" to their product, the question behind the question is almost always the same: how much does it cost, how long does it take, and what can go wrong?

This article answers all three, no hedging. It draws on what I've built and operated: a Privacy Shield that masks PII before any LLM call (PromptVault), a CV-scoring pipeline with a capped cost per candidate (Matchr), a "cited or rejected" anti-hallucination engine (Vouch), and a forecasting case where the LLM lost to ML.NET (SaleCast).

The three cost centers — two of them invisible at POC stage

1. Development (the one you see)

Calling an LLM is three lines of code. What takes time is everything that makes the call production-worthy:

• the prompt and its context: assembling the right business data, in the right window, without leaking anything that must not leave your systems;
• the structured output: an LLM returns text; your product needs typed, validated objects, with a defined behaviour when the model answers off-shape;
• the fallbacks: timeout, provider rate-limit, invalid response — every failure path needs a product answer, not a stack trace;
• the provider abstraction: being able to switch OpenAI → Claude → Azure OpenAI without rewriting the feature. In .NET, Microsoft.Extensions.AI makes this clean — it's the ILogger of LLMs.

2. Tokens (the one you underestimate)

The unit cost of a call looks trivial. Multiplied by your users, per request, per day, it becomes a COGS line that moves with your usage — and that nobody caps at POC stage.

On Matchr, every CV analysis has a cost capped by construction: a model sized to the task, context trimmed to what scoring needs, caching on the invariants. That's not premature optimization: it's what lets you advertise a price per candidate without discovering your margin after the fact.

Two rules that hold:

• the most expensive model is almost never needed everywhere — route simple tasks (classification, extraction) to a fast model and reserve the big model for the steps that create value;
• every AI feature must have a token budget per operation, measured and alerted — otherwise you discover the overrun on the invoice, six weeks later.

3. Continuous evaluation (the one you discover too late)

This is the most neglected center. A prompt is not code: there's no compiler or type system to tell you a change broke a case. Without an evaluation set (real inputs, expected outputs, a score), every prompt improvement is an invisible potential regression.

Concretely, from production day one you need:

• an eval set versioned with the code, replayed on every prompt or model change;
• drift monitoring: providers update their models, and a silent update can degrade your outputs without a single line of your code changing;
• application-level guardrails: on Vouch, every generated statement must cite its source document or it's rejected — the model isn't allowed to improvise in a compliance-audit context.

Build vs API: that's rarely the real question

"Do we use a provider API or host our own model?" is rarely the right first question. For the vast majority of SaaS products, the answer is a provider API (directly or via Azure) — self-hosting is only justified by strong sovereignty constraints or a volume that changes the order of magnitude of the bill.

The real scoping questions are:

1. What data leaves for the provider, and in what form? If your prompts contain personal data, you have a GDPR problem before you have an AI problem. That's exactly what masking PII before the call solves — reversible on the way back, so the user sees the real value but the model doesn't.
2. Is the LLM even the right tool? On SaleCast I pitted nine algorithms against each other on e-commerce forecasting: classical ML (ML.NET) beats the LLM on that task, for a fraction of the inference cost. An LLM is not a reflex, it's a choice you justify task by task.
3. What happens when the model is wrong? Not "if" — "when". The answer defines your bar for guardrails, and therefore a significant share of the budget.

Realistic timelines

Orders of magnitude observed across my projects, for an existing .NET team:

• Honest POC: ~2 weeks. "Honest" means: on your real data, with the ugly cases, and a mini eval set. A POC on three hand-picked examples proves nothing — it only buys the budget for the real POC.
• Production feature: 6 to 10 weeks. The delta between the two is everything this article describes: structured outputs, fallbacks, cost ceilings, evals, monitoring, and clean integration into your existing product.
• Industrialization (several AI features): that's a platform. Provider abstraction, prompt observability, centralized budget management — built once, amortized across every following feature.

The five traps that cost the most

1. The demo drives the roadmap. An LLM demo is always spectacular. Deciding on the demo instead of an eval set means buying the peak of the enthusiasm curve.
2. No cost ceiling per operation. COGS discovered on the invoice, after launch, when the public price is already announced.
3. PII leaks into the prompts. The issue shows up in no demo and surfaces at the first audit — or the first incident. Mask before the call, log what leaves, done.
4. No provider exit strategy. Prices and models move fast. Without an abstraction layer, every market shift is a project instead of a config change.
5. AI bolted where a rule would do. If the task is deterministic, an LLM adds cost, latency and uncertainty. Half the value of a scoping audit is the list of what does not need AI.

When I say "no AI here"

Three cases where I say it explicitly during scoping:

• the task is deterministic — a business rule does it better, for nothing;
• the cost of an error is unacceptable and unverifiable — if no guardrail can validate the output, the LLM has no place on that critical path;
• the volume makes inference absurd — classical ML trained on your data does the job at near-zero marginal cost.

Saying no to AI on one perimeter is often what makes the yes credible on the rest.

Key takeaways

• Budget three centers: development, tokens, continuous evaluation. The POC only shows the first.
• Demand a POC on your real data with an eval set — 2 well-scoped weeks beat a perfect demo.
• Cap the cost per operation before launch, not after the first invoice.
• Handle PII before the LLM call, not in a post-incident ticket.
• And accept that part of the scope doesn't need AI — that's a sign of healthy scoping, not a failure.

If you have a GenAI subject in a .NET stack and want a straight opinion on what's worth doing — the first conversation is free.