Almost every product team we talk to now has "add AI" somewhere on the roadmap. The teams that get real value from it aren't the ones that move fastest — they're the ones that are clear about what problem the AI is solving and honest about where it can go wrong. Here's the approach we use.

Start with the job, not the model

The first question is never "which model?" It's "what task are we automating or assisting, and how will we know if it worked?" AI features tend to land in a few buckets: summarizing or drafting content, answering questions over your own data, classifying or routing things, and extracting structure from messy input. Each of those has a measurable outcome. If you can't state the outcome, it's too early to build.

A good filter: would a knowledgeable human do this task well, and is it repetitive or slow enough to be worth assisting? If yes, it's a strong candidate. If the task requires guarantees of correctness that even a careful human couldn't give confidently, that's a warning sign.

Retrieval beats fine-tuning for most products

When teams want the AI to "know our business," their first instinct is often to fine-tune a model. In practice, retrieval-augmented generation (RAG) — pulling relevant snippets from your own content and giving them to the model as context — is cheaper, easier to update, and easier to reason about for the vast majority of product features. You change an answer by changing a document, not by retraining. We reserve fine-tuning for narrow cases like enforcing a very specific output style or format at scale.

Design for the model being wrong

This is the part hype skips. Language models are probabilistic; they will occasionally produce confident nonsense. A production-grade AI feature is defined less by the prompt and more by what surrounds it:

  • Keep a human in the loop where the cost of an error is high. Draft, don't send. Suggest, don't act. Let people approve.
  • Show your work. Cite the sources a RAG answer came from so users can verify, and so wrong answers are caught quickly.
  • Constrain the output. Ask for structured formats you can validate, and reject or retry responses that don't fit.
  • Have a fallback. Decide what happens when the model is unavailable, slow, or low-confidence — the feature should degrade gracefully, not break.

Cost, latency, and privacy are product decisions

Every AI call has a price and a delay, and often sends data to a third party. These aren't just engineering details — they shape the product. We think about them early: caching repeated requests, choosing a smaller model where it's good enough, streaming responses so the interface feels fast, and being explicit about what data leaves your systems. For sensitive domains, that last point can decide the entire architecture.

Ship narrow, then widen

The strongest AI features we've shipped started embarrassingly small — one well-defined task, behind a flag, for a subset of users, with a clear way to measure whether it helped. That gives you real usage data and builds trust before you expand. Trying to launch a sprawling "AI assistant" that does everything usually produces something that does nothing particularly well.

The unglamorous conclusion

Good AI features look a lot like good software features: a clear job, honest handling of failure, and attention to cost and privacy. The model is a powerful new ingredient, but the engineering discipline around it is what turns a demo into something people rely on.