The Gap Between the Conversation and the Reality
I talk to business owners every month who are somewhere on a spectrum between "AI is going to replace everything" and "AI is a toy for tech companies." Both positions lead to the same outcome: no meaningful action, no competitive advantage, and a strategy vacuum that will eventually be filled by someone in their market who figured it out.
The practical reality of AI in business operations — specifically large language models (LLMs) integrated into existing workflows — is far more mundane than the hype and far more valuable than the skeptics allow. It is not magic. It is not transformation. It is a set of specific capabilities that map well to specific operational problems, and the businesses that identify that mapping correctly are getting measurable results today.
What LLMs Actually Do Well
Before any discussion of ROI, I want to be precise about the capability, because misapplied AI generates costs, not returns.
LLMs are exceptionally good at processing and transforming unstructured text — reading a document and extracting structured data from it, classifying a block of text according to a set of categories, generating a first draft from a structured input, and summarizing a long document into key points.
They are also good at handling variations in natural language — interpreting what a user means even when they express it inconsistently, which makes them useful for customer-facing interaction and internal search over unstructured knowledge bases.
They are not good — and should not be trusted — for precise factual recall, arithmetic, deterministic rule execution, or any task where the output must be verifiably correct 100 percent of the time without a validation layer. Anyone who tells you otherwise either does not understand how LLMs work or is selling something.
For a broader picture of how AI agents can be structured around these capabilities, my article on AI agents for business covers the architectural layer beyond individual LLM calls.
Three Integrations That Deliver Measurable Returns
1. Document classification and data extraction
If your team processes inbound documents — contracts, purchase orders, invoices, support tickets, insurance claims, applications — and someone is manually reading each one to extract key fields and route it to the right place, you have an LLM use case.
A well-built extraction pipeline reads the document, pulls the relevant fields (dates, amounts, parties, categories, urgency signals), validates the output against a schema, and routes the document automatically. Human review is reserved for low-confidence extractions and edge cases.
I have built this type of pipeline for a logistics firm processing inbound shipping documents and for a professional services company classifying and routing support requests. In both cases the accuracy exceeded 90 percent on first pass, and the remaining 10 percent required human review that was faster than reviewing 100 percent of documents — because the AI had already pre-populated the fields and the human was confirming rather than entering.
2. Internal knowledge retrieval (RAG)
Retrieval-augmented generation — usually called RAG — is the pattern where an LLM answers questions by first retrieving relevant context from a document store and then generating an answer grounded in that retrieved content. This is the practical architecture behind "chat with your documents" use cases.
For businesses with a large body of internal knowledge — policy documents, technical manuals, product specifications, historical project notes — this pattern allows staff to ask questions in natural language and get accurate answers without reading through dozens of documents. The system cites the source, so the answer is verifiable.
The ROI here is onboarding speed and reduction in expert time spent answering repeated questions. When a new team member can get a correct answer to "what is the approval threshold for non-standard pricing" at 11pm without waiting for someone to be available, both the employee experience and the operational velocity improve.
3. Drafting and response generation from structured inputs
This is the highest-volume, lowest-risk LLM application for most businesses: generating first-draft text from structured data that already exists in your systems.
Proposal drafts generated from CRM deal data and a prompt template. Status update emails generated from project management system fields. Job posting drafts generated from a role spec form. Product descriptions generated from a specification sheet.
In every case, a human reviews and edits before anything goes out. The LLM does not produce the final output — it produces a first draft that is 70 to 80 percent of the way to finished. The time saving is on the blank-page problem, which is often the most expensive part of writing.
The Architecture That Makes It Work
Three decisions determine whether an LLM integration creates value or creates a maintenance problem:
Deterministic validation on outputs. LLM outputs should be validated against a schema before they are acted on. If the extraction pipeline is supposed to output a date in ISO 8601 format and it outputs "next Tuesday," the system should catch that before it writes to the database. This validation layer is not optional — it is what makes AI integration reliable rather than probabilistic.
Human-in-the-loop for consequential decisions. Any LLM output that triggers a consequential action — sending a communication to a customer, modifying a financial record, routing a document for a payment run — should pass through a human review step. The AI does the work; the human holds accountability. Getting this boundary wrong is where AI integrations create the most organizational risk.
Observable pipelines. Every LLM call in a production system should be logged: the input, the output, the model version, the latency, and whether the output passed or failed validation. Without this, debugging failures and tracking quality drift over time is impossible. This is also what a compliance audit looks for, and if you operate in a regulated industry you need this from day one.
For businesses also managing broader automation initiatives, the same event-driven architecture I describe for API integrations applies here — LLM calls are just another step in an orchestrated workflow.
The Cost Side of the Equation
LLM integrations are not free, and the cost structure is different from traditional software. You pay per token (per word, roughly) on inference — which means costs scale with usage in ways that traditional infrastructure costs do not.
For most SMB-scale use cases with current major models, the inference cost is surprisingly low — typically a few cents per document processed, not dollars. But costs compound at scale, and a poorly designed prompt that sends 10x more tokens than necessary will run 10x more expensive than it needs to.
The real cost in LLM projects is development and iteration time. Getting the prompt right, building the validation layer, handling edge cases, and making the pipeline robust takes longer than most teams expect. Budget accordingly — and be skeptical of any estimate that treats LLM integration as a one-week project.
The Question I Ask Before Starting Any AI Project
"What specific decision or output will this AI integration produce, and what happens if it is wrong?"
If the answer is "it will extract the invoice amount and post it to the accounting system, and if it is wrong the accountant will catch it in the monthly reconciliation," that is a manageable risk profile and a viable project.
If the answer is "it will generate the contract terms and send them to the client automatically," that is a risk profile that requires more careful architecture, a clear human review step, and probably a legal conversation before it goes anywhere near production.
The businesses I see getting practical value from AI are not the ones chasing the broadest possible application — they are the ones who identified one specific, well-bounded operational problem and built a focused solution for it. Start there.
If you want to think through where AI could realistically fit into your operations, I am happy to have that conversation. No pitch, no hype — just a practical assessment of what makes sense for your specific context.