LangChain implementation agency for LLM applications in production

LangChain & LangGraph RAG pipelines Agents & tools LangSmith observability

We build LangChain applications that survive the step from Jupyter notebook to production. From LCEL chains with streaming and async, to LangGraph agents with state machines, retries and human-in-the-loop. Including LangSmith tracing, eval suites and cost monitoring per chain.

Book a technical intake Our AI approach
PromptTemplate ChatModel Retriever VectorStore OutputParser Pydantic RunnableSequence

When to use LangChain, and when not to

LangChain is a framework for composing LLM applications. It doesn't supply models, but abstractions: prompt templates, chat models, retrievers, output parsers, memory, tools and callbacks. The value lies in the combination. A raw OpenAI or Anthropic call is fine for one-shot tasks; once you need retrieval, multiple models, structured output, fallback strategies, streaming and observability all at once, a framework wins.

For pure document Q&A with a single vector store and little logic, LlamaIndex is often more compact. If you're building agent flows with multiple tools, conditional paths and human approval steps, however, LangGraph (part of the LangChain ecosystem) offers a more explicit graph API that is easier to debug in production than an implicit AgentExecutor. For very simple wrappers around a chat completion, using an SDK directly without a framework is fine: fewer dependencies, a smaller attack surface.

Our advice depends on your stack and team. We work equally well with the Python and TypeScript implementations of LangChain, and we have references from clients who started with LangChain and later migrated parts to raw clients. We have no ideological preference; we build what fits the problem. Also read our page on AI implementation as a partnership.

Building blocks we use every day

LangChain has a large surface area. These are the patterns that come back in almost every production application, along with the choices that will save you pain in the long run.

LCEL and RunnableSequence

The LangChain Expression Language is the standard for composition. prompt | model | parser returns a Runnable with built-in streaming, batching, async and parallelism. We write new code exclusively in LCEL; legacy LLMChaincode we migrate where it delivers a return on investment.

Retrievers and RAG

BaseRetriever as the interface, hybrid search (BM25 plus dense embeddings), MMR for diversity, and contextual compression to filter out noise. We integrate Qdrant, Weaviate, pgvector or Pinecone depending on scale, hosting requirements and filter logic.

Output parsers with Pydantic

No regular expressions on LLM output. We use PydanticOutputParser, StructuredOutputParser or the newer with_structured_output() on chat models. The result: typed objects, automatic validation and clear retry paths on parse failures.

Agents and tools

ReAct agents for exploratory flows, OpenAI functions or tool-calling agents for tightly controlled task execution. We prefer create_tool_calling_agent over older prompt-only ReAct implementations: more reliable and cheaper. We write tools as validated Pydantic functions.

Memory and sessions

ConversationBufferMemory for short sessions, summary buffer for longer ones, and vector-store-backed memory for persistent per-user knowledge. In LangGraph we manage state explicitly via StateGraph; we avoid implicit memory in production.

Callbacks and streaming

Callbacks for logging, tracing and token counters. Streaming via astream and server-sent events to the frontend, with proper backpressure. Building async-first avoids expensive refactors later, when throughput needs to go up.

From prototype to production: what really changes

A working notebook is not a working system. We go through these phases as standard in every LangChain implementation project.

01

Architecture and selection

Which models, which vector store, which hosting model. Privacy scope, expected volumes, latency requirements. The choice between an LCEL chain and a LangGraph agent depends on the complexity of the flow.

02

Build with evals

An eval set from day one. We use LangSmith datasets and custom evaluators (correctness, hallucination rate, latency). No prompt iteration without a measurable regression test.

03

Hardening

Rate limiting, exponential backoff, fallback models, semantic caching, cost caps per tenant. Prompt injection mitigation and input sanitisation. Idempotency where it fits.

04

Observability

LangSmith or OpenTelemetry for traces, metrics to Prometheus, alerts on p95 latency and cost per user. Versioning of prompts and chains so you can roll out and roll back with precision.

Production considerations you won't see in a POC

This is where most in-house LangChain projects stumble. We have the patterns ready.

Rate limiting and retries

OpenAI, Anthropic and Azure OpenAI each have their own quota models. We build client-side rate limiters with token buckets, exponential backoff on 429 and 5xx errors, and automatic fallback to a second provider if the primary times out. Tenacity or LangChain's native retry config, depending on where you want the logic to live.

Caching at multiple layers

Exact-match caching for identical prompts, semantic caching via embeddings for near-duplicate queries, and HTTP caching for retriever results. In the right places, this saves 30 to 70 percent on API costs with no loss of quality.

Cost tracking with LangSmith

Per chain, per tenant, per feature flag. We attach tags and metadata to every run, so in LangSmith you can see which prompt version or which retriever strategy is pushing costs up. Cost anomaly alerts prevent surprises at the end of the month.

Prompt versioning and A/B testing

Prompts in code and in LangSmith Hub, linked to a version tag. A/B tests run in parallel on a percentage of traffic, with statistically significant winners based on your own evaluators. No more ad-hoc "I've got a better prompt" deploys.

Streaming to the frontend

Server-sent events or WebSockets, with token-by-token output and correct shutdown on cancellation. For agents we also stream tool calls and intermediate steps, so end users can see what the system is doing. Working out backpressure and timeouts prevents hanging connections.

Async and concurrency

Virtually all Runnables support ainvoke, abatch and astream. A well-designed async pipeline gets several times the throughput out of the same rate-limit budgets at the same cost. We use FastAPI or LangServe for the HTTP layer and uvicorn workers with proper tuning.

Use cases where LangChain really pays off

Not every LLM problem calls for a framework. For these patterns, we choose LangChain or LangGraph by default.

Enterprise RAG over internal knowledge

Documents from SharePoint, Confluence, a DMS or your own API, with metadata filtering per user role, hybrid retrieval and source citations. LangChain's retriever abstraction makes swapping the vector store a matter of hours, not weeks.

Multi-step research agents

An agent that analyses a question, formulates sub-questions, searches in parallel, summarises the results and cites its sources. LangGraph turns this into an explicit state machine with retry branches and a human in the loop for borderline cases.

Data extraction from unstructured text

Contracts, emails, customer service tickets, reports. Pydantic schemas as output parsers deliver type-safe records that fit straight into a database or ETL pipeline. When in doubt, the chain falls back to a second model or escalates to a human.

Conversational interfaces with context

Chatbots that do more than FAQ matching: personalised advice, transactional actions via tools, and memory across sessions. See also our page on AI chatbot implementation for the broader approach.

Workflow automation with LLM steps

An existing BPM or ETL process where one or more steps are handled by an LLM: classification, summarisation, translation, quality control. LangChain fits here because retries, fallbacks and logging are available natively.

Multi-modal pipelines

Text, images and audio in a single flow: GPT-4o, Claude with vision, Whisper for transcription. LangChain's message format supports content blocks, so you can route different modalities through the same chain cleanly.

Not sure about a large project yet?

Test your idea first: a working prototype in 1 day

With OneDayBuild, we make your idea tangible in a single day for €1,150, so you know whether further development is worth the investment. Decide to go ahead with the full build? Then we deduct the cost in full.

View OneDayBuild →

Technology and integrations

An overview of what we often combine within a LangChain stack. We don't force anything on you; your existing infrastructure drives many of the choices.

LangChain (Python & TS) LangGraph LangSmith LangServe OpenAI Anthropic Claude Azure OpenAI Mistral Llama 3 via vLLM Qdrant Weaviate pgvector Pinecone Redis (semantic cache) FastAPI Pydantic v2 OpenTelemetry Prometheus Docker / Kubernetes Terraform
AI chatbot implementation AI implementation partner Interim AI tech lead AIOps platform AI consultant AI document processing What AI implementation costs

Jargon you'll find in our code and commits

For tech leads who want to know whether we speak the same language.

LCEL and Runnables

RunnableSequence, RunnableParallel, RunnablePassthrough, RunnableLambda, RunnableBranch. Composition via the pipe operator. Type checking with input and output schema introspection.

Prompts

ChatPromptTemplate, MessagesPlaceholder, few-shot templates, partial variables. Prompt Hub for versioning. format_messages and format_prompt we know inside out.

Retrieval

BaseRetriever, EnsembleRetriever, ParentDocumentRetriever, MultiQueryRetriever, ContextualCompressionRetriever. Re-ranking with Cohere Rerank or a custom cross-encoder where it helps.

Agents and tools

AgentExecutor, create_tool_calling_agent, create_react_agent, create_openai_functions_agent. Tools as validated Pydantic objects via @tool or StructuredTool.from_function.

LangGraph

StateGraph, MessagesState, add_node, add_conditional_edges, checkpointers (Postgres or SQLite) for recoverable flows, interrupts for human-in-the-loop, sub-graphs for reusable modules.

Memory

ConversationBufferMemory, ConversationSummaryBufferMemory, VectorStoreRetrieverMemory. In LangGraph: state keys with reducers, checkpointers per thread_id.

Why Appfront for your LangChain implementation

We build LLM applications that your operations and security teams will accept. Not demos, but systems that keep working at half past two in the morning.

Production-first mindset

Async, observability and evals are not an afterthought but day-one practice. We write lean, sturdy code that will last ten years, not a glossy prototype.

Deep LangChain expertise

We follow the release notes, know the breaking changes between 0.1, 0.2 and 0.3, and know which abstractions will demand ongoing maintenance. We fork ourselves where that's faster than waiting for upstream.

Experience with multi-provider stacks

OpenAI, Anthropic, Azure OpenAI, Mistral, local models via vLLM. Routing based on cost, latency and privacy. Fallback strategies that have actually been tested, not just configured.

Frequently asked questions about LangChain implementations

Do you prefer working with LangChain or LlamaIndex?
For pure document Q&A systems with a single vector store, we sometimes choose LlamaIndex because its retrieval abstractions go deeper there. For agent flows, multi-tool orchestration or complex state, we choose LangChain plus LangGraph. We've built with both and also migrate between the two when the use case calls for it.
When do you use LangGraph and when just LCEL?
LCEL is for linear or lightly branching chains: prompt to model to parser, possibly with a retriever in front. Once multiple agents, conditional paths, retries on specific branches or human-in-the-loop come into play, we choose LangGraph. The explicit state machine makes debugging in production far more manageable.
How do you organise evals and regression tests?
We build an eval set in LangSmith from the first week. Custom evaluators for correctness, hallucination rate, format compliance and latency. Every prompt or chain change runs against that set before a PR can be merged. Clients can follow along in the runs and give feedback.
Do you host models yourselves or always via an API?
Both, depending on the requirements. For PII or medical data, we often run open-source models via vLLM or Ollama on your own Kubernetes cluster or with a Dutch hosting provider. For general use cases, a managed API is more cost-efficient. We advise based on privacy scope, volumes and the specific quality requirements for each task.
How do you handle prompt injection and data leaks?
Input sanitisation, system prompt isolation, output filtering and a separate category for tools that perform write actions (which always go through an approval layer). For RAG: source isolation per user role via metadata filters in the retriever. For sensitive sectors, we build custom guardrail layers based on the OWASP LLM Top 10.
Do you work in Python or TypeScript?
Both. In our experience, Python is slightly ahead in terms of feature parity, but LangChain.js is more than sufficient for most applications and fits better with Node stacks. With hybrid teams, we choose the language per service that causes the least friction. We document our choices in an ADR per project.
What do you do when API costs get out of hand?
Three layers: per-tenant and per-feature cost caps with automatic throttling, semantic caching for repeated queries, and routing to cheaper models for branches where that is possible without losing quality. LangSmith tags show where the money goes. A well-configured stack typically comes in 30 to 60 percent below a naive implementation.
Do you also take over existing LangChain codebases?
Yes. Many teams have an internal POC that never makes it to production. We start with an audit of chain design, eval coverage, costs, observability and security, and deliver a prioritised list. After that, we either help with the hardening or take over completely, depending on what you want. See also interim AI tech lead.

Ready for a LangChain implementation that stays up in production?

We schedule a one-hour technical intake. You outline the use case, we ask sharp questions and give you honest advice on architecture, stack and approach the same week. No sales talk, just workable answers.

Schedule a technical intake See our AI approach

Edit content