Prompt engineering course for companies
Prompt engineering is engineering, not copywriting
In production AI systems, the prompt decides whether a feature works or the helpdesk gets flooded. A good prompt engineer thinks in system prompts, few-shot examples, JSON schemas, retrieval context and evals, not in clever phrases for ChatGPT. Our course teaches technical teams how to design, version, test and monitor prompts as part of a serious software stack.
Discuss an in-house course View curriculumWho this course is for
This is not a "learn to write ChatGPT prompts" workshop. It is a technical training for teams that have LLMs in production or are about to deploy them, along with all the versioning, eval and monitoring issues that come with that.
The course is designed for developers, data engineers, ML engineers, content teams working with structured generation, and customer service leads who manage LLM bots. Prerequisites: participants already work with an LLM API (OpenAI, Anthropic, Azure OpenAI or an open-source model via vLLM/Ollama), know the difference between a chat completion and a completion endpoint, and have built at least one feature in which an LLM sits in a request-response cycle. We go beyond the basics: function calling, JSON mode, schema-constrained decoding, RAG pipelines and multi-agent handoffs are the norm.
What we explicitly don't do: share prompt tips you can find on LinkedIn, practise "act as a senior consultant" hacks, or claim that a good prompt solves every LLM problem. We treat prompt engineering as one layer in a larger AI system, alongside model choice, retriever tuning, guardrails and monitoring. If you only want to learn how to write a better ChatGPT prompt, this isn't the right course; other courses cover that.
Curriculum: five modules, building on each other
The course runs from prompt architecture to agent orchestration. Each module combines theory with hands-on exercises based on participants' own use cases, with no toy examples and no made-up tasks.
Prompt architecture
How to build a prompt for production: the difference between system and user prompts, role-based prompting, context management in long dialogues, and when you should truncate or summarise context. We cover prompt components (instructions, context, examples, schemas, output format) and how to pour them into a reusable template system so developers don't reinvent every prompt.
- System versus user prompt: what belongs where
- Role-based prompting and persona stability
- Context window management and token budgets
- Prompt templates and variable substitution without injection risk
Few-shot, zero-shot, chain-of-thought and ReAct
When do you choose which approach? Few-shot works for classification and structured extraction, but costs tokens; chain-of-thought improves reasoning tasks but increases latency; ReAct is powerful for agent loops but unstable without tight schema control. We build all four and benchmark them against each other on a real task.
- Few-shot example selection and in-context learning
- Chain-of-thought (CoT) prompting and self-consistency
- ReAct: reasoning + acting in one loop
- Trade-offs between accuracy, latency and cost
Function calling, tool use and structured output
A production LLM doesn't produce free text but a validated JSON object you can use downstream. We cover function calling (OpenAI, Anthropic), tool use with a tool router, JSON mode, and schema-constrained decoding via libraries such as Outlines or Instructor. This includes retry logic for invalid outputs and how to stop the model from emitting half-finished tool calls.
- Function calling spec and tool-routing patterns
- JSON mode versus schema-constrained decoding
- Pydantic / Zod schemas as the single source of truth
- Validation, retries and fallback strategies
Guardrails, jailbreak prevention and prompt injection
Every LLM system that processes user input is an attack surface. We cover prompt injection (direct and indirect, for example via retrieved RAG documents), jailbreak patterns and the well-known defences: input sanitisation, sandwich prompting, output filtering, and separating trusted and untrusted context. Including practical demos with current jailbreaks and how to block them without wrecking your legitimate use case.
- Direct and indirect prompt injection
- Jailbreak categories and mitigation patterns
- Output filtering and content moderation
- Separating trusted and untrusted context
RAG prompting, agent prompting and multi-agent handoffs
Retrieval-augmented generation sounds simple (just throw documents into the context), but in production all the complexity lies in the prompting: how do you instruct the model to use only cited facts, how do you handle conflicting sources, and how do you stop retriever noise from confusing the model? After that we build an agent loop with tool use and cover multi-agent handoffs: when does agent A pass the baton to agent B, and how do you make the shared state persistent?
- RAG prompts: citation enforcement and source grounding
- Integrating retriever output without hallucination
- Agent loops: planning, action, observation
- Multi-agent handoffs and shared memory
Evals: how do you know a prompt works?
A prompt that "feels good" during development is not a prompt that works in production. Evals are the unit tests of AI systems: without evals, you don't know whether your new prompt version is better or worse than the previous one. This module covers eval frameworks and how to build them into your CI pipeline.
Promptfoo
Open-source CLI for prompt testing. Define test cases in YAML, run them against multiple models and compare outputs side by side. Well suited to regression testing of prompt changes in CI. Works well for structured-output use cases where you can define an expected JSON per case.
Langfuse
Tracing and eval platform for LLM apps. It logs every call (prompt, response, latency, cost) and lets you run evals on production traces. Open-source and self-hostable, which matters for teams with EU residency requirements. Integrates with LangChain, LlamaIndex and the raw OpenAI/Anthropic SDKs.
Helicone
Observability layer that sits between your app and the LLM API. Gives you per-request insight into latency, cost, cache hits and error rates. Combines well with Promptfoo or Langfuse: Helicone for production telemetry, Promptfoo for offline regression.
Custom evals: LLM-as-judge
For open-ended outputs, a keyword match doesn't work. LLM-as-judge uses a strong model to score another model's output against rubrics (correctness, tone, format compliance). We cover judge bias, calibration and how to prevent your judge from making the same mistakes as your production model.
Golden datasets
Eval quality is dataset quality. We teach participants how to build a golden dataset from real production interactions: edge cases, regression cases and adversarial inputs. Including annotation workflows and how to detect dataset drift.
A/B testing in production
Offline evals don't cover everything. We cover feature flags for prompt versions, traffic splitting per user cohort, and how to measure a statistically significant prompt uplift without harming your users if a variant performs badly.
Production issues you won't learn anywhere else
A prompt that works on your laptop is not the same as a prompt that runs stably for six months. This module covers operational reality: versioning, drift, cost and caching.
Prompt versioning is not optional but mandatory. Every prompt that touches production gets a version number, a changelog and a rollback path, just like code. We cover prompt versioning in Git (prompts as code), in a prompt management tool (Langfuse, PromptLayer) and hybrid models where prompts live in a database with an audit trail.
Drift monitoring is the second part. A prompt that works today can perform worse in three months because the underlying model has been updated (silent model updates by API providers), because the input distribution has shifted, or because an upstream system has changed its output format. We cover drift detection through continuous evals on a fixed set, semantic similarity checks on outputs, and alerting when scores degrade.
Cost optimisation is the third. Naive LLM implementations can be far more expensive than necessary. We cover prompt caching (Anthropic prompt caching, OpenAI's automatic caching, Redis-based response caching), model routing (a cheap model for simple queries, a strong model for complex ones), batch processing for async workloads, and how to enforce a cost budget per request without crippling your service during peak usage.
Finally, the practical things nobody ever writes about. How do you handle provider rate limits (exponential backoff, request queueing, multi-provider fallback)? How do you debug a prompt that fails "sometimes"? How do you roll out a prompt update without downtime? And how do you explain to compliance that you are not generating made-up content, but a grounded answer from your RAG pipeline?
Hands-on: participants work on their own use cases
The course is not a series of slides. Every module has hands-on assignments in which participants write, evaluate and revise prompts for their own production use cases. No abstract exercises: the prompt you build in the course is one you can deploy on Friday.
CRM AI: lead enrichment and classification
A common use case: the CRM system needs to automatically classify incoming leads, enrich them with public data and route them to the right sales rep. Participants build a function-calling prompt with a JSON schema, a retry loop for invalid outputs, and evals on a dataset of historical leads. This includes edge-case handling (incomplete data, multi-language input, ambiguous cases).
Customer service bot with RAG
A customer service bot that draws its answers from a knowledge base: manuals, FAQs, policy documents. Participants design the RAG prompt (citation enforcement, fallback when there is no match, escalation to a human), build an eval set with edge cases and test prompt injection resistance with adversarial inputs.
Internal wiki RAG for engineering teams
A second RAG case: a Q&A system that searches internal documentation, such as Confluence, Notion and Google Docs. The challenges differ from customer service: more technical jargon, code snippets in the retrieval, multi-document answers, and ACL compliance (which user may see which documents). Participants build a prompt that draws exclusively from permitted sources.
Document extraction pipeline
Extracting structured data from PDFs, contracts or forms using function calling and schema-constrained decoding. Participants define a Pydantic schema, write the extraction prompt, build retries for parse failures and measure extraction accuracy against a manually labelled set.
If you don't have a use case of your own, you will be given a realistic case study, for example a document extraction pipeline on publicly available contracts, or a RAG bot on open-source documentation. But most teams arrive with several production cases they want to tackle during the course.
Tools and frameworks we cover
The course is provider-agnostic in its core concepts, but in the hands-on sessions we work with the tools most commonly used in production. Participants gain practical experience with multiple models and frameworks so they can make informed choices, rather than simply choosing "what we happen to know".
Important: we teach participants why a tool fits a use case, not how to name-drop every tool. LangChain is powerful but overkill for simple apps; LiteLLM is a thin abstraction for multi-provider routing; Instructor is great for schema-constrained extraction but adds a layer. Which combination suits your stack depends on team skills, deployment environment and data residency requirements, and that is what we guide participants through.
Case walkthrough: from naive prompt to production-grade
A typical exercise pattern from the course, in brief: how a quote extraction prompt evolves from a 200-token zero-shot attempt into an instrumented pipeline with evals, caching and monitoring.
Iteration 1: naive zero-shot
A developer writes a prompt: "Extract the customer name, order ID and total amount from this email." It works on some of the inputs. On the rest, the model extracts the wrong fields, hallucinates IDs or refuses to answer on multilingual emails. No schema, no retries, no evals.
Iteration 2: few-shot with a JSON schema
Add a few few-shot examples that cover diversity (English, Dutch, with VAT, without VAT), plus function calling with a Pydantic schema that defines customer_name, order_id and total_amount_eur as strictly typed fields. Accuracy improves noticeably, but costs double because of the extra context tokens, and with unusual formats the model still returns invalid JSON.
Iteration 3: schema-constrained decoding and retries
Replace it with schema-constrained decoding via Instructor: the model can structurally no longer produce invalid JSON. Retry logic with exponential backoff on parse failures. An eval set in Promptfoo makes every prompt change measurable. Latency barely increases.
Iteration 4: production instrumentation
Langfuse logs every call with prompt version, cost and latency. Helicone caches similar requests (identical email templates produce cache hits). An A/B test in production between two prompt versions (an extra disambiguation instruction for amounts in currencies other than EUR). A drift eval runs every night on new production traces to detect regressions.
What you learn from this flow
- Never start with a complex prompt: measure the baseline first.
- Schema-constrained decoding solves a whole class of problems that retries cannot fix.
- Evals are the blocker: without an eval set, you cannot tell iteration 1 from iteration 4.
- Caching is a free cost saving as soon as you call the same model several times with similar input.
- A prompt is never "done": drift monitoring catches the regressions you never see in dev.
Frequently asked questions about the prompt engineering course
Plan a prompt engineering course for your team?
Tell us which LLM features you have in production or want to build, what level your team is starting from, and which deployment environment you use. We put together a curriculum that fits your stack and use cases: no template, no generic slides.
Schedule an intake call