The dev platform for production LLM apps
Bring features powered by Large Language Models to production with tools for prompt engineering, semantic search, model versioning, quantitative testing, and performance monitoring. Compatible across all major LLM providers.
Rapid prompt engineering with Vellum
Perform side-by-side comparisons of multiple prompts, parameters, models, and even model providers across a variety of test cases.
Cross-Provider Support
Compare how the same prompt performs using any of the major LLM providers.
Test Cases & Quantitative Evaluation
Build up a bank of test cases so that with each iteration of your prompt, you get closer to your ideal output.
History Tracking & Collaboration
Each permutation you try is saved to your history and has a unique url so that you can revisit or share with others at any time.
Use your data to personalize experiences with Search
Language models are trained on the internet. They don't know factual information about your company or your customers and by default, tend to "make facts up" or "hallucinate." Vellum Search solves this.
Fully-managed semantic search
Good semantic search allows you to reliably retrieve relevant data specific to your company and use it as context in LLM calls. Home-grown systems are easy to prototype, but often fall short once in production.
Powerful defaults, highly configurable
Vellum provides powerful defaults for things like chunking strategy, embedding model, and hybrid search weightings, but exposes advanced configuration at each step along the way – all while managing the underlying infra for you.
Tightly integrated with the entire AI stack
Great search means nothing if it doesn't power a great end-user experience. Vellum Search is tightly integrated with the rest of Vellum's AI stack so you can quickly iterate on the full experience holistically.
Make changes with confidence using Deployments
LLM's non-deterministic outputs make it hard to build (and modify!) production systems around them. Vellum's framework for testing, versioning, and monitoring changes help you iterate with confidence.
Simple API interface
Vellum's provider-agnostic, high-reliability, low-latency API allows you to make changes to the prompt, or even the underlying model/provider, without making any code changes.
Quantitative testing & version control
Build up banks of test cases and quantitatively evaluate changes to prompts at scale. Every update is version-controlled and can be easily reverted, no code-changes necessary.
Observability and monitoring
Every model input, output, and end-user feedback is captured and made visible at both the row-level and in aggregate. You own your data. Export it at any time!
Iterate on and deploy prompt chains with Workflows
Quickly prototype, deploy, version, and monitor complex chains of LLM calls and the business logic that tie them together.
Rapid Experimentation
Experimenting with prompt chains in code is tedious, time-intensive, and usually bottle-necked by engineering. Vellum’s low-code workflow builder UI allows you to efficiently test hypotheses and iterate on your chains.
Managed Orchestration
Workflows can be deployed through Vellum and invoked via a simple streaming API. No need manage complex infrastructure for schedulers or event-driven execution. Updates are versioned and require no code code changes on your end.
Production-grade observability
Vellum logs all Workflow invocations as well as valuable debugging info for each step in the chain. Debugging and troubleshooting problematic chains has never been easier.
