Welcome to another Vellum product update! September brings a wide variety of enhancements across most of Vellum, but had a special focus on Workflows, Security, and Support. Let’s dig in!
Workflow Sandboxes
We released Workflows last month to help you quickly prototype, deploy, and manage complex chains of LLM calls and the business logic that tie them together. The reception has been amazing and we’ve already been pushed by many of you to make Workflows even more powerful.
API Nodes
You can now make HTTP requests as part of a Workflow using the new “API Node.” API Nodes are critical for building agents, or for integrating more deeply with your own application. For example, maybe you need fetch data about a user through an authenticated API that you host and then feed the response into a Prompt for use by an LLM.
Renaming Workflow Scenarios
Scenarios are used to define example sets of input variable values that represent the types of inputs you might see in production. You can swap between scenarios to manually test how changes to your Workflow effect its outputs for that Scenario.
You can now rename Scenarios so that you can easily tell which case each Scenario represents.
Workflow Deployments
You can deploy a Workflow so that you can invoke complex chains of LLM Prompts with a single API call to Vellum, with all the monitoring, versioning, and scaling benefits that you’re used to with Vellum Prompt Deployments.
Now, we’re bringing more and more features from Prompt Deployments to Workflow Deployments.
Save Execution as Scenario
Once you’ve integrated a Workflow Deployment with your application, Vellum will track all requests made against it – their inputs, outputs, latency, etc. Now, if you spot an edge case, you can save it as a Scenario to your Workflow Sandbox so that you can iterate on your Workflow until it outputs what you expect.
Supplying Actuals
Just like you can “supply actuals” to a Prompt Deployment to capture end-user feedback, you can now do the same for Workflow Deployments. You’re able to supply a quality score (between 0.0 - 1.0) for each Output within the Workflow, as well as a “Desired Output” representing what you wish the output had been.
You can do this either manually from the UI.
Or programmatically via the API (which is particularly useful for capturing feedback directly from your end users).
Displaying Errors
You can now see when errors occur in production on individual Workflow Deployment Executions, as well as a description of the error.
Security
Security and data privacy is treated with the utmost of care within Vellum. We continue to invest in making Vellum an enterprise-ready solution for building AI applications.
Secret Management + Auth Headers
You can now define “Secrets” in one central place in Vellum (e.g. API keys, service tokens, etc.) and then dynamically reference them elsewhere in Vellum. These secret values are securely encrypted and stored using Google Cloud Platform’s KMS service.
You might then use these Secrets as authorization header values in API Nodes within Workflows and in Webhook Evaluation Metrics within Test Suites.
HMAC Signatures
You can now provide a secret token that can be used to verify that outgoing API requests are in fact coming from Vellum. This is especially helpful if you’re standing up endpoints for use by API Nodes or Webhook Evaluation Metrics and want to ensure no one is trying to maliciously impersonate Vellum.
Deployments
Prompt Deployments got some UI/UX love this month.
Filters, Column Visibility, and Refresh
There are now visual indicators when the Completions table of a Prompt Deployment has hidden columns and/or has filters applied. As a result, it’s also easy to update column visibility and filter criteria. You can also now refresh the table’s data with the new “Refresh” button.
Prevent Deletion of Deployments Used in Workflows
Every Prompt Node within a Workflow Deployment is backed by a Prompt Deployment. We now show a warning when you try to delete a Prompt Deployment that’s still in use by a Workflow Deployment.
API
Retrieve Document Index
There is now an officially supported API (in Beta state) to retrieve a Document Index by it’s ID/Name. This is useful if you’re programmatically creating indexes and want to check to see if one already exists prior to creating it.
Support
We’ve made a big push this past month to get the help you need while using Vellum. We can’t claim it’s perfect yet, so please always feel free to reach out to us on Slack, Discord, or email us at [email protected]!
New Help Docs
We have a brand new Help Docs site thanks to our friends at Fern. You can expect richer documentation and tighter integration with our API Docs over the coming months.
In-App Support via Pylon
You can now chat with us directly within the Vellum application thanks to our friends at Pylon. If you’re a Vellum customer with a shared Slack channel with us, then these messages will sync and carry over to the channel.
Conclusion
And that’s a wrap! Thanks as always to all of you who help push us to build a better product and become a better business every month.
We have exciting plans for October – stayed tuned!
Latest AI news, tips, and techniques
Specific tips for Your AI use cases
No spam
Each issue is packed with valuable resources, tools, and insights that help us stay ahead in AI development. We've discovered strategies and frameworks that boosted our efficiency by 30%, making it a must-read for anyone in the field.
This is just a great newsletter. The content is so helpful, even when I’m busy I read them.
Experiment, Evaluate, Deploy, Repeat.
AI development doesn’t end once you've defined your system. Learn how Vellum helps you manage the entire AI development lifecycle.