No items found.

Legal RAG chatbot

Chatbot that provides answers based on user queries and legal documents.

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form.

Click to interact

This workflow creates a legal RAG chatbot that generates comprehensive legal answers based on user queries and relevant legal documents. It allows users to search legal documents, format the results, and provide answers with proper citations in both markdown and JSON formats.

‍

How it Works / How to Build It

Legal Search: This node performs a search on the specified legal documents using the user’s query. It retrieves relevant documents based on semantic and keyword matching.
Format Search Results: This node formats the search results, including citation metadata, to prepare them for analysis.
Legal Answer Generator: This node generates a legal answer based on the formatted search results and the user’s query. It outputs the answer and citations in JSON format.
Format Answer Markdown: This node takes the JSON output from the Legal Answer Generator and formats it into a markdown response, including the answer and citations.
Final Output Answer: This node outputs the formatted answer in markdown.
Format Citations Json: This node extracts and formats the citations from the JSON output into a clean JSON format.
Final Output Citations: This node outputs the formatted citations in JSON.

What You Can Use This For

Legal teams can quickly get answers to legal questions based on internal documents.
Law firms can automate responses to common legal inquiries.
Compliance departments can ensure that responses are backed by proper documentation and citations.

Prerequisites

Vellum account
Access to a set of legal documents (e.g., PDFs or text files)
Basic understanding of how to use the Vellum workflow builder

How to Set It Up

Clone the "Template_Legal RAG Assistant" workflow from the Vellum library.
Configure the LegalSearch node to point to your legal document index.
Set up the Inputs to include your document set, chat history, and user query.
Connect the nodes in the specified order: LegalSearch → FormatSearchResults → LegalAnswerGenerator → {Format Answer Markdown, Format Citations Json}.
Test the workflow by inputting a sample query and reviewing the outputs from Final Output Answer and Final Output Citations.

Created By

Nicolas Zeeb

Last Updated

October 14, 2025

Discover more agents

Review my roadmap based on team capacity

Detect declining usage trends ahead of renewals

Track team progress without standup meetings

Help me write SEO optimized articles

Flag suspicious Stripe transactions in Slack

Automate KYC checks and send reports to Slack

Summarize my clients’ portfolios weekly

Review my contracts and generate risk summaries

Highlight NDA deviations and send alert to Slack

Review DPAs or privacy policies for compliance

Run review when new prior auth packets arrive

Review claims for compliance and errors

sucCCESS STORIES

Hear it from our customers

We know the power of AI, but how do we make it secure and ensure that we're not compromising privacy and security while still providing value? Vellum has been a big part of accelerating that experimentation part, allowing us to validate that a feature is high-impact and feasible.

Pratik Bhat

ai Product manager

We sped up AI development by 50 percent and decoupled updates from releases with Vellum. This allowed us to fix errors instantly without worrying about infrastructure uptime or costs.

Jordan Nemrow

Co-Founder & CTO @ Woflow

Vellum helped us quickly evaluate prompt designs and workflows, saving us hours of development. This gave us the confidence to launch our virtual assistant in 14 U.S. markets.

Sebi Lozano

Sr. Product Manager @ Redfin

FAQ

1. Can I adapt this workflow for my own legal documents or internal knowledge base?

Yes, the Legal Search node is fully configurable. You can connect it to any document index, whether that’s internal memos, contracts, or case law databases. Once connected, the workflow will automatically ground answers in your organization’s content instead of public data.

2. How does this workflow make sure answers are grounded and properly cited?

Every response is based on the retrieved legal documents. The Legal Answer Generator references those documents directly, while the Format Citations Json node extracts verifiable citations. This ensures transparency you need to always trace a claim back to the source.

3. Can I change how the answer is formatted or what metadata it includes?

Definitely, the Format Answer Markdown and Format Citations Json nodes are designed for easy editing. You can add sections like “Summary,” “Relevant Statutes,” or “Jurisdiction Notes,” or export results in other formats such as plain text or HTML.

4. What types of legal content does this work best with?

This agent performs best with well-structured legal documents — such as contracts, case law, regulatory filings, or policy handbooks. It can handle both text and PDF inputs, as long as the documents are properly indexed or parsed before retrieval.

5. How can I extend this workflow beyond legal use cases?

The RAG pattern here is universal. You can reuse the same structure for policy research, compliance Q&A, or academic summarization by swapping the document index and updating prompts. It’s an easy way to turn any text repository into a reliable, cited knowledge agent.