Q&A RAG Chatbot with Cohere reranking

How it Works / How to Build It

1. MostRecentMessage: This node captures the latest user message from the chat history.
2. SearchNode: It performs a search query against a document index (specifically, "vellum-trust-center-policies-2") using the latest message as the query. It retrieves up to 8 relevant documents based on semantic similarity and keywords.
3. CohereAPIRerankChunks: This node takes the search results and reranks them using the Cohere API, enhancing the relevance of the retrieved documents based on the user's query.
4. FormattedSearchResults: It formats the reranked search results into a structured output, preparing them for the next step.
5. PromptNode: This node generates a final answer by prompting a machine learning model with the question and the formatted policy quotes, ensuring the response includes citations.
6. FinalOutput: It outputs the final answer generated by the PromptNode.

What You Can Use This For

• Answering employee questions about company policies.
• Providing quick access to compliance information for legal teams.
• Assisting HR with policy-related inquiries from staff.
• Supporting customer service representatives with policy clarifications.

Prerequisites

• Vellum account.
• Document index created in Vellum
• Cohere API key for reranking functionality.

How to Set It Up

1. Create a new workflow in your Vellum account.
2. Add the MostRecentMessage node to capture the latest user input.
3. Connect the MostRecentMessage node to the SearchNode to perform a search based on the captured message.
4. Link the SearchNode to the CohereAPIRerankChunks node to rerank the search results.
5. Connect the CohereAPIRerankChunks to the FormattedSearchResults node to format the output.
6. Link the FormattedSearchResults to the PromptNode to generate a final answer.
7. Finally, connect the PromptNode to the FinalOutput node to display the answer to the user.