How it Works / How to Build It

1. URL Node: Initializes the workflow with a predefined URL for the evaluation report.
2. APINode: Sends a request to the specified API endpoint to retrieve example data.
3. Example Node: Processes the API response to extract relevant example data for generating test cases.
4. PromptNode: Constructs a prompt for the AI model, incorporating user inputs and examples to generate test cases.
5. TestCases Node: Formats the output from the PromptNode into a JSON structure suitable for API requests.
6. CURLBody Node: Executes a script to update the IDs of the generated test cases to ensure they are unique and deterministic.
7. BulkURL Node: Creates a URL for bulk API requests based on the test suite name.
8. PastedFromCURL Node: Sends the formatted test cases to the API endpoint using a POST request.
9. FinalOutput Node: Outputs the final result of the workflow, which includes the generated test cases.

What You Can Use This For

• Generating test cases for AI model validation in software development.
• Creating synthetic datasets for machine learning experiments.
• Automating the testing process for AI pipelines in data science teams.
• Providing a structured format for API requests in testing environments.

Prerequisites

• Vellum account.
• API key for authentication with Vellum Test Suites
• Number of test cases to generate.
• Optional additional context for test case generation.

How to Set It Up

1. Clone the workflow template in your Vellum account.
2. Input the workflow_purpose, test_suite_name, and number_of_test_cases in the Inputs section.
3. (Optional) Add any additional_context that may help in generating test cases.
4. Ensure your API key is correctly set in the APINode for authentication.
5. Run the workflow to generate the test cases and retrieve the final output.