Guides

April 23, 2024

Tutorial: Setting Up OpenAI Function Calling with Chat Models

Co-authors

No items found.

What is Function Calling
Function Calling Example
Describing Functions
Generating Function Arguments
Parallel Function Calling
Calling Functions
Conclusion

LLMs are great at complex language tasks but often produce unstructured and unpredictable responses, creating challenges for developers who prefer structured data. Extracting info from unstructured text usually involves intricate methods like RegEx or prompt engineering—thus slowing development.

To simplify this, OpenAI introduced function calling to ensure more reliable structured data output from their models.

After reading this tutorial, you'll understand how this feature works and how to implement various function calling techniques with OpenAI's API.

Let's get started.

‍

What is Function Calling?

With function calling you can get consistent structured data from models.

But wait, don't be misled by the name—this feature doesn't actually execute functions on your behalf. Instead, you describe the functions in the API call, and the model learns how to generate the necessary arguments. Once the arguments are generated, you can use them to execute functions in your code.

So now that we’ve cleared that, let’s show you how to set it up.

‍

OpenAI Function Calling example

In this tutorial, we'll show you how to dynamically generate arguments for two arbitrary weather forecast functions. We'll show you how to:

Use the OpenAI's tool parameter to describe your functions;
Run the model to generate arguments for one or multiple functions;
Use those arguments to execute arbitrary functions in your code;

💡 Please note that this tutorial primarily focuses on configuring the "function calling" feature and does not include instructions for setting up the OpenAI environment. We assume that you already have that covered; if not, please refer to this documentation here. In the sections below, we'll detail each step and share the code we used. If you'd like to run the code while you read, feel free to use this Colab notebook.

‍

Describing your Functions

First we need to describe our functions in the tools parameter in the OpenAI's Chat Completions API call.

For this example, we'll describe these two functions:

get_current_weather(): Obtains the weather of a given city at the time of request.some text
- location: A string indicating the city and state (e.g., San Francisco, CA).
- format: A string enum specifying the temperature unit, either as celsius or fahrenheit.(the model will automatically derrive this from the location)
get_n_day_weather_forecast(): Returns the weather over n days at a given location. The function includes the parameters location and format, but also includes:
- num_days: An integer indicating the number of days for the forecast.

This is how our schema looks like:


tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "format": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit to use. Infer this from the users location.",
                    },
                },
                "required": ["location", "format"],
            },
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_n_day_weather_forecast",
            "description": "Get an N-day weather forecast",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "format": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit to use. Infer this from the users location.",
                    },
                    "num_days": {
                        "type": "integer",
                        "description": "The number of days to forecast",
                    }
                },
                "required": ["location", "format", "num_days"]
            },
        }
    },
]

Before using this schema, we’ll introduce a helper function to make calling the Chat Completions API easier. Our helper function will reduce code repetition, handle errors, and set a default model. In our Collab notebook, we’ve defined the GPT_MODEL as gpt-3.5-turbo-0613. Here's the helper function that we'll continue to use in the following sections:


# Helper function

def chat_completion_request(messages, tools=None, tool_choice=None, model=GPT_MODEL):
    try:
        response = client.chat.completions.create(
            model=model,
            messages=messages,
            tools=tools,
            tool_choice=tool_choice,
        )
        return response
    except Exception as e:
        print("Unable to generate ChatCompletion response")
        print(f"Exception: {e}")
        return e

‍

Generating Function Arguments

Now let's see how this schema works, as we pass a system and a user message.


# Define messages

messages = []
messages.append(
    create_message(
        "system",
        "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."
    )
)
messages.append(create_message("user", "What's the weather like today?"))

# Submit response

chat_response = chat_completion_request(
    messages, tools=tools
)
messages.append(chat_response.choices[0].message)
print(chat_response.choices[0])

In the example above, we instructed the model not to assume function parameters if they're not provided in the System message. This means the model won't generate a function call unless it has all the necessary parameter details. For instance, if the user message is "What's the weather like today," the model will ask the user for the location before it generates the function call output:


# Output 

Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Sure, could you please provide me with your current location?', role='assistant', function_call=None, tool_calls=None))

When the model is confident that it has all the required parameters that we defined in our schema, it will finally output the function calling arguments. You can tell a function has been called by observing the finish_reason and function flags in the response.

In our snippet below, we add our response to the messages list, which is then sent as a request to the API again:


# Define messages

messages.append(create_message("user", "I'm in San Francisco, CA"))
chat_response = chat_completion_request(
    messages, tools=tools
)

# Submit response

chat_response = chat_completion_request(
    messages, tools=tools
)
messages.append(chat_response.choices[0].message)
print(chat_response.choices[0])

Since we’re providing the last missing piece of information, this should be enough information for the model to return a function call with arguments:


#Output

Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_npQlZt0Ef84rYiT6Dat8V1xO', function=Function(arguments='{\n  "location": "San Francisco, CA",\n  "format": "celsius"\n}', name='get_current_weather'), type='function')]))

Noticed that the model automatically called the function for this user message?

That's because if multiple functions are present, the model will intelligently choose which function call to provide by default. This means that the tool_choice parameter will be set to auto. If there are no functions, the tool_choice parameter will be set to none.

Take a look at the following example, where we change the user's request. For instance, if we changed our prompt to:


...
messages.append(create_message("user", "What is the weather going to be like in San Francisco, CA over the next 5 days"))
...

The model will know to suggest our get_n_day_weather_forecast()instead:


Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_npQlZt0Ef84rYiT6Dat8V1xO', function=Function(arguments='{\n  "location": "San Francisco, CA",\n  "format": "celsius"\n}', name='get_current_weather'), type='function')]))

Forcing a model to choose one function

It's important to note that you can also force a model to choose only from one function. Here's how you can do that:


...
chat_response = chat_completion_request(
    messages, tools=tools, tool_choice={"type": "function", "function": {"name": "get_n_day_weather_forecast"}}
)
...

And here's the output that we get from it:


Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_FQSl6U46sIG2HyXHQ1UnNMrm', function=Function(arguments='{\n  "location": "Toronto, Canada",\n  "format": "celsius",\n  "num_days": 1\n}', name='get_n_day_weather_forecast'), type='function')]))

‍

Parallel Function Calling

In some cases, you'd like the model to run multiple function calls together, allowing the effects and results of these function calls to be resolved in parallel. This can be done by newer models like gpt-4-1106-preview or gpt-3.5-turbo-1106.

In our case, let's imagine that a user is asking for the weather in two locations:


messages = []
messages.append({"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."})
messages.append({"role": "user", "content": "What is the weather going to be like in San Francisco and Glasgow over the next 4 days"})
chat_response = chat_completion_request(
    messages, tools=tools, model='gpt-3.5-turbo-1106'
)

assistant_message = chat_response.choices[0].message.tool_calls
print(assistant_message)

This means that the model should output a list of two results, with two different arguments for the same function:


[ChatCompletionMessageToolCall(id='call_oEWfcqY5wiBNAGw8Rb6xlymf', function=Function(arguments='{"location": "San Francisco, CA", "format": "celsius", "num_days": 4}', name='get_n_day_weather_forecast'), type='function'), ChatCompletionMessageToolCall(id='call_yBIdc8jb2m4c3Z2zB4NUEofO', function=Function(arguments='{"location": "Glasgow", "format": "celsius", "num_days": 4}', name='get_n_day_weather_forecast'), type='function')]

‍

Calling Functions

Now that we know how to manipulate our API requests, it’s time to use this output to call our arbitrary functions.

Just to illustrate how this works, we only return an arbitrary text from each function. Then we wrote a function called execute_function_call() that contains if-else conditionals that check the LLM's output and calls the appropriate function based on that response.


import json

def get_current_weather(location, format):
    return "Call successful from get_current_weather()."


def get_n_day_weather_forecast(location, format, num_days):
    return "Call successful from get_n_day_weather_forecast()"


def execute_function_call(message):
    args = json.loads(msg.tool_calls[0].function.arguments)
    if message.tool_calls[0].function.name == "get_current_weather":
        results = get_current_weather(args["location"], args["format"])
    elif message.tool_calls[0].function.name == "get_n_day_weather_forecast":
        results = get_n_day_weather_forecast(args["location"], args["format"], args["num_days"])
    else:
        results = f"Error: function {message.tool_calls[0].function.name} does not exist"
    return results

Piecing everything together, let's send one more request to the API.

The code below:

Handles a user request
Submits the list of messages to the model via the chat_completion_request()
Parses the model's response
Calls the corresponding function with our defined function execute_function_call()
Finally, it prints the results in a structured format


# Define messages

messages = []
messages.append(
    create_message(
        "system",
        "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."
    )
)
messages.append(create_message("user", "what is the weather going to be like in San Francisco, CA?"))

# Submit response

chat_response = chat_completion_request(messages, tools)

# Parse response

msg = chat_response.choices[0].message
messages.append({"role": msg.role, "content": msg.tool_calls[0].function})
msg_func = str(msg.tool_calls[0].function)

# Call corresponding function

if msg.tool_calls:
    results = execute_function_call(msg)
    messages.append({"role": "function",
                     "tool_call_id": msg.tool_calls[0].id,
                     "name": msg.tool_calls[0].function.name,
                     "content": results
                     })
pretty_print_conversation(messages)

And we get this final output:


System: Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.

User: what is the weather going to be like in San Francisco, CA?

Assistant: Function(arguments='{\n  "location": "San Francisco, CA",\n  "format": "celsius"\n}', name='get_current_weather')

function (get_current_weather): Call successful from get_current_weather().

‍

Conclusion

In summary, the OpenAI API's function calling feature allows you to describe custom functions that the AI model can intelligently decide to call, generating structured JSON outputs containing the necessary arguments. This helps with more dynamic and interactive applications where the AI can perform specific tasks or retrieve information by invoking these functions based on natural language inputs.

Using this demo, you should be good to implement function calling for your use-case. If you have any troubles feel free to DM me on twitter.

If you want to get these insights in your inbox, subscribe to our newsletter here.

Join 10,000+ developers staying up-to-date with the latest AI techniques and methods.

🎉

Thanks for joining our newsletter.

Oops! Something went wrong.

Anita Kirkovska

Founding Growth at Vellum

Anita Kirkovska, is currently leading Growth and Content Marketing at Vellum. She is a technical marketer, with an engineering background and a sharp acumen for scaling startups. She has helped SaaS startups scale and had a successful exit from an ML company. Anita writes a lot of content on generative AI to educate business founders on best practices in the field.

About the authors

No items found.