OpenAI

Stream meeting audio to an OpenAI Realtime session and play its responses back into the meeting.

Recall.ai connects to the OpenAI Realtime API over WebRTC. Configure the model, voice, and instructions in the bot request. Your server can attach to the same session to handle tools and conversation logic while Recall.ai handles audio.

Setup

1. Get an OpenAI API key

Create a project API key in the OpenAI dashboard with access to the Realtime API and the model you plan to use. You do not need to create a saved assistant or agent first.

2. Add your credentials to Recall.ai

Open Agents → OpenAI in the Recall region where you create bots:

Select the workspace used by your Recall API key, then save your OpenAI project key.

Quickstart

Call Create Bot with your meeting URL and Realtime session settings. This example uses GPT-Realtime with automatic turn detection and interruptions.

We recommend setting the relevant meeting platform in variant to web_4_core when pairing this audio agent with a video provider or a kind: "webpage" source. See bot sizing guidance.

{
  "meeting_url": "MEETING_URL",
  "output_media": {
    "audio": {
      "kind": "openai",
      "config": {
        "session": {
          "model": "gpt-realtime",
          "instructions": "You are a helpful meeting assistant. Keep spoken replies brief.",
          "audio": {
            "input": {
              "turn_detection": {
                "type": "server_vad",
                "create_response": true,
                "interrupt_response": true
              }
            },
            "output": {
              "voice": "marin"
            }
          }
        }
      }
    }
  }
}

Once the bot joins, Recall.ai creates an OpenAI Realtime call, sends the meeting audio mix, and plays the model's replies into the meeting.

For recording the agent's voice, see Recording meeting agents. For response controls, see the Meeting agents FAQ.

Additional configurations

The output_media.audio.config object supports:

FieldRequiredDescription
sessionYesOpenAI Realtime session settings as a JSON object, up to 1 MiB when serialized. Must include model.

The API key is read from saved credentials. api_key, credential_id, and a saved assistant ID are not accepted in this config.

Configure the session

Recall.ai passes session to OpenAI when creating the WebRTC call. It sets type to realtime and output_modalities to ["audio"] if omitted. If supplied, those fields must have those values.

Common fields inside session include:

FieldDescription
modelRequired Realtime model ID. Must be 1–256 printable ASCII characters without whitespace.
instructionsInstructions for the conversation.
audio.output.voiceVoice supported by the selected model, such as marin.
audio.input.turn_detectionOpenAI turn-detection settings. Keep automatic responses enabled for a voice agent that responds without your server issuing response commands.
toolsTool definitions supported by OpenAI. Your server must handle custom function calls and any required approvals.

See OpenAI Realtime conversations for session settings. Recall.ai negotiates audio formats through WebRTC, so audio.input.format and audio.output.format are not accepted.

Add a camera or avatar

OpenAI is supported under output_media.audio. You can combine it with an independent camera or screenshare source. When audio is configured, the meeting audio mix goes only to the audio session; camera and screenshare sources do not receive it. This integration does not send meeting video to OpenAI.

To use an automatic avatar, combine output_media.audio with automatic_video_output, leaving output_media.camera and output_media.screenshare absent.

FAQ

Why does the bot join but the agent does not respond?

Check that the saved key can access the selected Realtime model, that the session uses audio output, and that automatic responses are enabled or your server is requesting them. If the model is waiting on a tool, confirm that your server returns its result. Save credentials in each Recall workspace and region where you create bots.

What happens when the OpenAI session ends?

OpenAI currently limits Realtime sessions to 60 minutes. The Zoom, Google Meet, Teams, or Webex bot can remain in the meeting after its OpenAI session ends, but the agent stops responding.

Recall.ai emits an openai_session_closed customer-visible log when it observes the session ending, except when output was explicitly stopped or replaced. It does not automatically reconnect. To start a fresh session, replace the audio source using the Output Media endpoint; the new session starts with the settings you supply and does not automatically inherit the previous conversation.


Did this page help you?