📘

Audio data streaming is currently supported in raw pcm format

Audio format is mono 16 bit signed little-endian PCM at 16khz.

This guide is for you if:

• You want to get the real-time mixed raw audio data from a meeting

Platforms Support

Implementation

Step 1: Create a bot

To get mixed audio per participant, you must set recording_config.audio_mixed_raw = {}. Below is an example of what it would look like in your request

curl --request POST \
     --url https://us-west-2.recall.ai/api/v1/bot \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'authorization: YOUR_RECALL_API_KEY' \
     --data '
{
  "meeting_url": "YOUR_MEETING_URL",
  "recording_config": {
    "audio_mixed_raw": {},
    "realtime_endpoints": [
      {
        "type": "websocket",
        "url": "wss://my-tunnel-domain.ngrok-free.app",
        "events": ["audio_mixed_raw.data"]
      }
    ]
  }
}
'

📘

Make sure to set the realtime endpoint's url uses a ws or wss endpoint.

Step 2: Receive websocket messages with audio data

Setup a websocket server and ensure it is publicly accessible. You will receive messages in the following payload format:

{
  "event": "audio_mixed_raw.data", 
  "data": {
    "data": {
      "buffer": string, // base64-encoded raw audio 16 kHz mono, S16LE(16-bit PCM LE)
      "timestamp": {
      	"relative": float,
        "absolute": string
    	}
    },
    "realtime_endpoint": {
      "id": string,
      "metadata": object,
    },
    "audio_mixed": {
      "id": string,
      "metadata": object
    },
    "recording": {
      "id": string,
      "metadata": object
    },
    "bot": {
      "id": string,
      "metadata": object
    },
  }
}

FAQ

Do muted participants produce audio?

No, muted participants do not produce any audio.

If a participant is unmuted but silent, you will receive empty audio packets.

Will bots receive audio from other bots?

Since bots are participants, if there are other bots in a call, the bot will receive audio from the bot like any other participant.

Since bots are muted by default, unless another bot is outputting audio, the bot will not receive audio packets from other bots

What is the retry behavior?

See the retry behaviors in Real-Time Websocket Endpoints for details.