This guide is designed to help developers understand the API Extension system and provide a clear path for those who want to take advantage of its new capabilities. This is completely optional since the existing API will be supported forever. This guide is targeted at developers with specific needs that are addressed by the new features in the API Extension.

The most significant difference between the existing API and the API Extension is the central object around which each system is designed:

• Existing API: Centered entirely around bots. Everything revolves around bot objects, and bots produce recordings. This approach works well when bots are your only source of data.
• API Extension: Centered around recordings. All data ingestion sources (bots, desktop SDK, and other future sources) produce recordings. Recordings are no longer tied to a specific data source.

This architectural shift allows for greater flexibility and consistency across different data sources.

Common API Extension Use Cases

Multiplexed Real-time data delivery

Using the API Extension, we can send multiple different types of data to the same websocket endpoint. This is convenient if, for example, you want to do your own diarization using active speaker events and raw audio.

Implementation Guide

1. Configure the Recall Webhook Dashboard to listen for bot.done events. Once you’ve fully completed the migration, you can stop listening for bot.status_change events.
2. Use the recording_config object in Create Bot endpoint. This signals that the bot should use API extension fields. Set up a websocket in the realtime_endpoints field so that you can listen for multiple different types of event on the same websocket connection.

// Create Bot
{
	meeting_url: ,
	join_at: ,
	
	// new recording_config field ✅
	recording_config: {

		// multiplex realtime streaming of raw audio
		// and active speaker events
		realtime_endpoints: [
			{
				type: "websocket", // webhook also supported
				url: "wss://....",
        events: [
          "participant_events.speech_on",
          "participant_events.speech_off",
          "audio_mixed_raw.data",
          // other events also supported
        ]
			}
		],
        audio_mixed_raw: {}
	}
	
	// fields not supported ❌
	transcription_options: , // use recording_config.transcript
	real_time_transcription: , // use recording_config.realtime_endpoints
	real_time_media: , // use recording_config.realtime_endpoints
	recording_mode: , // use recording_config.video_mixed_layout
	recording_mode_options: , // use recording_config.video_mixed_layout
	
	...(other fields)
}

1. Once the bot enters a call and starts recording, you’ll start receiving multiplexed data on the endpoint configured from the Create Bot request.

2. Receive the bot.done webhook (emitted once bot has finished recording) and fetch the recording (if desired) using recordings[0].media_shortcuts.video_mixed.data.download_url.

Transcribing without recording

The API Extension allows you to transcribe conversations without ever producing a video or audio recording of the conversation. This can be important when dealing with privacy-focused customers who never want a recording to be created.

Implementation Guide

1. Configure the Recall Webhook Dashboard to listen for bot.done events. Once you’ve fully implemented this workflow, you can stop listening for bot.status_change events.
2. Use the recording_config object in Create Bot endpoint. Explicitly mark video_mixed_mp4: null (ensures bot will not capture video in the meeting)

// Create Bot
{
	meeting_url: ,
	join_at: ,
	
	// **new recording_config field** ✅
	recording_config: {
		// configure bot to not record video
		video_mixed_mp4: null 

		// configure bot to generate transcript
		transcript: {
			provider: {
				// any one of the currently supported providers
				// can be used here with appropriate settings
				assembly_ai_streaming: {...assembly streaming options}
			}
		},
				
		// optional: realtime streaming of transcripts
		realtime_endpoints: [
			{
				type: "websocket", // webhook also supported
				url: "wss://....",
				events: [
					"transcript.data",
					// other supported events can be 
					// multiplexed in this realtime data stream
				] 
			}
		]
	}
	
	
	**// fields not supported ❌**
	transcription_options: , // use recording_config.transcript
	real_time_transcription: , // use recording_config.realtime_endpoints
	real_time_media: , // use recording_config.realtime_endpoints
	recording_mode: , // use recording_config.video_mixed_layout
	recording_mode_options: , // use recording_config.video_mixed_layout
	
	
	...(other fields)
}

1. Receive the bot.done webhook (emitted once bot has finished recording) and fetch the transcript using the Get Bot Transcript endpoint. You can also fetch the transcript by utilizing the new API Extension fields. To do this, you can call Retrieve Bot and download the transcript from recordings[0].media_shortcuts.transcript.data.download_url in the response. Keep in mind that this transcript will have a different schema than the schema returned by Get Bot Transcript.

// Retrieve Bot
{
	id: ,
	...(other fields)
	
	recordings: [
		// recording resource 
		{
			id: ,
			media_shortcuts: {
				video_mixed: null, // confirm no video recording was captured
				transcript: {
					id: ,
					...(other transcript fields),
					
					data: {
						// use to download diarized transcript 
						// (format: https://beta-docs.recall.ai/docs/download-urls#json-transcript-download-url)
						download_url: , 
					}
				},
				
				...(other media shortcuts)
			}
			
			...(other recording fields)
		}
	]
	
	
	**// fields not supported ❌**
	video_url: , // use recordings[0].media_shortcuts
	media_retention_end: , // use recordings[0].expires_at
	meeting_metadata: , // use recordings[0].media_shortcuts.meeting_metadata.data.download_url
	meeting_participants: , // use recordings[0].media_shortcuts.participant_events.data.download_url
	recording: , // use recordings
}

Supporting multiple types of data ingestion

You may want to integrate multiple different data sources as you capture conversation data. The API Extension allows you to capture data from bots, the desktop SDK, and other sources in a more seamless manner than the existing API does.

Implementation Guide

1. Configure the Recall Webhook Dashboard to listen for recording.done events. Once you’ve fully implemented this workflow, you can stop listening for bot.status_change events.
2. Use the recording_config object in Create Bot endpoint. This signals that the bot should use API extension fields.

// Create Bot
{
	meeting_url: ,
	join_at: ,
	
	// **new recording_config field** ✅
	recording_config: {

		**// fields not supported ❌**
		transcription_options: , // use recording_config.transcript
		real_time_transcription: , // use recording_config.realtime_endpoints
		real_time_media: , // use recording_config.realtime_endpoints
		recording_mode: , // use recording_config.video_mixed_layout
		recording_mode_options: , // use recording_config.video_mixed_layout
		...
	}	
	
	...(other fields)
}

1. After receiving recording.done you can retrieve the recorded video file using the Retrieve Recording endpoint with the recording_id field from the webhook. You can also listen for this webhook to kick off any post-processing that needs to be done on your end.

2. You can also optionally do asynchronous transcription after receiving the recording.done webhook by using the Create Async Transcript endpoint.

// Create Async Transcript
{
  "provider": {
  	// any one of the currently supported providers
		// can be used here with appropriate settings
    "assembly_ai_async": {
      ...
    }
  },
  // optionally attach metadata to the transcript
  "metadata": {}
}

New API Extension Endpoints and Fields

Endpoint Mapping

The API Extension introduces several new endpoints that are meant to be data-source agnostic.

One example of this would be the Create Async Transcript endpoint. While the existing Analyze Bot Media endpoint can be used to create a transcript for a bot, the Create Async Transcript can create a transcript for any recording. This is regardless of whether the recording was produced by a bot, the desktop SDK, or another source.

Below is a mapping between the bot endpoints and their API Extension counterparts:

Create Bot Request Field Mapping

📘

Translating Existing Create Bot Requests

If you want to translate your existing Create Bot request to use the API Extension fields, you can use this free tool.

To take advantage of the API extension, you’ll need to structure your Create Bot requests slightly differently. Here’s a mapping of the existing Create Bot fields compared to the alternative API extension fields:

Create Bot Response Field Mapping

When using the API Extension, the structure of the bot response object changes. Here's a mapping of existing API fields to their API Extension equivalents:

Webhook Changes

The API Extension also introduces a wide variety of new webhooks to give you more insight into the status of the resources

For example, instead of listening for the bot.status_change event and filtering for the done status on your end, you can listen for the bot.done event instead. You can also configure custom metadata to be attached and sent with these webhooks. Here’s an example of the structure you can expect from the bot.done webhook:

{
  "event": "bot.done",
  "data": {
    "data": {
      "code": string,
      "sub_code": string | null,
      "updated_at": string
    },
    "bot": {
      "id": string,
      "metadata": object
    }
  }
}