RTVI projects implement a client instance that:

  • Facilitates web requests to an endpoint you create.
  • Dispatches single-turn actions to a HTTP bot service when disconnected.
  • Provides methods that handle the connectivity state and real-time interaction with your bot service.
  • Manages media transport (such as audio and video).
  • Provides callbacks and events for handling bot messages and actions.
  • Optionally configures your AI services and pipeline.

​
RTVIClient

import { RTVIClient } from "realtime-ai";

RTVIClient is a base class that serves as a template for building transport-specific implementations.

For connected use-cases, you must pass a transport instance to the constructor for your chosen protocol or provider. See Transport for more information.

If you were looking to use WebRTC as a transport layer, you may use a provider like Daily. In this scenario, you’d construct a transport instance and pass it to the client accordingly:

import { RTVIClient } from "realtime-ai";
import { DailyTransport } from "@daily-co/realtime-ai-daily";

const dailyTransport = new DailyTransport();
const rtviClient = new RTVIClient({
  transport: dailyTransport,
});

All transport packages (such as DailyTransport) extend from the Transport base class defined in RTVI core. You can extend this class if you are looking to implement your own or add additional functionality.

​
Example

import { RTVIEvent, RTVIMessage, RTVIClient } from "realtime-ai";
import { DailyTransport } from "@daily-co/realtime-ai-daily";

const dailyTransport = new DailyTransport();

const rtviClient = new RTVIClient({
  transport: dailyTransport,
  params: {
    baseUrl: "https://your-connect-end-point-here",
    services: {
      stt: "deepgram",
      llm: "together",
      tts: "cartesia",
    },
    config: [
      {
        service: "tts",
        options: [
          { name: "voice", value: "79a125e8-cd45-4c13-8a67-188112f4dd22" }
        ]
      },
      {
        service: "llm",
        options: [
          { name: "model", value: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" },
          {
            name: "messages",
            value: [
              {
                role: "system",
                content:
                  "You are a assistant called ExampleBot. You can ask me anything. Keep responses brief and legible. Your responses will be converted to audio, so please avoid using any special characters except '!' or '?'.",
              }
            ]
          }
        ]
      }
    ]
  }
  enableMic: true,
  enableCam: false,
  timeout: 15 * 1000,
  callbacks: {
    onConnected: () => {
      console.log("[CALLBACK] User connected");
    },
    onDisconnected: () => {
      console.log("[CALLBACK] User disconnected");
    },
    onTransportStateChanged: (state: string) => {
      console.log("[CALLBACK] State change:", state);
    },
    onBotConnected: () => {
      console.log("[CALLBACK] Bot connected");
    },
    onBotDisconnected: () => {
      console.log("[CALLBACK] Bot disconnected");
    },
    onBotReady: () => {
      console.log("[CALLBACK] Bot ready to chat!");
    },
  },
});

try {
  await rtviClient.connect();
} catch (e) {
  console.error(e.message);
}

// Events
rtviClient.on(RTVIEvent.TransportStateChanged, (state) => {
  console.log("[EVENT] Transport state change:", state);
});
rtviClient.on(RTVIEvent.BotReady, () => {
  console.log("[EVENT] Bot is ready");
});
rtviClient.on(RTVIEvent.Connected, () => {
  console.log("[EVENT] User connected");
});
rtviClient.on(RTVIEvent.Disconnected, () => {
  console.log("[EVENT] User disconnected");
});

​
API reference

​
transport

​
transport
Class<Transport>
default: "undefined"required

Instance of Transport required to connect to your bot service (rtviClient.connect()). Transports define connectivity and state logic that manage the lifecycle of your session. Transports also define a protocol for message exchange between your client and bot service (such as actions, or generic messages.

If your transport package (e.g. @daily-co/realtime-ai-daily) exports multiple transports classes, you can specify which to use here.

import { RTVIClient } from "realtime-ai";
import { DailyTransport } from "@daily-co/realtime-ai-daily";

const dailyTransport = new DailyTransport();
const rtviClient = new RTVIClient({
  transport: dailyTransport,
});

​
params

A client-side mutatable object that is used to configure how your client connects or triggers actions.

Parameters can be updated at any time during the client lifecycle.

​
baseUrl
string
required

Base URL to a developer hosted API that exposes various endpoints to the RTVI application.

​
endpoints
Record<RTVIURLEndpoints, string>

Endpoints appended to your client base URL. RTVI will apply defaults if not provided.

PropertyDescription
connect:stringDeveloper hosted endpoint that triggers authentication, transport session creation and bot instantiation. Typically returns a valid authentication bundle required to join a session. Defaults to “connect/”.
action:stringDeveloper hosted endpoint that triggers disconnected action for single-turn operations. Typically implemented as a HTTP Stream or Websocket. Defaults to “action/”.

You should ensure your endpoints have the necessary CORS headers set to allow requests from your client application.

const rtviClient = RTVIClient({
  params: {
    baseUrl: "https://www.example.com",
    endpoints: {
      connect: "/connect", // https://www.example.com/connect
      action: "/action", // https://www.example.com/action
    },
  },
});
​
requestData
object

Custom request data that is passed to the baseUrl with any web request. This can be used to pass additional data to your server-side endpoint.

const rtviClient = RTVIClient({
  params: {
    baseUrl: "https://www.example.com",
    requestData: {
      customData: "my custom data",
    },
  },
});

await rtviClient.connect();

// POST https://www.example.com/connect
// --data '{ "customData": "my custom data" }'

await rtviClient.disconnect();

rtviClient.params.requestData = {
  customData: "my updated custom data",
};

await rtviClient.connect();

// POST https://www.example.com/connect
// --data '{ "customData": "my updated custom data"}'
​
headers
Headers

Optional Headers object that is used when making requests to the baseUrl.

​
config
Array <RTVIClientConfigOption[]>

Pipeline RTVI configuration passed to your bot pipeline. Must contain a valid RTVIClientConfigOption[] array.

Client configuration is optional and can be passed directly to the bot via your server-side endpoint (where sensitive information can be provided, such as API keys.)

The service order of your configuration array is important. See configuration.

​
Optional properties

​
callbacks
RTVIEventCallbacks

Map of callback functions. See callbacks.

​
timeout
number
default: "undefined"

Time (in milleseconds) to wait for the bot to enter a ready state after calling connect(). If the timeout period is elapsed, the client will return a ConnectionTimeoutError error and disconnect from the transport.

​
enableMic
boolean
default: "true"

Enable user’s local microphone device.

​
enableCamera
boolean
default: "false"

Enable user’s local webcam device. Note: please ensure you are using a transport package that supports video.

​
Custom Connect Handler

Calling rtviClient.connect() will trigger a default fetch query to your baseUrl/connect endpoint. If you need to override this behaviour, you can provide a custom connect handler.

​
customConnectHandler
(params, timeout, abortController) => Promise<void>

Override the default fetch query called by rtviClient.connect(). If your server-side infrastructure has specific requirements, you can provide custom behaviour here.

  • params:RTVIClientParams Params provided in client constructor.
  • timeout:Timeout | undefined Start timeout. You should clear this once your method resolves e.g. clearTimeout(timeout).
  • abortController:AbortController - Aborting connection via abortController.abort() will reject the inflight request, clear the connection timeout and set the client to an error state.
const rtviClient = new RTVIClient({
  // ...
  customConnectHandler: async (
    params: RTVIClientParams,
    timeout: number | undefined,
    abortController: AbortController
  ): Promise<void> => {
    const connectUrl = rtviClient.constructUrl(params.endpoints.connect);
    return await fetch(connectUrl, {
      method: "POST",
      mode: "cors",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ...`,
      },
      body: JSON.stringify({
        services: {...},
        config: [...],
        // ... your custom body params here
      }),
      signal: abortController?.signal,
    }).then((res) => {
      clearTimeout(timeout); // Clear the start timeout (if set)

      if (res.ok) {
        return res.json();
      }
      return Promise.reject(res);
    });
  },
});
SDK IntroductionClient Methods