Services - RTVI Client Web Documentation

Bots built with RTVI define and register various services as part of their pipeline. For example, a typical voice bot will likely use Language Models (LLM), text-to-speech, and speech-to-text services to provide voice-to-voice interactions. RTVI clients allow you to specify which of the available services to use in a session and how to configure them.

Understanding services

Your RTVI client can configure any services registered in an RTVI-powered bot using the configuration options it makes available. Service registration within your bot might look something like this pseudocode:

 def main(services:dict, api_keys:dict):
    rtvi_llm = RTVIService(
        name="llm",
        options=[
            RTVIServiceOption(
                name="model",
                type="string",
                handler=config_llm_model_handler),
            RTVIServiceOption(
                name="messages",
                type="array",
                handler=config_llm_messages_handler)])

    rtvi = RTVIProcessor(config=config)
    rtvi.register_service(rtvi_llm)

    def service_factory_get(service: str, name: str, api_key: str) -> Any:
        match service:
            case "openai":
                return OpenAILLMService(
                    name=name,
                    api_key=api_key)
            case "together":
                return OpenAILLMService(
                    name=name,
                    api_key=api_key,
                    base_url="https://api.together.xyz"
                )


llm = service_factory_get(services["llm"], "llm", api_keys[api_keys["llm"]] or "")

# ... pipeline code

The above bot file defines a service named llm with two config options, model and messages, as well as their associated handlers. Bots can define one or more services to a particular function. For example, we may want to run a pipeline that can be switched between different LLM providers e.g. OpenAI, Together, Anthropic etc.

Building a client requires knowledge of the services that have been registered and the corresponding names. This information is necessary to pass the appropriate configuration and API keys as string-matched values.

Names and providers

In the above example, we have a factory method that returns the relevant provider class for a specific service based on name string.

Service Name - An arbitrary string that references the service in your bot file, e.g. "llm"
Service Provider - A provider-specific implementation that gets included in the pipeline, e.g. OpenAILLMService

Selecting between services on the client

RTVI bots can be passed an optional services object at startup that can be used to specify which provider to use for the specified service name. In the above above example, we can configure a voice client to use Together like so:

const voiceClient = new VoiceClient({
  services: {
    llm: "together",
  },
  config: [
    {
      service: "llm",
      options: [
        { name: "model", value: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" },
      ],
    },
  ],
  // etc ...
});

Passing API keys

Service keys are secret, you should not set them on a client. To pass API keys to your RTVI bot, host a secure server-side endpoint that includes them as part of the config payload.

RTVI bots accept an api_keys object, mapping them to the relevant service account. Here is an example server-side route using NextJS:

// api/
export async function POST(request: Request) {
  const { services, config } = await request.json();

  if (!services || !config || !process.env.DAILY_BOTS_URL) {
    return new Response(`Services or config not found on request body`, {
      status: 400,
    });
  }

  const payload = {
    services,
    api_keys: {
      together: process.env.TOGETHER_API_KEY,
    },
    config: [...config],
  };

  const req = await fetch("your-bot-start-endpoint", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify(payload),
  });

  const res = await req.json();

  return Response.json(res);
}

You can also extend your config object here, if you wanted to provide some defaults outside of the client constructor.

In the above example, we’d set the baseUrl property of the voice client to point to this endpoint, and define which service’s to use in the bot’s registry like so:

const voiceClient = new DailyVoiceClient({
  baseUrl: "/api",
  services: {
    llm: "together",
  },
  config: [
    {
      service: "llm",
      options: [
        { name: "model", value: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" },
      ],
    },
  ],
});

config references the service name that it was registered with in your bot file.
services specifies which provider / service account to use for a specific name-matched service in the registry (in this case, llm.)
api_keys provides a key matched to the service account.

​Understanding services

​Names and providers

​Selecting between services on the client

​Passing API keys

Understanding services

Names and providers

Selecting between services on the client

Passing API keys