Services
Selecting and authenticating bot services
Bots built with RTVI define and register various services as part of their pipeline. For example, a typical voice bot will likely use Language Models (LLM), text-to-speech, and speech-to-text services to provide voice-to-voice interactions.
RTVI clients allow you to specify which of the available services to use in a session and how to configure them.
Understanding services
Your RTVI client can configure any services registered in an RTVI-powered bot using the configuration options it makes available.
Service registration within your bot might look something like this pseudocode:
def main(services:dict, api_keys:dict):
rtvi_llm = RTVIService(
name="llm",
options=[
RTVIServiceOption(
name="model",
type="string",
handler=config_llm_model_handler),
RTVIServiceOption(
name="messages",
type="array",
handler=config_llm_messages_handler)])
rtvi = RTVIProcessor(config=config)
rtvi.register_service(rtvi_llm)
def service_factory_get(service: str, name: str, api_key: str) -> Any:
match service:
case "openai":
return OpenAILLMService(
name=name,
api_key=api_key)
case "together":
return OpenAILLMService(
name=name,
api_key=api_key,
base_url="https://api.together.xyz"
)
llm = service_factory_get(services["llm"], "llm", api_keys[api_keys["llm"]] or "")
# ... pipeline code
The above bot file defines a service named llm
with two config options, model
and messages
, as well as their associated handlers.
Bots can define one or more services to a particular function. For example, we may want to run a pipeline that can be switched between different LLM providers e.g. OpenAI, Together, Anthropic etc.
Building a client requires knowledge of the services that have been registered and the corresponding names. This information is necessary to pass the appropriate configuration and API keys as string-matched values.
Names and providers
In the above example, we have a factory method that returns the relevant provider class for a specific service based on name string.
- Service Name - An arbitrary string that references the service in your bot file, e.g.
"llm"
- Service Provider - A provider-specific implementation that gets included in the pipeline, e.g.
OpenAILLMService
Selecting between services on the client
RTVI bots can be passed an optional services
object at startup that can be used to specify which provider to use for the specified service name.
In the above above example, we can configure a voice client to use Together like so:
const voiceClient = new VoiceClient({
services: {
llm: "together",
},
config: [
{
service: "llm",
options: [
{ name: "model", value: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" },
],
},
],
// etc ...
});
Passing API keys
Service keys are secret, you should not set them on a client. To pass API keys to your RTVI bot, host a secure server-side endpoint that includes them as part of the config payload.
RTVI bots accept an api_keys
object, mapping them to the relevant service account. Here is an example server-side route
using NextJS:
// api/
export async function POST(request: Request) {
const { services, config } = await request.json();
if (!services || !config || !process.env.DAILY_BOTS_URL) {
return new Response(`Services or config not found on request body`, {
status: 400,
});
}
const payload = {
services,
api_keys: {
together: process.env.TOGETHER_API_KEY,
},
config: [...config],
};
const req = await fetch("your-bot-start-endpoint", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify(payload),
});
const res = await req.json();
return Response.json(res);
}
You can also extend your config object here, if you wanted to provide some defaults outside of the client constructor.
In the above example, we’d set the baseUrl property of the voice client to point to this endpoint, and define which service’s to use in the bot’s registry like so:
const voiceClient = new DailyVoiceClient({
baseUrl: "/api",
services: {
llm: "together",
},
config: [
{
service: "llm",
options: [
{ name: "model", value: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" },
],
},
],
});
config
references the service name that it was registered with in your bot file.services
specifies which provider / service account to use for a specific name-matched service in the registry (in this case,llm
.)api_keys
provides a key matched to the service account.