Introduction
Helpers provide additional ways to interact with your bot pipeline. They offer a common interface for working with transports, events, and messages within the context of a specific service.
Since RTVI does not make any assumptions about which services a developer will leverage in their bots, helpers reduce unnecessary functionality in your voice clients and facilitate framework extensibility.
Some examples of where helpers can be useful include:
- LLM message context modificiation
- Direct text-to-speech output
- Multi-lingual workflows
- Custom convenience functions
It’s easiest to think of helpers as wrappers around existing voice client functionality that target a specific service, making your RTVI codebase less verbose.
Using a helper
Helpers must be imported, defined and registered to you voice client.
Here is an example using the included LLM helper:
import {
LLMHelper,
LLMHelperEvent
VoiceMessage,
} from "realtime-ai";
const voiceClient = new VoiceClient({
services: {
llm: "openai"
},
config: [
// ...
]
});
const llmHelper = new LLMHelper({
callbacks: {
onLLMContextUpdate: (messages) => {
// ...
}
}
});
voiceClient.registerHelper("llm", llmHelper);
// Events
voiceClient.on(LLMHelperEvent.LLMContextUpdate, (messages) => console.log(messages));
await llmHelper.appendContext({
role: "user",
content: "Tell me a joke!",
});
Registering a helper
Registering a helper provides it with a reference to your voice client. Helpers must be registered in order to use event and action dispatch / resolution.
voiceClient.registerHelper(service, helper);
Service targeted by the helper as part of your voice client constructor services
map.
An exception is thrown if no matching service can be found.
Note: targeting services is required for scenarios where a pipeline contains multiple declarations of the same service type. For example, your pipeline may have two or more LLM services registered.
A helper subclass instance of VoiceClientHelper
.
Retrieving a helper
The voice client holds a reference to all registered helpers. This can be useful if you do not want to hold / pass a reference across your codebase:
const llmHelper = voiceClient.getHelper("llm") as LLMHelper;
Unregistering a helper
A service in your pipeline can currently have reference to a single helper to avoid conflicting action dispatch.
You can unregister a helper from a service like so:
voiceClient.unregisterHelper("llm")
Unregistering helpers will not delete any references in your own code. Any subsequent attempts to use an unregistered helper against the service name will raise an exception.
If you register a new helper with the same service name and use a previous helper instance, you may encounter errors.
How are helper messages handled?
The VoiceClientHelper
base class requires that a helper define a handleMessage
method.
Any RTVI messages received from the transport that does not match native voice client message types are passed through to your client’s registered helpers:
public handleMessage(ev: VoiceMessage): void {
switch(ev.type) {
case MyHelperMessageType.SOME_MESSAGE:
this._options.callbacks.onSomeMessage(ev.data);
break;
// etc
}
}
Be mindful of defining a ‘default’ switch handler if you have multiple registered helpers listening for the same message type. Catching them will prevent lower order helpers from receiving the message.
Writing helpers
Creating your own helper classes can be useful when you want to create reusable code that targets a specific service (or series of services.)
The abstract VoiceClientHelper
class is intentionally lightweight to provide as much flexibility to the developer as needed.
A typical helper definition will likely include:
- A constructor
- Callback methods
- Event and message types
A basic implementation of a helper would look like this: