Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.rtvi.ai/llms.txt

Use this file to discover all available pages before exploring further.

Helpers provide developer convenience to extend the functionality of the base RTVI Client by wrapping actions, messages or other arbitrary features. They define a common interface for working with transports, events, and messages within the context of a specific service. For example, you may have a LLM helper, TTS helper, and so on. Since RTVI does not make any assumptions about which services a developer will leverage in their bots, helpers remove the need for unnecessary service-centric methods and handlers in the base client, and facilitate framework extensibility. Some examples of where helpers can be useful include:
  • LLM message context modificiation
  • Direct text-to-speech output
  • Multi-lingual workflows
  • Custom convenience functions
It’s easiest to think of helpers as wrappers around existing client functionality that target a specific service, making your RTVI codebase less verbose.
Since most pipelines will feature at least one LLM, the core RTVI web package includes a single LLM helper out of the box. Refer to LLM Helper for a quick-start introduction on how to work with LLMs in your project.

​
Using a helper

Helpers must be imported, defined and registered to your client. Here is an example using the bundled LLM helper: 
import {
  LLMHelper,
  LLMHelperEvent
  RTVIMessage,
  RTVIClient,
} from "realtime-ai";

const rtviClient = new RTVIClient({
  // ...
});

const llmHelper = new LLMHelper({
  callbacks: {
      onLLMContextUpdate: (messages) => {
          // ...
      }
  }
});

rtviClient.registerHelper("llm", llmHelper);

// Events
rtviClient.on(LLMHelperEvent.LLMContextUpdate, (messages) => console.log(messages));

await llmHelper.appendContext({
  role: "user",
  content: "Tell me a joke!",
});

​
Registering a helper

Registering a helper creates a reference with your RTVI client. Helpers must be registered in order to use event and action dispatch / resolution. rtviClient.registerHelper(service, helper);
​
service
string
required
Service targeted by the helper as part of your voice client constructor services map. An exception is thrown if no matching service can be found. Note: targeting services is required for scenarios where a pipeline contains multiple declarations of the same service type. For example, your pipeline may have two or more LLM services registered.
​
helper
RTVIClientHelper
required
A helper subclass instance of RTVIClientHelper.

​
Retrieving a helper

The client holds a reference to all registered helpers. This can be useful if you do not want to hold / pass a reference across your codebase: 
const llmHelper = rtviClient.getHelper("llm") as LLMHelper;

​
Unregistering a helper

A service in your pipeline can currently have reference to a single helper to avoid conflicting action dispatch. You can unregister a helper from a service like so: 
rtviClient.unregisterHelper("llm");
Unregistering helpers will not delete any references in your own code. Any subsequent attempts to use an unregistered helper against the service name will raise an exception. If you register a new helper with the same service name and use a previous helper instance, you may encounter errors.

​
How are helper messages handled?

The RTVIClientHelper base class requires that a helper define a handleMessage method. Any RTVI messages received from the transport that does not match native voice client message types are passed through to your client’s registered helpers: 
public handleMessage(ev: VoiceMessage): void {
  switch(ev.type) {
    case MyHelperMessageType.SOME_MESSAGE:
      this._options.callbacks.onSomeMessage(ev.data);
      break;
      // etc
  }
}
Be mindful of defining a ‘default’ switch handler if you have multiple registered helpers listening for the same message type. Catching them will prevent lower order helpers from receiving the message.

​
Writing helpers

Creating your own helper classes can be useful when you want to create reusable code that targets a specific service (or series of services.) The abstract RTVIClientHelper class is intentionally lightweight to provide as much flexibility to the developer as needed. A typical helper definition will likely include:
  • A constructor
  • Callback methods
  • Event and message types
A basic implementation of a helper would look like this: 
import { 
  RTVIClientHelper,
  RTVIClientHelperOptions
} from "realtime-ai";

// --- Events
// E.g. rtviClient.on(MyHelperEvents.SomeEvent, (message) => { ... });
export enum MyHelperEvents {
SomeEvent = "some-event",
}

// --- Message types
// E.g. await rtviClient.action({ service: "myService", action: MyHelperMessageType.SOME_MESSAGE, arguments: [...]});
export enum MyHelperMessageType {
SOME_MESSAGE = "some-message",
}

// --- Callbacks
export type MyHelperCallbacks = Partial<{
onSomeMessage: (message) => void;
}>;

// --- Interface and class
export interface MyHelperOptions extends RTVIClientHelperOptions {
callbacks?: MyHelperCallbacks;
}

export class LLMHelper extends RTVIClientHelper {
  protected declare _options: MyHelperOptions;

constructor(options: MyHelperOptions) {
super(options);
}

public handleMessage(ev: VoiceMessage): void {
switch (ev.type) {
case MyHelperMessageType.SOME_MESSAGE:
// dispatch action response etc
break;
}
}

public getMessageTypes(): string[] {
return Object.values(MyHelperMessageType) as string[];
}
}