RTVI Client Web includes a LLM helper for typical LLM tasks and workflows.

​
Using the LLM helper

import { VoiceClient, LLMHelper } from "realtime-ai";

const voiceClient = new VoiceClient({
  // ...
  services: {
    llm: "openai",
  },
});
const llmHelper = new LLMHelper({
  callbacks: {
    // ...
  },
});
voiceClient.registerHelper("llm", llmHelper);

​
Actions

All of the below are abstracted actions. As with all actions, they can be awaited or chained with .then(). If the bot is unable to process the action, it will trigger the onMessageError callback and MessageError event.

​
getContext()

llm:get_context Retrieve LLM context from bot. Returns Promise<LLMContext> 
const llmHelper = voiceClient.getHelper("llm") as LLMHelper;
const ctx = await llmHelper.getContext();

// >  { messages?: LLMContextMessage[]; tools?: []; }

​
setContext()

llm:set_context Replaces the current LLM context with the provided one. Returns Promise<boolean>.
​
context
LLMContext
required
LLMContext option to set.
​
interrupt
boolean
default:"false"
Interrupt the current conversation and apply the new context immediately.
const llmHelper = voiceClient.getHelper("llm") as LLMHelper;
await llmHelper.setContext(
  {
    messages: [
      {
        role: "system",
        content: "Your are a helpful assistant",
      },
      {
        role: "user",
        content: "Tell me a joke",
      },
    ],
  },
  false | true
);

// >  true | false (if error)

​
appendToMessages()

llm:append_to_messages Append a new message to the existing context. Returns Promise<boolean>.
​
context
LLMContextMessage
required
New message to apply to the context.
​
runImmediately
boolean
default:"false"
Apply the new message immediately, or wait until the current turn concludes.
const llmHelper = voiceClient.getHelper("llm") as LLMHelper;
await llmHelper.appendToMessages(
  {
    role: "user",
    content: "Tell me a joke",
  },
  false | true
);

// >  true | false (if error)

​
Callbacks and events

  • onLLMJsonCompletion: (jsonString: string) => void;`
  • onLLMFunctionCall: (func: LLMFunctionCallData) => void;
  • onLLMFunctionCallStart: (functionName: string) => void;
  • onLLMMessage: (message: LLMContextMessage) => void;