LLM Helper
RTVI Client Web includes a LLM helper for typical LLM tasks and workflows.
Using the LLM helper
import { VoiceClient, LLMHelper } from "realtime-ai";
const voiceClient = new VoiceClient({
// ...
services: {
llm: "openai",
},
});
const llmHelper = new LLMHelper({
callbacks: {
// ...
},
});
voiceClient.registerHelper("llm", llmHelper);
Actions
All of the below are abstracted actions.
As with all actions, they can be awaited or chained with .then()
.
If the bot is unable to process the action, it will trigger the onMessageError
callback and MessageError
event.
getContext()
llm:get_context
Retrieve LLM context from bot. Returns Promise<LLMContext>
const llmHelper = voiceClient.getHelper("llm") as LLMHelper;
const ctx = await llmHelper.getContext();
// > { messages?: LLMContextMessage[]; tools?: []; }
setContext()
llm:set_context
Replaces the current LLM context with the provided one. Returns Promise<boolean>
.
LLMContext option to set.
Interrupt the current conversation and apply the new context immediately.
const llmHelper = voiceClient.getHelper("llm") as LLMHelper;
await llmHelper.setContext(
{
messages: [
{
role: "system",
content: "Your are a helpful assistant",
},
{
role: "user",
content: "Tell me a joke",
},
],
},
false | true
);
// > true | false (if error)
appendToMessages()
llm:append_to_messages
Append a new message to the existing context. Returns Promise<boolean>
.
New message to apply to the context.
Apply the new message immediately, or wait until the current turn concludes.
const llmHelper = voiceClient.getHelper("llm") as LLMHelper;
await llmHelper.appendToMessages(
{
role: "user",
content: "Tell me a joke",
},
false | true
);
// > true | false (if error)
Callbacks and events
- onLLMJsonCompletion: (jsonString: string) => void;`
- onLLMFunctionCall: (func: LLMFunctionCallData) => void;
- onLLMFunctionCallStart: (functionName: string) => void;
- onLLMMessage: (message: LLMContextMessage) => void;