Skip to main content

Documentation Index

Fetch the complete documentation index at: https://lemonslice.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

The examples on this page use syntax specific to our LiveKit integration, but the same concepts apply to our other integrations.

​
Handle avatar connect/disconnect events

Read this section if you experience any of the following issues:
  • Empty room (user sees black screen)
  • User gets no video or audio, but audio/text input are still active
  • Avatar disappears
  • Avatar is delayed in speaking
Listen for the bot_ready message via DataReceived event and the ParticipantDisconnected event on the frontend to switch between your UI’s ringing, active, and finished states. LemonSlice will fire the bot_ready event when the avatar has joined the room and video frames have been generated and are visible to the user. Use this to trigger the UI to show the active call. Using bot_ready is more robust than ParticipantConnected because there is a small delay between when the avatar participant joins the room and when actual video frames are visible to the user. The bot_ready event only fires once video is fully ready, ensuring your UI transitions at the right moment. The ParticipantDisconnected event will fire if the avatar leaves, which happens if the pipeline stops for any reason (normal call completion, idle timeout, upstream errors, etc.). Use this to leave the active call UI and disconnect from the room.
LiveKit users: You should also listen to these events on the backend and call agent_session.generate_reply() when the LemonSlice avatar joins to prevent any idle time before the avatar speaks.
After the user initiates a call, stay in the “ringing state” until the avatar is ready (signaled through the bot_ready event). Then, if the avatar ever leaves (signaled through the ParticipantDisconnected event), move to the inactive call UI to allow the user to rejoin the call. Refer to our LiveKit agent quickstart example to see how these listeners can be registered on the backend. We provide a representative frontend snippet below:
JAVASCRIPT
import { Room, RoomEvent } from "livekit-client";

const AVATAR_IDENTITY = "lemonslice-avatar-agent";
const LEMONSLICE_RPC_TOPIC = "lemonslice";

const room = new Room(...); // Logic to join to the LiveKit room

room.on(RoomEvent.DataReceived, (payload, participant, kind, topic) => {
    if (topic !== LEMONSLICE_RPC_TOPIC) return;

    try {
        const data = JSON.parse(new TextDecoder().decode(payload));

        if (data === null || typeof data !== "object") return;

        if (data.type === "bot_ready") {
            console.log(`Avatar is ready! session_id = ${data.session_id}`);
            // TODO - swap over to active call UI.
            // We recommend staying in the "ringing" state until the avatar has joined. 
        }
    } catch (e) {
        console.warn("Unable to decode LemonSlice RPC JSON", e);
    }
});

room.on(RoomEvent.ParticipantDisconnected, (participant) => {
    if (participant.identity === AVATAR_IDENTITY) {
        console.log("LemonSlice avatar left");
        room.disconnect();
        // TODO - swap over to inactive call UI. Users can reinitiate a new call from here. 
    }
});

​
Handle room errors

Read this section if you experience any of the following issues:
  • Avatar fails to join a call
  • Crashed calls. Avatar leaves the call or audio cuts out.
Your frontend should listen for Disconnected events to handle any issues with the room. These can include network errors, WebRTC failures, or join failures.
JAVASCRIPT
room.on(RoomEvent.Disconnected, (reason) => {
  switch (reason) {
    case DisconnectReason.CLIENT_INITIATED:
      console.log("Clean disconnect");
      // TODO - you manually called room.disconnect()
      // swap over to inactive call UI
      break;
    default:
      console.log("Disconnected:", reason);
      // TODO: swap over to inactive call UI
  }
});

​
Catch pipeline errors

Read this section if you experience any of the following issues:
  • Avatar does not speak
  • Session dies unexpectedly
Your backend should catch STT, LLM, TTS errors by subscribing to the error event. You can log these errors and then handle them gracefully. Errors tagged with err.recoverable == False should end the pipeline gracefully as the pipeline is now dead.
PYTHON
@session.on("error")
def on_session_error(ev: ErrorEvent) -> None:
    err = ev.error
    if isinstance(err, TTSError):
        logger.error(
            f"AgentSession TTS error",
            exc_info=err.error,
        )
    elif isinstance(err, STTError):
        logger.error(
            f"AgentSession STT error",
            exc_info=err.error,
        )
    elif isinstance(err, LLMError):
        logger.error(
            f"AgentSession LLM error",
            exc_info=err.error,
        )
    else:
        logger.error(
            f"AgentSession error",
            exc_info=err.error
        )

​
Handle startup failures

Read this section if you experience any of the following issues:
  • The call never connects
Sometimes the LemonSlice avatar may fail to join the room. This happens very rarely. To handle this, you can send a message via the WebRTC room to indicate a failure. This allows the UI to gracefully exit the call if the LemonSlice participant never joins. Our LiveKit agent quickstart example shows how to monitor the agent status and send a message to the room if it does not join in time. This message can be caught on the frontend using code like below:
JAVASCRIPT
import { Room, RoomEvent } from 'livekit-client';

const ROOM_MESSAGE_TOPIC = 'lemonslice/message'

const room = new Room();
const decoder = new TextDecoder();
room.on(
  RoomEvent.DataReceived,
  (payload, participant, kind, topic) => {
    if (topic !== ROOM_MESSAGE_TOPIC) return;

    try {
      const message = JSON.parse(decoder.decode(payload));

      if (message.type === 'startup_failure') {
        console.log('Room closing event received');
        room.disconnect();
        // TODO - swap to UI where the user can restart the call (inactive call)
      }
    } catch (err) {
      console.error('Failed to parse data packet:', err);
    }
  }
);

​
Check timeouts

Read this section if you experience any of the following issues:
  • Avatar suddenly exits a call
  • Calls suddenly end after the same number of minutes
There are several LemonSlice timeouts that may affect a session, and likely others baked into third-party tools or integrations. Ensure these are all set to the values you intended:
  • LemonSlice idle timeout (default is 60 seconds). Resets when the avatar is talking.
  • LemonSlice GPU timeout (default is 30 minutes. If you regularly have longer calls, please contact support@lemonslice.com)
  • Third-party timeouts (e.g. LiveKit or ElevenLabs)
You can set idle_timeout to -1 to disable the LemonSlice idle timeout, but you should ensure sessions are properly terminated to avoid stale calls and runaway billing.
PYTHON
avatar = lemonslice.AvatarSession(
    agent_image_url="....",
    agent_prompt="a person talking.",
    idle_timeout=600,  # update this LemonSlice parameter
)
session_id = await avatar.start(session, room=ctx.room)