Production checklist for deploying LemonSlice avatars

Clip from our home page playground demo — a LiveKit agent built with the checklist on this page. We wrote these recommendations based on what we learned shipping it.

Use a ringing sound and UI

Avatar video takes a few seconds to initialize. A phone-call metaphor works well: users understand ringing, pickup, conversation, and hang-up.

Handle avatar connect/disconnect events

Read this section if you experience any of the following issues:

Empty room (user sees black screen)
User gets no video or audio, but audio/text input are still active
Avatar disappears

Use the @lemonsliceai/avatar npm package on the frontend to detect when the avatar’s first video frame has rendered, then switch from ringing to your active call UI. Listen for the avatar leaving to return to an inactive state so users can rejoin. See Use a ringing sound and UI for the UX pattern.

LiveKit
Daily (Hosted)

Also listen for ParticipantDisconnected when the avatar leaves (call completion, idle timeout, errors, etc.).

On the backend, listen for these events and call agent_session.generate_reply() when the LemonSlice avatar joins to prevent idle time before the avatar speaks. See the LiveKit starter project for a complete implementation.

import { useState, useEffect } from "react";
import { LiveKitRoom, RoomEvent, useRoomContext } from "@livekit/components-react";
import { LiveKitAvatarReadyWatcher } from "@lemonsliceai/avatar/livekit-react";

const AVATAR_IDENTITY = "lemonslice-avatar-agent";

function AvatarCall({ serverUrl, token }: { serverUrl: string; token: string }) {
  const [avatarReady, setAvatarReady] = useState(false);

  return (
    <LiveKitRoom serverUrl={serverUrl} token={token} connect>
      <LiveKitAvatarReadyWatcher onReady={() => setAvatarReady(true)} />

      {!avatarReady && <RingingUI />}
      {avatarReady && <ActiveCallUI />}

      <DisconnectHandler />
    </LiveKitRoom>
  );
}

function DisconnectHandler() {
  const room = useRoomContext();

  useEffect(() => {
    const handleDisconnect = (participant) => {
      if (participant.identity === AVATAR_IDENTITY) {
        room.disconnect();
        // Swap to inactive call UI — user can rejoin from here
      }
    };

    room.on(RoomEvent.ParticipantDisconnected, handleDisconnect);
    return () => room.off(RoomEvent.ParticipantDisconnected, handleDisconnect);
  }, [room]);

  return null;
}

LemonSlice events arrive on Daily’s app-message channel. When the agent participant leaves, handle participant-left to reset your UI.See the hosted Daily starter project for a full React implementation.

import { useState, useRef, useEffect, useCallback } from "react";
import { useDailyEvent, useMediaTrack } from "@daily-co/daily-react";
import { useAvatarReady } from "@lemonsliceai/avatar/react";

function AvatarCall() {
  const [avatarReady, setAvatarReady] = useState(false);
  const avatarTrack = useMediaTrack("LemonSlice Agent", "video");

  useAvatarReady(avatarTrack?.persistentTrack ?? null, {
    onReady: () => setAvatarReady(true),
  });

  useDailyEvent(
    "app-message",
    useCallback((ev) => {
      if (ev?.data?.type === "idle_timeout") {
        // Agent stopped due to inactivity — swap to inactive call UI
      }
    }, [])
  );

  useDailyEvent("participant-left", (ev) => {
    if (ev.participant?.user_name === "LemonSlice Agent") {
      // Swap to inactive call UI — user can start a new room
    }
  });

  return avatarReady ? <ActiveCallUI /> : <RingingUI />;
}

Handle room errors

Read this section if you experience any of the following issues:

Avatar fails to join a call
Crashed calls — avatar leaves or audio cuts out

LiveKit
Daily (Hosted)

Listen for Disconnected events to handle network errors, WebRTC failures, or join failures.

import { DisconnectReason, RoomEvent } from "livekit-client";

room.on(RoomEvent.Disconnected, (reason) => {
  switch (reason) {
    case DisconnectReason.CLIENT_INITIATED:
      // User ended the call — swap to inactive call UI
      break;
    default:
      // Unexpected disconnect — swap to inactive call UI
  }
});

Listen for daily_error app-messages and Daily’s top-level error/left events.

useDailyEvent("app-message", (ev) => {
  if (ev?.data?.type === "daily_error") {
    console.error(ev.data.error, ev.data.fatal);
    // Swap to inactive call UI if fatal
  }
});

useDailyEvent("error", (ev) => {
  console.error("Daily error:", ev);
  // Swap to inactive call UI
});

Catch pipeline errors

Read this section if you experience any of the following issues:

Avatar does not speak
Session dies unexpectedly

LiveKit
Daily (Hosted)

Subscribe to AgentSession error events on your backend. Errors with err.recoverable == False mean the pipeline is dead — end the session gracefully.

@session.on("error")
def on_session_error(ev: ErrorEvent) -> None:
    err = ev.error
    if isinstance(err, TTSError):
        logger.error("AgentSession TTS error", exc_info=err.error)
    elif isinstance(err, STTError):
        logger.error("AgentSession STT error", exc_info=err.error)
    elif isinstance(err, LLMError):
        logger.error("AgentSession LLM error", exc_info=err.error)
    else:
        logger.error("AgentSession error", exc_info=err.error)

The Hosted Pipeline manages STT, LLM, and TTS for you. Monitor pipeline issues on the frontend via LemonSlice app-messages — especially daily_error and video_generation_error. See the event reference for the full list.

Handle startup failures

Read this section if you experience any of the following issues:

The call never connects

If the avatar fails to join, give users a way to exit gracefully instead of waiting indefinitely.

LiveKit
Daily (Hosted)

The LiveKit starter project monitors agent startup and can send a failure message to the room. Catch it on the frontend:

const ROOM_MESSAGE_TOPIC = "lemonslice/message";

room.on(RoomEvent.DataReceived, (payload, participant, kind, topic) => {
  if (topic !== ROOM_MESSAGE_TOPIC) return;

  try {
    const message = JSON.parse(new TextDecoder().decode(payload));
    if (message.type === "startup_failure") {
      room.disconnect();
      // Swap to inactive call UI — user can retry
    }
  } catch (err) {
    console.error("Failed to parse data packet:", err);
  }
});

Set a client-side timeout after creating the room. If bot_ready has not fired within your threshold (e.g. 30 seconds), call leave() and show a retry UI. Also handle daily_error with fatal: true during startup.

Check timeouts

Read this section if you experience any of the following issues:

Avatar suddenly exits a call
Calls end after the same number of minutes every time

Several timeouts can affect a session. Confirm each is set to the value you intend:

LemonSlice idle timeout (default 60 seconds) — resets when the avatar is talking
LemonSlice GPU timeout (default 30 minutes) — contact support@lemonslice.com if you need longer calls
Third-party timeouts (LiveKit, Daily, ElevenLabs, etc.)

LiveKit
Daily (Hosted)

Set idle_timeout on AvatarSession:

Setting idle_timeout to -1 disables the LemonSlice idle timeout. Ensure sessions are properly terminated to avoid stale calls and runaway billing.

avatar = lemonslice.AvatarSession(
    agent_image_url="....",
    agent_prompt="a person talking.",
    idle_timeout=600,
)
session_id = await avatar.start(session, room=ctx.room)

const avatar = new lemonslice.AvatarSession({
  agentImageUrl: "...",
  agentPrompt: "a person talking.",
  idleTimeout: 600,
});
const sessionId = await avatar.start(session, room);

transport = LemonSliceTransport(
    ...
    session_request=LemonSliceNewSessionRequest(
        agent_image_url="...",
        agent_prompt="a person talking.",
        idle_timeout=600,
    ),
)

Idle timeout is configured on your agent in the LemonSlice web app. When it fires, you’ll receive an idle_timeout app-message — use it to reset your UI.End sessions explicitly with a force-end control event when the user hangs up:

sendAppMessage({ event: "force-end" }, "*");

Budget your latency

Read this section if your avatar feels slow to respond — including delayed replies after the user finishes speaking.

If the conversation lags, the bottleneck is usually your LLM — not LemonSlice video. Check LLM response times and tool-calling latency first.

Humans expect a quick back-and-forth. Measure each pipeline step (STT, LLM, TTS, avatar video) and set a latency budget upfront. LLM times often creep from sub-second to three or four seconds as you add function calls, reasoning, or multimodal inputs. If a step blows your budget, move it off the critical path — run it async, set expectations with the user, or defer it.

Use VAD to decrease perceived latency

Perceived latency matters more than absolute latency. Voice Activity Detection (VAD) lets you start reacting before a user has fully finished speaking, which effectively pulls your entire pipeline forward. Good turn detection means you can kick off generation as soon as intent is clear.

Optional: Show a transcription

Displaying transcription can improve usability, especially for catching STT errors and making the agent’s timing feel more predictable. Users can see when their speech has been “accepted,” which reduces ambiguity around when the agent will respond. However, transcription introduces its own UX risks. Many pipelines expose both fast, low-accuracy interim results and slower, higher-accuracy final transcripts (e.g., interim_results=false to suppress partials for Deepgram). When both are surfaced, users can see text rapidly change or correct itself, which feels unstable and undermines trust. We recommend disabling interim results if you choose to show transcriptions.

​Use a ringing sound and UI

​Handle avatar connect/disconnect events

​Handle room errors

​Catch pipeline errors

​Handle startup failures

​Check timeouts

​Budget your latency

​Use VAD to decrease perceived latency

​Optional: Show a transcription

Use a ringing sound and UI

Handle avatar connect/disconnect events

Handle room errors

Catch pipeline errors

Handle startup failures

Check timeouts

Budget your latency

Use VAD to decrease perceived latency

Optional: Show a transcription