Real-time Avatar Updates

Overview

With any LemonSlice avatar model, you can change the character’s reference image during an active call. Simply post an update-image event to our REST endpoint with either a public URL or the raw image bytes (base64-encoded). Call it from an LLM tool, your backend, a webhook, or any client. The LemonSlice video stream reflects the new image in under a second.

_{Latency in this demo comes from initiating the tool call (upstream of LemonSlice). Image changes take ~600 ms.}

Applications

Real-time image updates go beyond passive interactive avatars and make characters feel truly immersive: the avatar’s look can follow the narrative arc of the conversation, shifting appearance, scene, or even identity based on context as the call unfolds.

Use case	What you get
Change appearance	Outfits, accessories, hairstyles, or other looks for the same character
Change the setting	Move the character into a new scene as the story evolves
User-driven customization	Let users pick or upload a look and apply it live
Change characters	Switch to an entirely different character mid-call

End-to-end example

Our example repo provides a complete Next.js + LiveKit Agents demo with three entry points for image changes:

Tool call — LLM picks a new image based on conversational context
Upload — user supplies a new image (URL or local upload)
Prompt-based edit — an image-edit model generates a new image from a text prompt, then applies it

The repo also provides code for managing the lifecycle of image change requests (e.g. listening for completion events).

How to trigger

Send a POST to the LemonSlice control endpoint with event: "update-image" and exactly one of:

image_url — a publicly accessible URL to the new image
image_base64 — the raw image bytes, base64-encoded (a data:image/...;base64, data URL also works). The decoded image must be under 900 KB.

Send image bytes directly to skip a server-side URL fetch and save ~0.4 seconds.

curl -X POST "https://lemonslice.com/api/liveai/sessions/<session_id>/control" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $LEMONSLICE_API_KEY" \
  -d '{
    "event": "update-image",
    "image_url": "https://example.com/avatar.jpg"
  }'

import httpx

async def update_avatar_image(session_id: str, image_url: str, api_key: str) -> bool:
    url = f"https://lemonslice.com/api/liveai/sessions/{session_id}/control"
    async with httpx.AsyncClient() as client:
        response = await client.post(
            url,
            headers={
                "Content-Type": "application/json",
                "X-API-Key": api_key,
            },
            json={"event": "update-image", "image_url": image_url},
            timeout=10.0,
        )
        return response.is_success

import base64
import httpx

async def update_avatar_image_bytes(session_id: str, image_bytes: bytes, api_key: str) -> bool:
    url = f"https://lemonslice.com/api/liveai/sessions/{session_id}/control"
    async with httpx.AsyncClient() as client:
        response = await client.post(
            url,
            headers={
                "Content-Type": "application/json",
                "X-API-Key": api_key,
            },
            json={
                "event": "update-image",
                "image_base64": base64.b64encode(image_bytes).decode("ascii"),
            },
            timeout=10.0,
        )
        return response.is_success

async function updateAvatarImage(sessionId, imageUrl, apiKey) {
  const res = await fetch(
    `https://lemonslice.com/api/liveai/sessions/${sessionId}/control`,
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "X-API-Key": apiKey,
      },
      body: JSON.stringify({
        event: "update-image",
        image_url: imageUrl,
      }),
    },
  );
  return res.ok;
}

async function updateAvatarImageBytes(sessionId, imageBytes, apiKey) {
  const res = await fetch(
    `https://lemonslice.com/api/liveai/sessions/${sessionId}/control`,
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "X-API-Key": apiKey,
      },
      body: JSON.stringify({
        event: "update-image",
        image_base64: Buffer.from(imageBytes).toString("base64"),
      }),
    },
  );
  return res.ok;
}

A successful response looks like { "success": true }. That means the control event was accepted; the video stream switches to the new image in under a second.

Listen for completion

When the new image is loaded and LemonSlice has started pushing frames from it, we emit a message over the relevant transport channel. It can be helpful to listen to these events to trigger a follow-up response once the new image is live and surface failures:

Event	Meaning
`image_change_complete`	The new image is active in the video stream.
`image_change_error`	The update failed (e.g. image could not be downloaded).

Listen on LiveKit’s lemonslice data topic or Pipecat’s Daily app-message channel:

import json
from livekit import rtc

LEMONSLICE_RPC_TOPIC = "lemonslice"

def on_data_received(packet: rtc.DataPacket) -> None:
    if packet.topic != LEMONSLICE_RPC_TOPIC:
        return
    try:
        payload = json.loads(packet.data.decode("utf-8"))
    except (UnicodeDecodeError, json.JSONDecodeError):
        return
    if not isinstance(payload, dict):
        return
    if payload.get("type") == "image_change_complete":
        # New frames are live — clear any transitional UI
        ...
    elif payload.get("type") == "image_change_error":
        # Show error; optional: retry or revert UI state
        ...

room.on("data_received", on_data_received)

const LEMONSLICE_RPC_TOPIC = "lemonslice";

room.on(RoomEvent.DataReceived, (payload, participant, kind, topic) => {
  if (topic !== LEMONSLICE_RPC_TOPIC) return;

  try {
    const data = JSON.parse(new TextDecoder().decode(payload));
    if (data?.type === "image_change_complete") {
      // New frames are live — clear any transitional UI
    }
    if (data?.type === "image_change_error") {
      // Show error; optional: retry or revert UI state
      console.error(data.error ?? data.message);
    }
  } catch (e) {
    console.warn("Unable to decode LemonSlice RPC JSON", e);
  }
});

useDailyEvent("app-message", (ev) => {
  if (ev?.data?.type === "image_change_complete") {
    // New frames are live — clear any transitional UI
  }
  if (ev?.data?.type === "image_change_error") {
    console.error(ev.data.error);
  }
});

Recommended UI

The avatar stays fully functional during an image update — it can still receive audio and continue the call. You do not need to tear down video or block the room. The swap itself acts as an interruption (any audio currently playing is cut off), so trigger it during silence when possible. Since it completes in under a second, the update generally doesn’t need a transitional UI state: just let the video change. An exception is prompt-based edits, where an image-edit model like Nano Banana generates the new image on the fly. Image generation models still take several seconds, and that latency happens before LemonSlice is involved. For that flow, we recommend showing a transitional state while the image is being generated:

When the user submits a prompt, enter a transitional UI state
- Keep the existing video visible; overlay a subtle indicator, such as the iridescent shader in our example
When generation finishes, send the update-image event
On image_change_complete, clear the indicator and optionally generate a response
On image_change_error or timeout, clear the indicator and surface an error if desired

Additional Tips

Use images that follow our Avatar Image Tips.
Avoid unnecessarily large image files — bigger payloads add latency to the change.
Keep a small library of pre-hosted HTTPS images for predictable tool-driven updates; use an image-edit API (e.g. Fal) when you want to support open-ended or user-defined changes.
For full character changes (including voice and personality), coordinate LemonSlice’s update-image with your own TTS voice and LLM instructions outside LemonSlice.

LiveKit Integration

Pipecat Integration

Prompting guide

Advanced

Endpoints

Real-time Avatar Updates

Overview

Applications

End-to-end example

How to trigger

Listen for completion

Recommended UI

Additional Tips

​Overview

​Applications

​End-to-end example

​How to trigger

​Listen for completion

​Recommended UI

​Additional Tips

Overview

Applications

End-to-end example

How to trigger

Listen for completion

Recommended UI

Additional Tips