Character World Model 1

LemonSlice avatars always have natural, expressive hand gestures when they speak. Character World Model 1 takes this further by letting you trigger specific whole-body motions like waving, nodding, or touching hair. There are three ways to use actions, from fully automatic to fully manual.

Actions are an enterprise-only feature. Contact support@lemonslice.com to get access.

1. Action Machine (automatic)

Get the full power of actions with no custom logic required. LemonSlice automatically selects and triggers contextually appropriate actions based on the conversation state. Set enable_actions and action_engine when creating a session:

avatar = lemonslice.AvatarSession(
    agent_image_url="https://example.com/avatar.jpg",
    enable_actions=True,
    action_engine="natural",
)

session_id = await avatar.start(session, room=ctx.room)

async with aiohttp.ClientSession() as session:
    transport = LemonSliceTransport(
        bot_name="Pipecat",
        api_key=os.getenv("LEMONSLICE_API_KEY"),
        session=session,
        session_request=LemonSliceNewSessionRequest(
            agent_image_url="https://example.com/avatar.jpg",
            enable_actions=True,
            action_engine="natural",
        ),
    )

// Given Agora’s Conversational AI join endpoint
// with an avatar block under properties,
// add the parameter enable_actions and action_engine
{
  "avatar": {
    "vendor": "generic",
    "enable": true,
    "params": {
      "api_key": "<lemonslice_api_key>",
      "api_base_url": "https://lemonslice.com/api/liveai/agora",
      "avatar_id": "lemonslice",
      "agent_image_url": "<public_image_url>",
      "enable_actions": true,
      "action_engine": "natural",
      "video_encoding": "H264",
      "agora_uid": "<avatar_rtc_uid>",
      "agora_token": "<avatar_rtc_token>"
    }
  }
}

curl --request POST \
  --url "https://lemonslice.com/api/liveai/sessions" \
  --header "X-API-Key: $LEMONSLICE_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "transport_type": "websocket-livekit",
    "agent_image_url": "https://example.com/avatar.jpg",
    "enable_actions": true,
    "action_engine": "natural",
    "livekit_properties": {
      "livekit_url": "wss://your-project.livekit.cloud",
      "livekit_token": "<agent_access_token>"
    }
  }'

Enrich the Action Machine with user state (LiveKit only)

By default, the Action Machine knows two avatar states: IDLING and RESPONDING. You can unlock a third state, LISTENING, by publishing the user’s speech state from your agent code (e.g., agent.py). This lets the avatar react naturally while the user is speaking.

import asyncio
import json

USER_STATE_TOPIC = "ls.user-state"
LEMONSLICE_AVATAR_IDENTITY = "lemonslice-avatar-agent"

async def _publish_user_state(state: str) -> None:
    try:
        await ctx.room.local_participant.publish_data(
            json.dumps({"state": state}).encode("utf-8"),
            reliable=True,
            topic=USER_STATE_TOPIC,
            destination_identities=[LEMONSLICE_AVATAR_IDENTITY],
        )
    except Exception:
        logger.exception("failed to publish user state")

@session.on("user_state_changed")
def _on_user_state_changed(ev: UserStateChangedEvent) -> None:
    if ev.new_state == "speaking":
        state = "speaking"
    elif ev.new_state == "listening":
        state = "idle"
    else:
        return
    asyncio.create_task(_publish_user_state(state))

2. LiveKit data messages (manual, lowest latency)

Trigger actions yourself with full control over timing. LiveKit data messages are approximately 250 ms faster than the REST control endpoint, making them the recommended option for LiveKit users. Set enable_actions when creating a session:

avatar = lemonslice.AvatarSession(
    agent_image_url="https://example.com/avatar.jpg",
    enable_actions=True,
)

session_id = await avatar.start(session, room=ctx.room)

Publish a data message with topic ls.action and a JSON payload containing the action name from the available actions table:

const LEMONSLICE_ACTION_TOPIC = "ls.action";
const LEMONSLICE_AVATAR_IDENTITY = "lemonslice-avatar-agent";

const performAction = useCallback(
  async (action: string) => {
    const avatarParticipant = remoteParticipants.find(
      ({ identity }) => identity === LEMONSLICE_AVATAR_IDENTITY,
    );

    if (!avatarParticipant) {
      console.warn("LemonSlice avatar is not in the room");
      return;
    }

    const payload = new TextEncoder().encode(JSON.stringify({ action }));

    try {
      await room.localParticipant.publishData(payload, {
        reliable: true,
        topic: LEMONSLICE_ACTION_TOPIC,
        destinationIdentities: [avatarParticipant.identity],
      });
    } catch (error) {
      console.error("Failed to perform LemonSlice action:", error);
    }
  },
  [room, remoteParticipants],
);

LEMONSLICE_ACTION_TOPIC = "ls.action"
LEMONSLICE_AVATAR_IDENTITY = "lemonslice-avatar-agent"

async def _perform_action(action: str) -> None:
    try:
        await ctx.room.local_participant.publish_data(
            json.dumps({"action": action}).encode("utf-8"),
            reliable=True,
            topic=LEMONSLICE_ACTION_TOPIC,
            destination_identities=[LEMONSLICE_AVATAR_IDENTITY],
        )
    except Exception:
        logger.exception("failed to publish avatar action")

3. Control endpoint (manual, any integration)

Use the control endpoint to trigger actions from any integration (Daily, Agora, WebSocket). If LiveKit data messages are available to you, prefer those for lower latency. Set enable_actions when creating a session:

async with aiohttp.ClientSession() as session:
    transport = LemonSliceTransport(
        bot_name="Pipecat",
        api_key=os.getenv("LEMONSLICE_API_KEY"),
        session=session,
        session_request=LemonSliceNewSessionRequest(
            agent_image_url="https://example.com/avatar.jpg",
            enable_actions=True,
        ),
    )

curl --request POST \
  --url "https://lemonslice.com/api/liveai/sessions" \
  --header "X-API-Key: $LEMONSLICE_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "transport_type": "websocket-livekit",
    "agent_image_url": "https://example.com/avatar.jpg",
    "enable_actions": true,
    "properties": {
      "livekit_url": "wss://your-project.livekit.cloud",
      "livekit_token": "<LIVEKIT_TOKEN>"
    }
  }'

Once the session is running, send a POST request with the action name. Replace {session_id} with your active session ID and <ACTION_NAME> with any action from the table below.

curl -X POST "https://lemonslice.com/api/liveai/sessions/{session_id}/control" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $LEMONSLICE_API_KEY" \
  -d '{"event": "action", "action": "<ACTION_NAME>"}'

When actions are enabled, agent_prompt and agent_idle_prompt are ignored.

Available actions

Actions marked with ✅ work best when the avatar is silent. All other actions can be triggered at any time.

Action Name	Description	Best During Silence
`adjust_collar`	Adjusts shirt collar/necklace
`angry`	Angry expression
`arms_crossed`	Crosses arms over chest
`blow_kiss`	Blows a kiss	✅
`clasp_hands`	Clasps hands together
`deep_breath`	Takes a slow, deep breath	✅
`excited`	Excited expression
`explain`	Talking gesture using hands
`explain_finger_up`	Raises a finger to make a point
`explain_palms_up`	Talking gesture with palms up
`glance_down`	Glances downward
`glance_down_then_rub_eye`	Glances down then rubs eye
`glance_right`	Glances to the right
`glance_side_down_side`	Glances to the side then down
`glance_side_up`	Glances to the side then up
`glance_sideways`	Glances to the side
`glance_up`	Glances upward
`hands_on_hip`	Places hands on hips
`hold_chin`	Rests hand on chin
`laugh`	Laughs
`lean_long`	Sustained forward lean
`lean_slow`	Slow, gentle lean forward
`lean_strong`	Pronounced lean forward
`listen`	Acknowledging nods	✅
`looking_around`	Looks around the room
`move_more`	Pronounced body movement
`move_subtle`	Subtle body movement
`phone_call`	Talks on the phone
`raise_eyebrow`	Raises an eyebrow
`rub_eye`	Rubs one eye
`rub_eye_then_touch_hair`	Rubs eye then touches hair
`shoulder_shimmy`	Playful shoulder shimmy	✅
`smile`	Warm smile	✅
`texting`	Texts on the phone
`tilt_head_right`	Tilts head	✅
`touch_chin`	Touches chin thoughtfully
`touch_hair`	Touches or caresses hair
`turn_left`	Turns body to the left	✅
`wave`	Waves hello or goodbye
`yawn`	Yawns	✅

Integrations

Prompting guide

Advanced

Endpoints

Character World Model 1

1. Action Machine (automatic)

Enrich the Action Machine with user state (LiveKit only)

2. LiveKit data messages (manual, lowest latency)

3. Control endpoint (manual, any integration)

Available actions

​1. Action Machine (automatic)

​Enrich the Action Machine with user state (LiveKit only)

​2. LiveKit data messages (manual, lowest latency)

​3. Control endpoint (manual, any integration)

​Available actions

1. Action Machine (automatic)

Enrich the Action Machine with user state (LiveKit only)

2. LiveKit data messages (manual, lowest latency)

3. Control endpoint (manual, any integration)

Available actions