Emotion Engine

LemonSlice avatars always have natural, expressive hand gestures when they speak. Character World Model 1 includes a built-in emotion engine that decides what the character does and feels. Use a preset to give the avatar a more life-like presence, or trigger specific actions and emotions programmatically.

CWM-1 is available to users on Ultra and Enterprise tiers. See pricing.

Looping CWM-1 examples: any character, whole-body motion, collar adjustments, and hair interactions

There are three ways to use actions and emotions, from fully automatic to fully manual.

1. Action Machine (automatic)

Get the full power of actions with no custom logic required. LemonSlice automatically selects and triggers contextually appropriate actions based on the conversation state. Set model and action_engine when creating a session:

avatar = lemonslice.AvatarSession(
    agent_image_url="https://example.com/avatar.jpg",
    model="cwm-1",  # or "cwm-1-lite"
    action_engine="natural",
)

session_id = await avatar.start(session, room=ctx.room)

async with aiohttp.ClientSession() as session:
    transport = LemonSliceTransport(
        bot_name="Pipecat",
        api_key=os.getenv("LEMONSLICE_API_KEY"),
        session=session,
        session_request=LemonSliceNewSessionRequest(
            agent_image_url="https://example.com/avatar.jpg",
            model="cwm-1",  # or "cwm-1-lite"
            action_engine="natural",
        ),
    )

// Given Agora’s Conversational AI join endpoint
// with an avatar block under properties,
// add the parameters model and action_engine
{
  "avatar": {
    "vendor": "generic",
    "enable": true,
    "params": {
      "api_key": "<lemonslice_api_key>",
      "api_base_url": "https://lemonslice.com/api/liveai/agora",
      "avatar_id": "lemonslice",
      "agent_image_url": "<public_image_url>",
      "model": "cwm-1", // or "cwm-1-lite"
      "action_engine": "natural",
      "video_encoding": "H264",
      "agora_uid": "<avatar_rtc_uid>",
      "agora_token": "<avatar_rtc_token>"
    }
  }
}

# model: "cwm-1" or "cwm-1-lite"
curl --request POST \
  --url "https://lemonslice.com/api/liveai/sessions" \
  --header "X-API-Key: $LEMONSLICE_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "transport_type": "websocket-livekit",
    "agent_image_url": "https://example.com/avatar.jpg",
    "model": "cwm-1",
    "action_engine": "natural",
    "livekit_properties": {
      "livekit_url": "wss://your-project.livekit.cloud",
      "livekit_token": "<agent_access_token>"
    }
  }'

Enrich the Action Machine with user state (LiveKit only)

By default, the Action Machine knows two avatar states: IDLING and RESPONDING. You can unlock a third state, LISTENING, by publishing the user’s speech state from your agent code (e.g., agent.py). This lets the avatar react naturally while the user is speaking.

import asyncio
import json

USER_STATE_TOPIC = "ls.user-state"
LEMONSLICE_AVATAR_IDENTITY = "lemonslice-avatar-agent"

async def _publish_user_state(state: str) -> None:
    try:
        await ctx.room.local_participant.publish_data(
            json.dumps({"state": state}).encode("utf-8"),
            reliable=True,
            topic=USER_STATE_TOPIC,
            destination_identities=[LEMONSLICE_AVATAR_IDENTITY],
        )
    except Exception:
        logger.exception("failed to publish user state")

@session.on("user_state_changed")
def _on_user_state_changed(ev: UserStateChangedEvent) -> None:
    if ev.new_state == "speaking":
        state = "speaking"
    elif ev.new_state == "listening":
        state = "idle"
    else:
        return
    asyncio.create_task(_publish_user_state(state))

2. LiveKit data messages (manual, lowest latency)

Trigger actions yourself with full control over timing. LiveKit data messages are approximately 250 ms faster than the REST control endpoint, making them the recommended option for LiveKit users. Set model to "cwm-1" when creating a session:

avatar = lemonslice.AvatarSession(
    agent_image_url="https://example.com/avatar.jpg",
    model="cwm-1",  # or "cwm-1-lite"
)

session_id = await avatar.start(session, room=ctx.room)

Publish a data message with topic ls.action and a JSON payload containing the action name from the available actions table:

const LEMONSLICE_ACTION_TOPIC = "ls.action";
const LEMONSLICE_AVATAR_IDENTITY = "lemonslice-avatar-agent";

const performAction = useCallback(
  async (action: string) => {
    const avatarParticipant = remoteParticipants.find(
      ({ identity }) => identity === LEMONSLICE_AVATAR_IDENTITY,
    );

    if (!avatarParticipant) {
      console.warn("LemonSlice avatar is not in the room");
      return;
    }

    const payload = new TextEncoder().encode(JSON.stringify({ action }));

    try {
      await room.localParticipant.publishData(payload, {
        reliable: true,
        topic: LEMONSLICE_ACTION_TOPIC,
        destinationIdentities: [avatarParticipant.identity],
      });
    } catch (error) {
      console.error("Failed to perform LemonSlice action:", error);
    }
  },
  [room, remoteParticipants],
);

LEMONSLICE_ACTION_TOPIC = "ls.action"
LEMONSLICE_AVATAR_IDENTITY = "lemonslice-avatar-agent"

async def _perform_action(action: str) -> None:
    try:
        await ctx.room.local_participant.publish_data(
            json.dumps({"action": action}).encode("utf-8"),
            reliable=True,
            topic=LEMONSLICE_ACTION_TOPIC,
            destination_identities=[LEMONSLICE_AVATAR_IDENTITY],
        )
    except Exception:
        logger.exception("failed to publish avatar action")

3. Control endpoint (manual, any integration)

Use the control endpoint to trigger actions from any integration (Daily, Agora, WebSocket). If LiveKit data messages are available to you, prefer those for lower latency. Set model to "cwm-1" when creating a session:

async with aiohttp.ClientSession() as session:
    transport = LemonSliceTransport(
        bot_name="Pipecat",
        api_key=os.getenv("LEMONSLICE_API_KEY"),
        session=session,
        session_request=LemonSliceNewSessionRequest(
            agent_image_url="https://example.com/avatar.jpg",
            model="cwm-1",  # or "cwm-1-lite"
        ),
    )

# model: "cwm-1" or "cwm-1-lite"
curl --request POST \
  --url "https://lemonslice.com/api/liveai/sessions" \
  --header "X-API-Key: $LEMONSLICE_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "transport_type": "websocket-livekit",
    "agent_image_url": "https://example.com/avatar.jpg",
    "model": "cwm-1",
    "properties": {
      "livekit_url": "wss://your-project.livekit.cloud",
      "livekit_token": "<LIVEKIT_TOKEN>"
    }
  }'

Once the session is running, send a POST request with the action name. Replace {session_id} with your active session ID and <ACTION_NAME> with any action from the table below.

curl -X POST "https://lemonslice.com/api/liveai/sessions/{session_id}/control" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $LEMONSLICE_API_KEY" \
  -d '{"event": "action", "action": "<ACTION_NAME>"}'

When using model="cwm-1", the arguments agent_prompt and agent_idle_prompt are ignored.

Available actions

Actions marked with ✅ work best when the avatar is silent. All other actions can be triggered at any time. Select the eye icon to watch an example.

Action Name	Description	Best During Silence
`adjust_collar`	Adjusts shirt collar/necklace
`angry`	Angry expression
`arms_crossed`	Crosses arms over chest
`blow_kiss`	Blows a kiss	✅
`clasp_hands`	Clasps hands together
`clear_action`	Ends the current action
`deep_breath`	Takes a slow, deep breath	✅
`excited`	Excited expression
`explain`	Talking gesture using hands
`explain_finger_up`	Raises a finger to make a point
`explain_palms_up`	Talking gesture with palms up
`glance_down`	Glances downward
`glance_down_then_rub_eye`	Glances down then rubs eye
`glance_right`	Glances to the right
`glance_side_down_side`	Glances to the side then down
`glance_side_up`	Glances to the side then up
`glance_sideways`	Glances to the side
`glance_up`	Glances upward
`hands_on_hip`	Places hands on hips
`hold_chin`	Rests hand on chin
`laugh`	Laughs
`lean_long`	Sustained forward lean
`lean_slow`	Slow, gentle lean forward
`lean_strong`	Pronounced lean forward
`listen`	Acknowledging nods	✅
`looking_around`	Looks around the room
`move_more`	Pronounced body movement
`move_subtle`	Subtle body movement
`phone_call`	Talks on the phone
`raise_eyebrow`	Raises an eyebrow
`rub_eye`	Rubs one eye
`rub_eye_then_touch_hair`	Rubs eye then touches hair
`shoulder_shimmy`	Playful shoulder shimmy	✅
`smile`	Warm smile	✅
`texting`	Texts on the phone
`tilt_head_right`	Tilts head	✅
`touch_chin`	Touches chin thoughtfully
`touch_hair`	Touches or caresses hair
`turn_left`	Turns body to the left	✅
`wave`	Waves hello or goodbye
`yawn`	Yawns	✅

Frequently Asked Questions

What happens if I send an action while the Action Machine is running?

A manual action takes priority. Publishing ls.action, or sending an action event to the control endpoint, cancels the action the Action Machine is playing and starts yours. When your action finishes, the Action Machine resumes if action_engine is set.

How do I stop the current action?

Send clear_action on the same channel you use for other actions. It ends the action in progress and returns the avatar to its default behavior.LiveKit data message on topic ls.action:

{ "action": "clear_action" }

Control endpoint:

curl -X POST "https://lemonslice.com/api/liveai/sessions/{session_id}/control" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $LEMONSLICE_API_KEY" \
  -d '{"event": "action", "action": "clear_action"}'

How long does an action last?

Every action plays once for a fixed duration, before the avatar returns to its default state. Trigger the same action again to hold the action or expression longer.

Can I play two actions at the same time?

CWM-1 plays one action at a time, so angry and arms_crossed cannot run together. Wait for the current action to finish, or send clear_action before starting the next one.

Can I really trigger actions from a single photo?

Yes, CWM-1 works with any character. Use of actions or emotions does not require any character-specific fine-tuning.

Integrations

Prompting guide

Advanced

Endpoints

1. Action Machine (automatic)

Enrich the Action Machine with user state (LiveKit only)

2. LiveKit data messages (manual, lowest latency)

3. Control endpoint (manual, any integration)

Available actions

Frequently Asked Questions

​1. Action Machine (automatic)

​Enrich the Action Machine with user state (LiveKit only)

​2. LiveKit data messages (manual, lowest latency)

​3. Control endpoint (manual, any integration)

​Available actions

​Frequently Asked Questions

1. Action Machine (automatic)

Enrich the Action Machine with user state (LiveKit only)

2. LiveKit data messages (manual, lowest latency)

3. Control endpoint (manual, any integration)

Available actions

Frequently Asked Questions