> ## Documentation Index
> Fetch the complete documentation index at: https://lemonslice.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Websocket Integration

> Drive a LemonSlice Avatar from your own audio stack over WebSocket

## Overview

The Websocket integration lets you drive a LemonSlice avatar from any audio stack. Send PCM audio and control events to LemonSlice over a WebSocket. Synchronized audio and video is streamed to a [LiveKit](https://livekit.io) or [Daily](https://www.daily.co) room that you control.

## Prerequisites

1. A LemonSlice account and API key associated with an active subscription. Create a key at [lemonslice.com/developers](https://lemonslice.com/developers).
2. A reference image for your avatar. See [our guide](/docs/prompting-guide/avatar-image-tips) for recommendations on how to design your avatar image to achieve the best quality possible.
3. A LiveKit or Daily room with separate credentials for:
   * LemonSlice to join and publish audio and video.
   * Your client to join and subscribe to the avatar stream.

For LiveKit, [generate access tokens](https://docs.livekit.io/frontends/reference/tokens-grants/); the room is created when the first participant joins. For Daily, [create a room](https://docs.daily.co/reference/rest-api/rooms/create-room) and [create meeting tokens](https://docs.daily.co/reference/rest-api/meeting-tokens/create-meeting-token).

## How to use

<Steps>
  <Step title="Create a session">
    Send a request to the LemonSlice sessions endpoint with your avatar and the credentials LemonSlice should use to publish into your room.

    <CodeGroup>
      ```bash LIVEKIT theme={null}
      curl --request POST \
        --url "https://lemonslice.com/api/liveai/sessions" \
        --header "X-API-Key: $LEMONSLICE_API_KEY" \
        --header "Content-Type: application/json" \
        --data '{
          "transport_type": "websocket-livekit",
          "agent_image_url": "https://example.com/avatar.jpg",
          "livekit_properties": {
            "livekit_url": "wss://your-project.livekit.cloud",
            "livekit_token": "<agent_access_token>"
          }
        }'
      ```

      ```bash DAILY theme={null}
      curl --request POST \
        --url "https://lemonslice.com/api/liveai/sessions" \
        --header "X-API-Key: $LEMONSLICE_API_KEY" \
        --header "Content-Type: application/json" \
        --data '{
          "transport_type": "websocket-daily",
          "agent_image_url": "https://example.com/avatar.jpg",
          "daily_properties": {
            "daily_url": "https://your-domain.daily.co/room-name",
            "daily_token": "<agent_meeting_token>"
          }
        }'
      ```
    </CodeGroup>

    You can also provide `agent_image_base64` for a local image or `agent_id` to use an avatar created in the LemonSlice web app instead of `agent_image_url`.

    A successful response includes the WebSocket address:

    ```json theme={null}
    {
      "session_id": "...",
      "websocket_address": "wss://...",
      "control_url": "..."
    }
    ```
  </Step>

  <Step title="Open the WebSocket">
    Connect a standard WebSocket client to `websocket_address`:

    ```javascript theme={null}
    const socket = new WebSocket(websocketAddress);
    ```

    No additional authentication header is required. All messages are JSON text frames, and only one WebSocket client may be connected to a session at a time.
  </Step>

  <Step title="Stream an audio turn">
    Send mono PCM16 audio as base64-encoded chunks. We recommend chunks containing approximately 80–100 ms of audio. You can send them as quickly as they become available. They do not need to be paced in real time.

    <Tip>
      LemonSlice processes audio at 16 kHz. Send audio at that rate to avoid overhead associated with resampling.
    </Tip>

    ```json theme={null}
    {
      "command": "audio",
      "audio": "<base64_pcm16_mono>",
      "sampleRate": 16000,
      "encoding": "PCM16"
    }
    ```

    | Field        | Description                                                                                 |
    | ------------ | ------------------------------------------------------------------------------------------- |
    | `audio`      | Base64-encoded mono PCM16 audio.                                                            |
    | `sampleRate` | Sample rate of the audio in Hz. Required. Other rates are accepted and resampled to 16 kHz. |
    | `encoding`   | Must be `"PCM16"` when provided. Defaults to `"PCM16"` if omitted.                          |

    End every response with `audio_end`:

    ```json theme={null}
    { "command": "audio_end" }
    ```

    A complete turn is `audio` → `audio` → … → `audio_end`. The final command commits the turn so LemonSlice can finish rendering it. Without `audio_end`, trailing audio may be dropped and your avatar will freeze between turns.
  </Step>

  <Step title="Subscribe to the avatar">
    Join the LiveKit or Daily room from your client using its viewer credentials. LemonSlice publishes the generated avatar audio and video into this room.
  </Step>
</Steps>

## Handle playback completion

LemonSlice sends a `playback_finished` event over the WebSocket when avatar A/V playback completes:

```json theme={null}
{
  "command": "playback_finished",
  "playback_position": 4.2,
  "interrupted": false
}
```

Use this event as the source of truth for whether the avatar is still playing:

* After sending audio for a response, treat playback as active until you receive its `playback_finished` event.
* If the user interrupts before that event arrives, send an `interrupt` command even if your TTS provider has finished sending audio or the estimated playback duration has elapsed.
* Once `playback_finished` arrives, the response is complete and an `interrupt` for that response is unnecessary.

This avoids a desynchronization window between your TTS pipeline and the avatar's actual A/V playback.

## Interrupt playback

To stop the current avatar response, send an `interrupt` event over the websocket and drop any queued audio chunks that had not yet been sent over the websocket:

```json theme={null}
{ "command": "interrupt" }
```

## Heartbeats

Use heartbeats to confirm that the WebSocket is responsive:

```json theme={null}
{
  "command": "heartbeat",
  "event_id": "optional-id",
  "timestamp": 1710000000000
}
```

LemonSlice replies with `heartbeat_ack`, echoing `event_id` and `timestamp` when present.

## End the session

When the call is complete, shut down the LemonSlice session by sending a `terminate` event, then close the WebSocket:

```json theme={null}
{ "command": "terminate" }
```

<Warning>
  Always terminate sessions explicitly. Otherwise, the session and billing remain active until its idle timeout or maximum duration is reached.
</Warning>

## WebSocket event summary

| Client → LemonSlice | Description                                                                                      |
| :------------------ | :----------------------------------------------------------------------------------------------- |
| `audio`             | Base64-encoded mono PCM16 audio chunk. Include `sampleRate` and optional `encoding` (`"PCM16"`). |
| `audio_end`         | Commits the current turn after the last `audio` chunk.                                           |
| `interrupt`         | Stops the current avatar response mid-playback.                                                  |
| `heartbeat`         | Liveness check. Optional `event_id` and `timestamp` are echoed in the ack.                       |
| `terminate`         | Ends the LemonSlice session. Close the WebSocket afterward.                                      |

| LemonSlice → client | Description                                                                                                      |
| :------------------ | :--------------------------------------------------------------------------------------------------------------- |
| `playback_finished` | Avatar A/V playback for a response has completed. Includes `playback_position` (float) and `interrupted` (bool). |
| `heartbeat_ack`     | Reply to `heartbeat`, echoing `event_id` and `timestamp` when present.                                           |

## Starter projects

| Repo                                                                                                 | What's included                                                                                                             |
| :--------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------- |
| [`10-websocket-app`](https://github.com/lemonsliceai/lemonslice-examples/tree/main/10-websocket-app) | Fullstack web app that connects an ElevenLabs voice agent to LemonSlice via WebSocket and receives avatar A/V over LiveKit. |

## Additional resources

Use the [session control endpoint](/docs/api-reference/control-session) for mid-call triggers such as real-time avatar image updates.