Websocket Integration

Overview

The Websocket integration lets you drive a LemonSlice avatar from any audio stack. Send PCM audio and control events to LemonSlice over a WebSocket. Synchronized audio and video is streamed to a LiveKit or Daily room that you control.

Prerequisites

A LemonSlice account and API key associated with an active subscription. Create a key at lemonslice.com/developers.
A reference image for your avatar. See our guide for recommendations on how to design your avatar image to achieve the best quality possible.
A LiveKit or Daily room with separate credentials for:
- LemonSlice to join and publish audio and video.
- Your client to join and subscribe to the avatar stream.

For LiveKit, generate access tokens; the room is created when the first participant joins. For Daily, create a room and create meeting tokens.

How to use

Create a session

Send a request to the LemonSlice sessions endpoint with your avatar and the credentials LemonSlice should use to publish into your room.

curl --request POST \
  --url "https://lemonslice.com/api/liveai/sessions" \
  --header "X-API-Key: $LEMONSLICE_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "transport_type": "websocket-livekit",
    "agent_image_url": "https://example.com/avatar.jpg",
    "livekit_properties": {
      "livekit_url": "wss://your-project.livekit.cloud",
      "livekit_token": "<agent_access_token>"
    }
  }'

curl --request POST \
  --url "https://lemonslice.com/api/liveai/sessions" \
  --header "X-API-Key: $LEMONSLICE_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "transport_type": "websocket-daily",
    "agent_image_url": "https://example.com/avatar.jpg",
    "daily_properties": {
      "daily_url": "https://your-domain.daily.co/room-name",
      "daily_token": "<agent_meeting_token>"
    }
  }'

You can also provide agent_image_base64 for a local image or agent_id to use an avatar created in the LemonSlice web app instead of agent_image_url.A successful response includes the WebSocket address:

{
  "session_id": "...",
  "websocket_address": "wss://...",
  "control_url": "..."
}

Open the WebSocket

Connect a standard WebSocket client to websocket_address:

const socket = new WebSocket(websocketAddress);

No additional authentication header is required. All messages are JSON text frames, and only one WebSocket client may be connected to a session at a time.

Stream an audio turn

Send mono PCM16 audio as base64-encoded chunks. We recommend chunks containing approximately 80–100 ms of audio. You can send them as quickly as they become available. They do not need to be paced in real time.

LemonSlice processes audio at 16 kHz. Send audio at that rate to avoid overhead associated with resampling.

{
  "command": "audio",
  "audio": "<base64_pcm16_mono>",
  "sampleRate": 16000,
  "encoding": "PCM16"
}

Field	Description
`audio`	Base64-encoded mono PCM16 audio.
`sampleRate`	Sample rate of the audio in Hz. Required. Other rates are accepted and resampled to 16 kHz.
`encoding`	Must be `"PCM16"` when provided. Defaults to `"PCM16"` if omitted.

End every response with audio_end:

{ "command": "audio_end" }

A complete turn is audio → audio → … → audio_end. The final command commits the turn so LemonSlice can finish rendering it. Without audio_end, trailing audio may be dropped and your avatar will freeze between turns.

Subscribe to the avatar

Join the LiveKit or Daily room from your client using its viewer credentials. LemonSlice publishes the generated avatar audio and video into this room.

Handle playback completion

LemonSlice sends a playback_finished event over the WebSocket when avatar A/V playback completes:

{
  "command": "playback_finished",
  "playback_position": 4.2,
  "interrupted": false
}

Use this event as the source of truth for whether the avatar is still playing:

After sending audio for a response, treat playback as active until you receive its playback_finished event.
If the user interrupts before that event arrives, send an interrupt command even if your TTS provider has finished sending audio or the estimated playback duration has elapsed.
Once playback_finished arrives, the response is complete and an interrupt for that response is unnecessary.

This avoids a desynchronization window between your TTS pipeline and the avatar’s actual A/V playback.

Interrupt playback

To stop the current avatar response, send an interrupt event over the websocket and drop any queued audio chunks that had not yet been sent over the websocket:

{ "command": "interrupt" }

Heartbeats

Use heartbeats to confirm that the WebSocket is responsive:

{
  "command": "heartbeat",
  "event_id": "optional-id",
  "timestamp": 1710000000000
}

LemonSlice replies with heartbeat_ack, echoing event_id and timestamp when present.

End the session

When the call is complete, shut down the LemonSlice session by sending a terminate event, then close the WebSocket:

{ "command": "terminate" }

Always terminate sessions explicitly. Otherwise, the session and billing remain active until its idle timeout or maximum duration is reached.

WebSocket event summary

Client → LemonSlice	Description
`audio`	Base64-encoded mono PCM16 audio chunk. Include `sampleRate` and optional `encoding` (`"PCM16"`).
`audio_end`	Commits the current turn after the last `audio` chunk.
`interrupt`	Stops the current avatar response mid-playback.
`heartbeat`	Liveness check. Optional `event_id` and `timestamp` are echoed in the ack.
`terminate`	Ends the LemonSlice session. Close the WebSocket afterward.

LemonSlice → client	Description
`playback_finished`	Avatar A/V playback for a response has completed. Includes `playback_position` (float) and `interrupted` (bool).
`heartbeat_ack`	Reply to `heartbeat`, echoing `event_id` and `timestamp` when present.

Starter projects

Repo	What’s included
`10-websocket-app`	Fullstack web app that connects an ElevenLabs voice agent to LemonSlice via WebSocket and receives avatar A/V over LiveKit.

Additional resources

Use the session control endpoint for mid-call triggers such as real-time avatar image updates.

Integrations

Prompting guide

Advanced

Endpoints

Websocket Integration

Overview

Prerequisites

How to use

Handle playback completion

Interrupt playback

Heartbeats

End the session

WebSocket event summary

Starter projects

Additional resources

​Overview

​Prerequisites

​How to use

​Handle playback completion

​Interrupt playback

​Heartbeats

​End the session

​WebSocket event summary

​Starter projects

​Additional resources

Overview

Prerequisites

How to use

Handle playback completion

Interrupt playback

Heartbeats

End the session

WebSocket event summary

Starter projects

Additional resources