Study participants prefer LemonSlice over Anam, Tavus, and HeyGen

The aim of this study was to measure viewer preference between real-time avatar providers under conditions that remove every source of variation except the provider itself. Preference is a perceptual judgment, so we measured it the way perceptual judgments are usually measured: by showing people the stimuli and asking them to choose. Each study participant served as their own control, ranking the providers across the same series of avatars, so the comparison is within-subject and does not depend on differences between people.

Holding the stimulus constant required controlling the inputs at every stage. Each avatar originated from one source photo that was submitted to every provider. When a provider rejected a photo, we edited it only as far as needed to pass its checks, for example by cropping. Each line of dialogue was synthesized in advance and the same audio files were played to every provider. The result is a set of videos that differ only in how each provider animated the same avatar with the same speech.

We chose recorded output over live interaction deliberately. A live session introduces variation from the language model, the voice, and turn-taking that is unrelated to the avatar, and it cannot be replayed for a second provider with the same content. Recording each provider as it rendered in a real-time session preserves the rendering behavior of the live product while making an exact comparison possible. The trade-off is scope: the study characterizes audiovisual quality as perceived by viewers, but does not capture the impact of response latency. An earlier study showed LemonSlice to be as fast or faster than other providers.

The ranking task, the set of avatars, and the scoring procedure were specified before data collection and were not changed afterward. Study participants were recruited through Prolific, an independent research panel, were never shown provider names, and had no relationship to LemonSlice. We report every comparison, including the individual outcomes that make up each aggregate.

8 avatars were evaluated by each study participant. To improve variation and reduce the likelihood of a single video skewing results, multiple clips were generated for each avatar/provider combination. These included multiple transcripts and several takes of each transcript. A given study participant would see only one of these clips, randomly selected. Audio was sent to each provider during an interactive session. We recorded the resulting video track through LiveKit egress at the same resolution, frame rate, and audio sample rate published by the provider. This ensured the videos represent exactly what a user of that provider would see.

Every provider was tested on the latest real-time model it offered when the clips were generated, from a single photo and with the same audio.

Preference scores were first calculated at the participant level. For each head-to-head test, we calculated the percentage of avatars where LemonSlice was preferred over the other provider. For example, if a study participant saw Anam in 7 of 8 sets and ranked LemonSlice higher on 6 of them, they would have a score of 86%. This score was then averaged across all study participants to get an overall preference score. Significance was evaluated using a two-sided one-sample t-test against 50% (the expected score if participants had no preference). LemonSlice was preferred over all other providers, reaching statistical significance in each test (p < 0.001).