NVIDIA Audio2Face in Unity

Gawain, from Unity’s The Heretic — 322 blendshapes, not one of them named the ARKit way. Everything moving on his face was generated from the voice track.

Why this exists

NVIDIA Audio2Face produces excellent facial animation from speech, but getting it into Unity has never been straightforward. Unreal has an official LiveLink plugin; Unity does not, and the usual answers involve exporting caches through USD or Maya formats, or writing your own connector to receive streamed blendshape weights.

Audio Face Animator removes that step entirely. The model runs inside your Unity project, and the output lands where you want it — as a Unity AnimationClip you can drop straight into an Animator.

How it works

The plugin analyses a speech recording and drives ARKit-compatible blendshapes on your character, producing synchronised mouth movement and facial expression. Everything is computed locally. CPU is the default in both editions, with no NVIDIA GPU required; Full also offers an optional NVIDIA GPU backend, which needs a separate TensorRT installation. After setup, generation runs offline.

Your character does not have to be named the ARKit way. The 52 shapes the model outputs are mapped onto whatever your rig actually has — one ARKit shape can drive several of your blendshapes, or a bone — so a 322-shape face with its own naming scheme works as well as a stock ARKit head.

What you get

• Animation from audio filesPoint it at a recording and generate a performance — no keyframing pass.
• NVIDIA Audio2FaceThe real AI model, running locally inside Unity.
• Offline generationAudio stays on your machine; no cloud service or per-request fees.
• ARKit blendshapesWorks with the standard ARKit facial rig your character already uses.
• Saves AnimationClipsOutput is a normal Unity clip — reuse, edit and blend it like any other.
• Editor and runtimeBoth editions generate in the Editor; Full also generates in Windows x64 Players.
• EmotionsPlay a line angrily, or move through several moods inside one clip, with ten emotions the model knows.
• Eyes and blinkingGaze driven onto your rig's eye bones, an optional look-at target, and involuntary blinking.
• Blendshapes or bonesOne ARKit shape can drive several blendshapes, or a bone — so a rig whose jaw is a bone works too.

First run

Start at Tools > AudioFaceAnimator > Preflight Check. It opens by itself on a fresh project with common platform and model checks. Sync StreamingAssets, then select Checks > CPU for CPU readiness. Full users can separately select Checks > NVIDIA GPU to check their driver, video memory and TensorRT engine.

The AI model ships as ONNX. CPU generation loads it into a shared context; Prewarm can do this ahead of time. No GPU engine is built for CPU generation.

For optional NVIDIA GPU in Full, first install TensorRT 10.16.1.11 for Windows x64 / CUDA 12.9 separately, add its bin folder to PATH, and restart Unity and Unity Hub. The GPU setup guide covers each step. The first GPU generation then builds and caches a TensorRT engine matched to that GPU. In the Editor you can get it out of the way up front via Tools > AudioFaceAnimator > Setup TensorRT Engine, which shows your GPU and a progress bar. The same applies per machine in a shipped build: the first generation call on each player's PC builds and caches an engine, so show your own loading UI while it runs. This is precisely why the model ships as ONNX rather than a prebuilt engine — a prebuilt engine only loads on the GPU architecture it was compiled for.

Requirements

Generation runs on Windows x64. CPU needs no NVIDIA GPU; the GPU requirements below apply only if you explicitly select NVIDIA GPU in Full.

• Windows 10 or 11, 64-bit.
• CPU backend: works without an NVIDIA GPU, CUDA or TensorRT installation.
• Optional NVIDIA GPU: compute capability 7.5 or newer, a suitable driver (576 or newer recommended) and about 6 GB free VRAM.
• TensorRT for GPU only: install version 10.16.1.11, Windows x64 / CUDA 12.9 separately. Downloading it requires an NVIDIA account. CUDA is bundled with Full.
• Unity 6000.0 or newer. The sample UI also needs Input System 1.14.2.
• Imported asset size: about 1.58 GiB Full or 0.78 GiB Lite, plus the synced model copy and Unity's Library. GPU use also needs space for TensorRT and its engine.
• A character model with ARKit blendshapes.

Two editions

Videos

• Animating a complex facial rig — Gawain from Unity’s The Heretic: 322 blendshapes, no ARKit naming, and a whole performance taken from the voice track.
• Lip-sync from one audio clip — the same rig in three quarters of a minute, generated on the machine it plays on.
• Animate a face from audio alone — lips, brows, eyes and blinks generated from the voice track, with nothing keyframed by hand.
• Plugin overview — what the plugin does, what it produces, and how it runs locally.
• Character setup walkthrough — mapping your own character step by step.

All of it is on our YouTube channel.

Documentation

• Full documentation — setup, preparing your character, emotions, eyes and blinking, generating in the Editor and at runtime, the scripting API, and troubleshooting.

← See all 12 DevEloop assets