Running a model
sync.Video to videosync-3
Full-scene lip synchronization with global face understanding and obstruction handling
One of Auray's catalogue models: the same key, the same credits, the same job lifecycle as every other address.
Its request shape is published below. Whether it runs right now is GET /v1/catalogue's word, with no key needed: its addresses say connected for an address that runs, and a submit to any other is refused before anything is charged — described answers 501 model_not_connected, paused 503 model_disabled.
Category video-to-video · Scope video:write (or external:write on an older key) · Price Credits per call: 131.595. Plus 940.975 credits when the request sends inputs.video or inputs.audio (counted as any:inputs.video+inputs.audio). Plus 940.975 credits when the request sends settings.segments.[].audio (counted as any:settings.segments.[].audio). An MP4 or MOV clip in inputs.video that you uploaded through POST /v1/uploads, that one of your jobs made on a catalogue model or an auray-ai/ model, or that is a file of a Cinema Studio project you can open, is held for its own length instead, where its length can be read from the file: 15.682917 credits for each second it runs, rounded up to the next whole second, up to 60 seconds. Any other clip — a WebM, a file whose length cannot be read, a clip sent by any other link, or one that runs past 60 seconds — is refused before any charge. An MP4 or MOV clip in inputs.video that you uploaded through POST /v1/uploads, that one of your jobs made on a catalogue model or an auray-ai/ model, or that is a file of a Cinema Studio project you can open, is held for its own length instead, where its length can be read from the file: 15.682917 credits for each second it runs, rounded up to the next whole second, up to 60 seconds. Any other clip — a WebM, a file whose length cannot be read, a clip sent by any other link, or one that runs past 60 seconds — is refused before any charge. Quoted when you submit, at the most the request can cost; the charge settles to what the job cost and never above the quote. The submit's answer carries the quote as credits_quoted, and credits_charged says what has been taken.
#What it can do
lip-sync— Lip-sync: 1 source file; 1 track- Sound: makes its own sound
- Priced by:
call(the whole rule is under Price)
Find the addresses that do any of these with GET /v1/models?operation=<id>; the openapi.json carries them as data under x-auray-capability.
#Input
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
inputs | object | yes | — | |
inputs.video | string | yes | — | Video input (an http(s) URL). When the request also sends inputs.audio, or settings.segments[].audio, this model makes the person in a clip here speak or move: a clip you upload, or a link from anywhere else, is taken once your account has accepted, one time, that you have the right to the faces and voices you use — send acceptance: {"real_people": "2026-11-03"} with the request, or accept in the Auray app — and is refused 403 acceptance_required until then, before anything is charged. A clip made here — a job's result — asks nothing. While uploads are paused here, a clip you upload, or a link from anywhere else, is refused 403 likeness_not_made_here instead, before anything is charged. The files the examples here send are ours, and are taken. A URL. |
inputs.audio | string | no | — | Audio input (an http(s) URL; upload a local file first and send its read URL). A URL. |
settings | object | no | — | |
settings.temperature | number | no | 0.5 | Expressiveness of lip sync and facial movements. Documented range: 0 to 1. |
settings.activeSpeakerDetection | object | no | — | |
settings.activeSpeakerDetection.autoDetect | boolean | no | — | Automatically detect and target the active speaker. |
settings.activeSpeakerDetection.boundingBoxes | number[][] | no | — | Per-frame face bounding boxes across the clip. Each box is [x1, y1, x2, y2]. |
settings.activeSpeakerDetection.coordinates | number[] | no | — | [x, y] point on the target speaker's face in the selected frame. |
settings.activeSpeakerDetection.frameNumber | integer | no | — | Frame index corresponding to the provided face coordinates. Documented range: 0 or more. |
settings.emotion | string | no | — | Emotional tone for performance re-animation. One of "happy", "sad", "angry", "disgusted", "surprised", "neutral". |
settings.mode | string | no | — | Controls the region of facial animation applied during lip sync. One of "lips", "face", "head". |
settings.occlusionDetection | boolean | no | — | Enable occlusion handling for obstructed faces. |
settings.segments | object[] | no | — | |
settings.segments[].audio | string | no | — | Audio input (an http(s) URL; upload a local file first and send its read URL). A URL. |
settings.segments[].audioEndTime | number | no | — | End time in seconds within the source audio file. |
settings.segments[].audioStartTime | number | no | 0 | Start time in seconds within the source audio file. Documented range: 0 or more. |
settings.segments[].endTime | number | no | — | End time in seconds for the segment. Must be greater than startTime and within the source video's duration. |
settings.segments[].startTime | number | no | — | Start time in seconds for the segment. Documented range: 0 or more. |
settings.syncMode | string | no | "cut_off" | Synchronization strategy when audio and video durations don't match. One of "bounce", "cut_off", "silence", "remap". |
outputType | string | no | "URL" | Video output type. One of "URL". |
outputFormat | string | no | "MP4" | Specifies the file format of the generated output. The available values depend on the task type and the specific model's capabilities. - `JPG`: Best for photorealistic images with smaller file sizes (no transparency). - `PNG`: Lossless compression, supports high quality and transparency (alpha channel). - `WEBP`: Modern format providing superior compression and transparency support. - `MP4`: Widely supported video container (H.264), recommended for general use. - `WEBM`: Optimized for web delivery. - `MOV`: QuickTime format, common in professional workflows (Apple ecosystem). - `GIF`: Animated image format (no audio), suitable for short loops or previews. - `MP3`: Compressed audio, smaller file size. - `WAV`: Uncompressed, high-quality audio. - `FLAC`: Lossless compression. - `OGG`: Open-source compressed audio format (Vorbis codec). - `SVG`: Scalable Vector Graphics. - `TIFF`: High-quality output supporting layers. - `GLB`: Binary glTF format for 3D models, including geometry, textures, and PBR materials. > [!NOTE] > **Transparency**: If you are using features like background removal or LayerDiffuse that require transparency, you must select a format that supports an alpha channel (e.g., `PNG`, `WEBP`, `TIFF`). `JPG` does not support transparency. One of "MP4", "WEBM", "MOV". |
outputQuality | integer | no | 95 | Compression quality of the output. Higher values preserve quality but increase file size. Documented range: 20 to 99. |
safety | object | no | — | |
safety.checkContent | boolean | no | — | Enable or disable content safety checking. Increases total generation time. Always on: a false is sent as true. |
safety.mode | string | no | "fast" | Safety checking mode for video generation. Only fast or full is sent here: any other value is sent as fast. One of "fast", "full". |
idempotency_key | string | no | — | Your own name for this request. Send the same one twice and the second call is answered with the first request rather than charged again. Idempotency-Key: as a header is accepted as an alias — that is the spelling a retrying proxy replays, and the one to use when running a model on auray.run, where a request that outlives its connection is answered 504 and recovered by replay. Omit it and the platform mints one. 1 to 64 characters. |
acceptance | object | no | — | Your account's one acceptance, sent with the request that first needs it: a file of a real person's face or voice you upload, in a field this model makes speak or move, copies the voice of or puts into a clip, is taken once your account has accepted, one time, that you have the right to the faces and voices you use. Kept for your account before anything is charged, so no later request asks again — in any app or through the API — and asked of no file made here. You can accept in the Auray app (Account → Privacy) instead; an assistant cannot accept for you, so through MCP it is accepted in the app. |
acceptance.real_people | string | no | — | The version of the words you accept: that you have the right to every face and voice you upload, and to any voice or style you pick that sounds like a real person — it is yours, or you have permission — and that you are responsible for what you make with them. One of "2026-11-03". |
#Rules
Across fields, as the model's documentation states them. Each is judged before anything is charged — warned with a Deprecation header until enforcement starts — and listed as JSON Schema under x-auray-rules in the openapi.json.
- Do not send
settings.activeSpeakerDetection.boundingBoxestogether withsettings.activeSpeakerDetection.coordinates. - Send exactly one of
inputs.audioorsettings.segments. - Send
settings.activeSpeakerDetection.frameNumberwheneversettings.activeSpeakerDetection.coordinatesis sent. Sendsettings.activeSpeakerDetection.coordinateswheneversettings.activeSpeakerDetection.frameNumberis sent.settings.segmentsmust hold at least 1 item. In every item ofsettings.segments:settings.segments[].startTime,settings.segments[].endTimeandsettings.segments[].audioare required.
#Example
curl -X POST https://queue.auray.run/auray-serverless/sync/3-0 \
-H "Authorization: Bearer $AURAY_API_KEY" \
-H "content-type: application/json" \
-d '{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}'import { createClient } from "@auray/client";
const auray = createClient(); // reads AURAY_API_KEY
const result = await auray.subscribe("auray-serverless/sync/3-0", {
input: {
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
},
onQueueUpdate: (update) => console.log(update.status),
});
console.dir(result, { depth: null });from auray_client import Client
auray = Client() # reads AURAY_API_KEY
result = auray.subscribe(
"auray-serverless/sync/3-0",
input={
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
},
on_queue_update=lambda update: print(update["status"]),
)
print(result){
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "run_model",
"arguments": {
"address": "auray-serverless/sync/3-0",
"input": {
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}
}
}
}const res = await fetch("https://queue.auray.run/auray-serverless/sync/3-0", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.AURAY_API_KEY}`,
"content-type": "application/json",
},
body: JSON.stringify({
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}),
});
console.log(await res.json());import os, requests
r = requests.post(
"https://queue.auray.run/auray-serverless/sync/3-0",
headers={"Authorization": f"Bearer {os.environ['AURAY_API_KEY']}"},
json={
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
},
)
r.raise_for_status()
print(r.json())package main
import (
"bytes"
"fmt"
"io"
"net/http"
"os"
)
func main() {
body := []byte(`{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}`)
req, err := http.NewRequest("POST", "https://queue.auray.run/auray-serverless/sync/3-0", bytes.NewReader(body))
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("AURAY_API_KEY"))
req.Header.Set("Content-Type", "application/json")
res, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
out, _ := io.ReadAll(res.Body)
fmt.Println(res.Status, string(out))
}<?php
$ch = curl_init("https://queue.auray.run/auray-serverless/sync/3-0");
curl_setopt_array($ch, [
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
"Authorization: Bearer " . getenv("AURAY_API_KEY"),
"Content-Type: application/json",
],
CURLOPT_POSTFIELDS => json_encode([
"inputs" => [
"video" => "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio" => "https://v3b.auray.media/examples/samples/electronic-loop.mp3",
],
], JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE),
]);
$response = curl_exec($ch);
echo curl_getinfo($ch, CURLINFO_HTTP_CODE), " ", $response, PHP_EOL;require "net/http"
require "json"
uri = URI("https://queue.auray.run/auray-serverless/sync/3-0")
req = Net::HTTP::Post.new(uri)
req["Authorization"] = "Bearer #{ENV.fetch("AURAY_API_KEY")}"
req["Content-Type"] = "application/json"
req.body = {
"inputs" => {
"video" => "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio" => "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}.to_json
res = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) { |http| http.request(req) }
puts res.code, res.bodyimport java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
public class Main {
public static void main(String[] args) throws Exception {
String body = """
{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}""";
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://queue.auray.run/auray-serverless/sync/3-0"))
.header("Authorization", "Bearer " + System.getenv("AURAY_API_KEY"))
.header("Content-Type", "application/json")
.method("POST", HttpRequest.BodyPublishers.ofString(body))
.build();
HttpResponse<String> response = HttpClient.newHttpClient()
.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.statusCode() + " " + response.body());
}
}using System.Net.Http.Headers;
using System.Text;
using var client = new HttpClient();
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", Environment.GetEnvironmentVariable("AURAY_API_KEY"));
using var request = new HttpRequestMessage(HttpMethod.Post, "https://queue.auray.run/auray-serverless/sync/3-0");
var body = """
{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}
""";
request.Content = new StringContent(body, Encoding.UTF8, "application/json");
using var response = await client.SendAsync(request);
Console.WriteLine($"{(int)response.StatusCode} {await response.Content.ReadAsStringAsync()}");The same body on https://auray.run runs synchronously and answers the result — unless it usually takes longer than a run may hold its connection, which is refused 400 model_not_synchronous before anything is charged. Options — ?webhook=, ?wait=, priority and timeout headers — are on Submit a request.
#Machine contracts
- openapi.json — this page as an OpenAPI 3.1 document, public.
- llms.txt — the same, written for an agent.