Kling Lip Sync - AI Video Tools Tool

Overview

Kling Lip Sync is a lip-synchronization API hosted on Replicate that modifies a person's mouth movements in an existing video to match supplied audio or text-driven audio. According to the Replicate model page (https://replicate.com/kwaivgi/kling-lip-sync), the model takes a source video and either an audio file or text (for TTS-driven workflows) and generates a new video with adjusted lip motion to align with the provided audio. The model is intended for tasks like dubbing, content localization, marketing clips, and creating matched mouth movements for synthetic voices. The model routes processing from Replicate to Kuaishou for inference and billing, and Replicate's page notes that pricing is calculated per second of generated video. Public details such as exact per-second rates, supported maximum resolution, latency, and batch limits are not documented on the model page. Users should be aware this workflow sends video/audio data to a third-party provider (Kuaishou) for generation — check the model page and your privacy/compliance requirements before use. For up-to-date parameter lists, input formats, and any recent updates, consult the model's Replicate page linked above.

Key Features

• Re-synchronizes lip movements in an existing video to match provided audio or text input
• Accepts either separate audio files or text (for TTS-driven lip-sync workflows)
• Hosted on Replicate with inference routed to Kuaishou for generation and billing
• Intended for dubbing, localization, content marketing, and synthetic-voice alignment
• Per-second billing model (pricing details not published on the model page)

Example Usage

Example (python):

import os
import replicate

# Requires REPLICATE_API_TOKEN in environment
client = replicate.Client(api_token=os.environ.get("REPLICATE_API_TOKEN"))
model = client.models.get("kwaivgi/kling-lip-sync")

# Example inputs - actual parameter names may vary on the model page
with open("input_video.mp4", "rb") as video, open("input_audio.wav", "rb") as audio:
    output_urls = model.predict(video=video, audio=audio)

print("Model output:", output_urls)

# NOTE: This is a minimal illustrative example. Check the model page for exact input names,
# accepted file formats, and synchronous vs. asynchronous invocation patterns.

Key Information

• Category: Video Tools
• Type: AI Video Tools Tool