⚠️The LipDub platform has its strengths, but can also be challenging if you don’t know all its in’s and outs. Here’s a quick list of common user errors that can cause delayed or even poor results with Single Actor Projects.
✅Single Actor is essentially Multi-Actor Workflow but simplified. It combines all the advanced workflow steps into one, making it much easier to use and get a result. This ease of use & automation however may lead to un-intentional errors that cause the output to be poor. See below on the different scenarios where EasyDub may fail.
Scenario 1: Output is not lip-synced for the entire video
- What is happening? - the face in the background is actually being lip-synced, and not the main actress. This is a known face detection issue that can sometimes happen with Single Actor project type. LipDub face detection is highly sensitive, so its always best to ensure only 1 face is viewable to prevent this from happening.
- Solution -
- Option 1 - Please restart this project using Multi-actor Workflow. This advanced workflow gives users ability to select face detection tracks of which face you'd like to lip-sync, and which ones you do not want to lip-sync.
- Option 2 - If you really want to use Single Actor Project type you can! But please be sure to upload a different video where only 1 face is visible throughout.
Scenario 2: Output is only partially lip-synced
- What is happening? - There is a cut in the video which makes LipDub think there is 2 unique "identities". Single Actor Project Type only dubs one identity. So it only dubs one of the cuts.
- Solution - Please restart this project using Multi-actor Workflow.
Scenario 3: Output is not lip-syncing well
- What is happening? - the original source video does not have mouth movements. Therefore, the AI model is only trained on when the actor has closed mouth. This will yield to poor and unnatural lip-sync. LipDub requires some mouth movement of the actor before it trains an AI model.
- Solution:
- Option 1 - Create a new Single Actor project. This time upload a video that includes mouth movement for the actor.
- Option 2 - Create a new Multi-actor Workflow project. This time upload the closed mouth video in the Lipdub clips section & upload additional footage for training where the same actor in same lighting & environment is moving their mouth/talking.