Tencent Launches HunyuanPortrait, an Open-Source AI Model for Animating Portraits

Tencent launched a brand new synthetic intelligence (AI) mannequin on Tuesday that may animate nonetheless portrait photos. Dubbed HunyuanPortrait, the big language mannequin (LLM) is predicated on the diffusion structure, and may generate movies with lifelike animation based mostly on a reference picture and a guiding video. The researchers behind the mission highlighted that the mannequin can seize each facial information and spatial actions to precisely sync them into the reference picture. Tencent has now open-sourced the HunyuanPortrait AI mannequin, and it may be downloaded and run domestically from common repositories.

Tencent’s HunyuanPortrait Can Bring Still Portraits to Life

In a submit on X (previously often known as Twitter), the official deal with of Tencent Hunyuan introduced that the HunyuanPortrait mannequin is now accessible to the open group. The AI mannequin could be downloaded from Tencent’s GitHub and Hugging Face listings. Additionally, a pre-print paper detailing the mannequin can be being hosted on arXiv. Notably, the AI mannequin is out there for tutorial and research-based use circumstances, however not for business utilization.

HunyuanPortrait can generate lifelike animated movies utilizing a reference picture and driving movies. It captures the facial information and head poses from the video and interpolates them onto the nonetheless portrait picture. The firm claims that the sync of the motion is correct, and even the refined facial features adjustments are replicated.

HunyuanPortrait structure
Photo Credit: Tencent

On its mannequin web page, Tencent researchers detailed the structure of HunyuanPortrait. It is constructed on the structure of Stable Diffusion fashions alongside a situation management encoder. These pre-trained encoders decouple movement data and identification in movies. The information is captured as management indicators, that are then injected into the nonetheless portrait through a denoising unet. The firm claims this brings each spatial accuracy in addition to temporal consistency into the output.

Tencent claims that the AI mannequin outperforms current open-source options on the parameters of temporal consistency and controllability, however these metrics haven’t been independently verified.

Such fashions could be helpful within the filmmaking and animation industries. Traditionally, animators manually keyframe facial expressions or use costly movement seize programs to animate the characters realistically. Models like HunyuanPortrait will permit them to simply feed the character design and the goal actions and facial expressions, and will probably be capable of generate the output. Such LLMs even have the potential to make high-quality animation accessible to smaller studios and unbiased creators.