AI-based video generation has made great strides in recent years, but one of the most persistent obstacles has been maintaining the consistency of character identity across different scenes and actions. Many AI models, while excelling at creating visual content, often fail to preserve a character’s consistent appearance and behaviour, resulting in unrealistic and fragmented videos.
Tencent directly addresses this challenge with the release of HunyuanCustom, an open source AI model designed to generate videos with consistent characters using multimodal inputs such as text, images, audio and video. Based on the HunyuanVideo framework, HunyuanCustom introduces an advanced architecture that integrates image-text fusion modules and mode-specific injection mechanisms, ensuring that the identity of the subject remains constant throughout the video.
One of the distinguishing features of HunyuanCustom is its ability to handle multiple inputs to create realistic and consistent video. For example, a single image of a character, a script and an audio clip can be provided, and the template will generate a video in which the character speaks and moves naturally, while maintaining a consistent appearance and voice. This capability is particularly useful for applications such as virtual advertisements, singing avatars and customised videos.
HunyuanCustom also supports more complex scenarios with multiple characters. Using reference images for each subject and a textual description of the scene, the model can generate videos in which multiple characters interact with each other while maintaining the consistency of each one’s identity. This functionality opens up new possibilities for the creation of narrative and interactive content, where character consistency is crucial.
From a technical point of view, HunyuanCustom uses an LLaVA-based image-text fusion module to facilitate the interaction between images and text, enabling the effective integration of identity information into text descriptions. In addition, an identity enhancement module concatenates the image information along the temporal axis, exploiting the video model’s ability to temporally model to enhance the subject’s identity during the video.
How to test HunyuanCustom:
For those interested in experimenting with HunyuanCustom, the process is relatively simple:
-
System requirements: a machine with at least 16 GB RAM and an Nvidia GPU is recommended for optimal performance.
-
Clone the repository : access the official repository on GitHub: https://github.com/Tencent/HunyuanCustom and clone it in your local environment.
-
Install dependencies : use the command
pip install -r requirements.txtto install all the necessary dependencies. -
Load the model: download the pre-trained weights available on Hugging Face and load them into your environment.
-
Perform inference: use the scripts provided to generate videos based on text, image, audio or video input.
For those who prefer a simpler approach, you can use cloud platforms such as Google Colab or Paperspace, where pre-configured notebooks are available to run HunyuanCustom without the need for complex configurations.
The release of HunyuanCustom represents a significant step towards the democratisation of AI-based video generation. By offering an open source model with advanced identity maintenance capabilities, Tencent opens up new possibilities for content creators, developers and businesses around the world. In an era where personalisation and consistency are key, tools such as HunyuanCustom could become indispensable for the production of high-quality visual content.
