Imagine text coming to life on the screen and making dynamic what is normally a static communication. Animate text in such a way as to obtain images that are not only visually appealing but also semantically significant, is a formidable challenge. Using advanced techniques vector processinga team of academic researchers from the Hong Kong University of Science and Technology and Tel Aviv University is now proposing a truly pioneering approach.
How to animate text using advanced vector graphics processing
The dynamic typography it is an evolved scheme that merges two intricate tasks: on the one hand “deforming” the letters in order to transmit a semantic meaning and on the other hand develop movements based on the prompt provided by the user.
Using advanced techniques in vector graphic representations, the newly proposed method ensures that each animation is not only visually appealing but also semantically coherent. In another article we saw what it means to vectorize an image.
The process, described in great detail in the text Dynamic Typography: Bringing Text to Life via Video Diffusion Priorstarts with initializing the input letter using a set of Bézier curves connected cubics. The latter are a form of parametric curve commonly used in computational graphics and design. To better understand what they are, let’s look at some key concepts:
A cubic Bézier curve is defined by four control points: two extreme points and two intermediate points. This curve begins at the first control point, ends at the fourth control point, and is influenced by the two intermediate points, which determine the shape and orientation of the curve. We talk about “connected cubic” curves because the end point of one curve is the starting point of the next curve, creating a continuous path.
The method used therefore involves the application of various movements for each control point positioned on each frame. It is thus possible to effectively deform the letter to convey semantic meaning. The entire process is optimized using the technique video diffusion prior: It ensures that the final output faithfully interprets the user’s prompts while maintaining readability.
Test generating animated text using the technique locally video diffusion prior
The beauty of the research by academics from Hong Kong and Tel Aviv is that it is not a theoretical investigation. Interested parties can refer to the code and instructions published on GitHub to create their own texts that come to life and become dynamic.
All the demonstration images you find on this page were generated using a single card NVIDIA H800 with 80 GB of VRAM on board. To generate atext animation composed of 20 frames or more, however, you must use a GPU with at least 24 GB of VRAM.
Once these requirements are met, certainly not by everyone, it is possible clone the repository GitHub of the project on a Linux system then use the command conda env to set the working environment:
git clone https://github.com/zliucz/animate-your-word.git
cd animate-your-word
conda env create -f environment.yml
The command conda env is part of Conda, an environment and package management system for programming languages Python e R. In particular, conda env manages virtual environments, isolated workspaces that allow developers to work on projects with specific dependencies without affecting the system environment or other projects.
At this point, the following sample code can be used to generate a animated version of the letter “H” within the word father.
CUDA_VISIBLE_DEVICES=0 python dynamicTypography.py \
--word "father" --optimized_letter "h" \
--caption "A tall father walks along the road, holding his little son with his hand" \
--use_xformer --canonical --anneal \
--use_perceptual_loss --use_conformal_loss \
--use_transition_loss
Also at this address you will find some examples of transformations performed using the technique video diffusion prior.
In the opening image, an NVIDIA H800 card (source: NVIDIA).