NVIDIA, a global leader in GPU and AI technology, has unveiled a series of groundbreaking advancements in generative AI research. These innovations promise to revolutionize the way developers and artists bring their ideas to life, offering new possibilities in synthetic data generation for virtual worlds and enabling the production of high-quality visuals with unprecedented efficiency.
The generative AI landscape has been experiencing rapid innovation, with trailblazing tools such as Midjourney, Adobe Firefly, Stable Diffusion, DALL·E 2, and Runway leading the charge in image and video production. NVIDIA's latest research is poised to push the boundaries of what's possible even further.
At SIGGRAPH 2023, one of the premier computer graphics conferences, NVIDIA will showcase around 20 research papers advancing generative AI and neural graphics, in collaboration with over a dozen universities. This cutting-edge research spans generative AI models for text-to-image personalization, inverse rendering tools, neural physics models, and neural rendering models.
The first groundbreaking area of research focuses on customized text-to-image models. NVIDIA researchers and Tel Aviv University have developed two SIGGRAPH papers on streamlining the customization of text-to-image AI models. One paper showcases a technique that requires only a single example image to customize its output, slashing the personalization process from minutes to roughly 11 seconds on a single NVIDIA A100 Tensor Core GPU – a staggering 60x speed increase compared to previous methods.
A second paper introduces Perfusion, an innovative compact model that employs a few concept images to enable users to merge multiple personalized elements, such as specific objects or characters, into a single AI-generated visual.
The second area of research emphasizes generative 3D modeling and rendering. NVIDIA Research is devising AI techniques for converting 2D images and videos into 3D representations, including a real-time 3D avatar creation and video conferencing system, with outputs readily importable into 3D graphics applications for further editing.
For example, in collaboration with researchers at the University of California, San Diego, NVIDIA has developed technology capable of generating and rendering a photorealistic 3D head-and-shoulders model based on a single 2D portrait. This significant breakthrough brings 3D avatar creation and 3D video conferencing within reach, operating in real-time on consumer desktops and utilizing only conventional webcams or smartphone cameras.
Working with Stanford University, NVIDIA researchers have created an AI system that acquires realistic tennis skills from 2D video recordings and applies them to 3D characters. Their system is able to synthesize two physically simulated characters playing extended tennis rallies with simulated racket and ball dynamics.
Another paper delves into simulating intricate details like hair in high resolution and real-time using neural physics. This neural rendering research encompasses neural texture compression, NeuralVDB, and an AI system for representing photorealistic materials in real-time.
SIGGRAPH 2023, taking place from August 6-10 in Los Angeles, will feature NVIDIA presenting six courses, four talks, and two Emerging Technology demos, covering topics such as path tracing, telepresence, and diffusion models for generative AI.
As more companies integrate generative AI tools into their creative workflows, NVIDIA's latest research breakthroughs will undoubtedly serve as catalysts for redefining the creative industry and transforming the future of digital art and design.