The pace of technological development has increased over the past year, especially in AI. And in 2024, there was no better place to be a part of creating that outcome than NVIDIA Research.
NVIDIA Research is made up of hundreds of very bright people who are pushing the boundaries of knowledge, not only in AI, but in many areas of technology.
Last year, NVIDIA Research laid the groundwork for future improvements in GPU performance with major research findings in circuits, memory architectures and limited calculations. The group’s production of new imaging techniques continues to raise the bar for real-time rendering. And we’ve developed new ways to improve AI performance – requiring less power, taking fewer GPU cycles and delivering even better results.
But the most exciting development of the year was in artificial intelligence.
We are now able to produce, not just images and text, but 3D models, music and sounds. We are also developing better control over the output: generating realistic humanoid motion and generating image sequences with coherent content.
The use of artificial intelligence (AI) has resulted in more accurate weather forecasts than conventional weather models. AI models have given us the ability to accurately predict how blood sugar levels will respond to different foods. Emodied generative AI is used to develop autonomous vehicles and robots.
And it was just this year. What follows is a dive into some of NVIDIA’s biggest AI Research work in 2024. Of course, we continue to develop new AI models and techniques, and we expect exciting results and than next year.
ConsiStory: AI-Generated Images with Main Character Power
ConsiStory, a collaboration between NVIDIA researchers and Tel Aviv University, makes it possible to generate multiple images with a fixed main character – an important capability for story use cases such as creating images of humor or story telling.
The researchers’ approach has developed a technique called subject-divided attention, which reduces the time it takes to produce the same images from 13 minutes to 30 seconds.
Read the ConsiStory paper.
Edit 3D: Generative AI Enters a New Level
NVIDIA Edify 3D is a basic model that helps designers and developers quickly create 3D objects that can be used to express ideas and populate virtual worlds.
Edify 3D enables designers to quickly imagine, design and visualize deep environments with AI-powered assets. Both novice and experienced content creators can use text and image enhancements using the model, which is currently part of the NVIDIA Edify multimodal framework for developing visual AI.
Read the Edify 3D paper and watch the video on YouTube.
Fugatto: Flexible AI Sound Machine for Music, Lyrics and More
A team of NVIDIA researchers recently unveiled Fugatto, an AI-based model that can generate or transform any combination of music, words and sounds based on text or audio messages.
For example, you can make music beats based on text additions, add or remove instruments to existing songs, change the sound or feel of a voice recording, or create entirely new sounds. It can be used by music producers, advertising agencies, video game developers or language learning software developers.
Read Fugatto’s paper.
GluFormer: AI predicts blood sugar levels after four years
Researchers at the Weizmann Institute of Science, Tel Aviv-based Pheno.AI and NVIDIA led the development of GluFormer, an AI model that can predict a person’s future glucose levels and other health metrics. beauty based on previous blood sugar test data.
Researchers have shown that, after incorporating dietary information into this model, GluFormer can also predict how a person’s blood sugar levels will respond to certain foods and dietary changes, which is enabling proper nutrition. The research team validated GluFormer on 15 other databases and found that it generalized well to predict health outcomes for other groups, including those with type 1 and type 2 diabetes. 2, gestational diabetes and obesity.
Read the GluFormer paper.
LATTE3D: Enables Next-Generation, From Text to 3D Shape
Another 3D generator released by NVIDIA Research this year is LATTE3D, which converts text messages into 3D images in seconds – like a fast 3D printer. Created in a well-known format used for common translation applications, the generated formats can be easily used in virtual environments for developing video games, advertising campaigns, design projects or robot training sites.
Read the LATTE3D paper.
MaskedMimic: Reproducing Realistic Motion for Humanoid Robots
To advance the development of humanoid robots, NVIDIA researchers have developed MaskedMimic, an AI framework that works in painting – the process of reproducing detailed information from an incomplete, or closed view – to descriptions of movement.
Given incomplete information, such as a text description of movement, or head and hand position data from a virtual headset, MaskedMimic can fill in the gaps to provide full body movement. It has become part of NVIDIA Project GR00T, a research project to accelerate the development of humanoid robots.
Read the MaskedMimic paper.
StormCast: Enhancing Weather Prediction, Weather Simulation
In the field of weather technology, NVIDIA Research has announced StormCast, an AI model that generates atmospheric energy. While other machine learning models trained on global data have a spatial resolution of about 30 kilometers and a temporal resolution of six hours, StormCast achieves a 3-kilometer, scale of an hour.
Researchers trained StormCast on nearly three and a half years of NOAA weather data from the central U.S. When used with rain radars, StormCast provides forecasts with six-hour lead times which are up to 10% more accurate than the US. The National Oceanic and Atmospheric Administration’s 3-km regional climate forecast system.
Read the StormCast paper, written in collaboration with researchers at Lawrence Berkeley National Laboratory and the University of Washington.
NVIDIA Research Sets Records in AI, Autonomous Cars, Robotics
In 2024, models launched at NVIDIA Research set records across benchmarks for AI training, road optimization, autonomous driving and more.
NVIDIA cuOpt, an optimization AI microservice used for hardware improvements, has 23 world-record benchmarks. The NVIDIA Blackwell platform has demonstrated world-class performance in MPerf industry benchmarks for AI training and exploration.
In the field of autonomous vehicles, Hydra-MDP, the ultimate autonomous driving design by NVIDIA Research, has achieved first place in the End-To-End Driving at Scale track of the Autonomous Grand Challenge at CVPR 2024.
In robotics, FoundationPose, a unified foundational model for 6D object estimation and tracking, received first place on the BOP leaderboard for model-based pose estimation of unseen objects.
Learn more about NVIDIA Researchwhich has hundreds of scientists and engineers worldwide. NVIDIA Research teams focus on topics including AI, computer graphics, computer vision, self-driving cars and robotics.
#Research #Dissemination #Bringing #Advances #Simulation #Climate #Science #Sound #Engineering
