Nvidia's 'Graphics 3.0' Vision: AI-Powered Physical Productivity

Nvidia is championing a new era it terms “Graphics 3.0,” a vision where AI-generated visuals become fundamental to boosting productivity in the physical world, particularly within factories and warehouses. This concept shifts away from human-created graphics, instead leveraging generative AI (genAI) tools to produce the necessary imagery. Nvidia believes these AI-powered graphics will play a crucial role in various applications, from training robots for real-world tasks to assisting AI systems in automating the design and creation of equipment and structures.

“We believe we are now in Graphics 3.0…being superpowered by AI,” stated Ming-Yu Liu, vice president of research at Nvidia, during a keynote address at SIGGRAPH 2025, a prominent graphics conference held recently in Vancouver, BC. While Nvidia’s powerful GPUs are already widely utilized for text-based generative AI models and virtual assistants, the company envisions Graphics 3.0 extending AI’s influence directly into our physical environment. This includes enabling AI to manage robots, control traffic signals, operate home appliances, guide autonomous vehicles, and oversee equipment in diverse settings like offices, factories, and warehouses. Nvidia CEO Jensen Huang further emphasized this transformative potential in a video address, predicting that robots will soon “assist us in our homes, redefine how work is done in factories, warehouses, agriculture, and more.”

However, realizing Graphics 3.0 presents unique challenges. Unlike virtual AI, which often relies on abundant text data used to train large foundation models from companies like OpenAI and Google, physical AI necessitates pixel-based data. Such data is not as readily available or easily acquired in the real world. To bridge this gap, Nvidia is pioneering the creation of synthetic data by simulating comprehensive virtual worlds tailored for these applications. “Robots don’t learn from code. They learn from experience,” Huang explained, highlighting the core dilemma: “But real-world training is slow and expensive.”

To overcome these hurdles, Nvidia has developed advanced AI models and simulation tools designed to generate the precise pixel data needed to train robots, autonomous cars, and other physical AI devices. Aaron Lefohn, vice president of research at Nvidia’s real-time graphics lab, noted that these innovations demand “completely new tools so that artists can conceptualize, create, and iterate orders of magnitude more quickly than they can today.” Among these advancements are Nvidia’s Cosmos AI models, engineered to empower robots to interpret commands, sense their surroundings, reason, plan, and execute tasks within the physical domain. Sonia Fidler, vice president of research at Nvidia’s spatial intelligence lab, underscored how these models are crucial for injecting digital intelligence into the physical realm, adding that “Physical AI can’t scale through real world trial and error. It’s unsafe, time consuming and expensive.” A prime example is the training of autonomous vehicles in virtual environments, a far more feasible approach than repeatedly crashing physical cars to accumulate training data.

This week, Nvidia also unveiled Omniverse NuRec, a groundbreaking tool that converts real-world sensor data into fully interactive simulations. These simulations provide a safe and efficient virtual space where robots can undergo training and testing. Omniverse NuRec integrates various tools and AI models for the construction, simulation, rendering, and enhancement of detailed 3D digital environments. The virtual reconstruction of these worlds is achieved by processing 2D data collected from cameras and other sensors, with every pixel meticulously labeled based on a visual understanding of the incoming sensor data. Fidler acknowledged a critical nuance, however: “It is very important to stress here that visual understanding is not perfect and because of different ambiguities it’s hard to perfect.” Beyond simulation, the company also introduced new AI material-generation tools that facilitate the creation of highly realistic graphics, complete with authentic visual details like reflectivity and surface textures. These tools allow 3D experts and engineers to engage AI assistants using simple language to describe their design requirements, streamlining the creative process.