The new Cosmos platform promises to accelerate the development of robots and autonomous vehicles with advanced models and optimized data processing.
At CES 2025, NVIDIA launched Cosmos, an innovative platform designed to transform the development of Physical Artificial Intelligence (AI) systems, such as robots and autonomous vehicles (AVs). The platform includes generative foundation models, advanced tokenizers, and an accelerated data processing pipeline, optimized for NVIDIA data center GPUs.
Cosmos: Democratizing Physical AI
The development of physical AI traditionally requires vast resources in data and real-world testing, resulting in high costs and slow progress. With Cosmos, developers can generate photorealistic synthetic data based on physics to train and evaluate models. Additionally, the platform enables customization of these models with specific data, such as recordings of autonomous vehicles in various conditions or robots operating in warehouses.
According to Jensen Huang, founder and CEO of NVIDIA: “The transformative moment for robotics is near. Just like language models, foundational models of worlds are key to advancing the development of robots and AVs. Cosmos puts these tools within reach of all developers.”
Open and Versatile Models
The foundational models from Cosmos are available under an open license, fostering collaboration within the robotics and AV community. Developers can access these models in the NVIDIA API catalog or download them from NVIDIA NGC or Hugging Face for customization.
These models are ideal for tasks such as:
- Video search and understanding: Identifying specific scenarios, such as congestion in warehouses or snowy roads, within large volumes of data.
- Generation of photorealistic synthetic data: Creating videos based on controlled scenarios in NVIDIA Omniverse.
- Development and evaluation of physical AI models: Customizing models with reinforcement learning and assessing their performance in simulations.
- Simulation of possible futures: Generating multiple outcomes of scenarios to train models in selecting the best path.
Advanced Tools for AI Development
Cosmos includes tools designed to optimize the creation and processing of physical AI models:
- Accelerated data processing pipeline: Powered by NVIDIA AI and CUDA technology, it can process 20 million hours of video in just 14 days, compared to the three years it would take with a traditional pipeline based on CPUs.
- Cosmos Tokenizer: A state-of-the-art visual tokenizer that converts images and videos into tokens that are 8 times more compressed and 12 times faster than current tokenizers.
- NVIDIA NeMo Framework: A platform for efficiently training, customizing, and optimizing models.
Global Adoption by Leaders in Physical AI
Leading companies in robotics and transportation are adopting Cosmos technologies:
- Robotics: XPENG is using Cosmos to develop humanoid robots, while Agility creates photorealistic scenarios to train models without needing expensive real-world data.
- Autonomous vehicles: Companies like Waabi and Foretellix utilize Cosmos to generate high-fidelity test scenarios and training data.
- Autonomous mobility: Uber collaborates with NVIDIA to leverage Cosmos and its powerful data capabilities to develop more robust AI models.
Dara Khosrowshahi, CEO of Uber, stated: “Generative AI will be key to the future of mobility. Cosmos and NVIDIA DGX Cloud will significantly accelerate safe, scalable autonomous driving solutions.”
Commitment to Safe and Responsible AI
NVIDIA developed Cosmos following principles of reliable AI, prioritizing privacy, security, and transparency. The platform includes tools to mitigate harmful content and to mark AI-generated videos with invisible watermarks, reducing the risk of misinformation.
NVIDIA encourages developers to adopt responsible AI practices, improving safety and transparency solutions in their applications.
Availability
Cosmos is available in the NVIDIA NGC catalog and on Hugging Face. Models will soon be optimized as NVIDIA NIM microservices. The platform is also compatible with NVIDIA DGX Cloud for rapid deployments and enterprise support through NVIDIA AI Enterprise.
With Cosmos, NVIDIA is redefining the development of physical AI, elevating robotics and autonomous vehicles to new heights in innovation and accessibility.