NTT DATA, in its commitment to industrial digitalization, collaborates with NVIDIA and FANUC Germany to develop a digital twin of robotic arms. This project, driven by the NTT DATA Innovation Center established last August, is dedicated to introducing emerging and growing digitalization technologies, such as Digital Twins applied in the industrial metaverse.
The center, one of NTT DATA’s six global centers, aims to generate new business opportunities by collaborating on joint research and project development with clients. It also explores how technology can pragmatically transform society, making emerging and complex technologies transparent and useful for solving everyday problems.
NTT DATA has developed various innovative solutions using Digital Twins in different domains. For example, for a media client, they created a virtual replica of a data center that syncs with real ones to monitor various KPIs and simulate different hypothetical scenarios.
In a specific collaboration with NVIDIA, NTT DATA teams in DACH and Italy are developing specific industry use cases:
For an automotive OEM, NTT DATA developed an enhanced Product Configurator solution, connecting two phases of the automotive value chain, Design and Pre-sales, to allow customers to dynamically configure their desired car with all features and options available for the specific model. This use case enables the manufacturer to sell highly individualized products through direct channels like the internet, offering a customer experience almost identical to that of a visit to a local dealership.
For manufacturing clients, NTT DATA partnered with FANUC Deutschland GmbH, an expert in automation products and services such as robotics and computerized numerical control wireless systems. The project leverages the simulation and photorealistic rendering capabilities of NVIDIA Omniverse, a platform for connecting and developing tools and applications based on Universal Scene Description (OpenUSD). This platform allows companies to connect their 3D applications and tools for creation and collaboration, 5G connectivity, and AI algorithms. Customers benefit from the fact that, through NVIDIA Omniverse, FANUC’s robotic arm can be remotely controlled on a common platform that could potentially also connect other robots and machines, regardless of their specific product language. Additionally, the added value of having such a digital twin of the physical robotic arm is that it can be tested and configured in a purely virtual environment.
Technical Aspects
The project applies computer vision techniques to a 5G camera that captures real-world information, such as the position and orientation of objects (in this case, brick toys) to be picked up by the robot. The result demonstrates the importance of the Digital Twin for industrial robots, as it can optimize their performance, efficiency, and safety, as well as enable new scenarios for remote control and maintenance.
The NVIDIA Isaac Sim platform has been utilized as part of NVIDIA Omniverse to create and simulate the digital twin of the robotic arm. Using Omniverse technologies (e.g., Action Graph, extensions), a connection to FANUC’s proprietary platform has been implemented to simulate and execute the pick and place task with high precision and efficiency.
Artificial intelligence and computer vision techniques play a fundamental role in analyzing the captured images and determining the position and orientation of each object, as well as its color and shape. The information is then sent to the Digital Twin via the 5G connection, allowing the Digital Twin to simulate the same scene on its virtual table.