Arm, the renowned chip design company, has announced significant advances in its mission to facilitate the development of artificial intelligence (AI) and machine learning (ML) applications on devices based on its architecture. The company is closely collaborating with cloud service providers and leading frameworks to create an environment that streamlines the process of AI-accelerated software development on Arm hardware.
Kleidi: The Key to AI Performance on Arm
At the core of this initiative is Arm Kleidi, a technology that brings together the latest development tools and critical resources to drive technical collaboration and innovation in the field of machine learning. Since its launch just four months ago, Kleidi has already been accelerating development and unlocking significant AI performance improvements on Arm CPUs.
Integration with leading frameworks
Arm has worked closely with the PyTorch community to integrate Kleidi libraries directly into the framework. This allows application developers to automatically benefit from dramatic performance improvements as soon as new versions of the framework are released, without the need for additional steps.
The results of this integration are noteworthy:
- An Arm demo chatbot, powered by Meta’s Llama 3 language model and running on AWS Graviton processors, achieved real-time chat responses for the first time in PyTorch.
- A 2.5 times faster time to generate the first token was observed after integrating Kleidi technology into the PyTorch source code, measured on AWS Graviton4.
- By applying optimizations to torch.compile to efficiently utilize Kleidi technology, performance gains of 1.35 to 2 times were achieved on a variety of Hugging Face model inference workloads, measured on AWS Graviton3.
Strategic Collaborations
Arm is working closely with cloud service providers like AWS and Google, as well as the growing community of independent software vendors (ISVs) in machine learning, such as Databricks. The goal is to ensure that developers can keep pace with the rapid evolution of generative AI.
Edge performance improvements
In addition to cloud advancements, Arm has announced that KleidiAI will be integrated into ExecuTorch, the new PyTorch device inference runtime. This integration is scheduled to be completed by October 2024 and promises exciting performance improvements for edge devices across a wide range of applications.
The Future of AI Development on Arm
Alex Spinelli, recently appointed to the PyTorch board of directors, leads Arm’s mission to empower developers worldwide in creating cutting-edge AI capabilities and applications, leveraging the full potential of end-to-end AI on Arm.
With these initiatives, Arm continues its commitment to making the developer experience as seamless as possible, regardless of where they are in the machine learning stack. The company is doing the hard work so developers can focus on innovation and creating next-generation AI applications.
Source: ARM news