Red Hat, a global leader in open source solutions, has announced the acquisition of Neural Magic, a pioneering company specializing in software and algorithms designed to optimize generative AI (GenAI) models in inference environments. Neural Magic’s expertise in inference performance engineering and its commitment to open source align with Red Hat’s vision of providing high-performance AI solutions tailored to the specific needs of its customers, across any hybrid cloud environment.
Generative AI remains one of the most promising technologies, although it requires robust, high-capacity infrastructure to manage the large language models (LLMs) that underpin it. These models demand significant processing power and energy resources, often limiting their accessibility for many organizations. With this acquisition, Red Hat aims to facilitate access to generative AI through open and innovative platforms, integrating Neural Magic’s technology with its own portfolio of hybrid cloud AI solutions.
Democratizing AI through vLLM Technology and Open Source
Red Hat and Neural Magic will advance the vLLM project, an open-source initiative developed at the University of California, Berkeley, which enables the optimization of generative models across a wide range of hardware, from AMD and NVIDIA GPUs to x86 CPUs. Collaboration on vLLM will allow Red Hat to provide enterprises with an open path to creating AI strategies that can be implemented in their own data centers, in the cloud, or at the edge.
Founded in 2018 as a spin-off from MIT, Neural Magic specializes in inference software that enhances the performance of deep learning models. By joining Red Hat, the company will contribute to expanding Red Hat AI’s capabilities, allowing businesses to deploy language models more efficiently and with greater control, facilitating the use of AI in enterprise environments with high security standards.
Red Hat AI and Neural Magic: A Comprehensive Platform for Hybrid Cloud AI
The integration of Neural Magic’s technology will strengthen Red Hat AI’s offerings, a platform that includes open-source licensed models of up to 405 billion parameters, adaptable to any hybrid cloud environment. This platform allows for the customization of generative models, optimizing their performance and reducing operational costs.
Key technologies that will benefit from this integration include:
- Red Hat Enterprise Linux AI (RHEL AI): a platform for developing and deploying AI models on Linux servers.
- Red Hat OpenShift AI: a KubernetesKubernetes (commonly referred to in English as “K8s”) … environment for the development, training, and monitoring of AI models in distributed environments.
- InstructLab: an open-source project in collaboration with IBM that allows users to participate in refining the Granite LLM models.
A Future of Secure, Flexible, and Scalable AI
With this acquisition, Red Hat reinforces its commitment to open, secure, and accessible generative AI, backed by a community of contributors. Neural Magic’s technology will not only enhance the efficiency and performance of generative models but also enable companies to adapt AI to their needs while maintaining control over their data and security policies.
This transaction, which is subject to regulatory approval, marks a decisive step in Red Hat’s strategy to solidify its position as a leader in open source AI solutions in the hybrid cloud.