Microsoft has introduced Maia 300, its third-generation artificial intelligence (AI) accelerator, with a data center design that packs 96 chips per rack, high-bandwidth HBM4 memory and mostly liquid cooling. The company plans to begin production deployment in 2027 and turn this platform into one of its main systems for running inference for its products and services. The goal is to process more AI queries with less energy and reduce reliance on third-party accelerators.
Microsoft Maia 300 in 30 seconds
- Each accelerator integrates eight HBM4 memory stacks, with 288 GB of capacity and up to 18 TB/s of peak bandwidth.
- A rack packs 96 accelerators and reaches a design power envelope of 167 kW.
- Liquid cooling removes more than 80% of the thermal load from the accelerators, voltage regulators and switching chips.
- Microsoft promises up to seven times the token-per-rack performance of Maia 200 and 40% more tokens per megawatt than reference GPU-based systems.
- Production will begin rolling out in 2027, with a network architecture designed to connect up to 12,288 accelerators in a superpod.
The announcement, published by Microsoft on October 12, 2026, reflects how major cloud providers are moving from designing individual chips to developing complete AI systems. In Maia 300, the processor, memory, high-speed interconnects, power delivery and cooling are designed to work as a single platform.
The focus is especially on inference, the phase in which an already-trained model generates responses or carries out tasks. Microsoft cites long-context models, low-latency conversations and AI agents that perform several consecutive operations as its targets.
HBM4 and chiplet architecture to reduce bottlenecks
Each Maia 300 accelerator incorporates eight HBM4 memory stacks, with a total capacity of 288 GB and peak bandwidth of 18 terabytes per second. HBM, short for High Bandwidth Memory, is a stacked memory technology that allows large amounts of data to be transferred between memory and the processor.
Capacity and bandwidth matter because AI models need to continuously move parameters, context data and intermediate states while generating responses. When memory access limits how the system runs, adding compute capacity alone doesn’t guarantee a proportional performance increase.
Microsoft says Maia 300 delivers 2.5 times the memory bandwidth of Maia 200 and about 33% more capacity. The company also highlights an approximately twofold improvement in HBM bandwidth per watt compared with the previous generation. These comparisons come from figures provided by Microsoft and aren’t the results of an independent test.
The chip uses a multi-chiplet architecture, meaning several specialized silicon blocks working within the same package. The compute chiplets are built on TSMC’s N2P process, a technology in the 2-nanometer class, while memory control and input/output functions are spread across blocks tailored to their needs.
This split makes it possible to dedicate the most advanced manufacturing process to the compute units and use different solutions for other functions. Microsoft argues that the design increases efficiency and allows for better cost control compared with an architecture that packs every function into a single silicon block.
The system also incorporates low-precision numeric formats, including an improved version of MXFP4. These representations reduce the amount of data that needs to be stored and moved, although their usefulness depends on the model, the operations performed and precision requirements.
A 167 kW rack with liquid cooling
The full Maia 300 configuration packs 96 accelerators across 24 trays, each holding four chips. The rack has a power envelope of 167 kilowatts and integrates the processors, memory, storage and interconnect infrastructure into an assembly built for data centers.
Liquid cooling handles more than 80% of the thermal load generated by the accelerators, voltage regulators and switching circuits. Other elements, such as memory, storage and auxiliary components, keep using air cooling.
The system can connect directly to a facility’s water loop when the data center has that infrastructure in place. It can also use a heat exchange unit to remove heat in environments running air-cooling systems. This flexibility widens deployment options, although it doesn’t remove the need to correctly size the electrical and thermal installation.
This compute density brings significant demands. A 167 kW rack needs power delivery capable of handling that load, proper electrical distribution and cooling systems designed to keep temperatures within operating limits. Microsoft includes two power feeds, high-capacity power distribution systems and a 50-volt bus architecture to manage energy within the rack.
Inside the rack’s network, electrical connections link the accelerators within the trays and allow them to connect through switches built into liquid-cooled trays. To link different racks together, the system uses a second network tier with optical connections, needed to cover longer distances.
The architecture can scale up to a superpod of 12,288 accelerators, spread across up to 128 racks according to the design Microsoft describes. This capacity makes it possible to split inference workloads across different groups of chips and adjust how tasks are distributed to each model’s needs.
Microsoft promises up to seven times more performance per rack
The company states that Maia 300 can deliver up to seven times the token-generation performance per rack of Maia 200 in a full 96-accelerator configuration. It also estimates the system will process 40% more tokens per megawatt than the reference GPU-based systems it compares against.
These figures shouldn’t be read as a universal improvement across every workload. Actual performance will depend on the model, context size, concurrency, network configuration and how the load is distributed across accelerators. Microsoft doesn’t present an independent comparison in the announcement that would allow all these figures to be reproduced under the same conditions on different platforms.
The company says the design is aimed at services such as Microsoft Copilot, GitHub Copilot, Microsoft Foundry and its own AI models. Maia 200 is already in production for certain Microsoft workloads, while Maia 300 will begin its rollout during 2027.
Developing its own chips lets Microsoft tailor hardware to its models and services, rather than relying exclusively on commercial accelerators from NVIDIA or AMD. That doesn’t mean it will abandon those vendors: the company can combine different platforms depending on availability, cost and the needs of each workload.
The challenge will be translating the announced improvements into real operations at scale. HBM4 memory, liquid cooling and accelerator interconnects make it possible to build denser systems, but they also require infrastructure investment and precise coordination between hardware, software and data centers.
Frequently asked questions
What memory does Microsoft Maia 300 use?
Each accelerator integrates eight HBM4 stacks, with 288 GB of memory and up to 18 TB/s of peak bandwidth, according to the specifications published by Microsoft.
How many accelerators does a Maia 300 rack have?
Each rack packs 96 accelerators and has a power envelope of 167 kW. The configuration uses 24 trays of four accelerators each.
When will Maia 300 start rolling out?
Microsoft plans to begin production deployments in 2027. The platform is designed mainly to run AI inference for its services and products.
What are the advantages of liquid cooling?
It removes more than 80% of the thermal load from the accelerators, voltage regulators and switching chips. Memory and other auxiliary components continue to use air cooling.
