Madrid. 2025. AI is no longer just a lab demo: it’s a business strategy. But its downside is an unprecedented electrical and thermal demand. Next-generation GPU racks consume 50–80 kW continually — and the first pilots exceed 100 kW — while many legacy data centers are still designed for 5–10 kW/rack. Even hyperscale facilities in hubs like Ashburn (Northern Virginia), Dublin, or Singapore face moratoria or connection limits. In this context, bare-metal ceases to be a “niche” and becomes a central element for training and inference of LLMs at scale, without collapsing the electrical infrastructure or betraying ESG commitments.
“What we see in 2025 is simple: either you move to high-density bare-metal with liquid cooling and 415 V electrical architecture, or you won’t be able to scale your AI clusters in Europe without fighting the grid. And fighting the grid means months or years of delays,” summarizes David Carrero, co-founder of Stackscale (Aire Group), a European provider of private cloud and bare-metal.
Below, a practical guide —with observations from Carrero— to design, deploy, and operate 80 kW racks without breaking the network or degrading your SLA.
Why bare-metal for AI?
Direct hardware access. Virtualization overheads penalize precisely where it matters: latency between GPUs, access to HBM/DRAM, PCIe/NVLink, pinned memory, NUMA. In distributed training (all-reduce) and low-latency inference, every microsecond counts.
Real hyperdensity. With bare-metal you can pack HGX nodes or equivalent in 60–80 kW per rack with direct-to-chip cooling or immersion cooling, plus PDUs configured for 415 V three-phase. Virtualizing by default at these profiles is usually counterproductive.
Compliance and perimeter. GDPR, finance/health sectors, or sensitive IP require physical isolation, traceability, and control over data flow. Bare-metal anchors compliance and integrates with public clouds for elastic or non-sensitive parts.
“We are asked for two things: predictable performance and governance. Performance comes with direct access and well-cabled NVSwitch topologies; governance comes with dedicated hardware, metrics, and traceability per rack,” notes Carrero.
The electrical and thermal challenge (and how to overcome it)
1) Top-down electrical engineering
- 415 V three-phase and redundant busways capable of continuous loads of 80 kW.
- High-current PDUs (with phase measurement) and selective protection to prevent Cascade outages.
- Staggered power-on (sequenced startup) and peak management (inrush, brownouts).
- Granular measurement (per-rack, per-PDU, per-server) for Electrical FinOps (€ per kWh, per model, per token).
“If you lack per-PDU and per-rack measurement and don’t practice staggered startup, protective devices will trip. Electrical FinOps is the new discipline: knowing the cost each epoch, each fine-tune,” says Carrero.
2) Cooling: from air to liquid (and immersion)
- Direct-to-chip (D2C): cold plates on GPU/CPU/HBM/VRM; circuits primary/secondary with glycol or treated water; bypass and delta-T control.
- Two-phase immersion cooling for loads >80–100 kW/rack or when air is unfeasible; modular tanks, pumps, and heat exchangers to dry coolers.
- Less CRAC/CRAH: Hot/cold aisle containment alone is no longer enough at these densities.
“Liquid cooling becomes the new norm from 50 kW/rack. With immersion, we gain density and EER but we must re-learn fluid maintenance and safety,” warns Carrero.
3) Staying connected to the grid: PPAs and microgeneration
- Renewable PPAs (solar/wind) for real offset in Europe;
- Microgrids with fuel cells or batteries for peaks and resilience;
- Flat load curves and work orchestration to avoid coincident peaks.
“Green kWh requires long-term contracts. Where there’s restriction, we see microgrids with batteries for peaks. Also, orchestration: don’t start 300 nodes at once,” comments Carrero.
80 kW rack design (operational template)
Power and distribution
- Dual busways, 415 V/50–60 Hz, A/B PDU;
- Heavy-duty cords and physical lock;
- Thermal management of PDU (they also heat up).
Network and topologies
- NVLink/NVSwitch according to bill-of-materials from the manufacturer;
- InfiniBand NDR/HDR or Ethernet 100/200/400 GbE with leaf-spine architecture;
- Timing: PTP/SyncE if latency is critical.
Storage
- Scratch NVMe local (PCIe 4/5) per node;
- Burst buffers NVMe over fabric for shuffle stages;
- QLC layer for datasets “warm” and nearline HDD where latency allows.
Cooling
- D2C: plates, manifolds, quick-disconnects, leak detection;
- Immersion cooling: tanks, fluids with technical datasheets, EHS, and training.
Security and compliance
- Separated logical/physical zones, cameras, access, WAF/DCIM/BMS integrated;
- Intervention and reading records per rack/server (for audit).
And the grid? Coordination with the data center and utility
- Capacity plan for 24–36 months;
- Stable load factors (no spiky peaks);
- Flexibility: deployable 5–10 MW blocks in phases;
- Location: campus with contracted electrical capacity (or expandable) and water or dry-cooling available.
“The bottleneck is no longer the white room: it’s the transformer and the line. You need to go where there’s capacity or where it can be created,” sums up Carrero.
Cost, timelines, and why “do it yourself” often fails
- CapEx: a build-to-suit for AI can exceed hundreds of millions.
- Permits: years for high voltage.
- Technological risk: hardware changes (densities, TDP, interconnection) might make your system obsolete by the time you’re ready to deploy.
“Time-to-GPU matters more than projected CapEx. With bare-metal colocation, you can arrive in months and stay market-relevant. Building from scratch makes you late,” adds Carrero.
Operate without surprises: SRE for AI and electrical “FinOps”
SRE/Operations
- SLOs per job (latency, throughput, cost);
- Autoscaling and power-aware queues;
- Liquid/in-immersion maintenance (procedures, spare parts, sensors).
FinOps
- Cost €/kWh × kWh per epoch → €/model/€/token;
- Metrics for PUE/TUE and EER per pod;
- Rightsizing of HBM/DRAM, batching, quantization, sparsity: less memory → less kW → less €.
“You need to publish a cost per model and per fine-tune. Without it, prioritization is impossible. We see clients saving 20–30% with quantization and baking everything right,” notes Carrero.
Hybrid strategies that work (2025)
- Bare-metal for training and sensitive inference;
- Public cloud for prototyping and peaks;
- Edge for inference near the user;
- Datasets: landing zone in private object storage, selective replicas in cloud;
- Network: private interconnects and dedicated backhauls (avoid unexpected egress).
Real cases (Europe)
Healthcare / Life sciences
- Imaging pipelines, genomics, drug discovery with protected data (GDPR, HDS, etc.) and dedicated clusters; D2C reduces thermal noise and stabilizes SLA.
Finance
- Fraud inference and copilots with low latency and direct peering to markets; GPU pods anchored at regional financial centers.
AI-first
- LLMs and multimodal models in pods of 8–16 racks at 60–80 kW; mix of NDR IB + Ethernet 400G; local scratch NVMe + burst buffers.
David Carrero’s recommendations (short list)
- Choose location based on megawatts, not postal code. Ask for signed capacity and electricity delivery dates.
- Liquid cooling from the design phase. From 50 kW/rack, air does not scale.
- 415 V and A/B PDUs with measurement and sequenced startup.
- Financial metrics: assign € to epochs and tokens. Without measurement, improvement is impossible.
- Energy contract (PPA) and, if applicable, microgrid. Without energy, there’s no AI.
- Orchestration with power awareness: avoid unnecessary peaks.
- 3-4 year plan: hardware changes, but network stays relatively stable. Design for modularity.
“Responsible AI is not just about model governance; it’s about electric and thermal engineering accountability. Bare-metal is the tool to meet both,” concludes Carrero.
Conclusion
The question is no longer whether you can reach 80 kW/rack. It’s whether you can sustain it without breaking the network or compromising your SLA. With tailored bare-metal (415 V, liquid/immersion, dimensioned PDUs and busways), energy contracts, and measured operations, it’s possible to scale AI in Europe following a clear criterion: predictable performance, compliance, and footprint aligned with your ESG goals.
2025 and beyond will belong to those who combine model ambition with infrastructure discipline. Because training bigger models is easy; training better —and sustaining it— is where the real advantage lies.