NVIDIA has taken a significant step toward making Rust a first-class language inside CUDA. The company has introduced two projects for writing GPU kernels directly in Rust and compiling them to PTX: cuda-oxide, aimed at the traditional SIMT model, and cutile-rs, built around the newer CUDA Tile approach. Both are still under development, and NVIDIA warns that neither is production-ready, but the company plans to keep maturing CUDA Rust through 2027 and beyond.
CUDA Rust in 30 seconds
- NVIDIA now lets developers write GPU kernels directly in Rust and compile them to PTX, without keeping the kernel in CUDA C++.
- There are two paths: cuda-oxide for low-level SIMT programming and cutile-rs for working with blocks, or tiles.
- NVIDIA recommends starting with Tile when direct control over threads and memory isn’t required.
- Rust brings some of its ownership and aliasing guarantees to parallel GPU programming.
- Both projects are experimental; cuda-oxide is in early alpha.
The move addresses a somewhat awkward situation for teams already building AI infrastructure in Rust. The language has been gaining ground for a while in systems components, inference engines, and infrastructure tooling, but reaching the kernel that ultimately runs on the GPU usually meant switching languages.
With CUDA Rust, NVIDIA wants to remove that boundary.
The kernel can be written in Rust and compiled directly into PTX (Parallel Thread Execution), the intermediate representation CUDA uses for code destined for NVIDIA GPUs.
That doesn’t make Rust an immediate replacement for CUDA C++. NVIDIA itself describes CUDA C++ and CUDA Python as mature tools for enterprise environments, while its two Rust alternatives remain at an early stage.
Two different ways to program the GPU with Rust
NVIDIA is bringing to Rust the two programming models it’s currently developing within CUDA.
The first is SIMT (Single Instruction, Multiple Threads), the classic model associated with CUDA. The programmer describes what a single thread should do, then runs thousands of threads in parallel.
cuda-oxide covers this approach.
The second option is Tile, a higher level of abstraction in which the programmer describes operations on blocks of data and lets the compiler decide how to distribute the work across the physical threads of each architecture.
That’s where cutile-rs comes in. NVIDIA introduced the Tile abstraction itself as part of CUDA 13.1, before extending it to Rust with this new library.
| Feature | cuda-oxide | cutile-rs |
|---|---|---|
| Model | SIMT | Tile |
| Level of control | Low-level | More abstract |
| Compilation | Rust → MIR → Pliron → LLVM → PTX | Rust → CUDA Tile IR |
| Rust | Pinned nightly | Stable 1.89 or later |
| CUDA | 12.x or later | CUDA 13.3 |
| Minimum GPU | Compute Capability 8.0 | Compute Capability 8.0 |
| Custom LLVM | Yes / associated toolchain | No |
| Status | Early alpha | More advanced, but experimental |
| NVIDIA’s recommendation | When control is needed | General first choice |
NVIDIA’s recommendation is telling: try Tile first, and drop down to SIMT only when you really need to control threads, memory, or architecture-specific details.
It’s a different philosophy from traditional CUDA programming, where much of the performance depended on understanding how to manually distribute blocks, threads, and shared memory.
CUDA Tile tries to hand more of those decisions to the compiler.
The potential upside lies in portability across GPU generations. If the code expresses an operation on a data tile instead of fixing exactly how individual threads should work, the compiler has more room to adapt the kernel to a different architecture.
cuda-oxide keeps CUDA’s classic model
For developers who need that level of control, there’s cuda-oxide, a custom backend for the rustc compiler.
When it finds a function marked as a kernel, it routes it through Rust’s intermediate representations, the Pliron framework, and LLVM to generate PTX. The rest of the program continues using the regular compilation process.
One important feature is that host code and GPU-bound code can live in the same Rust file. There’s no need to keep a separate project just for the kernels.
The model remains recognizable to anyone who has worked with CUDA before:
#[kernel]
#[launch_bounds(256)]
pub fn vecadd(...) {
let idx = thread::index_1d();
// thread operation
}Each thread computes its index and works on its own slice of the data.
The difference lies in how Rust can apply its type and ownership system to certain errors that, in traditional CUDA, might only surface at runtime.
One example NVIDIA uses is DisjointSlice.
A regular &mut [f32] would represent mutable access to the entire slice, which isn’t well suited to thousands of threads trying to write simultaneously to different positions.
DisjointSlice<f32> conceptually splits that access so each thread can get exclusive permission over its own position.
The thread index isn’t handled as a plain arbitrary integer either. thread::index_1d() returns a specific type that can be used with get_mut, and the result comes back wrapped in an Option.
That turns certain out-of-range accesses into situations the program must handle explicitly, rather than leaving them as a memory error that’s hard to reproduce.
cuda-oxide also introduces launch contracts.
An annotation can declare that a kernel expects blocks of 256 threads and a one-dimensional domain. Before running it, the configuration is checked against that contract and against the device’s actual capabilities.
When a kernel has no contract, the launch is marked unsafe.
It’s a good example of just how far NVIDIA wants to push Rust’s safety ideas into CUDA without fully hiding the hardware.
cutile-rs lets the compiler manage the threads
cutile-rs takes a fairly different philosophy.
The programmer stops thinking primarily in terms of individual threads and instead works with data tiles.
If there’s a vector of 1,024 elements split into blocks of 128, for instance, the system produces eight tiles. Each one forms a logical unit the kernel runs against.
An operation like:
let z = api::zeros::<f32>(&[1024]).partition(&[128]);doesn’t just split a data structure.
The partition defines which region each tile can modify, sets the execution geometry, and gives the kernel size information.
From there, the compiler decides how many physical threads to use to run that operation on the GPU.
It also changes how mutable memory is used.
Each tile gets an exclusive region to write to. Two tiles shouldn’t be able to hold simultaneous mutable references to the same chunk.
NVIDIA is trying to leverage one of Rust’s core principles here: if a mutable reference exists, no other incompatible reference to that same data should exist at the same time.
That idea is especially valuable on the GPU.
Concurrency bugs can be extraordinarily hard to reproduce because thousands of threads execute operations in an order the programmer doesn’t fully control. A conflict over a memory address might only show up under specific workloads or hardware configurations.
Rust’s ownership system can catch some of these situations before the program ever runs.
That doesn’t mean Rust automatically makes every GPU kernel safe. In cuda-oxide, for example, direct use of shared memory still requires unsafe blocks — precisely one of the areas NVIDIA admits still needs work.
Rust is also making its way closer to the GPU
This isn’t an isolated bet.
NVIDIA notes that several parts of its infrastructure already use Rust. NVIDIA Dynamo, its distributed inference platform, has a core built in the language, while NVTX offers Rust bindings. The company also mentions work around the Nova driver for Linux.
Kernel programming was one of the pieces that still forced a jump to another language.
CUDA Rust tries to connect both worlds:
Rust application → CUDA runtime → Rust kernel → PTX → NVIDIA GPU
That could be especially appealing for inference engines, accelerated databases, scientific computing, or AI applications that have already chosen Rust for the rest of their stack.
It could also cut down on some of the complexity that comes from mixing Rust code with kernels maintained separately in CUDA C++.
NVIDIA doesn’t yet consider CUDA Rust production-ready
The announcement comes with a clear caveat: these projects don’t yet replace CUDA C++ in critical applications.
NVIDIA labels cuda-oxide as early alpha. It requires Linux, a GPU with Compute Capability 8.0 or later, CUDA 12.x, Clang, and a specific Rust Nightly version.
The installation process still reflects that status:
cargo +nightly-2026-04-03 install --git https://github.com/NVlabs/cuda-oxide.git cargo-oxideA project can then be created with:
cargo oxide new vecadd_demo
cargo oxide doctor
cargo oxide runcutile-rs is somewhat further along in terms of distribution. It’s published as a package and works with stable Rust, though it requires CUDA 13.3.
cargo new vecadd_demo
cd vecadd_demo
cargo add cutileNVIDIA also notes that cutile-rs is already being used outside the company, including in Grout, a Hugging Face inference engine, and the mistral.rs project.
Even so, CUDA coverage remains incomplete, and the interfaces may still change.
Work presented in Fearless Concurrency on the GPU offers some clues about the performance target. In tests published by its authors, cuTile Rust reached roughly 7 TB/s on elementwise operations and 2 PFLOPS on GEMM on an NVIDIA B200, about 96% of cuBLAS’s performance in that specific test. These are experimental results on particular workloads, not a guarantee that any given Rust kernel will automatically come close to the performance of optimized CUDA libraries.
NVIDIA also wants to add interoperability between CUDA Rust, CUDA C++, and CUDA Python. The intent is that choosing Rust for one kernel shouldn’t force the rest of an application into the same language.
The bet, then, is bigger than just shipping two new libraries. NVIDIA is trying to let Rust run through the entire stack, from system infrastructure to the code that ultimately executes on the GPU’s cores.
CUDA C++ will remain the reference for many developers who need to squeeze every bit of performance from the hardware for a long time to come. But if cuda-oxide and cutile-rs mature as planned, Rust could stop being just the language that prepares and launches work toward the GPU, and become the language that writes the kernel itself.
via: developer.nvidia
