Rust · WebGPU · no CUDA, no Python
The most efficient, portable runtime for neural networks.
One Rust crate and one set of WGSL kernels — no CUDA toolchain, no
Python interpreter, no vendor to be locked to. The same code trains and runs
models on every GPU wgpu reaches, from
a workstation to a browser tab to the edge devices you own end to end.
Portable and efficient is where this is going: one runtime on every device you own, and eventually across them. Scroll down and it will run on yours, and tell you how fast.
verified — CPU and WGPU
logits agree to 8.4e-5, and GPT-2's
output matches HuggingFace
transformers token for token
(tests/gpt2_e2e.rs, on an RTX A5000).
Watch it think
It writes one character at a time. Each one is a choice among 65, and to make it the model reads the characters already there. Both halves of that are below, as they happen: the shortlist it chose from, and the places it looked to build it.
What it chose
The model's own probability for each candidate, before temperature and top-k reshape the distribution it samples from — so the character it picks is not always the one at the top.
Where it looked
Attention for the position just computed, over the 32 characters before it. Bar heights are relative to the strongest position in that row; hover one for the exact probability.
Nothing has been computed yet
Every number here is produced by shakespeare-char, a 10.77M parameter model trained from scratch with Forge. Start it and the numbers arrive one character at a time — 43 MB of weights, fetched once and then cached by your browser.
The whole matrix, every position
Rows are queries, columns are keys, and the triangle is the causal mask — no position may attend to the future. Brightness is p0.45 so small weights stay visible; the number you hover is the raw probability.
Lower temperature is more confident and more repetitive;
top-k limits how many of the 65 characters can be chosen at
all. top-k 0 is greedy — the
single most likely character, every time.
Weights (43 MB) are fetched only when you press Run, then cached by your browser.
The calculation, in words
"Where it looked" could not be drawn in this browser. The model still ran on your GPU, and the shortlist and the numbers beside it are real. What the missing panel shows is this: each character becomes a 384-number vector — its token embedding plus a position embedding — and that vector is the residual stream every later stage reads and writes. A LayerNorm rescales it; one matrix multiply turns it into a Query, a Key and a Value per head; each Query is scored against every Key as q · kᵢ / √64; a softmax over those scores, masked so no position sees the future, decides how much of each Value to collect; and the weighted sum is the head's output. Six heads do that at once on different 64-dimension slices, their outputs are laid end to end back into 384 numbers, a projection mixes them, and the result is added to the stream rather than replacing it. A second LayerNorm, an MLP that widens 384 → 1536, applies GELU and comes back, one more addition — and that is one block, six blocks deep. A final LayerNorm and the embedding table used backwards turn the last position's 384 numbers into a probability for each of the 65 characters.
No server does the work — your GPU does. Forge needs no
SharedArrayBuffer, because
rayon is native-only, so this page is
plain static files.
What a token costs
10.77 million parameters is not a small language model by the industry's use of the phrase — it is three orders of magnitude below it. The research question is what that floor actually buys, and the only answer worth printing is the one measured on the machine in front of you.
Nothing here is filled in until you press Run above — every figure is produced by your own hardware, so none of it can go stale.
What is not in that budget is as much the point: no CUDA toolkit, no Python interpreter, no PyTorch, no ONNX runtime, and no server round trip. The cost of a token here is the cost of the arithmetic plus one WebGPU dispatch queue.
Runs where you are
One set of WGSL kernels, four places to run them. The browser tab and the workstation take the same code path — not an equivalent one, the same one.
| Where | Backend | What it needs |
|---|---|---|
| Linux workstation | wgpu → Vulkan | a Vulkan ICD (mesa or the vendor driver) |
| macOS, Apple silicon | wgpu → Metal | nothing beyond the OS |
| Windows | wgpu → D3D12 | nothing beyond the OS |
| This page | wasm32 → WebGPU | Chrome/Edge 113+ or Safari 26+ |
The wasm build is the library with a
wasm-bindgen facade over the async
inference API — not a port. The only native-only dependencies
(rayon,
pollster) are behind target gates,
and the crate has no optional dependency at all: everything built
with the runtime, including the two pages linked above,
is a separate crate in
tools/. That is why this page needs
no cross-origin isolation headers and ships as static files.
Why Forge is built this way
Three decisions, and what each one buys.
Rust, not Python
Explicit memory and dispatch, no interpreter in the loop, and one binary to run. Training and inference are the same code on the same tensors, so there is no second implementation to keep honest.
WebGPU, not CUDA
One set of WGSL kernels reaches every target above through wgpu, with no vendor toolchain to install — and no vendor to be locked to.
A CPU reference backend
Every kernel has a mathematically identical plain-Rust twin to check against. That is what makes the parity numbers mean something — and what makes the thing teachable, since the reference and the kernel can be read side by side.
The model, and how to run it
One Rust crate, small enough to read in an afternoon — all of it is on GitHub. Below is the model the page above just ran, and the two commands that serve this page on your own machine.
The model: shakespeare-char
Trained from scratch with Forge — one GPU, one command, no pretrained weights and no Python in the loop. 10.77M parameters: 6 blocks × 6 heads × 384 dims, a 65-character vocabulary, 256 characters of context, LM head weight-tied to the token embedding.
Held-out validation loss —, against nanoGPT's published 1.4697 for the same configuration.
The longest passage it shares with the text it was trained on is 24 characters, and 98% of the words it invents are real words from the corpus. It is composing, not reciting.
Run this page locally
./scripts/build_site.sh ./scripts/serve_web.sh # plain static files — no COOP/COEP headers needed
23 WGSL kernels in Rust
The complete compute surface, generated from
shaders/ at build time so this list
cannot drift from the code.
Forward — 15
- add
- dropout
- embedding
- gather_nll
- gelu
- kv_append
- layernorm
- matmul
- merge_heads
- scale
- softmax
- split_heads
- sum_rows
- unmerge_heads
- unsplit_heads
Backward — 6
- ce_bwd
- gelu_bwd
- layernorm_bwd_dp
- layernorm_bwd_dx
- scatter_add
- softmax_bwd
Optimizer — 2
- adamw
- sumsq
Train it yourself
The model in the section above is not a download — it is what this command produces. Training and inference are the same code on the same tensors, so there is no separate training stack to install and nothing that only exists at train time.
Tiny Shakespeare, from scratch
./scripts/download_shakespeare.sh # 6 blocks / 6 heads / 384 dims, 65-token vocab cargo run --release --example train_shakespeare -- \ --backend wgpu
What comes out
A safetensors checkpoint and its vocabulary, byte-identical in
layout to the one this page loads. Point
--model at it and the demo above
runs your weights instead — the forward pass, the KV cache and
the trace do not know the difference.
Quick start
Rust edition 2024 and a Vulkan, Metal, or D3D12 driver. No Python, no CUDA toolkit. The first command needs no download either — the model from the demo above is in the repository.
Generate — no download
# the 43 MB char model is in the repo cargo run --release --example generate -- \ --model assets/shakespeare_char --prompt "ROMEO:"
Same binary, real weights:
./scripts/download_gpt2.sh, then
drop the --model flag to run
GPT-2 124M — the run
tests/gpt2_e2e.rs checks against
HuggingFace.
Browse models in the terminal
# forge-top: model browser + live dashboard # tokens/s, VRAM, GPU util, temperature, power cargo run --release -p forge-top -- --path models/
A crate in tools/, not a feature
of the runtime — so nothing that draws a terminal can reach a
dependent's build.