Skip to content
CLAY CLOUD DOCSMachine learningConceptsStatusDashboard

Machine learning in Clay Cloud / chapter 6 of 8

6. Porting models

The weights file is the same for everyone. Porting means writing the recipe again, and proving it right.

Why porting is possible at all

Because a model is weights plus recipe (chapter 1), a new implementation of the recipe can use the original weights as they are: no retraining. Small implementations exist in C (llama2.c, llm.c) at a few hundred to about a thousand lines (docs/ml-workloads.md section 7). The work is well specified, checkable and repetitive, which makes it a good fit for agents. The risk is not writing it, but writing it wrong in a way that looks plausible: a model with a slightly wrong attention scale still produces fluent text. So we never accept a port by reading it.

The reference-diff harness

A port is accepted or rejected by a program, the reference-diff harness. It needs a trusted answer to compare to. Here that is an independent oracle: a pure-Python (standard library only) implementation, with its own tokenizer, written from a description of the model rather than from our port, computing in 64-bit floats over the same weights. Twenty prompts, chosen to include accents, Chinese characters, emoji, tabs, a 313-token paragraph and exactly 1,000 tokens, have their expected results stored as goldens. (packages/tensor-host/tools/gpt2_oracle.py, packages/tensor-host/golden/, packages/compiler/tools/gpt2-diff.ts.)

20 prompts oracle (Python, f64) Clay port (f32) compare goldens
Figure 6.1. The acceptance pipeline (docs/ml-m2-gpt2.md, "Acceptance").

What is compared, with thresholds fixed in advance in docs/ml-m0-scope.md and not loosened to make a port pass:

One rule was adjusted once, on real data, with a written reason: prompt p08 has two candidates only 2e-5 apart, far below float noise, so exact order of the top 20 cannot be demanded; the rule became "within 1e-3 of the reference's own score at that rank".

The real GPT-2 result

All 20 prompts pass. Worst case over all of them: last-position error 1.8e-4 (limit 5e-3), any-position error 2.7e-4, mean 1.4e-4 (limit 1e-3), greedy tokens exact for all 16 new tokens on each prompt (8 on the 1,000-token one). measured The tolerances were beaten by 20 times or more, so they were not tuned. Independently of both our oracle and our port, three outputs published in transformers' own tests are reproduced exactly, including the id sequence for "The dog" and the sentence "Hello, my dog is a little bit of a mess. I'm not sure if he's going".

The harness itself is tested by mutation: five deliberately wrong versions of the model (the wrong activation function, attention scale 0.25, no causal mask, layer-norm epsilon 1e-3, positions shifted by one) must each fail the harness on a prompt. If one passed, the harness would be too lax to trust.

The operation set, and the rule for growing it

A port runs if every operation it needs exists in the Tensor ability. GPT-2 needs 19 of the 22 (docs/ml-m0-scope.md section 2); a Llama-style block adds four (mul, rmsNorm, rope, repeat) and BERT needs a subset. The rule decided on 2026-11-11 (docs/ml-workloads.md section 14):

PyTorch or tinygrad as the source of ports

Porting from PyTorch means reading PyTorch model code and finding which of its many operators it uses; each port starts with an inventory of the operators it needs, and the cost is operator coverage. tinygrad is a different framework, and it is a candidate target the project owner raised on 2026-11-11. inferred, not yet evaluated It is believed (from memory, not checked against its source) to build everything from a small set of primitive operations of three kinds (elementwise, reductions, data movement), which would be a close fit to our 22. Two routes are to be evaluated at rung M4: write tinygrad-style models in Clay calling our operations directly, or record the operations with a graph-recording handler (rung M6) and lower them. Nothing about tinygrad has been read or run yet; the first task of M4 is to read its operation set and write the mapping (docs/ml-workloads.md section 14).

What agents can automate

The harness is what makes delegation safe: an agent can be told "port this architecture; you are done when the harness passes" and the answer is a measurable yes or no. The parts that suit automation are the operator inventory, the weight-name mapping, the forward pass, and the golden generation. The parts that still need a person: the first oracle for a new family (SmolLM2 and MiniLM have no published anchor values, so an oracle for them is unanchored, risk R5 in docs/ml-m0-scope.md), licence review, and the decision to expose a model to third parties. nothing here has been run for a second architecture yet: M4 is not started.

To remember. A port is accepted by an independent oracle with thresholds fixed beforehand; the harness is itself mutation-tested; the operation set grows only when a real port needs it, with versions.