What can a network do when its weights come from a program instead of a training loop?
Why test this now?
Published 4 November 2026. On 2 November, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab published early experiments on weight-space programming. They report that a large language model can inspect, change and construct neural network weights through reasoning and code. Their experiments include small game policies and edits to an existing language model. These results motivate a test. They do not establish a replacement for training at scale.
I asked a coding assistant to build a small image generator with exactly 100,000 parameter entries and no training loop. The first model produced a ringed planet. Its controls changed too little. I then asked for stronger variation, followed by different objects selected through text. The assistant wrote Python construction code, inspected rendered images and revised the weight assignments. This was an iterative engineering process with visual feedback.
Here, “no training” applies to the two target networks. Their weights were assigned by code, without a dataset, loss function, optimizer or gradient updates. The assistant used to write that code was pretrained. Its prior knowledge and the design effort remain part of the process.
Run the planet generator
Choose a preset, then move the three controls. Climate changes the palette and some surface features. Structure changes rings and lighting. Terrain changes size and surface features. These are coupled controls: one input can affect several visible properties. The images are stylized geometry, not a physical simulation of planets.
Every setting uses the same adapted checkpoint. The browser evaluates the network once per pixel. It does not select a stored image or construct new weights for each setting. The initial planet network had 5,003 nonzero entries. The adapted network has 5,429. A patch changes 7,846 of the 100,000 entries while keeping the layer dimensions fixed.
Move a control to run the same network again. These controls can change several features at once.
Loading the local renderer…
Run the text-to-image system
Enter a phrase such as “large blue sailboat”. A fixed parser maps the object, color and size to three numbers. Those numbers join the two pixel coordinates at the network input. All six object families use one fixed checkpoint.
The vocabulary contains six objects, nine colors and three sizes. The parser also accepts six aliases, such as “boat” for “sailboat” and “kitten” for “cat”. Missing color and size values receive explicit defaults. Unsupported objects, multiple objects and unsupported scene descriptions produce an error. The interface is text to image in this restricted sense. It has no learned text encoder or open-ended language understanding.
The object geometry is encoded in the weights. Changing 8,481 entries of the adapted planet checkpoint produces the object network. The layer dimensions and forward-pass operations stay the same. This change rewrites the computation. It is not evidence that the network learned to transfer knowledge from planets to cats.
Objects: rocket, sailboat, tree, house, mushroom, cat.
Colors: red, orange, yellow, green, cyan, blue, purple, pink, white.
Sizes: small, medium, large.
Try “large blue sailboat”. A fixed word parser selects the controls. It rejects unsupported objects and scenes.
Loading the local renderer…
Count the parameters precisely
Both models have the architecture 5 → 193 → 185 → 185 → 151 → 3. Four hidden layers use ReLU: ReLU(t) = max(0, t). The final affine layer returns three color channels. The renderer clips these values to [0, 1], then converts them to 8-bit RGB.
For a layer with m inputs and n outputs, the count is mn weights plus n biases. The five layers sum to exactly 100,000 entries. This is a dense-shape count. It includes deliberately zero connections. The adapted planet network has 5,429 nonzero entries; the object network has 5,228. The count does not mean that there are 100,000 independent, useful or learned values.
A dense float32 checkpoint needs 400,000 bytes for the tensor values, excluding metadata. The browser uses a sparse JSON representation that omits zero connections. It preserves the same mathematical network. The 100,000-entry budget excludes the parser, compiler, evaluator and interface code. We did not establish a minimum network size or an advantage over a conventional procedural renderer.
| Layer | Calculation | Entries |
|---|---|---|
| Input → hidden 1 | 5 × 193 + 193 | 1,158 |
| Hidden 1 → hidden 2 | 193 × 185 + 185 | 35,890 |
| Hidden 2 → hidden 3 | 185 × 185 + 185 | 34,410 |
| Hidden 3 → hidden 4 | 185 × 151 + 151 | 28,086 |
| Hidden 4 → RGB | 151 × 3 + 3 | 456 |
| Total per network | 100,000 | |
How geometry becomes weights
For each pixel, define h₀ = [x, y, z₁, z₂, z₃]. The hidden layers compute hₗ = ReLU(Wₗhₗ₋₁ + bₗ). The output is RGB = clip(W₅h₄ + b₅, 0, 1). Pixel centers use x = 2(u + 0.5)/N − 1 and y = 1 − 2(v + 0.5)/N, where N is the image width and height. This is a coordinate network: the same function maps each position and control tuple to a color.
ReLU supplies useful geometric operations. An affine expression ax + by + c defines a line. Its positive part measures a violation of one side of that line. Sums of these violations describe convex polygon boundaries. The identity clip(t, 0, 1) = ReLU(t) − ReLU(t − 1) supplies bounded ramps. Combinations of ramps produce masks, outlines and color transitions.
The object compiler defines 41 convex polygon parts with 176 edges. Early layers encode edge violations and object selection. Later layers combine coverage, occlusion, palette values, outlines and glows. The planet compiler uses polygonal radius approximations, bands, rings, surface regions and lighting ramps. These shape definitions run when the weights are constructed. Browser inference uses only affine operations, ReLU and output clipping.
The result is a procedural graphics program expressed as a neural network. The visual rules come from the construction code. The network did not discover them from examples. This gives us a small system whose weights and operations we can inspect directly.
What the checks establish
The saved checks verify tensor shapes and counts. Applying each weight patch reproduces its target checkpoint exactly. Saving and loading the checkpoint preserves its arrays. Dense and sparse NumPy evaluation agree to a maximum absolute RGB difference of about 1.1 × 10⁻⁶ on the recorded probes. This is a numerical consistency check, not an image-quality score.
All 162 object–color–size combinations produced distinct image hashes at 48 × 48 pixels. That checks routing and output differences across the supported combinations. It does not measure aesthetic quality, semantic correctness or generalization to new concepts. The supplied probes activate every hidden neuron and find an output path from each one. These checks do not prove that every neuron is necessary.
The JavaScript renderer was checked against NumPy on six planet settings and twelve text prompts at 64 × 64 pixels. The maximum difference was one 8-bit channel level in either model. Different float32 accumulation and rounding paths can produce small differences at color boundaries. The test allows at most two levels. The downloadable report lists every test input. Timing shown in each demo is local render time for that device, not a benchmark across hardware.
Where this fits in the research
Generating weights has a longer history. HyperNetworks, by Ha, Dai and Le (2016), use one trained network to produce weights for another. Text-to-LoRA (2025) trains a hypernetwork to produce task-specific adapters from text. The target adaptation can be fast, but training is still part of those methods.
Tracr compiles programs in RASP into transformer weights. Its compiler is a useful comparison for this experiment: both approaches map an explicit computation into network parameters. Our code targets a small ReLU coordinate network for graphics. It is not an implementation of Tracr, a trained hypernetwork or a general compiler for natural language.
The recent weight-space programming work asks how much of weight construction and editing a general-purpose language model can perform. This example explores one narrow case. It does not reproduce those research benchmarks, compare stronger and weaker assistants, or test a scaling law.
Limits and a useful next experiment
These models produce a fixed family of stylized images. They have no diffusion process, learned visual concepts or ability to compose arbitrary scenes. They do not show that training is obsolete. They also do not show that a 100,000-entry network can replace a modern image model.
A stronger next test would compare the compiled networks with direct procedural renderers and trained networks at matched storage and runtime budgets. It would define new geometry tasks in advance, keep the test tasks hidden during construction, and measure visual correctness, compilation effort and runtime. That would separate useful weight programming from the simple fact that a neural network can represent a hand-written function.
For now, the useful result is concrete: two fixed checkpoints run locally in this page, and the code explains where their behavior comes from. Download a checkpoint, change an input and inspect the forward pass. The experiment is small enough to audit.
