cgb/triposr-onnx-webgpu

TripoSR, ONNX for the browser

TripoSR exported to ONNX so it turns a photo into a 3D shape entirely inside a web browser on WebGPU. No server, no API key, no upload: the weights are cached by the browser and the reconstruction happens on the visitor's own GPU.

Try it without downloading anything

Make a 3D model at freegen.ai

Files

File	Precision	Size
triplane_q8.onnx + .data	int8 weight-only, block 32	485 MB
decoder.onnx	float32	0.2 MB

triplane.onnx takes an image and produces the triplane. decoder.onnx turns sampled triplane features into density and colour, and is run per batch of points while isosurfacing.

Inputs and outputs

• triplane: image [1,3,512,512] float32, RGB in 0..1, to triplane [1,3,40,64,64]. The DINO mean and standard deviation are inside the graph, so a caller does not have to know them.
• decoder: features [N,120] float32 to density [N,1], color [N,3].

How the image has to be prepared

This is not cosmetic, and it is the easiest thing to get wrong. The subject must be cut out, framed to 0.85 of the width, and composited onto mid grey:

cut the subject out            -> RGBA
crop to the alpha bounds, pad to a square
pad again so the subject is 0.85 of the frame
resize to 512 square
rgb = rgb * alpha + (1 - alpha) * 0.5

Composited onto white instead, the encoder reads the unbroken bright field as surface and the mesh comes back with a sheet of phantom geometry hanging behind the subject. Skipping the 0.85 framing gives a subject whose proportions are wrong, because the model reads scale off the picture.

Sampling between them

The grid_sample step that turns a 3D point into 120 features is deliberately not in either graph. ONNX Runtime's WebGPU backend has patchy GridSample coverage, and the arithmetic is small enough to do in the caller:

normalise the point into -1..1
for each of the three planes, taking coordinate pairs (x,y), (x,z), (y,z):
    bilinear sample that plane, align_corners=false
concatenate the three results  ->  120 features

What was changed in the export

• Both renderer activations are folded into the decoder. It returns density as exp(raw + bias) and colour through a sigmoid, so the caller gets values it can use rather than logits it has to remember to transform. Getting that wrong produces a shape that is quietly the wrong size.
• int8, not int4. Reconstruction is less forgiving than generation: an error in a triplane is a bump in the surface rather than a slightly wrong colour.
• Block size 32, not 128. Both score above cosine 0.999 on every tensor, which makes them look equivalent. Measured as occupancy, which is what a mesh is actually built from, block 128 agrees with the reference on 96.6% of occupied volume and block 32 agrees on 99.1%, for 39 MB more.

Verification

Every graph is compared against the PyTorch reference on a real photograph, not noise, because noise never exercises the range a trained encoder sees. The fp32 export matches at cosine 1.000000 on triplane, density and colour. The int8 build holds 0.9998 on the triplane and 99.1% occupancy agreement.

Licence

MIT, from the upstream weights and code. The DINO ViT-B/16 encoder folded into the triplane graph is Apache-2.0. Both permit commercial use.