Real-time novel views
- Fast differentiable rasterisation
- Sharp, high-frequency appearance
- Simple explicit primitives
- Smooth camera motion through captured scenes
π5 EXPLAINS · 01 · 5 MIN · 3 INTERACTIONS
Gaussian splatting,
in five minutes.
How can a computer rebuild a place from photographs—and then show it from a camera position that never existed? The answer is a cloud of soft, coloured 3D ellipsoids.
Scroll, drag, and experiment. By the end you will understand what a Gaussian stores, how a view is rendered, and how the cloud learns from images.
01 / 03
Many tiny blobs,
one coherent view. Move your cursor through the scene
one coherent view. Move your cursor through the scene
00:00 · THE PROBLEM
Photos show rays.
We need a world.
A photograph tells us the colour that arrived along each camera ray, but not exactly where that colour came from in 3D. One image cannot tell whether a red patch is a nearby leaf or a distant wall. Several overlapping views provide the missing clues: objects shift by different amounts as the camera moves.
Novel-view synthesis turns those photographs into a representation that can answer a new question: “What would this scene look like from here?” Gaussian splatting answers by reconstructing millions of small 3D volumes, then projecting them into any requested camera.
KNOWN PHOTOS
Different views expose parallax and occlusion.
LEARNED 3D CLOUD
Geometry and appearance live together.
NEW CAMERA
Render a view that was never photographed.
00:55 · ONE GAUSSIAN
A soft 3D ellipsoid
becomes a 2D ellipse.
An ordinary point has a position but no size, so it can leave holes when projected. A Gaussian is different: it is a tiny, fuzzy volume that occupies space. Its centre is μ; its scale, stretch, and direction are encoded by a covariance matrix Σ.
Think of Σ as instructions for turning a sphere into a rotated rugby ball. From a camera, that 3D ellipsoid projects to a 2D elliptical footprint—a splat. Its edges fade smoothly instead of ending abruptly, which lets neighbouring splats blend without seams.
ONE SPLAT · SCREEN SPACE
μ = (0.00, 0.00)
Drag the centre. The sliders change its projected footprint.
EDIT THE GAUSSIAN
Important: a Gaussian is not a coloured disc pasted on an image. It has a 3D position, 3D extent, orientation, colour, and opacity. This ellipse is its camera-facing footprint.
μ, Σ
A single Gaussian stores more than geometry. It also carries opacity and view-dependent colour. Position says where it lives; covariance says what region it covers; opacity says how strongly it blocks what is behind it; colour says what light it contributes.
Gaussian = position + shape + opacity + colour
02:05 · RENDERING A VIEW
Move the camera.
The cloud makes a new view.
When the camera moves, the Gaussians do not. Their fixed 3D centres are simply projected through the new camera. Nearby Gaussians shift more than distant ones, producing the parallax that makes the result feel three-dimensional.
Each projected ellipse covers several pixels. For every affected pixel, the renderer sorts contributors by depth and blends them from front to back. A strong, opaque splat hides more of what is behind it; a weak one lets more background colour through.
Geometry view: the left panel is the 3D cloud; the right panel shows the projected ellipses before they are blended.
- 01
Project each 3D ellipsoid through the current camera.
- 02
Sort the 2D footprints by camera depth.
- 03
Blend their translucent colours into the final image.
THE BLENDING RULE
Colour accumulates; visibility runs out. Start with all light available. Each Gaussian contributes its colour multiplied by its opacity and by the visibility left by nearer splats. Then it consumes part of that remaining visibility.
Cpixel = Σ Ti · αi · ci
Ti = light still visible before splat i
03:30 · LEARNING THE CLOUD
Training adjusts the blobs,
not the rules of rendering.
We begin with camera poses and a sparse set of 3D points estimated from matching features across the photos. Each point becomes an initial Gaussian. The first render is rough: splats are misplaced, colours are wrong, and empty regions remain.
Training repeatedly picks a known camera, renders the current cloud, and compares that render with the real photograph. The image error sends gradients back into every Gaussian’s position, covariance, opacity, and colour. Large Gaussians may split; useless ones may disappear. Try the simplified optimization loop below.
Initial cloud: the current splats disagree with the target, so the error panel is bright.
- 1
Render from a camera whose real photo is known.
- 2
Compare the rendered pixels with that photograph.
- 3
Update Gaussian position, shape, opacity, and colour.
- 4
Densify difficult regions by cloning or splitting splats.
04:30 · WHY IT MATTERS
Fast and beautiful.
Not magic.
A trained scene is explicit: millions of Gaussian parameters sit in memory, ready to be projected. Rendering avoids asking a neural network to predict every pixel from scratch, so high-resolution novel views can be produced in real time.
That speed comes with trade-offs. A large cloud consumes memory. The representation gives appearance rather than a clean surface mesh. Mirrors, transparency, very thin structures, and regions never seen by a training camera remain difficult. Editing individual objects is also less direct than editing conventional geometry.
Incomplete or changing worlds
- Unseen geometry and extreme extrapolation
- Reflections, transparency, and thin surfaces
- Large memory footprints
- Relighting and object-level editing