The Scan Is the Floor, Not the Building
Notes on spatial storytelling — Part 1
A scan, on its own, is just geometry. It's the floor — somebody still has to build the room. Notes on spatial storytelling, Part 1.
Everyone building in 3D right now is trying to answer the same question: how do you capture a place. I kept getting asked a version of it too, and at first it seemed like a perfectly reasonable question. It took me a while to notice it wasn't the one I actually cared about.
Every time I showed someone an early build of the spatial storytelling tool I've been working on, the first question was almost always about the scan. What app did you use. Mesh or Gaussian splat. How long did the capture take. Good questions — just not the ones I was trying to answer.
A Scan Is Not Yet a Story
There's a reason the capture question comes first. It's the visibly hard part. A 3D scan is a technical feat you can point at — polygons, point clouds, splats catching the light. Watching a photogrammetry app stitch a room together in real time feels like magic, and magic invites questions about how the trick was done.
But a scan, on its own, is just geometry. It has no beginning, no middle, no argument. You can walk through it, admire it, even get a little lost in it — but it doesn't tell you anything. It's the floor. Somebody still has to build a room on top of it, put furniture in the room, and decide who's allowed to sit where.
That's the part almost nobody was building for. Every tool I looked at was optimizing the floor — better capture, cleaner meshes, faster splats — and treating the room as an afterthought, if it was there at all.
Widening Who Gets to Archive
I come at this as an experimental photographer, not a heritage professional or a games technologist. What's always pulled me in is people and how they use a space — the worn corner of a stair, the shortcut across a courtyard, the way a building changes once a redevelopment plan is announced. Cities and redevelopment in particular: the moment before a place gets rebuilt is exactly when its stories are most at risk of just disappearing along with it.
That's how I ended up connected to Mono Tung, an architect and teacher at HKU. I shared an early version of the spatial storytelling tool at a workshop of his, built around real heritage documentation fieldwork in Hong Kong. Watching professionals and total non-experts sit at the same table, both trying to say something true about a place using the same scan, confirmed what I'd been aiming for: I want to build something both professionals and non-professionals can pick up and use to tell a story, not a tool that only works in expert hands.
That's why the tool is built around a single Markdown file, not a proprietary capture pipeline. One file holds the story; the scan just gets attached to it. The same story can be read as a scrolling article, with the model embedded like an illustration, or walked through as an immersive scene where advancing the narrative moves the camera to the next waypoint. No account, no lock-in, no software you need a license for just to read what someone else made.
More recently, I decided on the spot to point the tool at a university architecture exhibition, documenting the show booth by booth in 3D — a much lighter, more personal use case than heritage fieldwork, and a good sign that the same tool holds up at both ends.
An open call
I don't think capture is the hard problem anymore — splats and scanning apps keep getting better on their own. Authorship is the part still catching up: giving someone, expert or not, a way to stand in front of a scan and say something true about it.
That's the gap I'm building into, and I'd rather not build it alone. If you work in heritage, architecture, museums, or you're just curious about telling stories over 3D scans, I'd like to hear from you.
This is the first in a short series of notes on the spatial storytelling project.