“Games and entertainment have pointed us toward a new way of thinking about interacting with software: spatial interfaces.” — John Palmer, Spatial Interfaces
Spatial Interfaces presents a vision for the future of UX, in which “lots of the software we already use can be augmented by spatial interfaces”. Palmer envisions “spatial environments, often in 3D, populated by 3D avatars that move, interact, and relate to each other in space”.
He makes the case that although “we tend to associate these concepts with gaming”, the rapid growth of gaming and e-sports, as well as significant improvements in tools for spatial software (“WebGL, Three JS, Unity, Unreal, etc.”) means “we can take this kind of design outside of game application”, and “this is something that software designers should seriously consider”.
Palmer then goes on to present several interesting applications beyond gaming:
- “Who is everyone at my company meeting with right now?” Consider trying to answer such a question using Google Calendar compared to Sococo, a “virtual office for remote teams”:
- “Establishing a stronger feeling of closeness” in messaging software such as… Figma?! Given that “software with spatial concepts built in can be used reasonably well for social interaction, even if not the intended purpose”:
- “Leaving a trail of breadcrumbs through a browsing session”, in the spirit of Vannevar Bush’s “Memex”[1]:
- Creating “a spatial interface that would give us a new and intuitive way for navigating the Internet and understanding how websites are related to each other”, like Neal Stephenson’s “Metaverse”[2]
Palmer’s vision is compelling. Although the article lacks a concrete call-to-action, creative and motivated readers should easily be able to infer their own next steps.
Experiment with the tools (“WebGL, Three JS, Unity, Unreal, etc.”); iterate on the presented applications; keep an eye out for opportunities to incorporate spatial relationships into their own work; learn game development for ideas and insights; maybe even rewatch that one scene from Minority Report for further inspiration.
“Visions give people a direction and inspire people to act, and a group of inspired people is the most powerful force in the world.” — Bret Victor, A Brief Rant on the Future of Interaction Design
However, Palmer’s vision doesn’t push far enough.
Palmer writes of an “app that replicates a deck of cards”. He lauds “when the spatial affordance directly analogizes the real world”. He encourages the reader to explore the “territory where skeuomorphic spatial affordances can make our lives easier, solving problems by simulating real world spaces”.
Indeed, such replicas and simulations would be a significant technical accomplishment, as well as “a major force in the near future of software”. But why stop there?
In the three-step transition from physical to hybrid to digital[3], Palmer stops prematurely at the hybrid step. “Scanning” physical reality into virtual reality is a significant step. But in order to push[4] as far as we can and achieve the full potential of this transition, we must continue by asking ourselves:
What are the possibilities of a digital-native reality?
Digital reality isn’t bound by the same physics as physical reality.
Why set the speed of light at 300 thousand kilometers per second when we can reduce it to walking speed, in order to develop intuition for the laws of special relativity?
Why limit ourselves to three spatial dimensions when we, like the inhabitants of Flatland, can explore a four-dimensional world (not just 3 spatial dimensions plus time, but 4 spatial dimensions)?
Why limit ourselves to the “geographical distance between points on the surface of the earth” when “the distance between our minds is increasingly characterized by a completely different metric: the geodesic distance”?
“Have you noticed that none of the species that have achieved spaceflight are natural fliers? There are birds and insects and gliders on many planets — some of them quite intelligent — and yet none of them have breached their atmospheres. It is only the groundlings that stretch and struggle — the groundlings that, having overcome the first boundary, turn their eyes to the second, and the third, and the fourth.” — Duncan Sabien, r!Animorphs: The Reckoning, Chapter 50
Our tools and understanding have developed to the point where “2D interfaces living on 2D screens” is in our past, and “increasing the dimensionality to 3D” is the natural next step in the near future of UX. And as Palmer shows, this would already be a significant step.
“There are wavelengths that people cannot see, there are sounds that people cannot hear, and maybe computers have thoughts that people cannot think.’’ — Richard Hamming, You and Your Research
But let’s push as far as we can. What thoughts could we unlock if we combine our million years of evolutionary capability in spatial reasoning with the infinite possibilities of a digital-native world?
Footnotes
[1] “Moreover, when numerous items have been thus joined together to form a trail, they can be reviewed in turn, rapidly or slowly, by deflecting a lever like that used for turning the pages of a book. It is exactly as though the physical items had been gathered together from widely separated sources and bound together to form a new book. It is more than this, for any item can be joined into numerous trails.”
Vannevar Bush, As We May Think
[2] “When Hiro goes into the Metaverse and looks down the Street and sees buildings and electric signs stretching off into the darkness, disappearing over the curve of the globe, he is actualy staring at the graphical representations — the user interfaces — of a myriad different pieces of software that have been engineered by major corporations.”
Neal Stephenson, Snow Crash
[3] “So you can think of a piece of paper, and then you can think of a scanner and then you can think of a text file. Right. So you go from physical to this hybrid, like physical to digital scanner and then digital native.”
Balaji Srinivasan, The Future of Bitcoin and Ethereum, How to Become Noncancelable, the Path to Personal Freedom and Wealth in a New World, the Changing Landscape of Warfare, and More
[4] “Pushing is the art of understanding what the potential promise is of your game design, and following through to reach that potential. It involves not just looking at the easy conclusions behind your design idea, but to find things that are not obvious.”
Jonathan Blow, Push and Convey
3D worlds and the future of UX: A review of “spatial interfaces” was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.