Are you ready to trade in your mobile phone, smart TV and laptop screen for the next world-changing device? I don’t know what it will be exactly—perhaps a pair of AR glasses, a holographic display or an AI assistant—but I do know this: the next significant era in human-computer interaction is being imagined, designed and invented as we speak.
We’re at the event horizon of a new era of spatial computing—a world where digital experiences mesh with physical reality. Immersive, 3D technologies like AR and VR, along with voice and embedded sensors, are all converging into a new medium, powered by artificial intelligence.
It will dramatically expand our means for collaborating with one another and consuming information. It will profoundly change the way we work, live, learn, and play. And, its effect on society will be larger than desktop and mobile computing combined.
As a designer, I’m super excited (and sometimes a little scared) to take part in this extraordinary period of human history.
What is spatial computing?
Up to this point, digital experiences have been defined by the boxes and devices that power them, and design has been about shaping the form factor of those devices or pushing pixels around on two-dimensional screens.
Spatial computing is changing that – it’s not locked in rectangles but can flow freely in and through the world around us, unlike mobile computing and desktop computing before it. In other words, spatial computing uses the space around us as a canvas for digital experiences.
Most people experience spatial computing for the first time through virtual reality (VR) or augmented reality (AR). VR creates new three-dimensional worlds and AR meshes digital experiences with real physical spaces.
A simple way to think of it is that VR creates a digital space that takes the user somewhere else, while AR—or mixed reality—adds a layer of digital objects and experiences to the real-world physical spaces we already inhabit. As a result, it’s much more immediate and social.
By now, the image of a model wearing VR goggles is a familiar sight—mouth open in amazement, head tilted towards an invisible horizon and hands grasping for an unseen object. It’s reflective of common VR applications like 360 video and gaming. The experience is powerfully immersive, but there’s also an element of social isolation. In fact, half of the respondents in a 2018 survey around young adults’ atitudes toward VR believe that it will make people more isolated and anti-social.
A common portrayal of the VR experience is incredibly immersive, but socially isolated.On the other hand, augmented reality is more social and immediate, but consumer applications are still trapped in the screen. If you’re using AR today, chances are it’s with your mobile phone or tablet as a window into digital spaces, playing an AR-inspired game like Wonderscope, viewing a digital artifact or artwork in a public space, or scanning a QR code to pull up extra information or animation from a brochure or product box.
As useful and as compelling as these immersive experiences can be, they have yet to be transformative at the scale of mass consumer adoption.
Natural interaction as an accelerant
What’s still missing is the ability to inhabit, move through and interact with digital spaces in more natural and intuitive ways.
Just like desktop computing required the GUI and mouse to drive consumer adoption, and the mobile revolution is powered by touch-enabled smartphones, spatial computing is waiting for that “magic” combination of form and function to emerge.
“One of the biggest challenges to designing something like AR glasses is making sure they can help me do something I need to do for 12-18 hours at a time,” says Avi Bar-Zeev, Principal, RealityPrime and widely known as an AR pioneer and an original designer of the Microsoft Hololens. “That’s less about technical improvements like battery life or high-resolution displays and more about understanding things like social graces and utility. It’s about understanding the world.”
Creating experiences with that kind of understanding requires multiple technologies working together:
- Immersive display technology, like AR glasses or holographic projectors will improve our visual and social experience of digital spaces.
- The Internet of things, will embed compute power and a variety of sensors in everyday physical objects and spaces.
- New forms of human-computer interaction like voice, gesture and haptics will engage more of our senses and make interaction more natural.
- And Artifical Intellegence will emerge as the connecting force underneath the entire ecosystem that powers the interface in the different contexts—home, work, car, public and private.
Combined, these technologies will shape and power the era of spatial computing.
“New devices like AR glasses are just part of an intelligent system that needs to be able to recognize the things in the room, start to label objects, understand what they are for and apply that knowledge in a useful way,” Avi adds. “You can’t separate the social and cultural value from the technology. Good design has to start there.”
New responsibilities
From a designer’s standpoint, creating these types of experiences will bring a new set of responsibilities and complications. The foundation we lay now, will impact future generations and their relationship with technology.
Different inputs, surfaces and contextual needs all come into play. But more importantly, the designs we create will result in more natural experiences that increase emotional connection.
To lead the way, designers must craft a new design language for our senses and consider the ethical implications of designing for human behavior in these digital-physical experiences. We’ll tackle that subject in our next article on the power and challenges of sensory design.