If you were in the market for a new smartphone, what specs would you look for? Most of us are still captivated by specific brands, and some may be particular about technical details like the processor power. Others may favor aesthetic choices like shape, color options and quality of screen. But arguably the key competitive differentiator that has emerged is camera quality—a shift that highlights the camera’s importance in how it has shaped and redefined the way we interact with one another and the world around us.

I have actively studied the evolution of mobile augmented reality (AR) over the past decade, which has flourished to support this user behavior while also challenging conventional notions of UI or UX. And with 5G on the horizon to offer more bandwidth and stronger internet connection, what you can accomplish via AR on a mobile device will become even more sophisticated than the experiences we enjoy today—and may take an even more important role in daily life, too. A report from eMarketer, “Getting Ready for 5G,” notes that internet users are more excited for mixed reality’s possibilities in medical and educational use cases than in entertainment.

a snapchat filter of a kitten overlaid on a little girl's face
The Little Brush, Big Brush experience from Signal uses AR to make good dental hygiene habits more fun. Image credit Signal.

The possibilities are there, and there’s clearly excitement for new experiences that drive actual value that we haven’t seen before. But what makes a good AR experience anyway, and how can creative teams equip themselves to deliver? Thankfully, it’s never been easier to experiment with AR and 3D content than it is now: platforms like Adobe’s own Project Aero have made developing AR experiences—a process that requires both creative and technical skill—easy for designers to pick up.

But on a conceptual level, how can creatives begin to consider or explore the creative opportunities offered by the technology? Just as the mobile camera prompted a shift in user behavior, developing for AR will require that creatives likewise adopt a new, camera-first perspective in their approach to designing the digital experiences of tomorrow.

Anticipating the next new interface through camera-readiness

The shift to a more camera-focused mindset begins with a greater sense of purpose in imagining the experiences you want to offer. It’s not about using tech for the sake of it; it’s about realizing new possibilities and ways of connecting audiences with the world around them. How will the world around the user influence and direct the experience? This line of questioning helps to lay the foundation for designing more value-added experiences. From there, think carefully about how central the camera can be to that experience.

Take, for example, Snap identifying as a camera company, not a software one. “We believe that reinventing the camera represents our greatest opportunity to improve the way that people live and communicate,” wrote the company in its IPO filing. “Our products empower people to express themselves, live in the moment, learn about the world, and have fun together.”

The positioning may have turned heads, but it made sense; it meant the company was redefining the camera not as a piece of hardware that documents scenes or events, but as an interaction that enables users to better express themselves and engage with the world. When designing for AR, creatives should likewise begin at the camera and what it means for users—a process that requires they understand the user’s mindset and the inflection points that direct them throughout the journey.

But if we’re really taking a future-focused view, we must recognize that experiences won’t simply take place through the camera alone; instead, it will become a gateway to content and experiences that persist across an ecosystem. In building its ecosystem of smart-connected devices, for example, Google has built a vision of being helpful wherever users need it. It’s a vision that’s driven technologies like Google Lens, which identifies objects or text within a photo and is accessible directly through Google’s Camera app; the highly immersive cloud based service 6d.ai; or the recently-announced Flutter, an ambient-computing platform that lets developers build apps for any device using a single codebase.

My belief is that this latest transition is to ambient computing, this notion of having an always accessible computer right at your fingertips, that understands you, that can do things on your behalf to help you in different ways.

Rishi Chandra, VP of Product, Google

With AR’s ability to add a virtual data layer onto the surrounding environment, it’s exciting to think about what possibilities are available—or soon will be. AR forces creatives to truly think about what value the digital visualization delivers to the real environment, from the first to the last second. In that sense, it provides a challenge for most creatives to wrap their heads around, requiring they make the step from theoretical user-centric design to actual user-centric design.

Assessing the current AR ecosystem

We’re already living in a world of ubiquitous AR; it’s all over stories on Facebook, Instagram and Snapchat, and is even making its way into more traditional advertising formats: WeMakeUp’s mobile AR ads, which allow users to virtually try on lipstick shades within the Facebook newsfeed via AR, resulted in a 27.6-point uptick in purchases of the makeup, according to Facebook.

Meanwhile, Apple and Google have baked native AR into their mobile operating systems, allowing for even more helpful and advanced applications of the technology. I’ve been fortunate to have been involved in the beta program of Google’s AR search results from the start, over 10 years ago. It lets users place a life-size model of a searched object into the space around them. This is an excellent example of how the camera can serve as a lens through which to see and understand the world in a new way. Live View in Google Maps similarly provides help in a unique way by superimposing information (in this case, walking directions) on the user’s environment. Seeing this concept emerge to what it is today brings back fun memories of how we in the AR developer community had to jump through hoops to create experiences like this just because the phones could not deliver the right performance.

“Since winning the search engine wars, Google has played a central role in the way we learn and understand our world. Their growing focus on AR offers another, perhaps even more intuitive and interactive way to help users learn about the world,” says Samuel Snider-Held, Creative Technologist at MediaMonks. “This sentiment is driving the creation of software and tools for us creatives to overlay experiences on top of the real world.”

electric signage that reads Pharos AR
Made in collaboration with Childish Gambino, the Pharos AR experience transports a group of users into the same mysterious, liminal space. Image credit Childish Gambino.

These examples—paired with Google’s focus on ambient computing, or extending the user experience across devices—give a hint at how AR can become a powerful user interface in its own right. The impact of this is that we will be able to identify and instantly visualize connected demographical, historical, and contextually related data to any object around us in the real world by simply pointing our phone’s camera at it.

Imagine being at a ski resort looking around through your phone’s camera and getting a real-time view of its rich history; the cost of the ski lift; and a real-time augmented view of the mountain with the blue, red, and black pistes connected to the snow and air quality. Simply pointing your phone at the ski lift would enable you to instantly collect the pass that would get you to the top of the mountain, with a sport at the best restaurant reserved for your group and access to a challenging red piste that will make your day. But to achieve this effectively requires working in concert with other technologies and sources of data, like computer vision or perhaps even voice for a fuller, more ambient, and contextual experience.

Take the first step toward designing mature AR experiences

If you’re feeling inspired about AR or mixed reality content, where should you begin experimenting with it? The good news is that there are many platforms through which you can try your hand at building simpler experiences. Social use cases like selfie lenses or Facebook’s AR ads are both great places to start, as both formats are easily shareable and provide immediate user feedback for you to learn from.

When working with brands, quick wins like these can also ladder up into developing more mature AR experiences. If the camera is increasingly central to the user experience, how could that inform engagement with traditional content, like in-store displays, print ads or social shoppable content? It has never been easier for designers to experiment and explore these opportunities in a hands-on way, and it will be exciting to see where this leads us over the next few years—like seeing the world through a whole new lens.