Woof: The Future of Social AR is More Than Face Filters

The following was published in SHOTS in May 2021…. sign up for Greg’s weekly email for stuff like this every week HERE.

We’re at a fascinating crossroads of digital and social marketing, creative technology, and changing consumer habits.

TVs are getting smarter. Mobile ad tracking is being disrupted. Social platforms are all adding audio stories. TikTok is transforming how news spreads. Creators are monetizing like never before. Email is actually popular again.  

And meanwhile, there is a digital and social layer on the physical world all around us, just waiting to be unlocked by our phones—and soon eyeglasses, mirrors, windshields, and more.

source: Hyper-Reality by Keiichi Matsuda

The term “augmented reality” is a boring and unimaginative term for a transformative technology that can detect your face and turn you into a frog. Or learn your room’s layout and transform it into a palace. Or unlock hidden brand content behind a product logo, from a billboard, in a grocery store aisle, or on a pedestrian mall.

Originally popularized by Snapchat filters and Pokemon Go, augmented reality (AR) reached the mainstream without consumers realizing the tremendous nerdy science of machine learning, visual recognition, and 3D modeling required to bring these experiences to life. They just work. By 2025, nearly 75% of the global population, and almost all smartphone users, will be frequent AR users. (Snap, 2021) In other words, everyday consumers have elevated expectations for branded AR, which means marketers need to learn the technology, best practices, and begin to prepare for AR’s future today.

AR experiences create an emotional connection and reason to engage with a brand that is more playful and memorable than traditional ads. Interacting with products that have AR experiences leads to a 94% higher conversion rate. (Snap) It doesn’t matter whether the story is entertaining, social, educational, gamified, or utilitarian because AR experiences are immersive and highly personal, they have a greater likelihood of increasing brand relevance and recall.

AR In Your Pocket, In Your Feeds, In Your Browsers

Microsoft’s $3500 Hololens is a powerful AR visor for jaw-dropping mixed-reality experiences, but most smartphones support AR via the operating system, app, and mobile browser. Nowadays, everybody has a device in their pocket they can use to overlay the digital layer on top of the physical world! 

And since the hardware is so ubiquitous, almost every major platform has introduced these magical AR features—including branded filters and environmental effects on Facebook Camera, Instagram, TikTok, and Snapchat. Amazon’s AR feature lets you view thousands of products in your home before you buy them. Pinterest and YouTube have “try on” features.

source: eMarketer

Facebook Newsfeed now has AR ads where you can demo a product in your home or automobile in your driveway. Apple’s Measure app turns your phone’s LiDAR camera into a stunning accurate tape measure. Google Maps now offers a heads-up AR street view, and Google’s mobile browser results will literally project a 3D object in your space, no app required (e.g., try Googling “horse” on mobile and select “View in 3D”).  

Moving Beyond Face Filters in Social AR

Social AR is going to become even more collaborative and well, social, in the coming year. Apple Clips make scanning your face and space and sharing with friends as easy as pressing a button. Spark AR Studio makes building AR experiences as easy as drop-and-drop on Facebook and Instagram. Facebook’s Oculus is incorporating AR into their social VR products. The creators of Pokemon Go are preparing a new walking-style AR game with Nintendo’s Pikmin, and they just released an augmented reality developer kit that has extraordinary multiplayer abilities – intersecting and your real-world friends with digital and AR world content. 

Gamified brand experiences have always been popular with consumers, but social AR will create an opportunity for interplay between a brand and its consumer that will make a social comment or retweet seem trite. 

Apple’s new AirTag smart trackers will be used for more than just finding your lost keys. They feature an ultra-wideband chip that uses AR tech for precise location finding, which means developers will be able to use this chip to identify and map physical objects in independent AR experiences. Imagine embedding that chip in your product so consumers can interact with it both in the real world and in the augmented world. Launched just this month, Snapchat’s new Spectacles feature AR, and are currently only available to developers and creators interested in experimenting on the platform.

Although Snapchat was the first to leverage the iPhone 12 Pro’s upgraded LiDAR Scanner for AR, the iPhone 13 is rumored to feature LiDAR scanners for all models. LiDAR is the same technology self-driving cars use to “see” the road and identify objects. That technology coming to mainstream phones will create even more opportunities for marketers to build magical experiences in the years to come. 

The future of AR is Smart Glasses

Almost every major tech company is working on smart eyeglasses – Apple, Snap, Google, Facebook, Microsoft –  and all of them include some form of AR. And while you may immediately roll your eyes about the second coming of Google Glass, the patent leaks and use cases are increasing by the month. 

Imagine consumers walking down a busy pedestrian mall and “seeing” customized ads and messages projected on blank walls that they can only see with their AR glasses. Or shopping in a grocery store, where the products on shelves advertise their own coupons. Or a branded pop-up experience that features an AR game played with a brand mascot, or a live band concert “hologram” on an otherwise empty stage.

Of course, there will also be pushback against this type of AR adoption. Concerns about privacy, utility, and not necessarily wanting ads wherever you look with your smart glasses are valid. Some off-the-shelf algorithms have difficulty sensing BIPOC faces or are prone to misgendering, so it’s critical branded experiences are inoculated from discrimination. And you can bet AR graffiti will be a thing. But that’s probably a few years out. 

Although it may be a couple years before we’re living the world of Keiichi Matsuda’s, “Hyper-Reality,” there is a high likelihood that every brand will need to have an AR playbook sooner than later. Now is the time to plan ahead. “See” what I mean?

6 Steps to Prepare Your Brand for AR’s Immersive Future:

  1. What products or services do you need 3D mapped today (and tomorrow)? Establish a process for capturing, labeling, and ensuring proper usage of 3D assets the same way you capture 2D and video assets in your content library. You’re going to need them.
  2. Have you ensured your AR experiences are inclusive? Some off-the-shelf algorithms have difficulty sensing BIPOC faces or are prone to mis-gendering. Ensure your branded experience is inoculated from discrimination.
  3. How can you move beyond branded face filters and raffles? Branded face filters and “What _____ Are You?” raffle experiences have become so commonplace on Instagram and Snapchat that they are ignorable. Level it up for 2021!
  4. What AR ad units should you be testing? Explore Facebook and Google’s AR advertising offers and see how they could fit within your marketing mix. Consider a pilot to weigh performance and ROI.
  5. How can you use social media to highlight AR experiences from your website and app? Brands may find that GIFs and short video demos of AR tools perform better than even static images on social.
  6. When does it make sense to bring AR into your experiential footprint? Point of sale displays, at-shelf, and pop-up branded experiences can all benefit from the immersive nature of AR to give a consumer extra information or a reason to stop and engage.


Alexa Glasses: Amazon Echo Frames

Sign up for Greg’s email list here

We’re all going to be wearing smart glasses. Someday. 

Based on patent filings, leaks and acquisitions, it’s evident that some of the biggest tech companies – Apple, Microsoft, Amazon, Snap, and Facebook – are all working on new technology and innovation that will change how we think about wearables, hearables, and an A.I.-assisted lifestyle. Just this week, Mark Zuckerberg presented Facebook’s plans for smart glasses in 2021, with augmented reality glasses coming soon after.

This week Mark Zuckerberg said many companies are taking shortcuts when it comes to building AR glasses by basically showing some heads-up information. He said: “I call that ‘putting an Apple Watch on your face.'” Well, today we can at least get an Alexa assistant on our faces.

Yes, I wore Google Glasses. Yes, I wore Snap Spectacles. Yes, I desperately want a pair of Nreal Light Augmented Reality Glasses I tested at CES. Yes, this is a review of the new Amazon Echo Frames. Yes, I will buy the Facebook glasses and cannot wait for Apple’s rumored glasses to come out. That’s all on brand for me, right?

But if you know me, you know I love experimenting with these near-term innovation products NOT BECAUSE I’m dying for technology on my face. Rather, it’s because I’m fascinated with these small steps leading to the killer, smart, face-mounted device we will all use in the future. 

Think of all the smart phones that existed before the iPhone was introduced in 2007 and changed the paradigm of what a phone would be by 2020 and how our education, work, and lifestyle would revolve around these black mirrors.

So that’s how I view today’s smart glasses experiments… they are baby steps to a paradigm shift I believe is coming. Disclaimers aside, let’s chat about Amazon’s new smart glasses!

Amazon Echo Frames

Amazon’s Echo Frames are the most affordable, effective, and consumer-facing look around the corner of where smart glasses trends are headed. They look good. You can’t tell they are smart unless you’re told. And they are a remarkable step forward for getting our faces out of our phones and wrists in daily life. 

Available to consumers through Amazon’s invite-only Day 1 Editions program, Amazon Echo Frames are pre-priced at $179 (full retail will be $249), plus the cost of prescription lenses (~$100, depending where you order them). So not exactly Zenni prices, but certainly cheaper than Snap Spectacles ($380) and Google Glass ($1500). 

Frames have Alexa built-in, plus Siri or Google Assistant, and rely on a bluetooth connection to your phone for data. Similar to other Eco devices, Alexa is always listening for the ‘wake word,’ which then triggers A.I. assistant to spring into action and give you complete access to Alexa’s wide range of Skills, calls, drop-ins, and more. Or with a simple temple press, access Siri and its native access to your entire iPhone, testing, calls, apps and more. Same with Google Assistant on Android devices.

With four microspeakers and two beamforming microphones embedded in the stems, the audio quality is excellent except in heavy winds.For music, they are a little tinny. But the microphones are excellent for voice. I have to imagine Amazon’s engineers are exploring bone conduction audio for future iterations. But as long as you have the volume set low enough, they are pretty private.

In fact, the other day I was having a discussion with my wife and listening to a podcast at the same time. I’m a super nice husband. And I will not make that mistake again.

No screen! No camera!

The immediate initial questions I get asked about Echo Frames are: 1) Are you looking at me through a screen right now, and 2) Are you recording me? 

No! Frames are hearables in the purest sense. A small light inside the frames tells you when you’ve activated Alexa by voice or the backup stem button, but otherwise there is no visual display to look at or through (these are not augmented reality glasses) and no camera.

As for the audio side of “Are you recording me?” – well, that’s a stickier answer. This is an Amazon product, after all. 

Google Glass missed their PR window to educate consumers about privacy and use of its camera. Snap Spectacles attempted to inoculate privacy concerns by its placement of bright circling lights when they are recording.   

This 1st edition of Echo Frames doesn’t include any forward-facing indicator when the microphones are “listening” or sending audio to your phone, but I assume that’s intentional — the microphones aren’t facing outward and aren’t designed to record.

However, there are valid concerns about how Amazon stores and uses all of its Alexa-based data and recordings. That same scrutiny can be applied to Frames. And like all Echo products, there is a manual “mute” button with red light indicator that it is not listening. 

So after a month wearing Amazon Echo Frames off and on, how am I using them?

  • “Dropping in” on my kids while they’re doing school in their rooms. We have Echo Dots in each room of the house, and it’s pretty great to quickly be able to call them down for lunch or ask how school is going. 
  • Talking on the phone, texting, listening to music and podcasts, and recording Marco Polos – as a supplement to my AirPods.
  • Asking random factoids and information, like how many ounces are in a gallon – as a more intimate supplement to normal Echo requests. 

Of course, this test period is taking place during a global pandemic. So practical use in “normal” times, including during a commute, working from an office, or doing extra curricular activities has yet to be fully flexed. I more or less never leave my highly-connected home, where I can count eight always-listening Echo devices, plus Google Home and Facebook Portal. And my Echo Loop smart ring. So they are a tad bit duplicative in this season of life.

My biggest issues?

Battery life. Man, these batteries are not good. Maybe 3-4 hours battery life, depending on how much you’re using them. There’s nothing like wearing dead smart glasses for hours at a time in the afternoon. So the trade off of Echo Frames not being bulky and obnoxious is you are trading battery life.

You also MUST have your phone around to use them. Just like the Apple Watch’s evolution, you can imagine future iterations of Frames having their own data plan. Leave your phone at home and just head out with your Frames. It will happen.

And lastly, I do think there are valid privacy issues to anticipate with the mainstreaming of hearables, particularly as people become more educated on surveillance and sousveillance as it pertains to audio recordings. Where Google Glass dropped the ball on proactive PR, others need to learn and start working ahead on normalization and privacy controls.

BONUS: Now when my phone rings my laptop, phone, Apple Watch, Echo Loop ring, and Echo Frames glasses all ring! The future is amazing, you guys! I’ll never miss a telemarketer spam call again.

Should you buy Echo Frames?

Probably not. They aren’t necessarily intended for general public consumption. However, they are the highest quality, most affordable, best looking smart glasses I’ve worn. 

In fact, I genuinely like them. It took me a couple weeks to get prescription lenses, and now that I’ve got them I will wear them regularly.

On the other side, I mean… just look at these cool folks at the official Google Glass Meetup at SXSW in 2014…

Group photo from Official Google Glass Explorer Meetup at SXSW 2014

So Echo Frames Edition 1 are not the must-purchase, but they are a step to where we’re headed.

Where are we headed?

 As mentioned in the intro, I’m pretty bullish on smart glasses coming into the mainstream. When? It’s safe to say the next 10 years, although I predict it will be sooner. 

I’m fascinated with the promise and value of smart glasses, including new uses we can only start to imagine, including…

  • Notifications: Screenless and heads-up 
  • Real-time translation: via visual displays and/or audio
  • Smart lenses: dynamically changing prescription, focal length, blue light, and UV protection
  • Anticipatory A.I.: moving beyond notifications, reminders and lookups to offer proactive information, scheduling, and more
  • Standalone data: We could conceivably not need to carry a phone at all by 2030.
  • Augmented Reality: opening the digital layer of our world to its wearer for work, play, and daily use. Of note, Nreal is advancing this quickly.

Will the future look like Keiichi Matsuda’s short film, Hyper Reality (2016), experienced through smart glasses? We’ll find out soon enough. 

If you dig content like this, sign up for Greg’s newsletter here. 

Success! You're on the list.