GUEST COLUMN: Social Augmented Reality; Connecting People, Content & Context In New Worlds » mobilegroove

Editor’s note: Interest in Augmented Reality (AR) is on the upswing as companies harness it to super-charge their brand or make their offer more compelling. Malaysia Airlines has developed a mobile AR app to promote special flights; the magazine Time Out New York is using mobile AR to let smartphone users call up a singing chorus from Staten Island; and department stores are experimenting with AR to take the hassle out of trying clothes. (Consumers try on clothes by overlaying the clothes over an image of themselves.) Over in Asia, a region light years ahead in the adoption and acceptance of mobile AR, Japan’s mobile operator NTT DoCoMo is set to launch “chokkan nabi”, (stands for intuitive navigation), in September. The service uses mobile AR to help people find their way around megacities such as Tokyo and Osaka. While this progress is encouraging, Christine Perey, an independent consultant and industry analyst focusing on mobile AR, suggests that Social AR – the intersection of mobile social networking and AR apps – is the one to watch.

***

Mobile Augmented Reality (AR) applications started out allowing us to view useful or factual information associated with objects and places in the world around us. The classic example of this type of mobile AR is the Wikitude application (by Mobilizy), which associates georeferenced Wikipedia entries with a point of interest and presents the same Wikipedia text in a box overlayed on the live video window. Using platforms that empower individuals to post their thoughts or comments on objects and places in the real world, we now have the potential to be immersed in Social AR.

As I show in this column, social AR is a natural next step in the evolution of mobile AR and social networking.

We started out with social networks and microblogging services that allowed us to post our ideas and reactions to places and events. And then, with GPS in mobile handsets, we moved to services that allowed us to geolocate our cyber-graffiti, adding new meaning and context to the breadcrumb trail we left behind.

Then came the “see through” video streaming we know from using an Android handset or the iPhone4. This made it possible for us to “see” the same tags in the context of the real world and – more importantly – share our comments with anyone who cares to lift up their device, switch on their AR application and take a look.

Now we’re moving full steam into a new scenario. Imagine a thought or comment about an object or person which is not fixed in space, but which has unique features—a face, a pet, a product—which can be recognized from an image (using image recognition software) and tied or “stuck” to the target when it appears on your field of view. We are very nearly there and the implications for people and businesses will be profound.

TECTONIC SHIFTS

But before we talk about where this is going, it’s important to trace the milestones that mark our path to this point.

Dan Melinger, one of the founders of Kamida, a NY-based startup, was working on geo-located social media as early as May 2004. Later that year, Dan presented his vision via this poster at Ubicomp 2004  , and introduced the use of “sticky Shadows” as a way to bookmark our lives and literally leave our mark everywhere. The company released its commercial service, called Socialight, in December 2006.

Today the company is still pushing the envelope. In fact, a review of its clients (a list that includes Ford in Europe) tells us there is a lot of mileage in geo-tagged social media. Socialight’s platform has an open API, permitting developers to infinitely extend features (they can add a new symbol to the list of emoticons, tint/color different personal comments or objects) and deepen the user’s social experience with a brand or a place. This opens up Social AR to a larger audience, allowing public communities to build their own custom applications with filters.

In 2007 we saw photos show up on a live map, a development that gave geo-tagged mobile social applications new importance and impact. Back then Marc Davis was running Yahoo! Research Berkeley, and his prototype project combined geo-tagging with images. I got a demo of the maps-based view of comments and Flickr photos (only supported with Yahoo! ID users) when our paths crossed at an Informa Mobile Social Networking event in London. Back then the processor limitations of the handsets  and lack of an integrated compass meant that overlaying those same images on a live video stream could not be implemented as a feature for mainstream users. But it was on the roadmap!

A year later, at ISMAR 08 (International Symposium on Mixed and Augmented Reality), Sinem Guven and a group of researchers from IBM’s T.J. Watson Research Center, demonstrated a prototype mobile AR application aimed at transforming the shopping experience. And, in my view, it did.

In a nutshell, the application allows a consumer to scan the cover of a book or a barcode to access more information about the book from Amazon – with a personal touch. Magically, an animated figure appears (overlayed on the page of the book) and offers opinions and reviews about the book. It was the first sign that retail and AR were coming together to impact our lives.

TWITTER ENTERS THE PICTURE

Fast forward to 2009 and the arrival of the iPhone 3GS and smartphones with compass (and GPS from other handset makers) catapults AR to a new level.

By the fall of 2009 we started seeing a new wave of applications that combined AR and real-time or near real-time social media. Among these were path-breaking applications that harnessed the Twitter API to bring tweets to eye level.

In July 2009, Michael Zoellner, a designer working on AR applications at Fraunhofer IGD released TwittAround for Android handsets. Soon after an application was released for the iPhone

Not long after, Andrej Solar developed TweetAR for Android. But, for people who don’t live in highly concentrated urban environment, tags appear far apart from one another on the spatial scale. So, viewing a simple list view of your Tweets remains more convenient. The situation changes, however, when we find ourselves in a city center. One of the most densely populated areas on the planet is Tokyo, the birthplace of Tonchidot, the winner of a TechCrunch 2008 award and provider of a social AR platform and service, Sekai Camera for Android and iPhone.

Another leap forward for Social AR came in the fall of 2009, when Metaio, one of the players in industrial strength Augmented Reality, released the junaio application for iPhone 3GS. Getting social media into this platform was not incredibly easy, but it was possible. Users could follow a series of steps using their computer or the mobile phone app to tag the real world. What’s more, users could post personal comments or “insert” a photo or a 3-D object, which would then float in space over the place or object. Since then, junaio continues to improve and social AR is a major feature of some of the most popular channels on the service.

Around the same time, the design innovation company The Astonishing Tribe, aka TAT, upped the ante with an application that allowed users to tag themselves for others to see.

The concept of Augmented ID – the result of a collaboration with Polar Rose, a company specialized in facial image recognition and head tracking – brings a new dimension to social networking allowing users to choose and customize the profile they show to the outside world. In this video (click image above), we have the example of a user who links his image with suitable social media links, such as his Facebook page, in preparation for a night on the town.

The companies followed up on this application with Recognizer, which was shown during Mobile World Congress 2010.   Unfortunately, due to lack of a clear business model and concerns about privacy and lack of social acceptance, the service is not commercially available.

LAYAR’S IMPACT

Social AR is here to stay. But where is this genre of AR applications heading?

We got a glimpse of this future on June 18, 2010, when Layar, the Dutch provider of the popular Layar mobile AR browser, announced the development of animated 3D objects called Floaticons. The Layar browser provides information in the video camera field of view, organized into various “layers” such as entertainment, eating out, games, real estate, retail, social networks, tourism, transportation and weather.

Floaticons put people back into the “total information” picture, giving users of the AR browser the ability to post their thoughts in space with more than just a bubble or a photo.

Although Layar has not yet released them, the company website highlights several series of Floaticons, including Floatos Locos, Robots 3000, Classic Monsters and Sparkle & Shine.

These animated 3-D creatures are part of a dead-serious business strategy. As Layar puts it: “Floaticons are Layar’s first step into the business of Digital Goods.”

To ensure that only relevant messages remain visible over time, Layar limits the “lifespan” of a Floaticon message. The health of a Floaticon can be prolonged by feeding it a digital cookie. And, since Floaticons are tightly connected with the in-application micropayment system, it looks as if there will be a business model for Social AR!

Identifying a viable business model is crucial to the continued investment in mobile AR services in general and social AR in particular.

In my view, having end-users pay—directly or indirectly—for Social AR is a much cleaner option than advertising or sponsorship. Consumers are nervous about seeing too much advertising super-imposed over their physical world, and advertisers are jumpy about having their brands associated with user-generated content that isn’t tightly censored or manually filtered/approved.

User-controlled options do more than build brand; they also add to credibility of the social AR content because it is more likely that, when users pay for it, only the best or highest value content is seen. It’s a simple voting mechanism. If a lot of users benefit from a post in the AR view, and give it their thumbs-up, then perhaps credits go to the source of the Floaticon and the cookie monster gets fed!

OPEN FOR BUSINESS

In the coming months, with the improvement of visual recognition algorithms, we can expect new mobile AR applications to support and super-charge retail, making good on the user experiences IBM promised and prototyped several years ago.

Take the recent moves made by both Metaio and Layar to integrate their client applications and backend systems with image recognition technology provided by Zurich-based Kooaba.  It’s clear that shopping – already a social experience – is about to be transformed by Social AR.

As Metaio CTO Peter Meier put it in a recent interview with VentureBeat, we already have the object of desire in the field of view, and we can add on user reviews. The next logical step is to make it possible for the user of an AR application to purchase exactly what is in front of them through the AR payment system.

So, why not just buy it outside the AR service?

It’s easy to imagine incentives to both merchants and users to engage in this. It’s called Mobile Social AR Shopping and I saw it coming in 2009 when I wrote this post.

But we’re not there yet. From a user experience point of view, Social AR could make the screen very, crowded, to the point of being chaotic. So, filters will need to be developed and implemented that allow users to select their preferences and manage the noisy environment in urban centers, where people have tagged – well – everything.

Social AR will also have to address some thorny issues such as privacy management. Currently, the technology is so new and the number of social AR users is so small that we can’t point to a case in the courts. But it’s not difficult to imagine a situation in which privacy management in AR becomes as loaded and troublesome as it is in Facebook today.

But there are ways to turn a pending problem into a golden opportunity. (The topic is sure to be of interest to the participants of the upcoming ISMAR 2010 workshop. The Mobile AR Summit focuses on Mobile AR Commercialization strategies.)

One solution for managing social media sharing policies across federated networks, which the W3C Social Web Incubator Group is examining is the use of a third-party “privacy management service provider” to manage and protect an individual’s social media profile across the Social Web. Look for services like this to come on the market next year as distributed social media management tools, such as the One Social Web project from Vodafone, emerge.

Until then, enjoy a walk around and the chance to view the thoughts of your friends and family floating freely around a family pet or restaurant near you!
* * *
Christine Perey is an independent consultant and industry analyst focusing on mobile Augmented Reality, mobile social networking and smart spaces/intelligent objects. All these domains are leveraging Cloud Computing combined with small, ubiquitous end points which sense and process information about their environment for users. Christine provides business development/sales, management consulting and custom market research services to companies seeking to build upon these domains of expertise for the purpose of improving quality of life of end users worldwide.
You can contact Christine via email (cperey@perey.com).

This entry was posted on Tuesday, August 3rd, 2010 at 6:42 pm and is filed under Guest columns, Mobile Marketing, Mobile Social Media. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.