Audio, Haptics and other changes in the way we interact with technology

SXSW kicks off on the 9^th and I’m looking forward to seeing what particular hits, misses and hypes come up.

But beyond the usual “marketing tech” I think the most interesting areas at the moment are around the way we physically interact with technology and media.

Two of the most innovative areas are Audio and Haptic technologies.

Audio

Audio in interaction design terms has been a long neglected area that is now exploding in terms of possibility and the potential for mainstream exploitation and adoption. The poster-child is obviously Siri on the iPhone 4s or Iris on Android but Ford has been employing Microsoft Sync—which also uses voice control extensively—in its cars for a few years, even in lower cost vehicles aimed at younger drivers. However, it is the use of Audio based interactions in the real or entertainment worlds where the interesting applications apply.

Sonic Notify is an interesting example. It uses Audio detection to deliver tailored and synched media content to people in a store, in front of a TV or at a concert. The media is actually transferred over audio via “the communication of data in the ultra high-frequency inaudible range between any speaker and microphone. The [Sonic Notify] decoding algorithm enables a common process for extracting data from audio using any microphone on any smartphone or tablet.”

If you can cope with the intense American spokes-model-weirdo below there’s an interesting demo:

Media can effectively “piggy back” on TV soundtracks, store PA systems, or radio ads and offer a richer experience without people having to use their own data allowances.

Similar Audio driven interactions are found in the eBay Application which synchs content and goods displayed with the TV programme people are watching – more Retail TV than Social TV or IntoNow.

Haptics

Haptic technology is all about touch. It is incredibly personal and all about giving natural instantly understandable feedback to people. It is when technology starts to make a physical connection with people. It has been described as “doing for the sense of touch what computer graphics does for vision”.

We’re starting to see more high end implementations such Surgical Robots but the idea is leaking into more and more concept products such as the Seabird phone.

Or even concept haptic speakers.

Or potentially something (anything) that could save the Blackberry smart-ish-phone.

But ultimately Haptics is more than waving your hands or body in the air (Kinect) or trying to use a holographic projection or wall mounted Tablet (like Samsung’s CES Innovation award-winning Smart Window) which rapidly leads to a heavy, tired sensation, or “Gorilla Arm” as Human Computer Interface designers call it.

It is about creating a dynamic tactile medium which should be able to tangibly represent almost anything. That might be through a descendent of deformable materials (Shape Memory-alloy or Electroactive Polymers) or a descendent of haptic holography.

It is the Tangible User Interface and will play a big role in the future of mobile devices or even flexible eReader Paper.

So far, so Sci-fi, but could Haptic technology make the mainstream sooner than we think? If there’s anyone who can the technology into real people’s hands – not just those of early adopters – it’s the people at Apple.

The invite to the iPad 3 (or iPad HD?) launch hints that ”We have something you really have to see. And touch” and people are taking this to be a trial for possible use of haptics.

If so it could be “a technology from Senseg, a Finnish startup which has developed a system called E-Sense which appears to give texture to a touchscreen. By using “tixels” generated by electric fields from elements embedded around the screen, it can make areas of the screen feel rough, ridged or rounded – and change those just as the screen pixels can change.” Guardian.

We’ll see soon. It could be hype.

It could be a red herring.

it could be a disappointment like the iPhone 5.

Hopefully it won’t be as under-whelming and over-priced as iAds.

Facebook Privacy improvement

Facebook seems to have made “a few” privacy gaffs lately but it’s nice when something goes right.

The new iPhone App had a some interesting upgrades (not to mention Places) but what I liked was the active reminder of “who you’re broadcasting your Status to”.

At them moment you can select the usual Facebook defaults but I think in future it would great if you can create your own groups – that way you can finally manage your different relationships/identities and split work from home or old uni mates or strange stalkers…

iPhones and the App Economy – what, how and why?

In the spirit of sharing here’s a collection of stats and approaches we’ve been working with for clients.

2010 Digital Trends, Ideas and Technologies (Part 1)

Here is Part 1 (of 2, maybe 2 and a half) of our 2010 Digital Trends, Ideas and Technologies presentation that I finished off over Christmas. It’s based around 4 Themes, which are each broken into 2 areas of focus/exploration:

It is in Beta (or that’s my excuse for a couple of gaps) and draws on a lot of ideas from some interesting people who make the strategy and digital creative world a good place . I’ll be posting the full list of sources here but in the meantime any feedback, ideas or input is gratefully accepted.

————————

Update: The presentation has now been voted onto the homepage and then chosen as a Top Presentation of the Day of Slideshare.net and picked for the homepage of noteandpoint.com. Thank you very much.

Mobile Social Networking and Social Periphery/Ambient Intimacy diagram

Even two years ago many people wouldn’t have predicted that the primary selling point of a mobile handset in its own advertising would not be the hardware, the screen size, calltime or even brand message, but the fact that you can get Facebook (and other feeds) on it.

The current rapid growth of mobile social networks is a symptom of finally moving beyond the endless powerpoint based hype of “social media marketing” to a new realism. Mobile that takes social networking firmly away from sitting behind computers and means we can bring back the lost real social elements of face to face encounters and real world experiences.

Mobile coming together with social networks means that we can physically see these networks actual transformative value. It is one of the many things leading to what I hope is a sense of  ”won’t believe the hype” maturity in our industry (and wider society).

This “always on”, real-time world of Social periphery or ambient intimacy [explored in the diagram above] has been on the cards since the birth of Jyri Engeström’s Jaiku but it has taken easy and seamless mobile integration to really snowball into the mainstream.

Now the challenge is what do we do with it?

Recently I’ve been exploring the idea of brands needing to function as both enabler and filter for people in order to have a role beyond passive loyalty. They not only need a position within a market or category but must also have a clear and simple point of view on the world/culture in which they operate.

This evolving way we deliver and communicate is at the heart of the opportunity for mobile networks and technology providers in light of their move to become the enable and the filter for people’s social peripherial vision and networks.

But what will the work need to look like?

Technology lets the crowd raise-up the things it likes with links and tags and re-posts, and damn the things it doesn’t like with a pointed lack of attention. Old passive message, big idea, objective correlative creative with a big call to action, and series of key frame proof points doesn’t cut it anymore. There is too much noise: now things have to be good enough to share.

But just because something is good enough to share or inherently interesting doesn’t mean it will catch on and spread through networks. The work itself must be implicitly structured for the network (as illustrated by SharedEgg) and stimulate the growth or reshaping of the network, not just have an AddThis or send to a friend button.

• It must contain an idea that can be reprocessed and played with, passed on and owned.
• It needs to be an idea that people can use but not be completely dry and functional.
• It needs to provides more than the (important) phatic entertainment and brand story BUT ALSO vital real world reward and social currency.
• It must allow the crowd to create nodal points* within their part of the network.

Mobile is the tangible, first thing you see in the morning, last thing you check at night, always with you object, that will enable people to pull their networks and these smaller, higher frequency, locally relevant ideas together so that they can create their own digital nodal points or experience their friends’ ones while on the move.

This is why the work we do needs to be good enough to share with everyone, anywhere.

———————————————–

* If we can think of a Nodal Point as a (potentially distributed) collection of content, conversations and links that spread a meme/concept and cause the ideas, experiences and other journeys around it to be reshaped and “dragged” (just like a planet’s mass influences the passage of time around it), then it is a key point in an interactive experience or someone’s digitally enabled life.

Digital in the real world or the internet of things, Augmented Reality, RFID and Mobile

I’ve been doing a series of Augmented Reality meets RFID meets mobile workshops and seminars recently and thought I’d share the slides.

The good thing about this type of work and technology is that it slowly gets us away from the idea of “digital” being a distinct or separate medium. I think the problem is that once again we have allowed a useful encapsulation term – digital, social media, networking, UGC or earned media etc. – that we can use to explain a philosophy to clients to hijack the agenda. We labelled ourselves in the early days as a way of selling our services but the legacy is that we have encouraged people to think of it as a channel.

The new technologies are almost unrecognisable to the old 800×600 microsites that were a product of those cool, first generation “digital new media agencies”. Just like no-one thinks we are operating in a world of “new media” any more, we should stop thinking we are operating in a channel/sector called digital or social media.

It is just the reality of experience and marketing. It is what real people are doing now.

Augmented reality future of mobile search and other AR applications

Back in summer 2008 I wrote a whitepaper for some of our clients about Cloud and Crowd based creativity and the need to break digital out into the real world away from screens and desks. Back then I included an image by the excellent Petit Invention as a concept vision for the future of mobile search.

Technology moves quickly and a couple of months later there was already a working prototype in the form of Wikitude AR Travel Guide. Using the Android OS on the G1 phone…

“users may hold the phone’s camera against a spectacular mountain range and see the names and heights displayed as overlay mapped with the mountains in the camera. Users may look out of an airplane window to see what is down there. Users may walk through a city like Seville, Spain, holding the phone’s camera against a building and Wikitude tells what it is.”

The application shows that augmented reality can go beyond the initial wow factor (and the peering around the back of an A4 print-out pointed at your webcam while trying to see something on your screen) and the Second Life comparisons.

IBM’s Seer Android Beta produced to support it sponsorship of the Wimbledon tennis tournament also shows a commercial application going beyond “marketing fluff”. Using it, visitors can find facilities on ground (locating the nearest restrooms), but more impressively, they can “point the phone at a tennis court, find out the court number and also who’s playing and more crucially, who’s winning”.

Once AR comes to the iPhone properly in an easily accessible form then I think we’ll see a whole new level of usage. The 3G S’s compass combined with Google Latitude and Wikipedia overlays would definitely add up to a powerful and socially changing experience.

Priority Mail’s AR Box Simulator also points at a future for an entertaining application usage but in the meantime here is a collection of 10 augmented reality campaigns that are much more in the fun and novelty area.

2. Further convergence (2009 Trends)

• Phones & Music & PDAs are already one. iPhone Apps store is the steroid-pumped evolution of convergence and personalization. Next step is phones as electronic wallets or banks and becoming the main method of payment. Perhaps even your keys? Google Gphone barcode scanner?

• Mobile networks, handset makers becoming the media and comissioning content rather than the next slickly shallow microsite. Converged print, TV, radio, net business models in a bid to survive the advertising downturn.

• Integrated, internet driven home entertainment, comms and gaming ==> out of kids bedroom.

•  But be realistic…Do you really want your fridge to talk to your PC?

Next… 3. Play is social

Back to…2009 Interactive Trends

1. Clouds and Crowds (2009 Trends)

OK, I know this has been a favourite subject of mine for a while but…

• The cloud is…distributing, accessing and publishing everywhere and sharing with everyone – including and especially yourself on all your devices

• Your machine does very little, it contacts the net and displays information/applications in the most appropriate fashion to the device at hand.

• It is WiFi enabled interactivity broken out into the real world

• Mobile to be dominant platform for connecting to net worldwide (already is in Japan)

• Even voice calls powered by internet & SMS/Texts become IM clients

• More access means more people sharing information, entertainment and functionality

• Stronger connections between people than with brands. Brands must try to facilitate these connections. Show don’t tell.

• Bluring between the product, social element, interactive film, site, app etc.

• The Crowd is…people shaping products/services and trialing them digitally – active engagement & opensource. People using technology to share and take control.

Next…2 Further Convergence

Back to…2009 Interactive Trends

The Cloud – distributed creativity, information and access

The Cloud is about distributing, accessing and publishing everywhere and sharing with everyone – a principle embodied by many a creative project or social media tool or site. So far so Web 2.0. Where it differs is that the audience for accessing, publishing and sharing everything includes you yourself, on any and all your devices – especially mobile ones.

It is this proliferation of places and devices that is in turn contributing to the increased importance and influence of the information itself over the older organising technologies and access methods. We will have control rather than the applications, technologies and devices.

Cloud computing means your machine does very little except contact the internet and display information in the most appropriate fashion to the device and purpose at hand – everything is taken from applications and storage in cyberspace². This new world of the Cloud is the world enabled by WiFi rather than the old cumbersome, expensive copper wires and fibre-optics. It is not a fiction or a future, it is now – but it has happened quite quickly. In 2006 William Gibson was still using WiFi as a “science fiction trope” in his novel Spook Country. It was exotic. Now it is commonplace. In the last 18 months, through competition between the companies like Orange or the Carphone Warehouse, we’ve already seen that access everywhere is real – indeed I can detect 14 wireless networks, not all secure, when I’m sitting on my sofa at home.

So interactive has finally broken out and is in the real world, not just in the window that sits on your desk tied down by wires. But as we begin to interact in different places, we start interacting in different ways. Now interactive in reality is about the new ways of manipulating – with the emphasis on manipulate – information, services and entertainment.

So what is the reality? How does the Cloud intersect with the real world? How is it being implemented right now? Well, first we need a quick caveat from the world of ET and Mr Spielberg.

Eliot: He’s a man from outer space and we’re taking him to his spaceship.
Greg: Can’t he just beam up?
Eliot: This is reality, Greg.

“This is reality, Greg”, implies that reality is about limits, that the fantastic doesn’t happen, that the real world is dull. Interaction design and digital technology frequently confront this issue. In fact Digital in the real world is both fantastic and dull, sometimes at the same time. At the very least it starts as fantastic, but soon becomes the norm – it alters the way we see things and relate to each other.

One click sharing is an accepted norm but the places we can share with and our identities within them are multiplying. The Cloud aims to make the experience seamless and manageable.

Digital in the world away from the desktop is not an overnight jump to flights of fantasy – to Minority Report, Tricorders and jetpacks – but equally it is not totally mundane. Designers have been exploring it from the beginning of Interaction design, with greater and lesser degrees of success.

Bill Gavers’ Emotional Key Table and Picture Frame, MIT Tangible Media Group’s Prof Hiroshi Ishii’s original TUI diagram. MIT Tangible Media Group focuses on seamless couplings between physicality and virtuality.

From emotional tables that respond to the force with which you throw your keys down at the end of the day and recommend the appropriate drink, to Ars Electronica 2003’s Teleklettergarten, a building sized “keyboard” that enabled an anonymous collective of programmers and climbers to transform the “largely cerebral act of programming into a physical experience”, the move from Graphical User Interface to Tangible User Interface offers advantages for marketers beyond entertainment and being a replacement for a mouse – it changes the way people relate to their data and introduces a new playfulness.

We already have well-integrated touch technology on consumer electronics, from Apple’s multitouch devices (iTouch, MacBook Pro, the new Mac operating system), games consoles (Nintendo DS and Wii) and now even PCs and notebooks. HP has now released the TouchSmart, a desktop system with a touch interface, and in July 2008 Dell released a module to turn its Latitude XT Tablet computer into a full-blown touch-operated system. All these devices have been designed for use on the move and in the Cloud.

Aerie Networks vision of the mobile internet and broadband - circa 2002

When you are not sitting at a desk, when you are moving in the Cloud, then the keyboard, large screen and mouse are not appropriate. Interacting with the Cloud in the real world these days means using devices like the iPhone or Samsung’s Omnia. It is not a hologram in the palm of your hand, or the CG silver surfer of the BT Cellnet’s brave new world of WAP. It is Blackberry in Orange Shops and Argos selling Laptops for the first time because they see them as a volume product; even Acer has released its Aspire One “netbook”. The Cloud is Google’s open-source web-based software and Apple’s MobileMe service – a service that is “not up to Apple’s usual standard” but still manages to achieve a 700% increase in traffic and users.

People have been proclaiming the year of mobile since the afore-mentioned ads of the early 21st century but the real uptake has happened quietly in the background. When ICM Research finds that 45% of UK mobile phone owners browse the web daily this shows that people are already interacting with the Cloud, and their number will only increase now that network operators have finally enabled ‘all you can eat’ data plans. Indeed, network operators are currently investing heavily in Cloud based services/applications and even mobile TV as future revenue streams.³

By 2013 Jupiter Research estimates that 2.1 billion mobile users will be using handsets to pay for goods downloaded directly to their phones – from ringtones, music, and games to gig tickets and TV shows. So the real future of mobile content lies in useful applications or assets that can embody social “bragging rights”.

As the semantic web evolves and becomes a digital reality then it will cross over into the real world via people’s need to interact with the Cloud. Geolocation services such as Yahoo’s FireEagle, a service that bridges the gap between traditional and mobile computing, or The Geode project for Firefox already aim to meet this need, and as they cross over into social networks to become “social compasses” they will only get more useful and more widely adopted. Only then will the potential for real world contextual creative opportunities combined with the newly accepted manipulative technologies come together to make reality more like fantasy. Then creative can start to challenge the limits of “reality, Greg”.

Continued…The Crowd – Cloud linked collaborative communities, creativity and tools

^2. It is even a possible future for Nicholas Negroponte’s XO laptop – the OLPC project – that aims to bridge the digital divide and enable the real Crowd. The whole OLPC project could effectively live in the Cloud.
^3. It will take until analogue switch off in 2012 to leave room for the UHF frequency needed, but in the States the average subscription length is 2.5 months – people leaving as soon as they join – so this passive Cloud model may need deeper examining.