Everyone needs a hearing aid and glasses – PART 1

Well, maybe not glasses. It might be contact lenses, a layer on the retina or maybe something else. It’s not unlikely that people in just ten years will be wearing commercial products improving their innate senses. By using technology to enhance and add new features to the world that we see, hear or sense in any other way we will access an augmented reality (AR) of the world. And actually, it’s already here, at least there are some toddler stage products that are exploring visual- and audio AR.

We’ll take this in two parts – first, a quick overview of today’s scene, then we’ll set the stage for the hypothesis of the future.

Part 1 – AR today

  • AR for your eyes
  • AR for your ears
  • Making AR accessible to developers

Part 2 – AR in the future

  • Hypothesis of the future
  • Further development of AR
  • Applications of AR
  • Relationships
  • Augmenting the senses

Daniil Kuželev

AR for your eyes

AR vs VR?

Usually when we speak about AR we mean augmented visual reality as opposed to augmented [any other sense] reality. AR is similar to VR (Virtual Reality). The difference is that VR replaces the world we see whereas AR just adds virtual layers to the physical world that we see. Imagine that you put a virtual cup of coffee on a real table (looking through the camera in your phone); that is AR. It looks like there is a cup of coffee on the table but it’s just virtual. If the cup, the table and everything else is virtual; then it’s VR.

AR is here to stay!

It can be argued that AR has existed since the early 90’s and it has certainly existed for mobile phones since 2008. Today the technology is more refined and more accessible to developers, something I will come back to. The tech giants Apple, Google and Microsoft are heavily invested in AR. Microsoft surprised the world by showing that they are capable of exciting innovation when they presented HoloLens a couple of years ago. Google took a shot with their Google Glass in 2013 which was both hyped and later criticised. The Google Glass team have now pivoted its application to other segments. Apple has released AR technology to developers.

AR is definitely here to stay. And we have seen nothing yet!

AR for your ears

Hear your friend in a bar

In the past few years we have also seen some new interesting hearing aid products that help wearers understand speech in difficult listening situations better than people with normal hearing. One example of earbuds that close the distance between hearing aid products and headphones is the Here One earbuds. The idea with Here One is not only to provide wireless earbuds for listening to music or other audio but it also offers ways on how to augment the sound around you, i.e. AR for your ears. Here One is promising but falls short in battery life and has connection issues. Today there are several headphones that are able to cancel out a lot of the background noise. 

Hear in 3D, not only in stereo

Another technique to augment sound is to use localisation which will help the wearer tell the direction of the sound. So far, it’s mostly used in hearing aids although Here One also has this feature. If you are interested in trying out how localisation sounds you can listen to binaural recordings where you will not only hear if the sound is in stereo but you will also hear exactly where the sound is allocated in the 3D space around you. Here is one example with 3D sound. Just use your normal headphones and remember to close your eyes for full effect!

 

David Grandmougin

Making AR accessible to developers

“Developers! Developers! Developers!”

The quote above is from a classical speech made by Steve Ballmer nearly two decades ago. To my understanding the speech was about what Microsoft employees could do to help developers be more productive. Apple, Google and Microsoft all have to attract developers to their respective platform. No developers – no apps. No apps – no customers. In order to attract developers the platform has to be exciting to work with. When Apple and Google releases their respective SDKs, ARKit and ARCore, it makes those platforms more exciting for developers because it’s easier to develop what the customers want [updated 2017-09-06].

AR for existing phones

Apple recently released their ARKit for developers. ARKit simplifies the development significantly of AR apps and it will probably help boost the exploration of different applications for AR (although several other solutions for developers already exists). With ARKit Apple takes ownership of the difficult technology required to understand a 2D image and convert it to objects in a 3D space. Here you have some early experiments from developers.

AR requiring extra hardware

Google has developed their own approach to AR in the Tango platform. Google Tango uses another approach to AR and their solution uses additional sensors in order to better understand its environment. It’s likely that Tango is more precise than Apple’s ARKit but it’s not yet confirmed. Google has also created a new technology called Google Lens. Google Lens will help you understand what you see and help you take action. In the Google I/O developer conference Google presented a few applications of this technology. For instance, they let the camera see a flower and the phone showed what type of flower it was and you had the possibility to order the flower. Since Google Tango requires special hardware whereas Apple ARKit can be used by anyone with an iPhone 6s or later running iOS 11 it’s likely that Google Tango will not spread in the same pace. In practice millions of people will immediately be able to run apps using ARKit when iOS 11 arrives this fall. 

Microsoft have their own approach to AR with the Hololens Developer Kit. This platform is very capable but it’s still unclear when it will be more accessible for consumers. You can read a comparison of these different platforms here.

Summary

There are different kinds of AR, although we mostly talk about augmented reality from a visual point of view. AR is already here and have been here for phones since 2008. We can safely say though that it’s still in its infancy. In part 2 we will be looking at the future of AR and why it matters and where it can take us.

Continue to PART 2 ->

Leave a Reply