Technological Innovations in our Evolving World

1. Be My Eyes

The first of four sessions on technology was presented by Hans Jørgen Wiberg the founder of Be My Eyes, from Denmark.

Be My Eyes is a free app for smartphones that connects blind and low-vision people with sighted volunteers.  A live video connection between the blind person and a random volunteer means that visual assistance is available 24/7 in any language.  There are two buttons.  The first calls the first available volunteer and the second is used for specialised help.

Skype or Facetime requires you to call a specific person.  The benefit of a blind or low-vision person using Be My Eyes is that a notification is sent out to ten volunteers speaking the same language, who are in a time zone where it is daytime.  So even if a blind person needs assistance in the middle of the night, they have access to volunteers who are ready to assist.  A volunteer can choose whether it is convenient to accept the call, and the first volunteer to accept the call is connected to the person needing visual assistance.

Be My Eyes started four years ago, in January 2015.  It already has over 120,000 blind users and more than two million sighted volunteers.  It is available in over 150 countries and over 185 languages. 

A slide showing an illustration of one week of calls on a world map was covered with international connections, illustrating that many blind people need help outside of their daytime.

One lady was at a badminton match where her son was playing.  She wanted the match described to her.  She wasn’t able to tell the volunteer which one was her son by his appearance, but when he shouted something the lady was able to say “Oh, that’s him yelling now” and the volunteer then knew which one was her son.

Other uses include:

• The colour of clothes, and make up. It makes it a very personal experience for blind person and volunteer.
• Mail arriving in the post so it can be dealt with rather than having to wait for a sighted person to arrive at your house.
• Kitchen stuff, what it is, and how to cook it.
• Navigating surroundings, such as construction work, although it should not be used to help a user cross a busy road, as a two-second delay could make that unsafe.

Hans showed a video of an actual help situation.  A man was in the process of updating his computer.  The screen reader stops talking and the man needed to know what was happening.  It didn’t matter that the volunteer didn’t know much about computer updates.  The man knew what he was doing, he just needed the assistance of the volunteer to read him the messages on the screen.

Another unusual example was a Japanese toilet which had a variety of functions, such as variable water pressure, hot air, warm seat controls, etc.  The user needed to know what is going to happen to him when he pressed the button. 

Hans explained that Be My Eyes is totally free because they want to help people in low-income circumstances.  They have partnered up with Microsoft and Google, and those organisations pay a subscription to Be My Eyes.

The second button on the app gives the user access to specialised help.  It enables a user to call the support desk of Microsoft or Google directly and they get the video connection.  Instead of the helpdesk asking the user to describe what’s on the screen when help is needed, the support desk can view the screen via the video link.  That helps the support desk operators resolve issues more efficiently, and they can experience their own products from the perspective of the user.

Social media has played a big part in the rapid growth of Be My Eyes.  Volunteers are so excited about helping someone they often write about it on social media.  In Brazil, a volunteer helped someone, they received 68,000 ‘Likes’, the post was shared more than 33,000 times on Facebook.  Be My Eyes now has almost 15,000 volunteers in Brazil.

Use of Be My Eyes in India is picking up really fast, and there are quite a few users in West Africa.  Hans is hopeful that they can make a real difference there as their use of smartphones increases.

Hans shared a story about meeting Stevie Wonder at a San Diego Conference and was delighted to hear that Stevie Wonder had heard about the app.  He probably doesn’t need to use it because he has so many people around him, but wouldn’t it be cool to think that there is a chance you could help Stevie Wonder.

Every year the American news publication ‘Foreign Policy’ puts out a list of ‘100 Global Thinkers’.  In 2015 the list included the Pope, and President Putin.  Hans Jørgen Wiberg was on the list too. 

Hans was amused by this and remarked:

“It’s funny to be on the same list as the Pope.  None of my school teachers expected me to be on that list.  And, interestingly, none of my teachers ever got on the list!”

2. SUNU Band

Fernando Albertorio is the co-founder of SUNU Band, a device worn on the wrist which enables people who are blind or have low vision to travel with confidence, navigate, and connect to the world around them.

Fernando began his presentation by reflecting on the huge advances in technology that have taken place since he was a child growing up in Puerto Rico, Latin America.  With low vision albinism his eye sight was not good outside.  As a child he had the choice of a giant CCTV or a magnifier.  Nowadays we have powerful computers in our pockets.

In Latin America the percentage of people with guide dogs is very low, and access is tricky.  There is a high incidence of sight loss due to diabetes and other conditions.  Fernando explained that he created SUNU to empower independence.  It enhances and is complementary to other aids such as a white cane and guide dog.  It’s a technology that can take you further, where a cane may not reach, or where a guide dog needs a little extra help.

The SUNU band project started with observations of girls at different stages and grades at a community school.   The older girls were more sedentary, yet the younger ones were fearless, and were running around.  Even with the use of a guide dog or white cane, there are things that you may miss which can cause accidents to the upper body.  Running into signposts or tree branches can be frustrating and dangerous.  The children took to the SUNU band prototypes really fast, and he obtained funding to develop the product.

Fernando explained that the SUNU band lets you access information about where you are.  It operates like a smartwatch, with a sonar sensor.  It detects obstacles and relays haptic feedback vibration.  The more pulses you feel on your wrist, the closer you are to a signpost.  So you can ‘feel’ the environment around you.  Some guide dog users tell him that they love using it when their guide dog tries to jump the queue.  It enables them to move along at the same pace as people in front.

Other users go hiking and running.  It provides extra awareness and feedback as well as being a smart navigation aid powered by Google Maps.  It can tell you which way you are walking, like a compass.  Fernando demonstrated by pointing his arm and the verbal command said “north” and then “south west”.  It’s really useful if you want to find Starbucks! 

If there is a store you want to go to that is close by, you can follow the vibration path to get to your destination.  Free software updates are available through the mobile app.  The app allows you to adjust the range and sensitivity.

SUNU work with corporate sponsors to provide subsidies, so the cost is US$299.

SUNU’s mission is to continue to empower independence through wearable technologies.

Accessible Heathrow

Roberto Castiglioni is the Chair of Heathrow Access Advisory Group (HAAG) in the UK.

80m passengers travel through London Heathrow Airport every year.   1.4m persons with disabilities request assistance, hence the need to look for solutions which allow people to be independent.

Roberto explained that Heathrow Airport received a ‘Poor’ rating in the 2017 Quality Standards Report for the Civil Aviation Authority.  This prompted the setting up of the HAAG.  The “Accessible Heathrow” project was created to completely overhaul services and facilities available to people with disabilities and vulnerable passengers.

Engaging with individuals during open days helped the advisory group to understand whether a solution was viable.

An Experience and Engage Event was held on 11 October 2018, World Sight Day. The journey and services for visually impaired people at Terminal 2 were showcased, and visually impaired volunteers took part in a trial of the AIRA software.  By using a smartphone app (or wearing a pair of smart glasses), they were connected to a trained professional agent in another location who provided remote assistance.  The trial was a great success and everyone who attended gave positive feedback.  Deputy Manager of Accessibility for British Airways, John Fishwick, said it was the first time in two years that he was able to travel through the airport independently. 

The AIRA product was launched to everyone at the beginning of December with the cost being absorbed by Heathrow.  The app has made a real difference for passengers being able to navigate independently and safely. 

They also tried a system called Room Mate, which was less successful.  It’s also called the talking toilet, and the layout of the toilet is described vocally as you enter the accessible toilet.  Although visually impaired people benefited, a person with Alzheimer’s became frightened because suddenly the toilet was talking to them!  A child with autism had a meltdown – so that’s not a solution.

Another project involves retraining all agents based on feedback on how vulnerable people with visual impairments feel when they are left alone at the gate.  If an agent escorts them and says “I will be back in 20 mins”, those 20 minutes turn into two hours in the mind of the visually impaired person, and creates unnecessary stress.  Procedures are being revised to enable the agent to stay with the person until they board the aircraft.

Another development is for airside dog toilets.  Roberto explained that when he met Judith Jones and David Adams at the European Disability Forum they were very vocal about the need for dog relief areas at airports.  Dog relief areas are necessary for connecting flights and also when there are flight delays too. 

In European legislation there is no guidance about dog relief areas so they referred to the US guidance on how to build Service Animal Relief Areas (SARAs).  Roberto sought justification for the financial investment which led to the conclusion in July 2018 that the HAAG recommends SARAs airside in all newly built terminals and all existing airside terminals undergoing major infrastructural renovation.  Major terminal innovations are meant to take place in older terminals T3 and T4 and within the scope we should see the creation of SARAs.

Roberto concluded by saying that this is the outcome of great collaboration between people who like to get things done.  It’s about driving change.

4. Argus II Retinal Prosthesis System

Amanda Hayhurst from Second Sight Medical Products, UK, fascinated the delegates with her presentation about the Retinal Implant called the Argus II.

She explained that it doesn’t replace the eye but the system uses an implant about half of the size of a child’s fingernail.  60 electrodes are planted on top of a retina.

The user wears a standard pair of Oakley sunglasses, with a small camera in the centre, connected to small computer (the size of a packet of cigarettes) which can be carried in a handbag or pocket.

The external components work with the implant in the eye. 

It is currently only available for patients with end stage Retinitis Pigmentosa and where the condition has led to total blindness.  The retinal implant creates light perception only and creates the ability to re-connect with shapes and objects

The implant is tiny and delicate.  It is inserted during a three hour operation which is no more risky than cataract operation. Two to three weeks after surgery the user receives vision training such as identifying whether large lines on a screen are vertical or horizontal.

Amanda explained that a camera connects with the implant in the eye to detect objects like a doorway.  Only a small section of the room can be seen, the size of an A4 piece of paper held at arm’s length.  That’s the window of light.  As that’s all a patient can see, they have to move their head to scan in order to see something as big as a doorway.  A patient can do this with their eyes closed.  They need good posture to be able to use it effectively.

The implant delivers a series of pixelated flashes of light, ranging from 4 seconds to less than a second ranges, to illuminate the room.  It does not give colour vision.  Patients learn to interpret these visual patterns. 

There are 350 users worldwide, mostly in the US but increasing in Europe.  Only one patient has had the implant taken out.  A study in Anglia Ruskin University in Cambridge using five years of data showed that nearly 80% of users could cross the road alone safely using the system.  Some could read very large letters and short sentences.  In a FLORA study (Functional Low Vision Observer Rated Assessment) users rated the implant as either neutral or positive.  Others said they had not used it.  Most were very positive:  “It’s a wonderful feeling to see something, as little as it is.”

Amanda recounted a moving story from the US where a gentleman called Larry, in his 60s, cried because he hadn’t seen his wife’s face for 20 years.  He could see the outline of her face, but not the detail.

A long cane user and guide dog owner, called Jeroen, navigated his way across a beach in Cornwall containing about a thousand rocks.  Jeroen was able to point to boats on the horizon and saw St Michael’s Mount.

Integrating Argus II with Guide Dogs

A UK trial looked at the impact of using an Argus II retinal implant alongside having a guide dog.  Kim, one of the seven people on the UK trial, with dog Harley met Amanda about five years ago.  At that time Kim said that she wasn’t using the implant very much because she had a guide dog and she knew where things were at home.  Kim is now using her guide dog Harley and Argus II together and says that it has really changed her life now that she is using it all the time.

Amanda explained some considerations for using Argus II alongside a guide dog:

• As the guide dog user has more control, the way dogs are trained alongside this technology needs to be more flexible.
• It may involve new instructions to get the dog to wait to give time for you to use the head scan.
• It may be necessary to slow the dog down to integrate with the guide dog owner’s own information.
• The dog needs to have a more flexible temperament, in order to respond to corrections from owners.
• It’s better if the user is an Argus II user before they get a guide dog so that the dog is specifically trained alongside having artificial vision.
• Argus II can be slow to react, especially for fast walkers.

Improvements and software upgrades in the future will be able to be made to the external camera rather than the implant.  It is hoped that the developments can include colour vision and depth perception.

Technology Innovations – Delegate Questions

After the four technology innovation presentations Sean Dilley, journalist, EGDF Board Member and representative of Light Into Europe, Romania facilitated enthusiastic questions from delegates.   Here are some of the issues discussed:

Isabel Holdsworth wondered whether the promise of technology would lead to false hope, bearing in mind that Argus II isn’t suitable for every blind person.

Hans Jørgen Wiberg used the example of driverless cars to say that it is still some years out before we have them.  When the trains are driving themselves, then we are close to having driverless cars.

Roberto Castiglioni said that sometimes our hopes are far-fetched but the biggest mistake we can make is to stop hoping.  That’s our future.

Assistance Dogs entering the UK

Jesús Ángel Méndez Pas asked whether it was true that there are additional requirements for assistance dogs entering the UK via a London airport that are not applicable for other airports, as stated on the City of London website.

It was felt that this related to the normal entry requirements for all UK airports (pet passport, worming between five days and 24 hours before arrival, a valid rabies injection, and certification that the dog is a properly trained assistance dog).  However, entry into the UK through the London group of airports happens to be monitored by the Animal Reception Centre (ARC) and their administration requirements are listed on the City of London website.

It is the job of the airlines to let the ARC know that a dog is on its way to the UK.  David Adams’s preferred approach is to contact the ARC as he boards the plane as his experience is that this speeds things up on arrival.

Future Technology Advancements

Roxana Van Mourik asked whether the SUNU band gives vibration cues for turning left and right.  Fernando Albertorio said that right now it provides audio instructions but they are currently doing research into how to improve the haptic technology, specifically to provide cues.

On a general debate about differences between machine learning technology and human involvement, there was broad agreement about the importance of individual needs and people having choices of different solutions.  Fernando Albertorio said that people will continue to have a role in training the machine and saw opportunities wider than navigation – what if you could get a description of the historical place that you have navigated to?

David Adams asked about possible advances of the Argus II system, referring to the advances that have occurred since the introduction of basic hearing aids.

Amanda Heyhurst said that work is already being done to advance the 60 electrodes array to the chip, and field of use.  The wet environment of the eye is the main challenge.  Another product called ‘Orion 1’ is in the initial clinical trial stage.  It also requires the user to have a visual memory.  It is a cortical implant, so bypasses the eye and connects to the visual cortex in the brain.

Technology Costs

Sean Dilley referred to the reducing cost of technology, using the JAWS screen reading software as an example, and asked whether the price of being blind is coming down.

Hans Jørgen Wiberg said the only way Be My Eyes is able to be a free app is that the volunteers are giving their services for free.

Amanda Heyhurst accepted that with a price tag of €150,000, Argus II is hugely expensive, but it’s cutting-edge technology.  The company is not profit making but is funded by shareholders who want to help visually impaired people to be able to see.  She added that they don’t want the patient to bear the cost, which is why they are working with governments to try to get them to fund it.  In some countries Argus II is available on the national healthcare scheme.

Fernando Albertorio has seen the cost of electronics coming down.  In respect of the SUNU band, it is built into the company’s social mission to make it affordable.

Roberto Castiglione added that creating efficiencies, and curbing the need for human interaction will lead to a business case to invest in improvements.