Visual disability Low vision 2015 Estimated blind people 2020 - PowerPoint PPT Presentation

Visual disability Low vision 2015 Estimated blind people 2020 Visually impaired 285 M Blind 54 M Blind 39 M Global data souce: WHO, IBU

See the world through the eyes of a visually impaired person

Normal vision

Cataract

Glaucoma

Macular degeneration

Diabetic Retinopathy

Complete blindness

One goal: independence

Text recognition

Object recognition

Mobility assistance

Scene and photos description

Face recognition

Possible approaches Smartphone Single Stereo Stereo Stereo App Camera Camera CPU Camera FPGA Camera GPU Day-lasting battery OTA Upgrades Real time performance (offline) Obstacle perception

How Horus works 1 Externalinput identification Image are transferred 3 to the computing unit Cameras 2 aquire images 6 Sound output Audio is transferred back 5 to the headset 4 Information extraction

What Horus does Horus can help the user with: Scene description Face recognition Text reading Object recognition Mobility assistance

User interaction Navigation menu Scene description Text reading Navigationbuttons Face recognition Object recognition ON/OFF button Mobility assistance After powering Horus, the user can choose the desired functionality by navigating a vocal menu using the navigation buttons.

Image description with Deep Learning A sunset over the mountains Language model Convolutional network Whole process runs on NVIDIA TK1

Results on TK1 CPU vs GPU (CNN + LSTM) Processing time on Memory footprint Processing time on GPU GPU CNN: 22ms CNN: 595ms 300 MB LSTM: 498ms LSTM: 1200ms TOTAL: 520ms TOTAL: 1795ms ~3.5X faster on GPU

Reporting obstacles Low-pitched sound center High-pitched sound High-pitched sound left right User Horus uses 3D sound to report the presence of obstacles during movement. The space in front of the user is divided in different sectors: lateral obstacles generate high-pitched sounds in one of the two speakers, while central obstacles generate low pitched centered sounds. These sounds are repetitive and they increase in repetition frequency as the obstacle gets closer.

Results on TK1 CPU vs GPU (SGBM @480p) Processing time on Processing time on GPU (Visionworks) CPU 116 ms 5650 ms ~48X faster on GPU

Example of audio feedback If the text is located in the upper part of the fieldof If the text is located in the lower part of the fieldof view, Horus emits a high-pitched sound to tell the view, Horus emits a low-pitched sound to tell the userto lower the text userto raise the text

Face recognition Source frame Face detection CNN Classification Tracking

Future improvements 3D reconstruction of faces 3D undistortion of sheets Object recognition Multi language LSTM models

www.horus.tech info@horus.tech