An Approach of a Research Tool based in a Shamanic Interface What - PowerPoint PPT Presentation

An Approach of a Research Tool based in a Shamanic Interface

What is a shamanic interface? Application with gestural interfaces normally use a mimicking approach. (e.g. dancing games in Kinect) 2

What is a shamanic interface? Shamanic interface concept was proposed in 2014 based in the science fiction novel Freedom TM . "It’s called the shamanic interface because it was designed to be comprehensible to all people on earth, regardless of technological level or cultural background." 3

What is a shamanic interface? The idea is creating a gestural interface that can be used by users with different cultures, through meaningful gestures, like cultural gestures (emblems). 4

What is a shamanic interface? 5

Motivation Natural User Interfaces suggest a natural interaction. With globalization, the same interface is used by users with different cultures. Different cultures implies different gestures. 6

Problem There is not any research tool that allows researchers to perform empirical evaluations and validations on the cultural awareness in gestural commands 7

Goals Create an application based on a cultural gestural interface, as a tool for empirical research on the shamanic interface concept. 8

Background Leap Motion SDK & Accord.NET Framework 9

Background - Leap Motion SDK 10

Background - Leap Motion SDK Frame Hands ... Hand Hand ... Position Direction Fingers ... 11

Background - Accord.NET Framework It provides methods and algorithms through different libraries on diverse scientific computing techniques of computer vision and artificial intelligence such as machine learning, mathematics, statistics, computer vision and computer audition. 12

Approach 13

Application based on a SI A game that use commands that explore the different gestures between cultures. Maze game with gestures as: selecting, indicate a number, moving, rotating, drink, grab, mute/unmute, pause/resume, quit. 14

Approach - Gesture Detection The capture of the hand’s user is made through a Leap Motion Controller . 15

Approach - Gesture Detection 6 features (18 values) - direction of all 5 fingertips + normal vector to the palm 16

Approach - Creation of models Each gesture is represented by an HMM. Using the Accord.NET Framework it is possible to initialize, teach, save and load HMMs. 17

Approach - Record lists of sequences In order to teach the HMMs, samples of each gesture must be stored with a recorder of gestures. 18

Approach - Gesture classification Through the Accord.NET Framework it is possible to create a HMM classifier to evaluate between several HMMs the most promising model to create a given sequence (gesture). 19

Approach - Cultural layer The cultural layer is the main core on a SI. It is responsible for create the relation between the gesture and the command, based on the user’s culture. 20

Approach - Cultural layer Previous approach: New approach: (Gesture, Culture) (Command, Culture) ↓ ↓ Command Gesture 21

Approach - Cultural layer N = 6 N = 20 P(x) = 1/6 P(x) = 1/20 22

Approach - Implementation in the game The SI is implemented in the game through: - A Cultural Layer responsible for store all the association between commands, gesture names and respective cultures. - A map between gesture name and HMM - A variable to store the user’s culture 23

Approach - Implementation in the game 24

Implementation The work is divided in three different targets Recorder SI Library Game 25

Implementation - Prototype 0 Game’s structure and the game’s mechanics. 26

Implementation - Prototype 1 - SI library; - Recorder; - Record and implemented some gestures (moving and rotating) 27

Implementation - Prototype 2 Finished to record and implement all gestures. New feature: normal vector to the palm. 28

Implementation - Prototype 2.5 New feature: Variation in the hand’s position. Result: static gestures when performed were misrecognized by dynamic gestures. 29

Implementation - Prototype 3 Added generic set of gestures, based on default gestures available in Leap Motion . 30

Playability tests with users Divided in three different groups, 4 users each, to test the game’s playability with all set of gestures: - A group with 4 Portugueses - A group with 4 Dutch - A group with a Dutch, a Belgium, an Indian and a Brazilian 31

Playability tests with users 32

Conclusions ✔ An application as a tool for empirical research on the shamanic interface concept. ! The application can still be improved. 33

Future work - Perform some empirical research in the SI concept; - Test other methods in the approach used; - Try different paths in the SI concept, such as identify the culture of a user based on the gestures. 34

An Approach of a Research Tool based in a Shamanic Interface