REAL-TIME AI FOR ENTITY RESOLUTION Jeff Jonas Founder and CEO - PowerPoint PPT Presentation

REAL-TIME AI FOR ENTITY RESOLUTION Jeff Jonas Founder and CEO jeff@senzing.com

Entity Resolution (ER) Conversely, recognizing when two Recognizing when two observations observations do not relate to the relate to the same entity, despite same entity, despite having been having been described differently. described similarly. Rob E. Smith Robert Smith Jr. Robert Smith S r. 123 E Main St 123 Main Street 123 Main Street Related +1 (703) 554-1214 703.554.1214 703.554.1214 And, the ability to remember relationships between entities. 2

NORA: Non-Obvious Relationship Awareness ü 15M+ customers ü 20k+ employees ü 18 watch lists 24 active players were known cheaters 23 players had relationships to prior arrests/incidents 192 employees had possible vendor relationships 7 employees were the vendor 3

NORA Grows Up G2 Aspirations § Real-time AI for entity resolution – No training, no tuning, no experts – Self-tuning; self-correcting § Speed and scalability – Scale vertically and horizontally 2005 – 10’s billions of records Identity Insight – 1k’s of transactions a second G2 2009 § New data sources, entity types, attributes on the fly § Privacy by Design (PbD) 4

Early “G2” Success Fake Identity Detection : 600k fake students detected; $300m saved Maritime Domain Awareness : Malacca Straits monitoring for the Singapore Ministry of Defense 5

ENTITY RESOLUTION IN SLOW MOTION 6

Do these three entities become one, two or three resolved entities? 2 3 1 Robert Smith Bob Smith Rob Smith 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 4 5 6 7 8 9 7

Despite variations in name, address and phone, and the date of birth transposition, they become resolved entity E1. 2 3 1 Robert Smith Bob Smith Rob Smith 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E1 4 5 6 1 2 3 7 8 9 Fuzzy Matching 8

New entity or belongs to the known entity E1? 2 3 1 Robert Smith Bob Smith Rob Smith 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E1 4 5 6 Patricia Smith 1 2 3 smith@email.com 7 8 9 9

When an email address is the same but the names are completely different, a new but related entity E2 is created. 2 3 1 Robert Smith Bob Smith Rob Smith 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E1 4 5 6 Patricia Smith 1 2 3 smith@email.com 7 8 9 4 Discovered Relationships E2 10

How does this entity integrate with the existing entity graph? 2 3 1 Robert Smith Bob Smith Rob Smith 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E1 4 5 6 Patricia Smith Kim Smith 1 2 3 smith@email.com kim@email.com 3 Spouse: 7 8 9 4 E2 11

Entity E3 is created with a relationship to entity E1. E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 Disclosed 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 Relationships smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E1 4 5 6 Patricia Smith Kim Smith 1 2 3 smith@email.com kim@email.com 3 Spouse: 7 8 9 4 E2 12

What becomes of this entity? E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E1 4 5 6 Patricia Smith Kim Smith Bob R. Smith 1 2 3 smith@email.com kim@email.com AKA Bobby Jones DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 4 E2 13

Many people have the same name and date of birth, thus entity E4 is created as a possible match. E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E4 E1 4 5 6 Patricia Smith Kim Smith Bob R . Smith 1 6 2 3 smith@email.com kim@email.com AKA Bobby Jones DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 4 Possible Matches E2 14

What becomes of this entity? E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E4 E1 4 5 6 Patricia Smith Kim Smith Bob R. Smith 1 6 2 3 smith@email.com kim@email.com AKA Bobby Jones DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 4 smith@email.com E2 15

If E2 did not exist this would resolve to E1, and vice versa. E5 is created as it’s a possible match to both E1 and E2. E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E4 E1 4 5 6 Patricia Smith Kim Smith Bob R. Smith 1 6 2 3 smith@email.com kim@email.com AKA Bobby Jones DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 Possible Match 7 4 (“Ambiguous”) smith@email.com E5 E2 16

What becomes of this entity? E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E4 E1 4 5 6 Patricia Smith Kim Smith Bob R. Smith 1 6 2 3 smith@email.com kim@email.com AKA Bobby Jones DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 7 4 B. Smith smith@email.com 1515 Adela Lane bsmith@work.com E5 E2 1.702.919.1600 17

This reveals that entities E1 and E4 are the same, causing the entity E4 to conjoin with entity E1. E3 2 3 1 Robert Smith B ob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St Self-correcting DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 (“Re-resolve”) 703.554.1214 1.702.919.1600 4 5 6 E1 Patricia Smith Kim Smith Bob R . Smith 1 6 8 2 3 smith@email.com kim@email.com AKA Bobby Jones DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 7 4 B. Smith Rob Smith Sr. smith@email.com 1515 Adela Lane DOB: 3/31/1954 bsmith@work.com ID: 00112233 E5 E2 1.702.919.1600 18

What becomes of this entity? E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 4 5 6 E1 Patricia Smith Kim Smith Bob R. Smith 1 6 8 2 3 smith@email.com kim@email.com AKA Bobby Jones DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 7 4 B. Smith Rob Smith Sr. smith@email.com 1515 Adela Lane DOB: 3/31/1954 bsmith@work.com ID: 00112233 E5 E2 1.702.919.1600 19

This is evidence that record 3 is not entity E1, causing the “Sr.” records to become the new entity E6. E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St Self-correcting DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 (“Un-resolve”) 703.554.1214 702.919.1600 E6 E1 4 5 6 Patricia Smith Kim Smith Bob R . Smith 1 3 8 smith@email.com kim@email.com AKA Bobby Jones 6 2 9 DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 7 4 B. Smith Rob Smith Sr. smith@email.com 1515 Adela Lane DOB: 3/31/1954 bsmith@work.com ID: 00112233 E5 E2 1.702.919.1600 20

What should happen to this entity? E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E6 5 E1 4 6 Patricia Smith Kim Smith Bob R. Smith 1 3 8 smith@email.com kim@email.com AKA Bobby Jones 6 2 9 DOB: 12/11/1978 3 Spouse: bsmith@work.com 7 8 9 10 7 4 Bob Jones B. Smith Rob Smith Sr. 123 Main Street smith@email.com 1515 Adela Lane DOB: 3/31/1954 702.919.1600 bsmith@work.com ID: 00112233 E5 E2 bjones@email.com 1.702.919.1600 21

This observation resolves to E1 because E1 contains a sufficient number of matching values. E3 2 3 1 Robert Smith Bob Smith Rob Smith 5 123 Main Street 1515 Adela Lane 123 E Main St “Entity-centric DOB: 12/11/1978 DOB: 11/12/1978 (703) 554-1214 Learning” smith@email.com 1.703.554.1214 ID: 00112233 703.554.1214 702.919.1600 E6 8 E1 6 9 Bob R. Smith B. Smith Rob Smith Sr. 1 3 10 8 AKA Bobby Jones 1515 Adela Lane DOB: 3/31/1954 6 2 9 DOB: 12/11/1978 bsmith@work.com ID: 00112233 bsmith@work.com 1.702.919.1600 10 7 4 Bob Jones 123 Main Street 702.919.1600 E5 E2 bjones@email.com 22