Reinventing Fraud Prevention & Underwriting with Machine - PowerPoint PPT Presentation

Reinventing Fraud Prevention & Underwriting with Machine Learning Ido Lustig – VP Risk Lendit April 2016 Propriety and Confidential

BlueVine – flexible business lines of credit and invoice factoring 08/2013 03/2014 06/2014 12/2014 12/2015 Founded + Beta launch Full launch Series B Series C Seed Tel Aviv , Israel $64M in equity and debt financing to date • R&D, Risk • 32 Employees Palo Alto , CA • Biz, Ops, Sales • 34 Employees Propriety and Confidential

3 QUESTIONS underlie our underwriting process Propriety and Confidential

? Propriety and Confidential

Machine-human interaction is the key for scale and accuracy We need to ask the right questions 
 and answer them like (smart) humans would have. Propriety and Confidential

Machine-learning capabilities continually advancing http://www.bloomberg.com/news/articles/2016-01-03/after-winning-at-chess-this-computer-may-help-decide-on-loans Propriety and Confidential

But it’s 
 still not perfect Propriety and Confidential

- First and last name correlation with loss - Number of letters in each word - Total number of letters - Number of times each letter appears - Order of letters - ….. Propriety and Confidential

Problem #1 - overfitting Observations https://shapeofdata.wordpress.com/2013/03/26/general-regression-and-over-fitting/ Propriety and Confidential

Problem #2 – Equal Credit Opportunity Act: things you can’t use (and end up using…) https://www.washingtonpost.com/news/wonk/wp/2015/05/26/what-your-name-says-about-your-age-state-job-and-political-leanings/ Propriety and Confidential

Problem #2 – Equal Credit Opportunity Act: things you can’t use (and end up using…) https://www.washingtonpost.com/news/wonk/wp/2015/05/26/what-your-name-says-about-your-age-state-job-and-political-leanings/ Propriety and Confidential

Problem #2 – Equal Credit Opportunity Act: things you can’t use (and end up using…) http://fivethirtyeight.com/features/how-to-tell-someones-age-when-all-you-know-is-her-name/ Propriety and Confidential

Problem #2 – Equal Credit Opportunity Act: things you can’t use (and end up using…) https://www.washingtonpost.com/news/wonk/wp/2015/05/26/what-your-name-says-about-your-age-state-job-and-political-leanings/ Propriety and Confidential

Problem #2 – Equal Credit Opportunity Act: things you can’t use (and end up using…) So what’s OK to ask? And what would we be better off not asking at all? Propriety and Confidential

Problem #3 – understanding the outcome • Clear rejection reasoning (ECOA) • Debrief and improve your policies Propriety and Confidential

Our 2¢ Insight driven and data backed automation process Propriety and Confidential

Map Questions Retrain Automate models Answers Fine tune Expose to features analysts Get feedback Propriety and Confidential

Ask the right questions (fraud example) 
 Is the person who she claims Does the she is? Are documents business exist? doctored? Any evidence of Does the business criminal activity? have a decent website? Does the activity match the client’s industry? Propriety and Confidential

Does the business have a decent website? (automation) ✓ Use user provided (www.idosbiz.com) Guess ✓ Use email domain (sales@idosbiz.com) URL ✓ Use search engine API (search ido (AND biz OR business) Crawl ✓ Download website ✓ Classify using internal model Website ✓ Use Industry as a standard ➢ Down Website ➢ Not found score ➢ Weak ➢ Medium ➢ High Propriety and Confidential

http://glo4led.com/ Propriety and Confidential

http://www.valleyisleaquatics.com/ Propriety and Confidential

Does the business have a decent website? 
 Does the business have a decent website given the industry? http://www.royalgranitesandgems.com/ Propriety and Confidential

• Not self-derived from the data • Answer critical questions • Fine tuned, highly accurate • High coverage Propriety and Confidential

Map Questions Retrain Automate models Answers Fine tune Expose to features analysts Get feedback Propriety and Confidential

Propriety and Confidential

Propriety and Confidential

• Hold both calculated and analyst values • Auto-retrain low performing variables Propriety and Confidential

Map Questions Retrain Automate models Answers Fine tune Expose to features analysts Get feedback Propriety and Confidential

• Same process for features, models, and decisions • For high level models – use tagging (fully automated) Propriety and Confidential

Featur Featur e e Deploy Deploy Human Reality Auto Auto Feedb Feedb Retrain Retrain ack ack Propriety and Confidential

Automated Decision Rate and Accuracy 100% 75% 50% 25% 0% Q2 2015 Q3 2015 Q4 2015 Q1 2016 Q2 2016 Coverage Accuracy Propriety and Confidential

Thank You Propriety and Confidential