On Human Predictions with Explanations and Predictions of Machine - PowerPoint PPT Presentation

On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection Vivian Lai and Chenhao Tan @vivwylai | @chenhaotan vivlai.github.io | chenhaot.com University of Colorado Boulder deception.machineintheloop.com

Risk assessment: COMPAS https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Most previous studies are concerned with the impact of such tools used in full automation https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Judges are required to take account of the algorithm’s limitations in Wisconsin In the end, though, Justice Bradley allowed sentencing judges to use Compas. They must take account of the algorithm's limitations and the secrecy surrounding it, she wrote, but she said the software could be helpful ”in providing the sentencing court with as much information as possible in order to arrive at an individualized sentence.” https://www.nytimes.com/2017/05/01/us/politics/sent-to-prison-by-a-software-programs-secret-algorithms.html

Full automation is not desired

How judges make decisions with COMPAS? https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

How humans make decisions with machine assistance in challenging tasks? Full Full human automation agency

A spectrum between full human agency and full automation Showing only explanations Showing machine (by highlighting salient predicted labels and information) explanations Showing machine Showing machine predicted labels predicted labels and Full human Full suggesting high accuracy agency automation

Deception Detection as a Case Study Showing only explanations Showing machine (by highlighting salient predicted labels and information) explanations Showing machine Showing machine predicted labels predicted labels and ~50% 87% suggesting high accuracy

I would not stay at this hotel again. The rooms had a fowl odor. It seemed as though the carpets have never been cleaned. The neighborhood was also less than desirable. The housekeepers seemed to be snooping around while they were cleaning the rooms. I will say that the front desk staff was friendly albeit slightly dimwitted.

I would not stay at this hotel again. The rooms had a fowl odor. It seemed as though the carpets have never been cleaned. The neighborhood was also less than desirable. The housekeepers seemed to be snooping around while they were cleaning the rooms. I will say that the front desk staff was friendly albeit slightly dimwitted.

The machine predicts that the below review is deceptive I would not stay at this hotel again. The rooms had a fowl odor. It seemed as though the carpets have never been cleaned. The neighborhood was also less than desirable. The housekeepers seemed to be snooping around while they were cleaning the rooms. I will say that the front desk staff was friendly albeit slightly dimwitted.

Can explanations alone improve human performance? Showing only explanations Showing machine (by highlighting salient predicted labels and information) explanations Showing machine Showing machine predicted labels predicted labels and suggesting high accuracy

Explanations alone slightly improve human performance Control 51.1% p=0.056 Examples 54.4% p=0.006 Highlight 55.9% p<0.001 Heatmap 57.6% Machine 87% 45 55 65 75 85 Accuracy (%)

Predicted labels > explanations Showing only explanations Showing machine (by highlighting salient predicted labels and information) explanations Showing machine Showing machine predicted labels predicted labels and suggesting high accuracy

Explicit accuracy improve human performance drastically Control 51.1% p<0.001 Heatmap 57.6% p<0.001 Predicted label 61.9% without accuracy p<0.001 Predicted label 74.6% with accuracy Machine 87% 45 55 65 75 85 Accuracy (%)

Tradeoff between human performance and human agency Showing only explanations Showing machine (by highlighting salient predicted labels and information) explanations Showing machine Showing machine predicted labels predicted labels and suggesting high accuracy Higher agency, Lower agency, lower performance higher performance

Can explanations moderate this tradeoff? Showing only explanations Showing machine (by highlighting salient predicted labels and information) explanations Showing machine Showing machine predicted labels predicted labels and suggesting high accuracy

Predicted labels + explanations ≈ explicit accuracy Predicted label 61.9% without accuracy p<0.001 Predicted label 72.5% & heatmap p<0.001 Predicted label 74.6% with accuracy Machine 87% 45 55 65 75 85 Accuracy (%)

How much do humans trust the predictions? Showing only explanations Showing machine (by highlighting salient predicted labels and information) explanations Showing machine Showing machine predicted labels predicted labels and suggesting high accuracy

Explanations help increase humans trust on predictions Predicted label 64.4% without accuracy p<0.001 Predicted label 78.7% & heatmap p<0.001 Predicted label 79.6% with accuracy 45 55 65 75 85 Trust (%)

Humans are more likely to trust predictions when they are correct 65.1% Predicted label without accuracy 60% Predicted label 79.4% Correct & heatmap 74.1% Incorrect 81.1% Predicted label with accuracy 69.8% 45 55 65 75 85 Trust (%)

Other analysis 50 60 70 Heterogeneity between participants Showing varying accuracies

Vivian Lai and Chenhao Tan Takeaway @vivwylai | @chenhaotan vivlai.github.io | chenhaot.com University of Colorado Boulder deception.machineintheloop.com Explanations alone only Explanations can slightly improve human moderate the performance tradeoff Showing machine Showing machine predicted labels predicted labels and suggesting high accuracy Higher agency, Lower agency, lower performance higher performance