Computer Mediated Transactions Hal Varian Google April 7 1 - PowerPoint PPT Presentation

Computer Mediated Transactions Hal Varian Google April 7 1

Outline -- what does CMT enable? There is now a computer in the middle of most economic transactions. What does this enable? 1. Data extraction and analysis 2. Personalization and customization 3. Experimentation and continuous improvement 4. Contractual innovation

Data extraction and analysis

Initial claims: good leading indicator for recessions Grey bars indicate recessions

Google Correlate with initial claims data

Initial claims and [unemployment filing]

Nowcasting initial claims Predict NSA initial claims (y t ), using lagged values of initial claims and contemporaneous queries on [unemployment filing] (x t ) Base: y t = a 0 + a 1 y t-1 + a 52 y t-52 + e t Trends: y t = a 0 + a 1 y t-1 + a 52 y t-52 + b x t + e t Result: R 2 goes from 80.8% to 87.6%

How can we make variable selection easier? Big data Rows or columns? How to choose best predictors? Simple correlation? Judgment? Stepwise regression? Lasso, LARS, Elastic Net? Spike-and-slab regression Kalman filter for trend and seasonality George-McCulloch [1997]) ;Madigan-Raftery [1994] for regression Prior probability variable is included (spike) Prior probability distribution over coefficient value (slab) Sample from simulated posterior, average to get prediction See Scott and Varian (2012, 2013) for details Download R package from CRAN ( BoomSpikeSlab, bsts )

New Home Sales in US

Raw correlation

Predictors chosen by model

Incremental fit plots Visualize how much each predictor contributes to model fit model: y t = trend t + seasonal t + b 1 x 1t + b 2 x 2t plot1: y t = trend t plot2: y t = trend t + seasonal t plot3: y t = trend t + seasonal t + b 1 x 1t plot4: y t = trend t + seasonal t + b 1 x 1t + b 2 x 2t

Trend

Seasonal

[appreciation rate]

[irs 1031]

[century 21 realtors]

[real estate purchase]

[80-20 mortgage]

One month ahead forecast Does 23% better than simple AR1 model

Geo-amplification You can do the same thing for any geographically distributed variable Find out queries or query categories that are predictive of that variable Make predictions/extrapolations to other geographies Many applications Social science Policy Marketing Politics Example: New York Times index of “hard places” (June 26, 2014)

Where are the hardest places to live in the U.S.?

What queries are associated with “hard places”? Based on state level data and Google Correlate

What queries are associated with “easy places”? Based on state level data and Google Correlate

Customization and personalization

Assembled in America

Predictors of survey response

Top and bottom cities' predicted score Top Bottom Kershaw, SC: 83.2 % Calipatria, CA: 40.2 % Summersville, WV: 82.8 % Fremont, CA: 40.2 % Grundy, VA: 82.8 % Mountain View, CA: 40.8 % Chesnee, SC: 82.7 % San Jose, CA: 41.4 % Duffield, VA: 82.5 % Berkeley, CA: 41.4 % Norton, VA: 82.3 % Redmond, WA: 41.5 % Jonesville, VA: 82.2 % Glendale, CA: 41.5 % Walnut Cove, NC: 82.2 % Cupertino, CA: 41.6 % Weston, WV: 82.2 % Palo Alto, CA: 41.7 % Ennice, NC: 82.1 % Daggett, CA: 41.9 %

Assembled in America by DMA

Experimentation and continuous improvement

Causal inference “To find out what happens when you change something, it is necessary to change it.” George Box

Experiments: gold standard for causality What goes wrong with observational data? y t = x t b + e t = observed + unobserved Correlation: if you observe x what is a good prediction for y? Causality: what happens to y if you change x? Confounder: something unobserved that affects both x and y

Advertising Q: How do your know your advertising works? A: Every December I increase my ad spend...

Advertising Q: How do your know your advertising works? A: Every December I increase my ad spend...and every December my sales go up!

Advertising Q: How do your know your advertising works? A: Every December I increase my ad spend...and every December my sales go up! “Christmas holidays” are a confounding variable. Here the solution is obvious, but what happens if you can’t observe the confounders?

Train, test, treat, compare 1. Train a model on historical data 2. Test the model on a holdout 3. Apply treatment at some time 4. Compare observed outcome with the treatment to the counterfactual prediction of model

Compare outcome to counterfactual

Actual and natural experiments You want randomized experiments to reduce systematic effects. Sometimes you get randomization “for free”. Impact of class size on performance ● Why are classes larger in some schools than others? ● In Israel maximum class size is 40. Classes with 41 are split in two. ● Can identify causal effect of class size on performance Impact of ad impressions on movie revenue Super Bowl facts ● Ads are bought long before teams are chosen ● Home cities of participating teams see elevated viewership ● Natural randomization

Experimentation capability should be coded in static code: const threshold = 3.14 if (x > threshold) do something learning code: param threshold = {3.13, 3.14, 3.15) performance = (num_right, num_wrong) if (x > threshold) do something report performance Research challenge: How to turn legacy code into learning code? Nice example: Keith Winstein et al, An Experimental Study of the Learnability of Congestion Control

Contractual innovation

What is a contract? “If you do this, I’ll do that.” But how do you verify “this” and “that”? Can only contract on things that can be observed and verified...

What is a contract? “If you do this, I’ll do that.” But how do you verify “this” and “that”? Can only contract on things that can be observed and verified… But with a computer in the middle of the transaction, lots more can be verified.

Examples of contracts ● “You take me to my hotel on the best route, I will pay you.” ● “You use the car and send me a monthly payment.” ● “You drive this rental car safely, I will give you a discount.” ● “You display an ad that brings someone to my store, I will pay you.”

Summary 1. Data extraction and analysis a. Can use searches to nowcast economic activity 2. Personalization and customization a. Can customize ads to different geos 3. Experimentation and continuous improvement a. Can use ML to estimate causal impact via train-test- treat-compare cycle 4. Contractual innovation a. As more things become observable, more contracts become viable

Appendix

Advertise a movie about surfing Honolulu: $1 ad spend $10 ticket sales Fargo: $0.10 ad spend $1 ticket sales Ticket sales = 10 x ad spend fits the data perfectly...

Advertise a movie about surfing Honolulu: $1 ad spend $10 ticket sales Fargo: $0.10 ad spend $1 ticket sales Ticket sales = 10 x ad spend fits the data perfectly... But do you really believe that if you increased spend to $1 in Fargo, you would get 10 times the ticket sales?

Ads and confounders “Interest in surfing” is a confounding variable Happens all the time in economics since people choose x (observing things you don’t observe.) Causal effect of college on education? Causal effect of fertilizer on yield? Causal effect of health care on income?

Superbowl as a natural ad experiment 1. Viewership in home cities of teams that are playing is about 10-15% higher than elsewhere. 2. Ads are purchased long before it is known who is playing Advertiser buys ad slot, then 2-3 months later two “random” cites get 10-15% more ad exposure.

Regression discontinuity Impact of class size on performance ● Why are classes larger in some schools than others? ● In Israel maximum class size is 40. Classes with 41 are split in two. ● Can identify causal effect of class size on performance What would happen to auto fatalities if you changed the minimum drinking age? ● 20.5 year olds are a lot like 21.5 year olds ● So looking at people on each side of the threshold can give estimate of causal effect

Regression discontinuity