10 — Analytics & Optimization From Code to Product gidgreen.com/course
Lecture 10 • Introduction • Data collection • Website metrics • Optimization • Competitive intelligence • Surveys • Tools and books From Code to Product Lecture 10 — Analytics— Slide 2 gidgreen.com/course
Why analytics? • Quantify success/failure – For yourselves – For investors – Against competition • Scientific decisions – No blind faith – Fewer arguments – Avoid HiPPO = highest paid person’s opinion From Code to Product Lecture 10 — Analytics— Slide 3 gidgreen.com/course
Good analytics • Simple • Few in number • Relevant • Unambiguous • Actionable • Instant (or nearly) • Repeatable From Code to Product Lecture 10 — Analytics— Slide 4 gidgreen.com/course
AARRR — Metrics for pirates A cquisition Site visit or app download A ctivation Registration or usage Dave McClure, 500 Startups R etention Repeat usage R eferral Brings other people R evenue Generate cash From Code to Product Lecture 10 — Analytics— Slide 5 gidgreen.com/course
Some quotes “What gets measured, gets managed.” — Peter Drucker “The only metrics that entrepreneurs should invest energy in collecting are those that help them make decisions.” — Eric Ries, The Lean Startup From Code to Product Lecture 10 — Analytics— Slide 6 gidgreen.com/course
Lecture 10 • Introduction • Data collection • Website metrics • Optimization • Competitive intelligence • Surveys • Tools and books From Code to Product Lecture 10 — Analytics— Slide 7 gidgreen.com/course
In-app analytics • Home rolled or third party • Store usage information locally – ‘Call home’ when online • Privacy concerns – Confirmation dialog? • Complete access to device – But you will be caught! • Problem: slow iteration From Code to Product Lecture 10 — Analytics— Slide 8 gidgreen.com/course
In-app integration From Code to Product Lecture 10 — Analytics— Slide 9 gidgreen.com/course
Reporting app events From Code to Product Lecture 10 — Analytics— Slide 10 gidgreen.com/course
Web analytics • All activity visible to site – Users don’t expect privacy • Web servers log requests – Also: Javascript solutions • Page view centric – Other events require integration – Coffee break? – Events not sessions From Code to Product Lecture 10 — Analytics— Slide 11 gidgreen.com/course
A web server log line www.websudoku.com 24.186.55.113 [06/May/2012:08:13:02 -0400] "GET / HTTP/1.1” 200 1045 "http://www.google.com/search?q=sudoku” "Mozilla/5.0 (iPhone; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B179 Safari/7534.48.3" From Code to Product Lecture 10 — Analytics— Slide 12 gidgreen.com/course
Javascript tracking code <script type="text/javascript”> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-1165533-3']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script> From Code to Product Lecture 10 — Analytics— Slide 13 gidgreen.com/course
Web metrics alternatives Server logs Javascript Home-made Integration None Via HTML Server code Download + Web-based Convenience Up to you analyze access Delay None Up to 24 hours Up to you Reporting Varies Advanced Up to you Other events Hard Via API Easy Data leakage None Total! None From Code to Product Lecture 4 — UI Design— Slide 14 gidgreen.com/course
Track web users by… • IP address – Given for every web request – Good for geography – But: proxies, classrooms, router resets • Cookies – Track user browser over long term – But: clearing, multi-browsing, first request – Customization of web server From Code to Product Lecture 10 — Analytics— Slide 15 gidgreen.com/course
Track web users by… • Log in – Reliable for registered users – But: anonymous users, multiple accounts – Requires custom logging tools • Solution: combine! – Intelligently tie IPs, cookies and accounts – Example: user registration • Data always incomplete From Code to Product Lecture 10 — Analytics— Slide 16 gidgreen.com/course
Lecture 10 • Introduction • Data collection • Website metrics • Optimization • Competitive intelligence • Surveys • Tools and books From Code to Product Lecture 10 — Analytics— Slide 17 gidgreen.com/course
Basic website metrics From Code to Product Lecture 10 — Analytics— Slide 18 gidgreen.com/course
Immediate questions • When does one visit end? – GA: 30 minutes without activity • What makes a visitor unique? – GA: Tracking cookie • How is duration calculated? – GA: Time between first and last pages • What makes a visitor new? – GA: Never visited your site before From Code to Product Lecture 10 — Analytics— Slide 19 gidgreen.com/course
Geography From Code to Product Lecture 6 — BM — Advertising— Slide 20 gidgreen.com/course
Demographics From Code to Product Lecture 6 — BM — Advertising— Slide 21 gidgreen.com/course
Frequency report From Code to Product Lecture 10 — Analytics— Slide 22 gidgreen.com/course
Sources of traffic • Type-in (no referrer) – Includes browser bookmarks • Search engines – Navigational search = type-in • Referrals – Website links or social media • Paid advertising • Email campaigns From Code to Product Lecture 10 — Analytics— Slide 23 gidgreen.com/course
The multitouch problem • There’s history before the referrer – Who deserves the credit, e.g. affiliates • So who gets the credit? – Last click (standard) – First click (unrealistic) – Even split – Split weighted to last From Code to Product Lecture 10 — Analytics— Slide 24 gidgreen.com/course
Search engine queries Also: internal site search From Code to Product Lecture 10 — Analytics— Slide 25 gidgreen.com/course
Popular pages From Code to Product Lecture 10 — Analytics— Slide 26 gidgreen.com/course
Landing/entry pages “You can’t choose your home page” — A. Kaushik From Code to Product Lecture 10 — Analytics— Slide 27 gidgreen.com/course
Clickmaps and heatmaps From Code to Product Lecture 10 — Analytics— Slide 28 gidgreen.com/course
Conversion funnel Source: www.searchenginejournal.com From Code to Product Lecture 10 — Analytics— Slide 29 gidgreen.com/course
Sampling methods • Popular site => lots of data – Burden to collect, slow to analyze • Don’t record all events – Choose important pages – Random subset of visitors – Random subset of pageviews • Sub-sample when analyzing – By page or visitor From Code to Product Lecture 10 — Analytics— Slide 30 gidgreen.com/course
Staleness due to changes in… • Content • User familiarity – Early adopters vs ... • Search engine rankings • Market needs • Devices • Cookies From Code to Product Lecture 10 — Analytics— Slide 31 gidgreen.com/course
Lecture 10 • Introduction • Data collection • Website metrics • Optimization • Competitive intelligence • Surveys • Tools and books From Code to Product Lecture 10 — Analytics— Slide 32 gidgreen.com/course
Optimization • You don’t know how users behave – Example: show price early on? • Small changes => big results – But which small changes? • Use a scientific methodology – Easy to set up – Easy to get report – Statistical significance From Code to Product Lecture 10 — Analytics— Slide 33 gidgreen.com/course
Wording example you_should_follow_me_on_twitter.html Source: http://www.dustincurtis.com/ From Code to Product Lecture 10 — Analytics— Slide 34 gidgreen.com/course
A/B testing • Two parallel variations – Current vs challenger • Assign randomly and evenly – What about previous visitors? – Repeat requests within a session? • Set test length in advance – Length of time or number of visits • Chi-squared (or similar) test From Code to Product Lecture 10 — Analytics— Slide 35 gidgreen.com/course
Contingency table Product Not purchased purchased 9 575 13 563 From Code to Product Lecture 10 — Analytics— Slide 36 gidgreen.com/course
Multivariate testing Source: http://www.getelastic.com/testing- part-1/ From Code to Product Lecture 10 — Analytics— Slide 37 gidgreen.com/course
Multivariate testing • Best to use third-party tool • Full factorial vs partial factorial – Certainty vs efficiency From Code to Product Lecture 10 — Analytics— Slide 38 gidgreen.com/course
Optimization pitfalls • Preconception driven – Too many similar tests – Checking before it’s done • Wrong goal – e.g. started vs completed purchases • Unfair test – Different time periods – New vs returning users From Code to Product Lecture 10 — Analytics— Slide 39 gidgreen.com/course
Recommend
More recommend
