POLLING POST MORTEM British Polling Council / NatCen 8th December - PowerPoint PPT Presentation

EU REFERENDUM POLLING POST MORTEM British Polling Council / NatCen 8th December 2016

FINAL POLLS Modal difference that hasn’t been seen since online polls first started to show UKIP overtaking the Lib Dems in 2012/2013 Leave % minus Remain % 2 2 Telephone -2 -2 Online -4 -8 -8 -10 Populus ComRes ORB Ipsos MORI Survation YouGov TNS Opinium 2

VOTING INTENTION JAN 2012 – MAY 2015 European Party propensity Parliament 45% weighting elections 40% 35% Con 30% Lab 25% UKIP 20% Lib Dem 15% 10% 5% 0% Trend line smoothed to 4-poll moving average 3

“LEAVE” STARTED WELL AHEAD Party propensity Actual question weighting text used 60% 50% 40% 30% 20% 10% 0% Leave Remain 4

THE TURNOUT PROBLEM Political Studies Association turnout predictions 80% Stephen Bush: “Broadly, a moderately 72.20% 70% high turnout (say 60 per cent all the 63% 61% 61% 59% 59% way up to 80 per cent) is good for 60% Remain, and anything below 45 is good 50% for Leave. On a low turnout, the 40% preferences of the elderly and the 30% devout are exaggerated – in this 20% contest, both those groups aid Leave. ” 10% 0% PSA Mean Academics Pollsters Journalists Other Actual turnout 5

SOME FARED WORSE THAN OTHERS Ad Adam Dr Drummond (Opiniu ium): “Instinctively I’d expect turnout between 45% and 55%, with Remain’s chances of victory increasing in almost direct proportion. At risk of creating hostages to fortune I’m almost tempted to go with a rule of thumb that turnout below 50% means Brexit, and above 50% means we stay in. ” Damian Lyons Lowe 60% Ben Page 59% “but, like I say, – Surv Survation – Ip Ipso sos MORI I’m not unused Luke Taylor 65% Martin Boon 40-50% to being – TNS – ICM ICM horribly wrong” Andrew Hawkins >60% Joe Twyman “Closer to 60% than 30%” – Com ComRes – You ouGov https://www.buzzfeed.com/albertonardelli/how-britains-pollsters-think-eu-referendum-might-g?utm_term=.hppR119Mr5#.xwVz556GpY Published April 11 th 2016 6

GENERAL ELECTION TURNOUT MODELS Effects of retroactively applying GE turnout filters to EU Ref Polls 60% 53% 51% 49% 50% 47% 40% 30% 20% 10% 0% Demographic weights with no GE adjustment + Party Propensity Demographic weights with GE adjustment + Party Propensity Remain Leave 7

“NUDGING” Results of “nudge” question And if you were forced to choose, would you say you lean more towards the UK 50% remaining part of the European Union or 45% more towards the UK leaving the European Union? 40% 35% 1. Lean more towards the UK 30% REMAINING in the EU 25% 2. Undecided but more likely to vote 20% REMAIN than LEAVE 15% 3. Undecided but more likely to vote 10% LEAVE than REMAIN 5% 4. Lean more towards the UK LEAVING the EU 0% 5. No opinion % of sample Lean REMAIN Lean LEAVE Still no view 8

“NUDGING” Results of “nudge” question And if you were forced to choose, would you say you lean more towards the UK 25% remaining part of the European Union or more towards the UK leaving the European Union? 20% 1. Lean more towards the UK 15% REMAINING in the EU 2. Undecided but more likely to vote 10% REMAIN than LEAVE 3. Undecided but more likely to vote 5% LEAVE than REMAIN 4. Lean more towards the UK LEAVING the EU 0% 5. No opinion Lean REMAIN Lean LEAVE 9

SOCIAL ATTITUDES BE BES Face ace to o face ace Pho hone Onl Onlin ine Racia Ra ial l equ equali lity Not gone far 31.9% 40.0% 24.9% enough About right 42.9% 39.5% 39.3% Gone too far 18.7% 15.3% 26.4% Na Natio ional l ide dentit ity More British 20.9% 27.2% 19.2% Equally British and 47.6% 42.7% 38.8% English More English 23.8% 24.4% 31.7% 10

HOW OUR NUMBERS COMPARED Online samples were more socially conservative but OUR online samples even more so. “Have efforts to counter racial discrimination… 50.0% 42.9% 45.0% 40.0% 39.5% 39.3% 40.0% 36.5% 35.2% 35.0% 31.9% 30.0% 26.4% 24.9% 25.0% 20.5% 18.7% 20.0% 15.3% 15.0% 10.0% 5.0% 0.0% Not gone far enough About right Gone too far BES Face to face Populus Phone Populus Online Opinium (Online) 27th May 16 11

HOW OUR NUMBERS COMPARED Online samples were more socially conservative but OUR online samples even more so. “Would you say that you feel…” 47.60% 50.00% 42.70% 45.00% 40.40% 38.80% 40.00% 35.80% 35.00% 31.70% 30.00% 27.20% 24.40% 23.80% 25.00% 20.90% 19.20% 20.00% 14.30% 15.00% 10.00% 5.00% 0.00% More British Equally British and English More English BES Face to face Populus Phone Populus Online Opinium (Online) 27th May 16 12

EFFECTS OF ATTITUDINAL WEIGHTING Without attitudinal Introduction of weights attitudinal weighs 60% 60% 50% 50% 40% 40% 30% 30% 20% 20% 10% 10% 0% 0% Leave Remain Leave Remain 13

EFFECT OF ATTITUDINAL WEIGHTING WITH WITHOUT attitudinal attitudinal weights weights All who expressed a voting intention 60% 53% 51% 49% 50% 47% 40% 30% 20% 10% 0% Remain (with attitudinal Leave (with attitudinal - Remain (no additudinal Leave (no attitudinal weights) weights) weights) weights) 14

EFFECT OF ATTITUDINAL WEIGHTING WITH WITHOUT attitudinal attitudinal weights weights 10 out of 10 likely to vote 60% 55% 52% 48% 50% 45% 40% 30% 20% 10% 0% Remain (with attitudinal Leave (with attitudinal - Remain (no additudinal Leave (no attitudinal weights) weights) weights) weights) 15

CONCLUSIONS • The EU EU referendum was a sui sui generis event almost specifically designed to trip up pollsters. There is a limit to the extent to which we can apply lessons to general election polling • No adju No justment we we would have made pre-referendum would have pushed our final poll into the sort of clear Remain lead territory that telephone polls had • Attitudinal weighting had a similar effect to the sort of turnout model we considered applying and which would have made our 2015 GE result much more accurate • Open vs vs. closed is something we’re going to have to take more account of and as time goes on as it transcends party loyalties, particularly as we move on from the referendum and past vote weighting becomes less and less applicable 16

OPEN VS. CLOSED The “Leave” vote is much more unified than the “Remain” vote. EU Referendum vote by “tribe” 90% 85 85% 80% 80 80% 74 74% 66 66% 70% 62 62% 59 59% 60% 51% 51 47% 47 50% 39 39% 40% 34 34% 31 31% 32 32% 26% 26 30% 20% 12 12% 11% 11 10% 3% 3% 0% Democratic Socialists Community Progressives Swing Voters New Britain Free Liberals Common Sense Our Britain Remain Leave 17

THANK YOU FOR ANY QUERIES CONTACT: ADAM DRUMMOND SENIOR RESEARCH MANAGER +44 (0)20 7566 3194 ADAMDRUMMOND@OPINIUM.CO.UK 24A ST JOHN STREET | LONDON | EC1M 4AY