Have the bookies adjusted for Ashcroft?

Last Wednesday social media was ablaze with Lord Ashcroft’s latest set of Scottish polls, which suggest that Labour are still on course for a Scottish wipeout on May 7. Has this affected what the bookies have to say?

As before, we look at mean implied probabilities for bookmakers, and this time consider the markets for banded ranges of seats for Labour. The impact of worse than previously anticipated polling in Scotland ought to be reflected in a lower seat expectation than previously. Betfair, Bet365, SkyBet, Ladbrokes and William Hill report markets on bands of seats a party wins at the election, and the bands are

  • less than 200 seats
  • 201-225 seats
  • 225 seats and under
  • 226-250 seats
  • 251-275 seats
  • 276-300 seats
  • 301-325 seats
  • 326-350 seats
  • 326 seats or over
  • 351-375 seats
  • 351 seats or over
  • 376-400 seats
  • 401 or more seats

Clearly the options towards the bottom of that range are hugely unlikely (bookies rate anything above 375 seats as less than 5% likely to happen), but it’s the upper half of the range when the action has been:

lab_bands

The black vertical line is March 4, when the Ashcroft Polls were released. Hence prices have moved since the announcement, but with the range 276-300 falling in likelihood only from 35% to 34% and 301-325 from 23% to 21%, the impact doesn’t appear to have been dramatic. Lower seat totals like 251-275 increased from 28% to 32%. Less likely events saw bigger moves, with 326-350 seats falling from 12.5% to 7% today.

Overall, the numbers would appear to suggest that the Ashcroft polls are reinforcing the current trends, at least in terms of bookmaker prices; an update to the plot of bookmaker implied probabilities for most seats from two weeks ago emphasises this:

most_seats_2015-03-09

Can Polls Predict Seats?

Opinion polls are increasingly common; UK Polling Report lists 125 polls for both the 1974 elections combined, between 1970 and 1974, while the same website lists 1868 for the 2015 election, with still 67 days to go. However opinion polls only report the vote share implied by the surveying of the pollster; while undoubtedly vote share will influence election outcomes, anomalies are still possible. For example, in February 1974 the Conservatives won more votes yet fewer seats than Labour, an outcome it is reported the Tories are preparing for this time around.

Seats matter more, and trends vary across the country for the different parties; many Tory heartlands are supposedly at threat from Ukip, while Labour’s Scottish seats appear lost to the SNP. Constituency polling, led it seems by Lord Ashcroft since 2010, is viewed as the way forward. Despite this, traditional nationwide polls continue to attract attention – not least the current neck-and-neck nature of Labour and the Tories. Can we glean anything from such polling?

Is there any kind of relationship between how much a party is polling, and how many seats they can expect to get? There naturally is, but the more pertinent question is how strong and robust is that relationship over the years? We focus on polls since 1970, hence 10 elections and 8,253 polls. We include information on the time horizon until the election (number of days), the political party (in case of any biases), and consider any kind of incumbency effect, and we interact all these variables together in a linear regression model in order to see whether the resulting model had any explanatory power. Surprisingly enough, it manages to account for almost 90% of variation in seats won, and I’m happy to provide any interested party with the regression output (I’m working on tidying up a more general set of code for this).

A few plots to help:

Poll Shares and Seats

Firstly this is a cross plot between actual opinion poll shares and election outcomes in terms of seats – the black circles signify such points. They show the Lib-Dem cluster below 100, and the Labour/Tory cluster spread between 200 and 400. It suggests something of a non-linear relationship in the Labour/Tory cluster, since polling shares in the 50s are consistent with seat totals of 300 and 400. The red apparent scribbles are the fitted values, or predicted values, from our linear regression model. They show that within the sample, to some extent, the two clusters are captured. Clearly improvements are possible, but it’s a reasonable model to begin with.

How did the model fare in 2010? Here’s the implied seat forecasts from each poll against the actual seat totals:

Model forecasts in 2010

Note this is the same model estimated over all polls prior to the 2005 election – any poll for the 2010 election is excluded in order that this is actually a forecasting exercise rather than an in-sample fitting exercise. The resulting forecasts for seats are quite surprising in that Labour were forecast as the election neared to get more seats. This appears to be the incumbency effect, which is strongly significant in the regression model. On average, forecast errors were about 11 seats for this election, but clearly biased up for Labour, and down for the Tories and Lib Dems. The same was true, but on a smaller scale, in 2005.

What does all of this mean for 2015? It’s harder to tell whether there’s an incumbency effect since there is no comparable full coalition term; the model automatically attributes this to the Conservatives. The forecasts look like:

Seats forecast 2015

Hence unlike much of the current media narrative (e.g here, here), this very simple model appears to point towards the Tories not just being the largest party in a hung parliament, but potentially winning a majority; the most recent polls indicate based on historical data that the Tories will win somewhere between 280 and 300 seats, but Labour only 220-240. Could the incumbency effect be too strong here, as it appears to have been in forecasts in 2010?

There are no Ukip forecasts since there is no recent historical precedent for Ukip, hence no data upon which to base a forecast. We could apply the model, estimated over Lib-Dem historical performance, to Ukip, but there’s only so far one should take a very basic statistical model such as this. On the most important question of most seats between Labour and the Tories, it has already provided a thought provoking forecast.

Are the bookies internally consistent?

Seventy one days until the election, arguably the most unpredictable in a generation, and perhaps we might hope that the bookmakers, renowned for putting their money where their mouths are, might be on the pulse.

A co-author of mine, Leighton Vaughan Williams, of the Betting Research and Political Forecasting unit at Nottingham Trent University, says that at least £40m will be staked on the forthcoming election, hence the bookmakers must surely have their prices right?

Making the news recently has been the story epitomised by the plot below:

Most Seats Bookies Prices

This plot is of the price 20 bookmakers have been posting since 2010 of the bet for most seats at GE2015. It’s the reciprocal of those decimal odds, which gives the implied probability, and the thick blue and red lines are the Conservatives and Labour respectively, while the different shades of thinner lines are individual bookmakers. The pattern seems clear, that the Tories have shot into the lead recently, and quite some lead.

But bookmakers, and in particular Betfair’s fixed odds wing, have been offering bets in all constituencies around the country (William Hill, Paddy Power and Bet365 also offer north of 100 constituency markets). What can we learn from these?

The following plot, hard to decipher, shows the implied probabilities (vertical axis) for parties across all constituencies (horizontal axis):

Odds in each constituency

 

What should be clear, hopefully, is that there’s plenty of variation! There’s still 73 days, after all. Digging a little deeper though, if we took the following rule of thumb: a party whose betting price implies a likelihood of victory of above 60% wins that seat, then we get the following outcome on May 7:

Labour 278

Conservative 247

Lib Dem 26

SNP 35

Ukip 4

Green 0

Plaid Cymru 2

In other words the hung parliament we all expect, but Labour with the most seats…?

Clearly this discrepancy (Tories favourites to win most seats, but constituency seats implying Labour to win most) is not one offering a particularly easy arbitrage opportunity, but it does point to the complexity of the forthcoming election – how will the individual fights in constituencies influence the overall outcome?