Wednesday, April 26, 2017

The ghost of #Pounds4McDougallGate

I must admit I burst out laughing when I heard that Blair McDougall is going to be Labour's general election candidate in East Renfrewshire, but once I regained my composure I realised that this is a symptom of a major dilemma for the unionist camp.  There's no doubt that the SNP are vulnerable in East Renfrewshire, but they'll start looking a hell of a lot less vulnerable if the Labour/Tory vote is split right down the middle.  Budding unionist tactical voters can't do much until they work out who the real challenger is, and that's far from clear at the moment.  The political history of the area, taken in combination with current opinion polls, would suggest the seat ought to be a straight fight between SNP and Tory, and that Labour's resilience in 2015 was a one-off due to Tories temporarily lending their support to Jim Murphy on a mass scale.  And yet anyone 'tactically' voting for the Tories this time will have to take a leap of faith and assume that natural Tory voters won't be coaxed by the electrifyingly charismatic McDougall and his trusty Lib Dem-style bar-charts into believing that "only Labour can beat the SNP here".  There will be a similar dilemma in East Dunbartonshire, where diehard unionists will have to guess whether Jo Swinson's candidacy means there will not be the mass-switch from Lib Dem to Tory that you might otherwise expect in an area with such a strong Tory tradition.

* * *

SCOT GOES POP POLL OF POLLS

The second update of our Poll of Polls for Scottish voting intentions at the general election is based on two full-scale Scottish polls (from Panelbase and Survation), and eight subsamples (two from ICM, one from Ipsos-Mori, one from Panelbase, one from ComRes, one from Survation, one from Opinium and one from YouGov). The GB-wide poll from Kantar/TNS has had to be excluded because no geographical breakdown was provided.

SNP 43.6% (-0.7)
Conservatives 30.4% (+6.1)
Labour 15.3% (-0.7)
Liberal Democrats 7.0% (+0.7)


(The Poll of Polls uses the Scottish subsamples from all GB-wide polls that have been conducted entirely within the last seven days and for which datasets have been provided, and also all full-scale Scottish polls that have been conducted at least partly within the last seven days. Full-scale polls are given ten times the weighting of subsamples.)

Tuesday, April 25, 2017

Drama as No-friendly TNS poll finds half of the Scottish public want an independence referendum

I've been having a look at the newly-released datasets from TNS to see if there is any potential explanation (other than the obvious one of data collection method) for why they are contradicting three other pollsters in showing a big swing to No.  What leaps out at me are the weightings for recalled Holyrood vote, and the highly unusual way TNS treat respondents who say they didn't cast a vote last year.  Most pollsters who weight by recalled vote do not try to upweight abstainers until they reflect the actual abstention rate.  There are two very good reasons for that approach : a) people are embarrassed to admit that they didn't vote, meaning a significant proportion will lie and claim they did turn out, and b) people who do openly admit they didn't vote are particularly unlikely to vote again in future elections anyway.

That being the case, you'll quickly spot the problem in the fact that only 27% of the unweighted TNS sample either said they didn't vote last year or can't remember how they voted, and that TNS decided to massively upweight that group to 42%.   It looks highly likely that disproportionate weight has been given to a hard-core of non-voters.  That doesn't explain all of the swing to No by any means - there is movement in that direction almost across the board among voters for most parties.  But the swing among abstainers from last year is very large - they've gone from being virtually split down the middle in the last TNS poll to being in favour of No this time by a 21-point margin.  The massive upweighting will obviously have artificially magnified the effect of that.

The biggest downweighting on the recalled vote is among people who say they voted SNP - 38% of the unweighted sample were SNP voters, and they were scaled down to count as just 27%.  That obviously has a significant detrimental effect on the reported Yes vote.  It's not unreasonable to speculate that 'embarrassed abstainers' who falsely claim to have voted last year may have defaulted to saying they voted for the winning party, so while it's possible that TNS may have interviewed too many SNP voters by chance, it's also possible that this group has been downweighted too much, leading to distorted headline numbers.

In addition, there's a very familiar problem with respondents who recall voting for an "other" party - meaning a party other than the SNP, Tories, Lib Dems or Labour.  This small group often ends up being very sharply downweighted, because people are asked how they voted on the constituency ballot, but instead find themselves recalling their vote for the Greens or UKIP on the list.  In the new poll, this had led to them being scaled down from 3.4% of the raw sample to count as just 0.6%.  It's blindingly obvious that TNS aren't giving them sufficient weight, and as it happens, they are the only group that didn't show any movement to No at all.  They also broke only very marginally for No overall.  If there had been a more realistic target figure for "others" to take account of the confusion between the constituency and list vote, this factor alone could conceivably have slightly reduced the reported swing to No.

You'll have seen a lot of hysterical coverage today about how this poll supposedly shows that the public don't want an independence referendum.  You probably won't faint with amazement to discover that it shows no such thing.  Excluding Don't Knows, 49% of respondents chose one of the four pro-referendum options provided by TNS, and 51% chose the sole anti-referendum option provided.  That's within the margin of error, so must be regarded as a 'statistical tie', and is strikingly similar to the findings of recent Panelbase polls which have also shown voters split down the middle.  It's also worth pointing out that if the TNS poll does turn out to be a rogue poll with too many No voters in the sample, the 49% in favour of holding a referendum is likely to be an underestimate.

Nostalgia night as TNS reverts to being an extreme No-friendly outlier

You might remember that throughout much of the long campaign leading up to the 2014 independence referendum, TNS was one of the group of No-friendly pollsters, before sensationally swinging in the opposite direction just before polling day.  That dramatic reversal survived into the post-referendum period, with a TNS poll in the autumn of 2015 putting Yes in the outright lead.  Now, incomprehensibly, and in total contradiction of the vast bulk of polling evidence from other firms, TNS has returned to its old ways by showing a very large No lead.

Should Scotland be an independent country? (TNS)

Yes 40% (-7)
No 60% (+7)

This is by some distance the worst poll for Yes from any firm since September 2014.  The previous low was Yes 43%, No 57% from YouGov.

The ever-reliable buffoons who insist that "only the last poll we set eyes upon matters" will inevitably lose all sense of perspective over this, but those of us who are a little more level-headed will recognise an indisputable fact here - that this poll can't possibly negate the much more favourable polls we've seen for Yes over the last few days, for the simple reason that it was conducted earlier.  TNS polls are always way out of date by the time that we see them, and this one is no exception - fieldwork started in late March and concluded two weeks ago, which dates it well before the Panelbase and Survation polls.  The majority of interviews also took place before BMG found a virtual 50/50 tie.  So the verdict from those three online pollsters is clear enough - they are more up-to-date, and they do not corroborate the findings of TNS.

That's not to say that if a more recent TNS poll had been conducted, it would necessarily have produced a healthier result for Yes.  TNS traditionally use a distinctive face-to-face data collection method, and that could largely explain why they've suddenly bolted off in a different direction from other firms (assuming this isn't an outright rogue poll, which always has to be considered a possibility when the numbers are this unexpected).  And yet it seems highly unlikely that the new No-friendly trend is going to be seen across all non-online polls, because as recently as early March, a telephone poll from Ipsos-Mori put Yes in a slight outright lead - a better result, ironically, than has been seen in any online poll so far this year.  It's going to take time to make sense of what's happening, because at the moment there's just no comprehensible pattern in any of this.  The BMG, Survation and Ipsos-Mori numbers are simply not reconcilable with TNS - the standard 3% margin of error can't explain such a big divergence.

You would have to say that the balance of probability is that Yes are trailing at the moment, but whether they are trailing by 20% as TNS say, or by 2% as BMG say, or whether the truth is somewhere in between those extremes, is anyone's guess.  We mustn't forget just how absurdly far adrift Leave were in most telephone polls before pulling off victory in the EU referendum last June - so it's perfectly possible that online polls are more accurate these days.

*  *  *

SCOT GOES POP POLL OF POLLS

Should Scotland be an independent country?

Yes 45.7% (-1.1)
No 54.3% (+1.1)

(The Poll of Polls is based on a rolling average of the most recent poll from each firm that has reported at least once within the last three months. The firms included in the current sample are Panelbase, BMG, Ipsos-Mori, YouGov, TNS and Survation.)

Monday, April 24, 2017

Some more information about the Panelbase poll

The datasets for Saturday night's Panelbase poll have been published, and the big surprise is that it appears as if 16 and 17 year olds were included in the sample.  At one point, the poll was listed on Wikipedia as definitely excluding under-18s, so if that was dud information, my apologies for repeating it here.

Looked at one way, this is bad news, because it means there's less reason for suspecting that the 45% Yes vote on the independence question may be a slight underestimate.  It also means that, if anything, the SNP's share of the vote in Westminster voting intentions could be a slight overestimate - because of course, young people are disproportionately likely to support the SNP, and under-18s won't be able to vote in June.  However, here comes the good news - both Yes and the SNP suffered in this poll from the effect of rounding.  It looks like the SNP must have been just a tiny fraction away from being rounded up to 45%, rather than rounded down to 44%.  The Tories were also rounded up a touch to 33%.  On the unrounded numbers, the SNP lead over the Tories is a little over 11.5%, rather than the 11% reported on the headline numbers.  The effect of rounding on the independence question was less significant, but nevertheless the unrounded numbers are a tad better : Yes 45.2%, No 54.8%.

Once again, the extent to which the rump Labour vote under Jim Murphy in 2015 was Tory-leaning has been laid bare by this poll - less than half of 2015 Labour voters are planning to stick with the party, while almost a quarter have moved direct to the Tories.  Only 8% have moved to the SNP.  It would be fascinating to ask the 13% of the electorate that are still sticking with Labour who their second-choice party would be.  That group is so small in number that it's hard to even guess who they are and what motivates them.  It certainly doesn't seem to be a 'working class' thing, though - Labour's support is only a little higher among the less affluent part of the sample than among the most affluent.  As you'd expect, they do significantly better among No voters from 2014, although mystifyingly, 8% of Yes voters have stayed loyal to them.  (On the plus side, that means there may still be some limited scope for a further Labour-to-SNP swing, unless that 8% is composed almost entirely of people who have changed their minds on independence since the referendum.)

Sunday, April 23, 2017

Terror strikes Tyrannical Theresa as astonishing BMG poll suggests Yes has practically drawn level

Should Scotland be an independent country? (BMG)

Yes 43% (+2)
No 45% (+1)

UPDATE : When I first put this post up, I thought I'd only have a few minutes to wait before more details appeared on the Herald website (the last few BMG polls have all been commissioned by the Herald), revealing what the numbers with Don't Knows excluded are, whether there are Westminster voting intention numbers, etc, etc.  Instead all we've got to prove the poll even exists is a single tweet from the Britain Elects account - which admittedly is normally reliable.  It looks like we'll have to wait until the morning for clarity.

As with Survation and Panelbase last night, what's most important about this poll is what it doesn't show.  A few weeks ago, there were YouGov and Panelbase polls published close together which both reported that Yes had slipped below the 45% achieved in the 2014 referendum, thus giving the impression that something had genuinely changed.  But if that was really the case, you'd have expected yesterday's Survation poll to show Yes slipping below the 47% figure that has been so typical in recent months.  You'd certainly have expected BMG to show a drop - and perhaps quite a sharp one - from the heady heights of 48% or 49% recorded in the firm's last two polls.  That hasn't happened.

Of the three polls we've seen this weekend, only Panelbase can arguably be reconciled with the "slippage for Yes" narrative.  Although the 45% Yes vote in that poll represents a 1% increase, it remains slightly below the recent norm.  But even that can potentially be explained away by the rare exclusion of 16 and 17 year olds from the sample.

In short, there is no longer much hard evidence that Yes have suffered any drift at all.  The minimal evidence that does exist is pretty much confined to YouGov polls, and it's possible there's a firm-specific explanation for that.

UPDATE II : Having applied a magnifying glass to a screenshot of the Herald front page, I've finally been able to work out what the BMG figures are with Don't Knows excluded...

Should Scotland be an independent country? (BMG)

Yes 49% (+1)
No 51% (-1)

So with Don't Knows included, the Yes vote is up 2%, and with Don't Knows excluded, it's up 1%.  Let me just gently observe that this renders the Herald's choice of headline ("Independence support fails to rise") more than a touch bizarre!

*  *  *

SCOT GOES POP POLL OF POLLS

Should Scotland be an independent country?

Yes 46.8% (+0.4)
No 53.2% (-0.4)

(The Poll of Polls is based on a rolling average of the most recent poll from each firm that has reported at least once within the last three months. The firms included in the current sample are Panelbase, BMG, Ipsos-Mori, YouGov and Survation.)

Tories left reeling as new polls suggest support for independence has INCREASED

It shows you how much the calling of a snap general election changes our priorities, but we've just had two new polls in which the question about independence has been treated as an afterthought.  Let's put that right, because the findings make for moderately pleasant reading.  Here are the Survation numbers...

Should Scotland be an independent country? (Survation)

Yes 46.9% (+0.1)
No 53.1% (-0.1)

A 0.1% swing in favour of Yes is obviously not remotely significant, but here's the thing - this poll is not directly comparable with the last Survation poll, because 16 and 17 year olds were excluded this time.  There's a semi-reasonable excuse for that, because the independence question was a supplementary in a poll that was primarily interested in voting intentions for an election from which under-18s will be excluded.  But it does mean there are reasonable grounds for suspecting that the increase in the Yes vote might otherwise be a tad bigger.  At the very least, there doesn't seem to have been any recent slippage in support for independence.

The Panelbase datasets aren't out yet, but it appears that 16 and 17 year olds were also excluded from that poll.  In spite of that in-built handicap, Yes manages a small increase.

Should Scotland be an independent country? (Panelbase)

Yes 45% (+1)
No 55% (-1)

Basically the Survation numbers are par for the course, and the Panelbase numbers perhaps remain a little below par - but in combination the two polls give the lie to any notion that support for independence is consistently slipping below the 45% achieved in the 2014 referendum.  For what it's worth, it remains the case that the only published telephone poll of the year so far actually gave Yes a very slight outright lead.

*  *  *

I'm still not sure how the Greens fared in the Panelbase poll, but we're not going to get an answer from Survation, who seemingly just lumped the Greens in with UKIP and others in a general "some other party" category.  That said, including the Greens can also produce a distorted outcome, because people might indicate that they are planning to vote Green when there isn't even a Green candidate in their own constituency.

*  *  *

Scottish subsamples of GB-wide polls are obviously much less meaningful than full-scale Scottish polls.  Nevertheless, the first inkling we had of the Scottish Tory surge over the last few days came from subsamples, which makes it interesting that the two most recent subsamples we have right now are somewhat less favourable for the Tories.  Both are based on fieldwork that is slightly more up-to-date than the two full-scale polls from Survation and Panelbase.  Today's YouGov subsample has the SNP on 49% and the Tories on 27%, while the subsample from today's Britain-wide Survation poll has the SNP on 45% and the Tories on only 19% (behind even Labour).

Saturday, April 22, 2017

SNP lead the Tories by 15% in first full-scale poll of the campaign

The first word on Scottish voting intentions for the general election comes tonight from Survation...

Westminster voting intentions (Survation poll) :

SNP 43.1%
Conservatives 27.9%
Labour 17.8%
Liberal Democrats 8.8%

A Panelbase poll will apparently shortly reveal a similar picture, but with the Tories on 30% or above.  I don't think there's any great surprise that there has been a boost for the Tories since the election was called - that's happened across the UK, and we're not immune to UK-wide trends.  However, it's startling that Survation seem to be suggesting that the Tories have taken support from the SNP rather than Labour.

Believe it or not, there is an important upside to this - we'll now see an enormous amount of hype about the prospect of the Tories winning ten or more seats, which will turn the expectations game on its head.  If the Tory surge recedes (sudden surges often have a habit of receding) and if the SNP end up with 50+ seats, that will now look like the spectacular triumph that it is, rather than a slight disappointment.

UPDATE : The Panelbase numbers have finally been revealed, although we've had a fair idea of what was coming for a couple of hours...

Westminster voting intentions (Panelbase poll) :

SNP 44%
Conservatives 33%
Labour 13%
Liberal Democrats 5%

The good news here is that, contrary to the expectations that had been building up as the night wore on, the SNP share of the vote is actually holding up fractionally better in the Panelbase poll than in the Survation poll, meaning that at least some of this much larger Tory surge has come from Labour. The combined vote for the three main unionist parties is just 51%, compared to 54.5% in Survation. The bad news obviously is that the SNP lead is "only" 11%, but with Survation showing something much less dramatic, we can't rule out the possibility that the Panelbase poll will eventually be looked back on as an extreme outlier that led us completely astray.

I've tended to assume that Labour will probably hold their sole seat due to tactical voting, but there comes a point where their national vote share is so low that all bets are off.  Panelbase have them a full 10% lower than they managed even in the Holyrood constituency ballot last year.

There's also no consensus between Panelbase and Survation on how the Lib Dems are faring - the difference between 5% and 9% could be truly mammoth in terms of the party's hopes of picking up a few seats.

So far I haven't been able to find the Green share of the vote from either poll, and that number will be very significant - the Greens won't be standing in every constituency, so a lot of their vote in the other seats (not all of it by any means) could in reality be heading to the SNP.

*  *  *

Although I think there is a very good chance that the Scottish Tory surge is built on sand and will recede as polling day approaches, we mustn't forget that there's another polling day less than two weeks away, and I'm far less convinced that the surge will have receded by then.  In the light of tonight's polls, I cannot stress enough how vitally important it is that as many SNP supporters as possible use all or most of their preferences in the local elections to make sure that other parties and independents are ranked ahead of the Tories.  It's inevitable that there's going to be a significant increase in the number of Tory councillors, but we can minimise that increase by using our lower preferences, and we can do it at no risk at all to the SNP.

Thursday, April 20, 2017

Introducing the Scot Goes Pop Poll of Polls for the general election

Just for the sheer hell of it, we might as well have a Poll of Polls for Scottish voting intentions at the general election. I was going to check which methodology I used last time around...and then I realised I knew exactly which methodology I used, because the last general election was only about fifteen seconds ago.

As before, Scottish subsamples of GB-wide polls will be taken into account, because they often represent the only up-to-date information we have.  That's obviously unsatisfactory because the figures are not properly weighted, but an average of several subsamples shouldn't lead us quite as wildly astray as an individual subsample might.  Whenever a full-scale Scottish poll comes along, that will be given ten times the weight of a subsample.  At the moment all we have from the last week are three subsamples (two from ICM and one from YouGov), so the following figures should be treated with great caution...

SNP 44.3%
Conservatives 24.3%
Labour 16.0%
Greens 7.0%
Liberal Democrats 6.3%
UKIP 1.7%

*  *  *

I have a new article at the TalkRadio website arguing that the election could be an opportunity in disguise for Jeremy Corbyn, and a risk in disguise for Ruth Davidson.  That's not a prediction of how I think things will play out, but just a reminder that the outcome of this campaign is not yet set in stone.  You can read the article HERE.

*  *  *

Kezia Dugdale describing the Labour party as a "progressive alliance" reminded me of Voltaire's joke about the Holy Roman Empire - "it was neither holy, nor Roman, nor an empire".

Your STV questions answered

Over the last few days, I've received a number of emails asking for clarification on various aspects of the Single Transferable Vote system that will be used to elect councillors on 4th May.  Apologies that I haven't been able to respond to everyone individually, but it probably makes more sense to answer a few of the questions in a public post anyway.

* First of all, someone contacted the Edinburgh Election office to ask whether he should or shouldn't rank certain candidates if he doesn't want any portion of his vote to be transferred to them.  I'm not quite sure what the question was getting at, because having your vote transferred is basically a good thing rather than a bad thing.  Even if you genuinely think three or four candidates are all equally awful, ensuring that your vote doesn't transfer to any of them is a neutral thing rather than a good thing, because by that point they will be the only candidates left in contention for one (or more) seats, meaning that one (or more) of them will still be elected.  The only effect of your non-transferred vote will be to deprive you of any influence over which one is successful.

However, I'm delighted to report that the Edinburgh Election office gave a scrupulously accurate answer (which is rather refreshing giving the amount of misinformation being pumped out by people and organisations that should know better).  They correctly indicated that no part of your vote can be transferred if you haven't given a ranking to any of the candidates remaining in contention for the seats yet to be filled.  Why you would want to prevent your vote being transferred is a bit of a mystery, but for what it's worth that's the definitive answer to the question posed.

* A number of people seem to be deeply troubled by the idea that even if they rank a Tory candidate last (by which I mean absolute last without leaving any preferences blank), part of their vote could technically transfer to the Tory at the end of the counting process.  That's true, but the operative word is "technically" - it really is a complete irrelevance.  If your vote ever reaches the point of being nominally transferred to your bottom-ranked candidate, that means by definition that the candidate in question has effectively already been elected, because all of the other candidates in contention for the last seat in the ward have been eliminated.  The final transfer of votes is just a meaningless formality, and it doesn't in any way affect the popular vote totals reported in the media, which will be based on first preference votes only.  If it bothers you, rank all but one of the candidates rather than all of them - that will allow you to maintain your purity without making any difference to the final seat allocation.  But seriously, don't worry about it - if you use all of your preferences and rank a candidate absolute last, you are emphatically voting against them, and maximising the chances that they will not be elected.

* Someone asked me if it might be a good idea to trawl through actual results from the recent Northern Ireland Assembly election, conducted under STV, to give concrete examples where the DUP only won a seat because nationalist voters didn't use enough of their lower preferences.  At this point, I'll just have to say that life is too short - but I don't have the slightest doubt that such examples exist.  Even in Northern Ireland where this system is so much better understood than it is here, there are many, many voters from both sides of the sectarian divide who do not bother using their lower preferences.  If one community was significantly more likely to use lower preferences than the other, they would gain a telling advantage in the final seat numbers.

* This isn't strictly speaking a reader's question, but I really do need to say something about the dangerous misinformation that has been put out on a certain SNP Facebook page.  Whoever runs that page is using the veneer of authority to mislead people into thinking that giving a lower preference to an independent or unionist candidate can somehow help that candidate overtake an SNP candidate you have ranked higher.  If you've read that claim, IGNORE IT.  It is totally without foundation.  If there are two SNP candidates in your ward, and you rank them 1 and 2, then none of your preferences from 3 onwards will EVEN BE LOOKED AT until and unless both SNP candidates have been either elected or eliminated.  Ask yourself this very simple question : how can a lower preference possibly help a unionist overtake an SNP candidate who has already been elected or eliminated?  It can't.  It's physically impossible.

If you only rank the SNP candidates, then all you are doing is abstaining in contests for seats that the SNP are no longer in the running for.  You might, for example, be abstaining in a straight fight between the Tories and the Greens for the final seat in the ward.  How does that help?

Wednesday, April 19, 2017

Tyrannical Theresa is snapping Scotland out of it

A second 'quick note' of the day, this time to let you know that I have a new article at the International Business Times, about the breathtaking arrogance that Theresa May has displayed over the last 24 hours - refusing to take part in TV debates, peddling fantasies about the UK being "united" behind her, and imagining that she can just wave away the laughable contradictions in her own statements.  You can read the article HERE.