Duppy polls, obfuscation, ignorance and simpletons

Bill Johnson, GUEST COLUMNIST

Kevin O'Brien Chang's attempts to besmirch my reputation in The Sunday Gleaner (December 4, 2011) are based on:

1) A poll I never conducted.

2) A deliberate obfuscation of survey details.

3) An ignorance of sampling methodology.

4) An apparent belief that most Jamaican voters are simpletons.

Duppy Polls

Chang starts off by criticising me for a poll he claims I conducted for the 2002 general election. However, the alleged poll is just another in a series of 'duppy' polls that have been attributed to me over the years that I did not conduct.

Just last Sunday, I was told about a national poll I was supposed to have recently conducted for the People's National Party (PNP). This poll purports to show the Jamaica Labour Party (JLP) in front by three points. All of this is very interesting, except for the fact that I did not conduct this poll for the PNP or for anybody else.

( I imagine that several years from now some irresponsible columnist will probably cite this duppy poll as evidence to disparage the relevancy or accuracy of my polls.)

Also, in the months leading up to the 2007 election, one radio station reported on a series of constituency polls it said I had conducted for the PNP. There was just one problem. I had not conducted the polls in question for the PNP or for anybody. In fact, I hadn't even surveyed in one of the constituencies that were mentioned.

To me, these are examples of the dirty tricks that have unfortunately become part of many political campaigns. I would think that a responsible columnist would at least fact-check the information that he is using for character assassination.

Let me make it perfectly clear: the 2002 poll prominently cited by Chang in his column as mine was not conducted by me, if, in fact, it was conducted by anybody at all.

Speaking of 2002

In 2002, I was polling for the PNP. The party had sufficient confidence in the accuracy of my polls that, immediately after the election, I was retained by it to conduct a massive 'post-mortem' survey in order to ascertain the effectiveness of their campaign and the campaigns of their opponents. Would it have commissioned me to do this survey if it had any doubt at all about the accuracy or integrity of my polls?

Would the PNP use me for more than 15 years to do the bulk of its polling if it did not have confidence in my integrity and in the accuracy of my results?

Chang talks about the "paucity of pollsters". This is just not true.

Political polling is a very competitive business in Jamaica and the rest of the Caribbean. Besides half a dozen respected home-grown survey-research firms in the Caribbean, there are dozens of reputable firms in the United States and the United Kingdom which would like nothing more than to do the polling for one of the various political parties or media houses in the country.

Comparing Apples to Ackees to June Plums

Chang continues his attempted character assassination of me with a comparison between three polls conducted before the 2007 general election; a survey of mine (an apple) conducted for The Gleaner, a survey of Ian Boxill's (an ackee) conducted, I believe, for the RJR Group, and a survey of Don Anderson's (a June plum) conducted for CVM TV. He says these were "election call" polls, an expression that can only be attributed to an overactive imagination.

Certainly, The Gleaner never described my poll (the apple) which was conducted over the weekend of August 25 and 26 as an "election call" poll. The election was held on Monday, September 3, and because of the exigencies of newspaper publishing, it was not possible for us to conduct a survey on Saturday, September 1 and have the results ready for publication by press time for Sunday's paper. (The Gleaner did not want to publish its poll results on the day of the election itself.) Therefore, our last poll was a 'snapshot' of the electorate eight to nine days before election day and certainly not what an irresponsible person would call an "election call" poll.

Polling for CVM, Don Anderson had much more flexibility, and I believe his last poll (the June plum) was conducted on Saturday, September 1.

I don't recall when Ian Boxill and RJR conducted their last poll (the ackee), but I believe it was at least a week before the election.

Not listing the dates on which the three polls were conducted, which is one of the commandments of credible reporting, and then falsely calling all of them "election call" polls, is just another example of bad and biased journalism.

Even though The Gleaner decided to stop conducting its surveys more than a week before the election, and because of this our last poll did not pick up a late slight bump for the PNP (some of which was probably caused by the hurricane-relief cheques that were sent out to all parts of the country by the PNP government the week before the election), our results were still within the actual margin of sampling error.

Chang's 'Margin of Error' Error

Speaking of sampling error, Chang (along with others) makes the mistake of confusing the sampling error of a poll with a so-called 'margin of error'. The two are not the same. The sampling error of a poll can be determined scientifically by known formulae. There is no way to scientifically determine a so-called margin of error.

Besides sampling errors, all polls are subject to numerous types of other errors, none of which can be estimated statistically. Some examples of these are: design errors, sample design errors, questionnaire bias, interviewer bias, respondent bias, refusal bias, analytical bias, and reporting errors. As much as I, and other reputable pollsters, attempt to eliminate or at least limit these possible errors, there is no way to validly statistically measure their possible impact.

Chang, the 'error authority', cites "the accepted three percentage-point margin of error (sic) ... ." The fact is that the sampling error of a sample of 1,000 where the responses are between 40 per cent and 60 per cent is approximately +/- 3 per cent at the 95 per cent confidence level for a sample in which every member of the relevant universe has an equal probability of being sampled.

To the best of my knowledge, all the pollsters in Jamaica use a cluster sampling technique for in-person interviewing in which certain numbers of persons are interviewed in pre-selected clusters (or communities or areas). This cluster technique increases the overall sampling error by approximately 50 per cent, or by 1.5 per cent for the aforementioned sample of 1,000, to 4.5 per cent.

The Gleaner survey, taken more than one week before the election, had a sample of 800 persons who stated who they would vote for on the day the survey was conducted. The sampling error for these 800 is +/- 3.4 per cent at the 95 per cent confidence level for a sample in which every resident of Jamaica age 18 and older had an equal probability of being selected. But in fact, we interviewed in 84 different clusters, so that the actual sampling error was approximately +/- 5.1 per cent. Despite the assertions of Chang, our week-old results fell within the margin of sampling error.

Chang also seems to confuse the 'sampling error' with the 'difference of means', but enough of the technical stuff.

Elections in the Caribbean are not won by the overall popular vote but by the number of constituencies each party wins.

Memo to Kevin O'Brien Chang: Jamaicans Are Not Simpletons

Chang seems to find it difficult to believe that Jamaican voters are not simpletons. He apparently finds it incongruous that a person can have a favourable opinion about a party leader and then not plan to vote for that party. He thinks that favourable perceptions of a party leader must equal intentions to vote for the party. If these numbers are not virtually the same and are "outside any margins of error" (there goes the 'error authority' again), he says that they "make no logical sense". Many would say that it is Chang himself who makes no logical sense.

Most students of political behaviour believe that under the Jamaican version of the Westminster form of government, a person's vote is determined by a combination of factors, including: perceptions of the possible prime ministers, perceptions of the parties, and perceptions of the local candidates for Parliament. Of course, who a person actually votes for can also be determined by the prevailing sentiment, if any, in the community (e.g. garrison communities), and whether or not a person actually goes to the polls and votes can also be determined by the proficiency of each party's get-out-the-vote effort.

The relative importance of each of these factors will often change during the course of the campaign.

Because of these multiple factors, we at Johnson Survey Research always ask specific questions (which Chang apparently feels are, or should be, redundant) on:

Who the respondent thinks would make the best prime minister.

Which party the respondent thinks would do the better job.

Which candidate for Parliament the respondent thinks would do the best job as his or her member of parliament.

If the respondent would vote, and if they would vote, how the respondent would vote if the election were being held on the day of the interview.

These questions are meant to both simulate and stimulate the individual voter's thought process.

Most would agree that it would only be natural if some Jamaicans had different individual responses to these different questions, particularly at a time when many are experiencing feelings of cognitive dissonance caused by a new young leader of the JLP.

For example, some persons may like Andrew Holness and feel he should he given the opportunity of leading the country, but at the same time, they are traditionally PNP supporters.

Others who are traditional JLP supporters may think he is too young and inexperienced to lead the country.

These are common examples of possible cognitive dissonance.

Chang says that in my June poll, there was a similarity between those who felt the PNP had the better leader, and was the better party, and their ultimate PNP vote (when Bruce Golding was the PM). However, he suggests that there was "some kind of internal contradiction" in my November poll because there was much less of a correlation between these three factors (when Andrew Holness was the newly chosen PM).

Many would agree that this reduced correlation was most likely caused by a simple and expected case of cognitive dissonance and not some mysterious "internal contradiction", as Chang intimates.

But then most wouldn't write a column of attempted character assassination without having at least a semblance of the facts.

Bill Johnson is a veteran pollster. Email feedback to columns@gleanerjm.com.