It seems antithetical to publish a survey that does not reflect reality. One might wonder why an organization would want to circulate a sloppy and misleading poll with its name in bold print at the top. The recently released NBC News-Wall Street Journal public opinion study is a textbook case of a national survey that is more of a hot mess than an indication of the election to come.
Conducted by Hart Research Associates/Public Opinion Strategies, this poll (and we use the term loosely) caused NBC News to publish a headline bellowing that former Vice President Joe Biden is 14 points ahead of incumbent President Donald Trump. So, what is the problem here?
How about everything.
This examination of the American public included 800 registered voters. Yes, a whopping 800 registered voters. Forget the mini sample size for a second and look at the type of voter questioned. There is quite a difference between registered voters and likely voters. Loads of people are registered, but when you conduct a survey, you want to talk to people who say they are likely to vote or have cast a ballot sometime in this century.
After the 2016 election, SurveyMonkey reported that Democratic-leaning registered voters who stayed home cost Hillary Clinton the election. The bottom line here is that the gold standard in polling is to query “likely,” not registered, voters.
The number of partisans sampled is another indication of whether you are looking at a poll that is worth its salt or one that should be flattened on the floor to paper-train a puppy. In general, a thoughtful pollster wants the quantity of Democrats and Republicans questioned to reflect an accurate portrait of the electorate.
According to a May 2020 study by the Gallup organization, 40% of Americans say they are Independent, 31% identify as Democrat, and 25% as Republican. It seems that 40% for Independents is a typo – but it is not. Causing further confusion in the party-affiliation equation is that 22 states do not require or permit registered voters to designate political party preference. Thus, it can be a challenge to figure out how many Democrats versus Republicans voted in the last election.
This NBC/WSJ survey questioned 45% who identify or lean Democratic and 36% who identify or lean Republican, and 13% who claim they are Independent. We can be sure that a 45%-36% Democrat/Republican split almost certainly did not occur in 2016. So how does this accurately reflect likely voters? It does not.
Then there is question number 12: “And, did the presidential debate make you more likely to support Donald Trump, more likely to support Joe Biden, or did it not make a difference in how you are going to vote?” The results are quite fascinating – not because 19% said they were more likely to support Biden versus 6% for Trump but because three out of four previous trends in this tab got it wrong.
So, for 2016, its trends showed that Hillary Clinton swayed 31% of voters toward her side after the first debate, Trump just 14%. For 2012, its poll after the candidates' initial head-to-head encounter showed 27% more likely to vote for Mitt Romney, 24% for Barack Obama. And following their first showdown in 2004, 33% said they were more likely to vote for John Kerry, 17% for George W. Bush. It’s a textbook on how to get it wrong every time.
In the article announcing this national survey, NBC quotes its pollster Jeff Horwitt of Hart Research Associates as saying, “The clear loser from the debate was Donald Trump.” That may very well depend upon which poll you cite, but the clear loser of this NBC/WSJ survey is not President Trump – but rather pollster Horwitt.
Myths and legends such as these have a way of metastasizing into the public consciousness. Educated voters should steer clear of the fairy tales or at least be aware that what they are reading is not an accurate portrait of the American electorate.
~
Read more from Leesa K. Donner.
The Fourteen-Point Myth
Conducted by Hart Research Associates/Public Opinion Strategies, this poll (and we use the term loosely) caused NBC News to publish a headline bellowing that former Vice President Joe Biden is 14 points ahead of incumbent President Donald Trump. So, what is the problem here?
How about everything.
This examination of the American public included 800 registered voters. Yes, a whopping 800 registered voters. Forget the mini sample size for a second and look at the type of voter questioned. There is quite a difference between registered voters and likely voters. Loads of people are registered, but when you conduct a survey, you want to talk to people who say they are likely to vote or have cast a ballot sometime in this century.
After the 2016 election, SurveyMonkey reported that Democratic-leaning registered voters who stayed home cost Hillary Clinton the election. The bottom line here is that the gold standard in polling is to query “likely,” not registered, voters.
The number of partisans sampled is another indication of whether you are looking at a poll that is worth its salt or one that should be flattened on the floor to paper-train a puppy. In general, a thoughtful pollster wants the quantity of Democrats and Republicans questioned to reflect an accurate portrait of the electorate.
According to a May 2020 study by the Gallup organization, 40% of Americans say they are Independent, 31% identify as Democrat, and 25% as Republican. It seems that 40% for Independents is a typo – but it is not. Causing further confusion in the party-affiliation equation is that 22 states do not require or permit registered voters to designate political party preference. Thus, it can be a challenge to figure out how many Democrats versus Republicans voted in the last election.
This NBC/WSJ survey questioned 45% who identify or lean Democratic and 36% who identify or lean Republican, and 13% who claim they are Independent. We can be sure that a 45%-36% Democrat/Republican split almost certainly did not occur in 2016. So how does this accurately reflect likely voters? It does not.
Then there is question number 12: “And, did the presidential debate make you more likely to support Donald Trump, more likely to support Joe Biden, or did it not make a difference in how you are going to vote?” The results are quite fascinating – not because 19% said they were more likely to support Biden versus 6% for Trump but because three out of four previous trends in this tab got it wrong.
So, for 2016, its trends showed that Hillary Clinton swayed 31% of voters toward her side after the first debate, Trump just 14%. For 2012, its poll after the candidates' initial head-to-head encounter showed 27% more likely to vote for Mitt Romney, 24% for Barack Obama. And following their first showdown in 2004, 33% said they were more likely to vote for John Kerry, 17% for George W. Bush. It’s a textbook on how to get it wrong every time.
In the article announcing this national survey, NBC quotes its pollster Jeff Horwitt of Hart Research Associates as saying, “The clear loser from the debate was Donald Trump.” That may very well depend upon which poll you cite, but the clear loser of this NBC/WSJ survey is not President Trump – but rather pollster Horwitt.
Myths and legends such as these have a way of metastasizing into the public consciousness. Educated voters should steer clear of the fairy tales or at least be aware that what they are reading is not an accurate portrait of the American electorate.
~
Read more from Leesa K. Donner.
.jpg%20MAGA&w=1920&q=75)
.jpg%20UK%20Elections&w=1920&q=75)
