Pollster Gallup has claimed since the year 2001 that the majority of Americans (averaging 59% between 2001-2018) hold much greater sympathy for Israelis than for Palestinians (averaging 17%) “in the Middle East situation.” Gallup releases its yearly results with titillating headlines, such as “Americans Remain Staunchly in Israel’s Corner” (2018 poll) and "Israel Maintains Positive Image in U.S." (2017 poll).
Gallup’s results are uncritically amplified in mainstream and Israel partisan news outlets as the single most reliable indicator of Israel’s ongoing high standing among Americans. Gallup also combines "sympathy" with other polling results to distill policy implications such as "Americans would lean more on the Palestinians to make peace." Gallup hasn’t missed a single year asking the sympathy question in surveys. This means Gallup alone can confidently tout historic high points, as it did in its 2018 survey results claiming "favorable views of Israel at 17-year high."
Gallup’s polls have been repeatedly cited in the benchmark Congressional Research Service annual report U.S. Foreign Aid to Israel to explain Israel’s status as the leading U.S. foreign aid recipient since 1948.
"Though aid to Israel has both supporters and detractors, overall U.S. public support for Israel remains strong. According to a February 2011 Gallup poll that measured Americans’ sympathies toward the disputants in the Israeli-Palestinian conflict, a near record-high 63% said their sympathies lie more with the Israelis."
Using Gallup results as a proxy is necessary, explained CRS report author Jeremy Sharp, because "There is less specific public polling data on support for aid to Israel."
Gallup 2001-2018 results: "In the Middle East situation, are your sympathies more with the Israelis or more with the Palestinians?"
1 Source: The Gallup Organization
How accurate are Gallup polling results in general? Like many other mainstream pollsters, Gallup wrongly predicted a win for presidential candidate Hillary Clinton over Donald Trump right up to the day of the 2016 election. Gallup more famously predicted in 1948 that Thomas Dewey would beat Harry S. Truman and also incorrectly forecast a slim victory for Gerald Ford over Jimmy Carter in 1976. While Gallup correctly predicted the winner of the 2008 presidential election, it was ranked by a Fordham scholar at 17th out of 23 polling organizations in terms of precision. In 2012 Gallup wrongly predicted a Mitt Romney win, and received a drubbing by poll analyst Nate Silver as the least accurate among 23 major firms, with a 7.2 point gap from the final result. More gravely, Gallup was criminally charged and paid fines in 2012 over well-documented charges of falsifying data.
In 2012 Gallup was criminally charged under both the False Claims Act and Procurement Integrity Act. The False Claims Act is the primary tool the U.S. government uses, leveraging insider whistle-blowers, to combat contracting fraud against the government. A 57-page complaint (PDF) documents Gallup’s intentional delivery of deceptive data to the U.S. Mint, the U.S. Department of State, and DHS (FEMA) to maximize profits on no-bid contracts. Among other acts alleged in the complaint were keeping two sets of books and making job offers to federal employees in positions to renew Gallup’s government contracts. At the time Gallup was charged, it was attempting to win Department of Defense contracts.
The objective of Gallup’s contract with the U.S. Mint was to assess U.S. public acceptance of new dollar coins. Gallup billed the U.S. mint for more than twice as many work hours as it performed (51,525 hours billed versus 21,627 actually performed according to the complaint). But overbilling was not Gallup’s most costly fraud. For all its self-proclaimed acumen as "the most trusted name in polling" Gallup also either vastly overestimated the public acceptance of dollar coins or massaged data to falsify the results. The Government Accountability Office incorporated results from a November 21, 2006 Gallup polling report titled "Americans Support Dollar Coins Featuring Past Presidents" claiming a "95 percent confidence that the maximum margins of sampling error are +/- 5" within its own 2011 GAO forecast (PDF) that growing public acceptance of the dollar coins would save the government (and taxpayers) $5.5 billion over three decades.
Gallup and GAO predictions soon led to disaster. U.S. Treasury Secretary Timothy Geithner announced that, because of budget constraints and increasing stockpiles of the unpopular Presidential dollar coins, circulation would be suspended on December 11, 2011. Some 1.4 billion of the dollar coins were stockpiled, a number that was on track to reach 2 billion in 2016.
The Gallup Organization avoided criminal prosecution and determination of liability for false claims only by paying a settlement of $10.5 million. The dollar coin was a very bad recipe cooked up by many chefs (including Congress), only to be served to U.S. government agencies and the American public with Gallup data setting the table. Because Gallup’s integrity as a pollster is questionable, it is wise to consider Gallup results with those of less problematic polling organizations.
Gallup vs Pew 2001-2018 average results: "sympathies more with the Israelis or more with the Palestinians?"
2 Source: The Gallup Organization and Pew Research data
Pew Research, a nonprofit charitable research organization funded by the Pew Charitable Trusts, has fielded a similar "sympathy" question since 2001, though data is unavailable for years 2008 and 2015, asking "In the dispute between Israel and the Palestinians, which side do you sympathize with more, Israel or the Palestinians?" Average 2001-2018 Pew Research results on "sympathy for Israelis" are a full 12 percentage points lower than Gallup, while "neither/both/no opinion" responses average fully 15 points higher. Pew found sympathy for Israelis as the majority response only three times (2014, 2015 and 2017) between 2001-2018 where Gallup claimed a majority every single year. Even though Pew lacks Gallup’s troubled recent history and inherent conflicts of interest, because Pew’s "sympathy" results, express more respondent uncertainty and lack splashy release PR, they enjoy far less circulation and prominence than Gallup’s. Even so, both results may not be accurate enough to inform bona fide policy debates.
Both Gallup and Pew still rely principally on telephone interviews to field survey questions. For the 2018 survey fielded February 1 – 10 Gallup contacted a random sample of 1,044 adults in all states plus the District of Columbia. It then weighted results with its own proprietary "computed design effects." Pew similarly polled 1,503 adults via landline and cell phone and used its own iterative weighting technique to match respondents to Census data.
Pew Research identified "nonresponse bias" as a major industry-wide problem in the 2016 the Clinton vs Trump polling debacle. It occurs, according to Pew, when certain kinds of people systematically do not respond to surveys despite equal opportunity outreach. Non-response can come from people who are alienated by the questions in some way or, according to Google Surveys, simply do not like answering surveys in general. Only stronger assurances of confidentiality, guaranteed aggregation and intelligent incentives can mitigate this type of bias.
In the 2012 election, a year Google Surveys was the second highest ranking in terms of accuracy, poll analyst Nate Silver noted that "polling firms that conduct their surveys online had strong results. Some telephone polls also performed well. But others, especially those that called only landlines or took other methodological shortcuts, performed poorly…" Telephone surveys inherently introduce "modal bias." This is when respondents try to be "more agreeable" with their interviewer, picking up cues and saying what they think the interviewer wants to hear.
What would the results of the "sympathy for Israelis versus Palestinians" be if the question were fielded through the cutting-edge, online-only, #2 in the 2012 ranking, Google Surveys, which taps a vast population of Internet users, who are appropriately incentivized to respond, less biased against answering questions and less inclined to shape their answers to please telephone interviewers? Tens of millions of Americans (PDF) are now accessible through Google Survey’s 1,500 partner websites and smartphone users. Spam and bogus answers are automatically removed.
To find out, IRmep fielded Gallup’s precise "sympathy" question wording through Google Surveys April 2-10 targeting 2,044 respondents. The weighted results, based on 1,609 responses, are vastly different than Gallup and Pew.
IRmep through Google: In the Middle East situation, are your sympathies more with the Israelis or more with the Palestinians?
3 Source: IRmep poll through Google Surveys
The IRmep poll fielded through Google reveals a majority of Americans (51.1%) unabashedly express "no opinion" on the matter, while only 20.7% express sympathy for the Israelis, 10.5% express sympathy for "both" and 9.7% say "neither."
It must be noted that the IRmep poll fielded through Google Surveys was a single question (Gallup and Pew field multi-question surveys). It was also conducted during the deadliest period thus far of the "Land Day" marches during which Israeli soldiers killed and wounded Palestinian demonstrators. But there is also some consistency with Gallup and Pew polls in that Palestinians still garner only roughly a third the amount of sympathy that Israelis receive. (Raw data from this poll and links to demographic data are available directly from Google Surveys and IRmep’s poll web page).
Google Surveys presents a strikingly different method – albeit with its own limitations – to gauge public opinion and avoid the inherent bias of phone-based interviews. Google, like Gallup and Pew, also must process raw data. Google’s highly automated platform processes response data against inferred (from Census data) and known (gathered directly from users with their permission) demographics to perform its own post-stratification weighting. Individuals or organizations paying to field such surveys cannot intervene or make their own adjustments to results, adding an additional layer of confidence.
Both Gallup and Pew would face challenges migrating to more accurate online/smartphone based polling platforms. One challenge would be that Google Surveys does not yet offer "answer piping" or changing subsequent questions based on prior answers. It also has limited capacity for the total number of questions. But the biggest single challenge for mainstream pollsters "is legacy." By migrating to better platforms, increased accuracy would drive a drastic change in their results. All their costs sunk into decades of results data would be forfeit the moment they made the switch. It simply would not be worth comparing to newer, more accurate data.
The stakes in the "Israeli Palestinian sympathy" question – and many others – are now far too high for pollsters – whether known to engage in false claims or those compromised by outdated survey methodologies – to be allowed to skew key public policy debates. Military conflagrations, billions in U.S. foreign aid and nonprofit charitable contribution dollars are all tied to public opinion and impact the fate of millions. Gallup users confidently trumpet American support for Israel as fact, when it is clearly fiction. There is also no longer any need for "sympathy" questions to serve as stand-ins for direct foreign aid questions. Multiple pollster results have confirmed that the majority of Americans, when asked in different ways, at times with comparative information, consistently say U.S. aid to Israel is either "too much" or "much too much." That most of the media – and especially the Congressional Research Service – staunchly refuse to recognize this is as harmful as it is unacceptable.
Grant F. Smith is the director of the Institute for Research: Middle Eastern Policy in Washington and the author of the 2016 book, Big Israel: How Israel’s Lobby moves America which is now available as an audiobook.
