JTA Asks: Which Iran Polling Should We Believe?

In recent days, numerous polls have been released claiming to state where the American Jewish population stands on the Iran nuclear agreement, many of which show drastically different results. With so much conflicting information, it can be difficult to know exactly who to trust. With this question in mind, Uriel Heilman of JTA set out to determine which poll was the most accurate.

Heilman writes:

A close look at each poll’s questions, methodologies and even sponsors quickly makes clear which of the surveys attempt to gauge public opinion and which are meant to influence the public. It’s the difference between honestly trying to find out what people think versus trying to produce a certain result in order to bolster a particular argument.

L.A. Jewish Journal

Note how the three key survey questions below are phrased. The formulations are neutral in their wording, and the interviewer is instructed to rotate the “good idea/bad idea” choices in the last question to ensure a bias-free result.

“As you may know, an agreement was reached in which the United States and other countries would lift major economic sanctions against Iran in exchange for Iran restricting its nuclear program in a way that makes it harder for it to produce nuclear weapons.”

Support the deal: 47.8 percent
Oppose the deal: 27.6 percent
Don’t know: 24.6 percent

“Should Congress vote to approve or oppose the deal?”

Approve: 53.6 percent
Oppose: 34.7 percent
Don’t know: 11.7 percent

[Rotate verbiage in parentheses] “In retrospect, was it a (good idea) or a (bad idea) for the U.S. to conduct negotiations with Iran, or are you not sure whether it was a good idea or a bad idea?”

Good idea: 57.9 percent
Bad idea: 18.2 percent
Not sure: 23.9 percent

The Jewish Journal used the polling research firm SSRS Omnibus, which conducts national, weekly telephone surveys on a variety of subjects. For this survey, SSRS called only respondents in its database who had identified themselves as Jewish in previous unrelated surveys.

Demographer Steven M. Cohen of Hebrew Union College-Jewish Institute of Religion, who does all kinds of Jewish community surveys, designed the questions. In all, 501 Jews were interviewed. The Jewish Journal used a similar methodology to survey non-Jewish Americans’ opinions on the deal (I won’t get into that here). The margin of error on the Jewish survey was 6 percent. The survey was conducted July 16-20.

The Israel Project

After an initial question about whether the respondent is a U.S. citizen, the survey asks:

“Now, generally speaking, would you say that things in the country are going in the right direction, or have they pretty seriously gotten off on the wrong track?”

Right direction: 37 percent
Wrong track: 53 percent
Don’t know/no opinion: 9 percent

Ira Sheskin, a University of Miami demographer who has conducted dozens of Jewish surveys, called it a terrible question.

Instead of framing the alternative of “right direction” as “wrong direction,” it articulates the negative as: “pretty seriously gotten off on the wrong track.”

“I would read that question and dismiss the entire survey,” Sheskin said. “They clearly are looking for particular responses when they do that.”

Let’s take a look at another question from the survey:

“Recently, the United States and five other countries (known as the P5+1) reached an agreement with Iran regarding the lifting of economic sanctions on Iran in exchange for concessions in Iran’s nuclear program. Based on what you know, do you approve or disapprove of this agreement?

Approve: 44 percent
Disapprove: 47 percent
Don’t know/no opinion: 9 percent

Suggesting that Iran agreed only to “concessions” on its nuclear program rather than agreed not to produce nuclear weapons bolsters the argument against the deal.

Finally, the most compelling reason to dismiss the results of The Israel Project poll come from the pollster himself, Nathan Klein. He told JTA he designed the questions to “educate” respondents about the Iran deal over the course of the questionnaire in order to gauge how their views would change once they were better informed. The following question was asked three times over the course of the survey, with the responses shifting over time:

“Now that you have some more information, in your own opinion, do you think that Congress should vote to approve the deal and lift sanctions on Iran or reject the deal and NOT lift sanctions on Iran?”

Near beginning of survey: 40 percent approve; 45 percent reject.
Middle of survey: 35 percent approve; 51 percent reject.
Near end of survey: 30 percent approve; 58 percent reject.

The survey was conducted via email and the questions were designed by Klein, founder of Olive Tree Strategies, former pollster for the National Republican Senatorial Committee and former director of research and messaging at The Israel Project. Klein used a third-party company that relied on previous email surveys it had conducted to find self-identified Jews in its database. Jewish respondents were sent an email blast with a link to the online survey. Only those who affirmed their religion as Jewish were able to complete the survey. In all, there were 1,034 respondents, and the poll was conducted July 21-26. The margin of error was 3 percent.

Though a cheaper method of polling, online surveys are inherently less reliable than telephone surveys, according to Sheskin, the Miami demographer, because studies show that those who feel most strongly about an issue are most inclined to complete online surveys (by contrast, telephone respondents are more likely to cooperate and complete surveys at random). In this case, the online survey probably favors those who oppose the Iran deal, because while the deal’s opponents tend to be adamant in their opposition, many Jewish proponents are openly ambivalent...

So, what’s the upshot?

Though it only has half as many respondents as the other two polls, the Jewish Journal poll clearly is the winner here — on the questions (neutral), the methodology (phone rather than email) and the source (the newspaper isn’t pushing a particular agenda).

You can read the entire piece at JTA here.