polling

Public Opinion 101 teaches that the wording of a survey question can make all the difference in the world. What happens when the surveyors misreport what they asked? This spring, on charitable choice, the journalists followed them over the cliff.

On April 10, the Pew Forum on Religion in Public Life issued the first in a planned annual series of religion-in-public-life polls. Conducted jointly with the Pew Research Center for the People and the Press, the survey produced a bevy of interesting, timely data, but the big news had to do with public attitudes toward President Bush’s "charitable choice" initiatives for faith-based social service providers.

"Faith-based Funding Backed, But Church-State Doubts Abound," announced the report, declaring on its first page, "Most notably, while the public expresses strong support for the idea of faith-based groups receiving government funding to provide social services, in practice, it has many reservations. Most Americans would not extend that right to non-Judeo-Christian religious groups" or to "groups that encourage religious conversion."

The poll report went on to say, repeatedly, that the public would not "allow" specific religious groups to apply for government social service funds. Also, a table listing the results for the different groups was headed: "Who Should be Eligible for Government Funds?"

In fact, however, the survey did not ask the public for an opinion on whether specific religious groups should be eligible to apply for government funds.

According to the published questionnaire, all it asked about eligibility was whether or not respondents favored, "allowing churches and other houses of worship to apply, along with other organizations, for government funding to provide social services such as job training or drug treatment counseling to people who need them."

It then moved on, asking for responses to a wide range of other matters (36 in all) before circling back to pose another question about faith-based funding. But this time, there was no mention of "allowing" or "not allowing" groups to apply. Instead, respondents were told: "I’m going to read the names of some specific religious groups. For each one that I name, please tell me whether you would favor or oppose this group applying for government funds to provide social services to people who need them."

Opposing a group’s applying might have been taken to mean opposing their right to apply. But it might also have been taken to mean not wanting them to apply, or not wanting them to succeed in their application. The question is highly ambiguous.

The public’s response to the general question of "allowing churches and other houses of worship to apply" was very positive. Seventy-five percent were in favor (up from 67 percent in a Pew Research Center poll of September 2000 that included the same wording).

Responses to the question about specific religious groups varied. Majorities said they were in favor of the following groups applying: "Catholic churches" (62 percent in favor, 32 percent opposed, and 6 percent don’t know/refused to answer), "Protestant churches" (61-31-8), "Jewish synagogues" (58-34-8), and "Mormon churches" (51-41-8). More people were opposed than in favor when it came to "Muslim mosques" (38-46-16), "Buddhist temples" (38-46-16), the "Nation of Islam" (29-53-18), the "Church of Scientology" (26-52-22), and "groups that encourage religious conversion as part of the services they provide" (32-59-9).

It is likely that respondents would have been more favorable—across the board—had they been asked if they favored or opposed allowing each of the above groups to apply. Unfortunately, they weren’t asked—and we don’t know the answer.

The Pew Research Center’s own September 2000 poll underscored how sensitive the public is to wording distinctions. Some respondents got a version of the funding question that asked about, "giving government funding to religious organizations so they can provide services…." (54 percent approved). But as this survey’s report noted, "There is considerably more backing (67 percent) when the issue is recast as allowing such groups to apply for government funding" (emphasis in original).

In short, the presence or absence of the phrase "allowing religious organizations to apply" can cause a sizeable shift. (A shift of only five percent from negative to positive in the more recent poll would have given both Muslims and Buddhists slim pluralities.)

Yet in April both the pollsters and the press reported the new data as if all versions of the question had been cast as "allowing" religious groups to apply. On April 10 Andrew Kohut, director of the Pew Research Center, told NPR’s Linda Wertheimer that while Americans approve the abstract idea of allowing religious organizations to apply they have "many reservations in practice. Most Americans would not favor allowing non-Judeo-Christian religious groups to apply for such funds."

Wertheimer then prompted Kohut to confirm the public’s reservations on specifics such as "whether groups should be allowed to proselytize or require participants in their programs to attend services." "All of these things," nodded Kohut. (In fact the survey did not mention several relevant specifics of the charitable choice policy, such as its requirement that religious services be optional, or its mandate for the availability of secular alternative providers.)

Taking their lead from the survey report, in the next day’s Washington Post Hanna Rosin and Thomas Edsall thought that the survey had demonstrated that most Americans "balked" when asked about the "specifics" of the initiative, "such as exactly which religious groups should be eligible for public funds and whether they could proselytize." "[T]he program faces major hurdles winning public support," they added.

Similarly, in the April 11 New York Times Laurie Goodstein wrote that, "while Americans favor the principle behind Mr. Bush’s initiative, they oppose many of the details…only 38 percent of respondents favored allowing Muslim mosques to apply for government funds, the same percentage that approved letting Buddhist temples apply." The Christian Science Monitor, Houston Chronicle, and the Washington Times were among the news organizations that also used such language in reporting the story.

Only Laura Meckler of the AP seemed to sense that something was amiss. After reviewing the list of non-Judeo-Christian groups that apparently made the public nervous, she noted that the survey "did not ask people whether their concerns about such groups were significant enough to scuttle support for the general idea."

The budget line suggested by the survey report—‘poll shows that public rejects specifics of charitable choice’—encouraged journalists to accentuate the negative. Both the Times’ Goodstein and the Post’s Rosin and Edsall singled out findings that a majority of the public agreed that an "important concern" about charitable choice was that welfare recipients "might be forced to take part in religious practices" (60 percent), and that the "government might get too involved in what religious organizations do" (68 percent).

But the authors failed to mention that three additional "important concerns" generated substantially less worry: interfering with separation of church and state (52 percent), failing to meet standards (47 percent), and increasing religious divisions in the country (48 percent). More importantly, they failed to mention the public’s positive response to a set of four "important reasons to favor" the idea: religious groups can do a better job because religion changes lives (62 percent), religious providers are more caring (72 percent) and more efficient (60 percent), and people who need services "should have a variety of options" (77 percent).

Put up a list of con reasons and people agree with most of them. Put up a list of pro reasons and they agree more strongly. What the survey really showed was that the American public was fairly positive about charitable choice. But that wasn’t the way it played.

"Critics of the Bush plan used the findings as ammunition," wrote Rosin and Edsall. That was putting it mildly. Through April and May, the survey report helped establish the conventional wisdom that the public didn’t want to allow some religious groups to apply for government money under charitable choice, and was very worried about the program’s specifics.

Into the many self- and enemy-inflicted wounds in the President’s faith-based initiative, the poll story rubbed some undeserved salt.

—Dennis Hoover

For the questionnaire, which was included as an appendix to the survey report, see http://pewforum.org/events/0410/report/topline.php3.

See the Letter to the Editor, and Dennis Hoover's reply, in the Fall 2001 issue.

See companion article in this issue, Faith-Based Update: Bipartisan Breakdown

Related Articles:

Faith-Based Ambivalence, Religion in the News, Spring 2001

Ten Issues to Keep an Eye On, Religion in the News, Spring 2001

Charitable Choice and the New Religious Center, Religion in the News, Spring 2000

A Different Spiritual Politics, Religion in the News, Summer 1999

Religion and the Post-Welfare State, Religion in the News, Summer 1998

Missing the Boat on Charitable Choice, Religion in the News, Summer 1998