I asked our Polling Director, Gary Langer, to weigh in on the Pentagon survey of troops about gays and lesbians serving openly in the military – and the criticism of it from the advocacy group Servicemembers United. (See earlier blog post today HERE.)
Langer says that “the use of agree/disagree questions is poor practice given the well-documented problems of acquiescence bias. Some other questions are hypothetical or ask respondents to guess what others think or would do, which likewise is not a good way to go. Other than these, the rest seem reasonable; phrasing is neutral and response categories are balanced. Characterizations of policy are always open to debate.”
Langer says that it is interesting that the Pentagon uses “gay and lesbian” and “homosexual” interchangeably. “Given questions and concerns about this language, which are hardly new, it would have been best if they had half-sampled each question.”
He disagrees with Servicemembers United, however, in its assertion that it’s a “well-established fact that the use of the term ‘homosexual” induces bias in survey research.”
Not true, Langer says. “It is a subject of discussion and current investigation in the field, with the most recent research indicating no effect, or no substantial effect. The subject was treated at length in a recent paper by a pair of graduate students in sociology at New York University: ‘We report no difference in support when respondents are asked about “gay,” “same-sex” or “homosexual” marriage and civil unions.’ They did find substantial differences in the strength, rather than overall level of such views; and said this effect was concentrated among middle-aged and older respondents.”
ABC News’ test of the language, last year, Langer says, “found perhaps a slight effect, but within the margin of sampling error for our survey, with a more clearly negative reaction to the term ‘homosexual’ among seniors. In spring ’09, we asked a random half of our respondents if it should be legal or illegal for ‘homosexual couples’ to get married; they divided 46-48 percent.
“We asked the other half about ‘gay or lesbian couples’; they divided 51-44 percent. Given our sample sizes the differences are not significant at the usual 95 percent confidence level.
“Directionally they do suggest that ‘homosexual’ could be somewhat pejorative, though certainly not broadly so; people asked about ‘gays and lesbians’ were 5 points more apt to support their being able to marry. Since this wasn’t replicated in studies cited by the NYU paper, it needs further study.”
- Jake Tapper