Gould’s Methodological Critique: Off-Base and Misleading

By April 20, 2007

In Jon B. Gould’s recent attack on FIRE in The Chronicle of Higher Education (subscription only)—already skillfully dissected by Robert (twice), Sam and Chris—Gould takes issue with FIRE’s report, Spotlight on Speech Codes 2006: The State of Free Speech on Our Nation’s Campuses, calling it “predictable” and “overly broad,” and accusing FIRE of “exaggerat[ing] the facts to make political hay.” These are bold statements indeed—but, like the rest of Gould’s arguments, they are supremely misleading.

Let’s look further at the basis for Gould’s claims about Spotlight on Speech Codes 2006. Specifically, Gould compares the results of FIRE’s study with data generated from his own empirical research, the findings of which were published in a law review article (The Precedent That Wasn’t: College Hate Speech Codes and the Two Faces of Legal Compliance, 35 Law & Soc’y Rev. 345) in 2001. Comparing the two studies, Gould writes:

Several years ago, I conducted an in-depth study of hate-speech codes at 100 colleges, applying categories previously tested by Vanderbilt University’s First Amendment Center. That project, using random sampling, found that 46 percent of four-year institutions had policies that could be used to restrict “hate speech,” meaning verbal attacks that target others on the basis of their immutable characteristics. However, only 23 percent of institutions not FIRE’s 96 percent [sic]—had rules that were inconsistent with the First Amendment. And even that number had to be narrowed: Only public institutions (and private colleges in California) are held to the First Amendment. Considered more precisely, just 9 percent of the institutions had unconstitutional speech policies, according to my study…

FIRE stands behind the results of our report unequivocally. Spotlight on Speech Codes 2006 is the product of thorough research and each determination about a school’s speech policy is the result of careful analysis performed by trained attorneys specializing in constitutional law.

So why are our results so different from Gould’s? He says only 23% of his sample maintained policies explicitly inconsistent with the First Amendment (9% after discarding schools not legally obliged to follow the First Amendment), whereas FIRE says 69% of schools do. (Gould’s claim that we found 96% of schools examined maintained unconstitutional policies is obtained by combining the percentages of schools rated both “red light” and “yellow light” by FIRE. There is a distinct difference, explained below.)

The short answer for the discrepancy is that FIRE and Gould are employing different methodologies: our study examines more policies from more schools in a more thorough and legally exacting manner than Gould’s research.

The long answer requires us to first compare the competing methodologies.

Examining the Research

Spotlight on Speech Codes 2006 relies on FIRE’s general classification system for rating a school’s speech code, using red, yellow and green “lights” to indicate the degree to which a school’s speech policy comports with the First Amendment (for public schools) or the school’s advertised commitments to free speech (for private schools). As Robert usefully explained earlier this week, FIRE doesn’t assign a ranking to private or sectarian schools that make it explicitly clear that other values—like faith, for example—are held in higher regard on that particular campus than freedom of expression.

Schools assigned a “red light” maintain “at least one policy that both clearly and substantially restricts freedom of speech. A ‘clear’ restriction is one that unambiguously infringes on what is or should be protected expression. In other words, the threat to free speech at a red light institution is obvious on the face of the policy and does not depend on how the policy is applied.” Schools assigned a “yellow light” enforce policies that, if enforced selectively, “could ban or excessively regulate protected speech.” Finally, schools are granted a “green light” if, after reviewing the school’s policies, FIRE cannot locate “any serious threats to students’ free speech rights in the policies on that campus.”

Comparatively, here’s Gould’s explanation of his rating system in his own words:

I borrowed categories from the Vanderbilt study to code the speech policies. Vanderbilt’s researchers used six ordered categories to code university speech policies. The rankings were on a sliding scale, with each group intended to reflect decreasing comportment with First Amendment norms. Eliminating only their code for obscenity policies, the categories I employed are


0. Did not adopt speech policies

1. Punished fighting words

2. Banned verbal abuse or harassment

3. Forbade verbal abuse or harassment against minorities

4. Proscribed offensive, demeaning, or stigmatizing speech

Each system of categorization seems straightforward enough, to be sure, and perhaps not all that different at first blush. However, as is usually the case, the distinction—and the explanation for how Gould’s research finds that nearly all schools maintain constitutional speech code policies, while ours shows that nearly 70% clearly do not—is to be found in the details.

FIRE Uses a Larger Sample

First, FIRE’s Spotlight on Speech Codes relies on a far larger sample than Gould’s research. Our study examined the speech policies maintained by over 330 schools. Gould’s sample consists of 100 schools selected at random from the Carnegie Foundation’s 1987 Classification of Institutions of Higher Education. Further, Gould notes his sample was “was stratified to account for the higher frequency of speech codes among large research universities and top liberal arts colleges”—a claim that, while perhaps true, he never proves.

It stands to reason that a larger sample will yield more accurate results about the prevalence of speech codes in higher education.

FIRE Looks at More Policies, More Carefully

Next—and I suspect this crucial detail is the single most important factor behind the differing results—Gould and FIRE rely on different views about which policies employed by a college may be analyzed for their impact on campus speech.

FIRE strongly believes in offering students the most complete summation of threats to their constitutional rights at a particular school as possible; to do less would be to shirk our institutional responsibility as a watchdog organization. Thus, when analyzing a school’s speech policies for constitutionality, we review any written policy maintained by the school with an impact on campus speech. That’s why a “red light” school like, say, Texas Tech University earns its red light not just for its sexual harassment policy (which curtails protected speech by failing to track either Supreme Court jurisprudence or guidelines issued by the Department of Education’s Office for Civil Rights (OCR), resulting in a policy that would allow students to risk potential punishment for a single “inappropriate comment”), but also for its “Web Page Standards” policy (which infringes upon constitutionally protected speech by requiring that students “not use Web pages to transmit images, sounds, or messages that could create an atmosphere of discomfort or harassment for others.”) While one policy may seem to be more “serious” that the other, and more geared towards “speech,” the simple fact is that students can and will be punished for constitutionally protected speech under either policy. FIRE has seen many such punishments meted out, and we know from experience that administrators don’t always cite a policy explicitly addressing “speech” when punishing protected expression. To imagine otherwise would be to turn a blind eye to clear violations of students’ fundamental civil liberties.

Gould’s study, on the other hand, is shockingly vague about what exactly constitutes a “college hate speech code” or a “speech policy,” and proper definitions of either term are never supplied. Judging from both the purposes of his study—to “estimate the prevalence of college hate speech codes, calculating the percentage—and types—that were created between 1987 and 1992 and then measuring them again in 1997 following five court cases that rejected many of the speech policies,” and to “find out why institutions behaved as they did”—and his failure to define the scope of his terms, it is safe to conclude that FIRE, whose institutional mission demands a comprehensive cataloguing of threats to speech on campus, has a broader understanding of “speech codes.”

FIRE Knows Conflicting Statements Do Not Absolve Unconstitutionality

Another detail likely responsible for the discrepancy in results is Gould’s apparent misunderstanding of the effect of what he calls “modifying words” on policies otherwise unconstitutional. In his article, Gould rails against FIRE’s study by accusing us of “failing to distinguish enforceable rules from exhortative statements, confusing examples with definitions, and taking statements out of context.” By way of example, he takes issue with our criticism of a University of Michigan policy that declares, “[i]ndividuals should not be unwittingly exposed to offensive material by the deliberate and knowing acts of others.” By failing to include the sentence immediately following (“The University is a community of individuals with diverse values, beliefs, and sensitivities. Individuals must be allowed to choose what they wish to access for their own purposes”), Gould accuses us of misleading readers and inflating our results. But as Robert so clearly demonstrated earlier this week, it is Gould who confuses “exhortative statements” with “enforceable rules.

Gould here also ignores the fact that such “savings clauses” cannot rectify the unconstitutionality of prior conditions; rather, they serve merely to muddy the water. Instead of being void for overbreadth, the offending clause is now rendered void for vagueness. As FIRE has noted elsewhere: “What could be more vague (indeed, self-contradictory) than a policy prohibiting all sorts of protected speech that then claims not to prohibit anything that is protected by the Constitution?” Coupled with his failure to recognize the critical distinction between aspirations and rules, Gould’s mistaken belief in the power of “savings clauses” cannot help but prompt yet further skepticism about the reliability of his study’s findings.

FIRE Understands Harassment Law

The final detail responsible for the difference between FIRE’s numbers and Gould’s is that while FIRE regards any university policy that impinges upon protected speech a speech code, Gould does not. Indeed, the distinction between FIRE’s understanding of unconstitutional speech codes and Gould’s is pointed out by Gould himself in his article, but proves a point contrary to his intent. Gould writes that “[w]here FIRE’s estimates are exaggerated, the reason can often be traced to the group’s categorization of sexual-harassment policies as ‘speech codes.’” Apparently, Gould believes that when a policy is labeled as a harassment policy rather than a “hate speech” code, any restrictions on protected speech contained therein are somehow legitimized. Luckily for students, this is not the case; one cannot lose the right to protected expression just because university administrators restrict it under sexual harassment policies. As Sam wrote this week:

The problem is that a large number of colleges and universities define sexual harassment to include speech that categorically does not meet the stringent legal definition of harassment and that is, by contrast, constitutionally protected speech. As Professor Gould must know, a university cannot prohibit otherwise constitutionally protected speech simply by defining it as harassment. But that is precisely what many universities do…Universities cannot simply make protected speech unprotected by deeming it “sexual harassment.”

It is essential to note that when FIRE makes a determination about the constitutionality of a university’s speech code, when the code in question is a harassment code, we still analyze the code’s legality, tracking the university’s definition of harassment against that promulgated and enforced by the Department of Education’s Office for Civil Rights, the government agency responsible for enforcing federal statutes outlawing racial and sexual discrimination and harassment on campus. We often find, as in the Texas Tech example above, that university harassment regulations go far beyond their legal bounds and infringe upon a student’s right to free speech. Indeed, as the OCR itself pointed out in a 2003 open letter to university administrators:

Some colleges and universities have interpreted OCR’s prohibition of “harassment” as encompassing all offensive speech regarding sex, disability, race or other classifications. Harassment, however, to be prohibited by the statutes within OCR’s jurisdiction, must include something beyond the mere expression of views, words, symbols or thoughts that some person finds offensive. Under OCR’s standard, the conduct must also be considered sufficiently serious to deny or limit a student’s ability to participate in or benefit from the educational program… No OCR regulation should be interpreted to impinge upon rights protected under the First Amendment to the U.S. Constitution or to require recipients to enact or enforce codes that punish the exercise of such rights.

By ignoring the fact that universities continue to punish students for protected speech under color of anti-harassment policies, Gould turns a blind eye to a very real threat to student speech. In fact, of the seven federal cases challenging speech codes at public universities that resulted in opinions from 1989 to today, six of them resulted in the successful invalidation of an unconstitutional harassment policy. As David French pointed out yesterday, FIRE’s critics are certainly free to “argue about legal interpretations all day long, but federal judges make the ultimate decision, and so far FIRE hasn’t gotten one wrong yet.”

But Gould isn’t concerned with such details; apparently, everything’s peachy keen on the American campus. In Gould’s view, remember, anyone concerned about free speech on campus should simply “t[ake] a deep breath” because universities are just “social institutions” “reflecting a popular norm”—so who needs the First Amendment, anyways? If a student is disciplined under a harassment policy for engaging in protected speech and voicing an unpopular political view on campus—well, hey, that student obviously just needs to get in line with the “large majorities” of freshmen who believe that their speech should be regulated beyond the boundaries of the First Amendment. Yes, in Gould’s America, students don’t need the First Amendment, because the popular majoritarian view will serve them just fine, thank you very much. In other words: sit down and shut up.

When examined and debunked, it seems Gould is the one exaggerating facts to score cheap political points. Because we examine more schools more completely with more attention to actual restrictions on student speech, FIRE’s research is by far the more accurate reflection of the threats to free speech on campus. We at FIRE certainly hope that one day our estimates of the number of unconstitutional speech codes at our nation’s universities will look as rosy as Gould’s. But we also promise that when and if they do, they will be the product of real change on campus, not narrow definitions and legal misunderstandings.