Table of Contents

No Comment: Public Universities' Social Media Use and the First Amendment

No comment: Social Media Report 2020

This report explains FIRE’s survey — covering over 200 public universities and colleges — concerning how these government institutions filter or block content and users on their official Facebook and Twitter accounts. Using examples of censorship identified in public records acquired from these institutions by FIRE, “No Comment: Public Universities’ Social Media Use and the First Amendment” discusses how the First Amendment applies to government actors’ use of social media and explains why their use of the filtering and blocking tools may violate the First Amendment.


Executive Summary

In October 2019, Facebook founder and chief executive officer Mark Zuckerberg spoke at Georgetown University, extolling the virtues of freedom of expression and noting in particular the importance of college students’ ability to “express who they were and what mattered to them,” including through “challeng[ing] some established ways of doing things on campus.”[1]

Because Facebook is a private entity, the First Amendment—which only limits government actors—does not require it to honor expressive freedom. Zuckerberg’s endorsement of freedom of expression as a principle is a welcome and encouraging development.

It is, however, at odds with the fact that Facebook provides governments the tools to censor. These actors include public universities and colleges which are bound by the First Amendment—those very campuses where students have “challenged some established ways of doing things.”

These tools include Facebook’s automated content filters, which allow state institutions to automatically “hide” users’ comments if they contain words included on Facebook’s undisclosed list of offensive words or the government actor’s customized list of prohibited words. These tools enable public universities—and other government actors—to quietly remove critical posts, transforming the Facebook pages into less of a forum and more of a vehicle for positive publicity.

This censorship has deleterious effects on campus and public discourse. Using public records requests, FIRE surveyed over 200 public universities and colleges across 47 states and the District of Columbia. Half of the surveyed institutions choose to use Facebook’s “strong” profanity filter, and nearly a third—55, or 27.8%—use the “medium” filter. This means that 77.4% of surveyed institutions choose to employ a blacklist of prohibited words not disclosed to the public.

Additionally, nearly a third of the universities surveyed (59, or 30.3%) use a custom blacklist, collectively censoring 1,825 unique words and phrases. These range from familiar profanities to words relating to matters of local and national concern—for example, blocking animal rights activists’ criticism of food vendors, curbing discussion of what to do about a campus Confederate monument, and shielding criticism of itinerant preachers, controversial faculty, politicians, and sports teams.

Even without these tools, public actors can prune their Facebook comments, manually hiding comments to manipulate and shape the apparent public discourse. Wright State University, for example, deleted comments supporting a faculty strike from its Facebook page, leaving behind a community “forum” that largely (and falsely) appeared supportive of the university’s administration and critical of striking faculty.

Because Facebook doesn’t alert a user when their post has been removed, or tell the public that comments have been censored, these commenters (and other users) may never know their words have vanished.

Further, both Facebook and Twitter allow government actors to block members of the public from participating in these public forums.

These automated methods of censorship are not only contrary to a commitment to freedom of expression, but also provide government actors with tools that—in light of recent federal court rulings concerning President Trump’s Twitter feed—violate the government actors’ legal obligations under the First Amendment.


Methodology

FIRE issued public records requests to five institutions in each state where public institutions of higher education are subject to public records laws, which are similar to the commonly known federal Freedom of Information Act (FOIA).

The five institutions per state were divided into two categories:

  1. The three public, four-year institutions with the largest undergraduate enrollment; and
  2. The two public, two-year institutions with the largest undergraduate enrollment.

Our approach has some logistical limitations. Some states, like Delaware and Pennsylvania, exempt some or all of their institutions of higher education from their public records laws, or limit the scope of the laws to specific expenditures. Other states’ public records laws expressly limit their public records laws to requests made by citizens of the state.[2] In other states, there are fewer than five institutions, or fewer than the three four-year and two two-year institutions this survey sought to cover. Finally, some institutions could not be contacted due to the potential for conflict with a pending FIRE lawsuit against the institution. In those cases, the institution was replaced with the public institution with the next-highest enrollment. A full list of exceptions may be found in Appendix B.

Altogether, FIRE’s survey issued public records requests to 224 public universities and colleges in 47 states and the District of Columbia. 198 institutions provided substantive responses. A full list of surveyed institutions may be found in Appendix A, which also identifies the institutions which failed or refused to produce records and includes data compiled from the survey.

Our public records requests identified the official Facebook and Twitter account for the institution and asked the institution to provide to FIRE the Facebook settings, list of blocked Facebook users, and list of blocked Twitter users. An example of a representative request may be found in Appendix C. We encourage journalists or interested members of the public to adapt our request to ask other public institutions—not just universities—about their records.

The calculation of the number of words on institutions’ customized blacklists excludes duplicates. Thus, if two institutions blocked the word “dog,” that word is counted only once. However, the calculation of the number of blocked Facebook users, as well as the number of blocked Twitter users, did not exclude duplicates. Accordingly, if one user were blocked by two universities, she would be counted twice.


Discussion

Facebook's tools

Facebook allows institutions, entities, organizations, and government actors to establish pages, operating as a forum where the public can share their thoughts with or about the organization. This includes public officials and government entities, ranging from the President of the United States to the City of North Pole, Alaska. As described below, public actors are bound by the First Amendment when they use tools provided by private entities.

When a business, organization, government entity, or public figure creates a “page,” Facebook allows them to create posts on their own page, gives the option of allowing others to add their own posts, and allows Facebook users to add comments to those posts.

Facebook provides page owners with four tools to limit user content:

  1. The profanity filter. Facebook’s “profanity filter” automatically hides visitors’ posts if they contain words on one of two lists—one for the “medium” setting and one for the “strong” setting.[3] The words on these lists are not publicly disclosed, but are composed of “the most commonly reported words and phrases marked offensive” by Facebook users.[4] The profanity filter is turned off by default.
  2. The customized blacklist. The “page moderation” filter allows an administrator to establish a custom list of blocked words.[5] Like the profanity filter, this filter automatically hides posts or comments if they contain a phrase on the custom list.
  3. The blocking function. An administrator of the page may ban particular accounts, after which “they'll no longer be able to publish to [the page], like or comment on your . . . posts, message [you] or like your [page].”[6]
  4. Manual removal. An administrator may manually hide specific posts or comments.

A user may not know that their comment has been hidden, as it remains visible to the poster.

Twitter's tools

Twitter offers fewer methods of regulating user interactions with government accounts:

  1. The blocking function. Twitter users can block others from interacting with them.[7]
  2. The muting function. Twitter users can mute users and terms, but this function does not prevent those users from interacting with the account, and a muted user’s tweets can still be seen by other visitors.[8]

The First Amendment limits public actors’ use of filtering tools

With the emergence of the internet and social media, courts have increasingly been called upon to apply the First Amendment to the digital realm. As the Supreme Court of the United States recently observed, “in the past there may have been difficulty in identifying the most important places (in a spatial sense) for the exchange of views,” but the answer today is “clear”: “It is cyberspace . . . and social media in particular.”[9]

Accordingly, a wide range of courts across the country have held government actors’ social media sites,[10] including those on Facebook and Twitter, to be subject to First Amendment limitations.[11] The most prominent of these decisions thus far is a successful challenge to President Trump’s practice of blocking critics from his Twitter account.[12] These rulings only limit government actors’ use of social media tools; the First Amendment does not impose legal obligations on social media sites like Facebook and Twitter, nor on other private parties when they use social media.[13]

What is the public forum doctrine?

The general thrust of the cases addressing public actors’ use of social media is that the “interactive space” constitutes a “public forum.” This is a term of art in First Amendment law, setting forth a “metaphor . . . first used in constitutional free speech cases as a way of explaining why the government cannot engage in . . . content discrimination with regard to speaking, picketing, or leafleting on city parks and sidewalks.”[14]

A public forum is most often seen as a physical site. For example, a public sidewalk or park would likely be seen as a “traditional public forum”: places where members of the public may freely gather and talk about whatever is on their minds.[15] In those places, attempts to limit viewpoint are never permissible, and attempts to limit content must meet “strict scrutiny”—that is, any limits must be necessary to address a compelling government interest and narrowly tailored to serve that interest.

Meanwhile, a meeting space set up by a government entity for discussion of particular subjects or use by particular groups might be a “limited public forum.” For example, school board meetings at which members of the public can speak are limited public forums, as they’re set aside for a particular purpose.[16] In these spaces, government has a freer hand to regulate the subject matter of speech or who may utilize the space, but can’t exclude people or speech based on the viewpoint expressed. Even if a particular space is privately owned, it is a public forum when a government actor exercises control over it, such as when a city leases a private theater.[17]

Public forum doctrine distinguishes between speech by the government and its employees—“government speech”—and speech by others that the government regulates.[18]

What are the different standards for different public forums?

Not every space opened to expression is a free-for-all in which anything goes. Different standards attach to the forum depending on its purpose, and the extent to which government actors may regulate expression within that forum depends on the forum’s purpose and the expression at issue:

  • Government actors can almost always limit expression that falls into one of the historic, narrowly-defined exceptions to the First Amendment: “incitement, obscenity, defamation, speech integral to criminal conduct, so-called ‘fighting words,’ child pornography, fraud, [and] true threats.”[19] This is true in all forums: traditional, designated, and limited.
  • Government actors can never limit expression because of its “point of view”[20] or where the “specific motivating ideology or the opinion or perspective of the speaker is the rationale for the restriction.”[21] This is also true in all forums.
  • If no effort is made to set aside the space for discussion by particular people or about particular subjects, the space is likely to be treated as a designated or “open” public forum.[22] In that case, restrictions on the “time, place, and manner of expression” are permissible only if they are “content-neutral, are narrowly tailored to serve a significant government interest, and leave open ample alternative channels of communication.”[23] If a burden on speech is content-based, the regulation is permissible only if it is “necessary to serve a compelling state interest and that it is narrowly drawn to achieve that end.”[24]
  • If a policy is established and consistently enforced[25] to limit use of the forum for particular people or discussion of certain subjects, the space is more likely to be treated as a limited public forum, in which the government has greater leeway to regulate speech.[26] There, restrictions need not be content-neutral, but must be viewpoint-neutral, “reasonable in light of the purpose served by the forum,”[27] and must “comport with the definition of the forum”[28]—that is, the government cannot exclude speech for which, or speakers for whom, the forum was opened.

These standards are also important for evaluating whether the regulation of online expression is permissible under the First Amendment.

How does the public forum doctrine apply to social media?

While public forums are most easily conceptualized as physical spaces—such as open areas, meeting rooms, and bulletin boards[29]—the public forum doctrine has also been applied by the Supreme Court to forums which consist “more in a metaphysical than in a spatial or geographic sense.”[30] For example, the same principles have been applied to—albeit with varying outcomes—student activity fees pooled for student organizations,[31] schools’ internal mailing systems,[32] and email systems.[33] As governments have ventured into cyberspace, courts have applied these principles to government websites, distinguishing government-published websites from websites that allow members of the public to exchange views.[34]

Courts have used the legal framework provided by public forum doctrine to analyze government social media use. While the government actor has control over its own posts (which are quintessential “government speech”), the “interactive space” that follows those posts and allows for response from the public constitutes a public forum subject to the First Amendment’s protection.[35] In other words, while a public university student or faculty member might have a right to post a relevant comment on an existing thread, they could not compel the university to “share” their post in the same manner as the university publishes its own posts.

Here’s a visual demonstration of how “government speech” and the interactive digital spaces work on Facebook (left) and Twitter (right):

Social Media report 2020: figures 1 and 2

Because the First Amendment protects users’ speech in these interactive spaces, government actors like public colleges and universities cannot censor user comments in that space because they disagree with the viewpoint expressed. That means that—at a minimum—government actors cannot block users,[36] remove posts,[37] or otherwise “burden” speech[38] because they find the expression offensive, disagreeable, or wrong.

That does not mean that every digital space associated with a government actor is a free-for-all forum where any content may be shared. How government actors characterize and treat digital spaces is important, as regulations on interactive spaces will be analyzed based on whether they are—as with physical spaces—traditional, dedicated, or limited public forums.

If the interactive space is a limited public forum, some restrictions will be permissible. For example, a prohibition on “clearly off topic” comments is a “self-evidently viewpoint-neutral” regulation “limiting a forum to discussion of selected topics,” and is “reasonably related — indeed, integral — to the forum’s purpose.”[39] As such, these kinds of viewpoint-neutral limitations are constitutionally permissible and do not violate the First Amendment.

But the government’s ability to regulate speech in a limited public forum has boundaries. If, in crafting a policy governing social media content, the policy affords too much discretion to the institution’s content moderators, those standards will inevitably be abused to censor criticism of the institution. The University of Connecticut’s policy, for example, reserves to officials “the right” to remove comments they believe to be “offensive,” expression that “aligns with hate speech,” or posts that are “otherwise objectionable.”

These types of broad, ambiguous policies will lead to viewpoint discrimination and may render the policy itself unconstitutional.[40] For example, Wright State University created “guidelines” barring comments deemed to be “propaganda,” “foul,” “trolling,” “offensive,” or “inflammatory,” giving staff members—overseen by university administrators—unfettered discretion to manually hide comments criticizing the administration’s positions during a faculty strike, such as those disclosing senior administrators’ salaries.[41] (At the time of publication, Wright State officials had communicated to FIRE that the university was conducting an internal review of its Facebook guidelines in response to a letter from FIRE raising concerns about the policy.)

In addition to the ability to limit expression by policy and manual enforcement, Facebook provides a variety of automated content-policing tools to administrators of Facebook pages.

However, the tools provided by Facebook are not adequately tailored and do not respect the contours of the First Amendment, and their use by government actors will almost certainly lead them to violate the First Amendment rights of students, faculty, and the general public. When used by public institutions, Facebook’s tools amount to always-on sentinels tasked with identifying and automatically censoring student and faculty speech—and, for years, government actors have been using these tools to effectuate online censorship.

Public institutions are using Facebook and Twitter tools to violate the First Amendment

It has long been settled law that the First Amendment is binding on public colleges and universities, including when they act to regulate the otherwise-protected speech of students, faculty, and organizations comprised of students or faculty members, such as student clubs or faculty unions.[42] When public universities open digital spaces for discussion by faculty, students, and the general public, regulations of those digital spaces must meet First Amendment scrutiny.

Beginning in October 2018, FIRE issued requests under public records laws to 224 public universities and colleges in 47 states, plus the District of Columbia. The selected institutions represent the three four-year institutions and the two two-year institutions with the highest enrollment in the state.[43]

Of the 224 institutions surveyed, 198 (or 88.4%) provided substantive responses.

FIRE found that many public universities and colleges are using automated tools to regulate online expression in ways that do not comport with the First Amendment.

Most public colleges use Facebook’s ‘profanity’ filters, secret blacklists of words that risk violating the First Amendment.

Half of the responding institutions—99 of 198—use Facebook’s “strong” profanity filter, and nearly a third—55, or 27.8%—use the platform’s medium filter. This means that 77.8% of surveyed institutions use some version of the profanity filter.

Because Facebook does not disclose the list of words or phrases on either the “medium” or “strong” profanity filters, it is not clear what content is automatically scrubbed from the pages of institutions that utilize the filters created by Facebook. (The custom filters, described below, can be identified through public records requests.)

Facebook, again, does not violate the First Amendment when it creates these filters and offers them to their users. Instead, the First Amendment protects the rights of private entities like Facebook to decide what speech to publish or refuse.[44]

However, when a government agency uses these tools to filter particular words or phrases, it raises serious First Amendment concerns. The premise underlying this type of restriction—that blocking individual words is justifiable under the First Amendment because it will improve public discourse—has been expressly rejected by the Supreme Court. In Cohen v. California, the Court held that a jacket emblazoned with the words “Fuck the Draft,” worn in a courthouse hallway populated with “women and children,” was protected speech.[45] “[W]e cannot indulge the facile assumption that one can forbid particular words without also running a substantial risk of suppressing ideas in the process,” the Court explained, as “governments might soon seize upon the censorship of particular words as a convenient guise for banning the expression of unpopular views.”[46]

This risk is most colorfully illustrated by institutions’ use of customized blacklists to suppress discussion of local controversies, as discussed below. However, the broader use of Facebook’s own profanity filters—a secret list of disapproved words automatically compiled from Facebook users’ feedback—will have the same effect, while failing to meaningfully protect equitable, civil discourse.[47]

First, because the list is generated by compiling the terms “most commonly reported” as offensive, it necessarily limits speech that is in the greatest need of protection from censorship because it is disapproved by those whose views and sensibilities are more likely to be majoritarian or mainstream. As the Supreme Court observed in holding that burning the American flag is protected expression, “[i]f there is a bedrock principle underlying the First Amendment, it is that the government may not prohibit the expression of an idea simply because society finds the idea itself offensive or disagreeable.”

The use of these databases may also, incidentally, inure to the detriment of minority speakers. One recent study, for example, found that “tweets by African American authors are 1.5 times more likely to be labeled ‘offensive,’” and tweets in African American English are “more than twice as likely to be labeled as ‘offensive’ or ‘abusive.’”[48] As a result, automated means of identifying human speech, when relying on human-generated data, may introduce “human biases [which] can easily result in a skewed distribution in the training data,” yielding “unintended bias in the resulting models, and therefore potentially unfair applications.”[49] Because datasets of offensive language may yield a “systemic racial bias,” they may “have a disproportionate negative impact” on minority communities, effectively creating systems that “may discriminate against the groups who are often the target of the abuse we are trying to detect.”[50]

Second, an automated system is unlikely to interpret the context and meaning of uses of words that may be offensive in some contexts and inoffensive in others. Content “moderation is, inherently, a subjective practice,” rendering “content moderation at scale . . . impossible to do well.”[51]

As a journalist reporting on recent studies on attempts to create filtering mechanisms for offensive speech observed:

[W]hat is considered offensive depends on social context. Terms that are slurs when used in some settings — like the “n-word” or “queer” — may not be in others. But algorithms — and content moderators who grade the test data that teaches these algorithms how to do their job — don’t usually know the context of the comments they’re reviewing.[52]

Similarly, automated systems are incapable of making value judgments, resulting in the suppression of speech that makes use of offensive words in order to appropriate or report on that speech. While a filter may be intended to prevent, for example, the use of a racial slur directed to another user, it will also suppress posts by a student who, targeted by use of that same slur, posts about their experience on their university’s Facebook page to call on the university to take action. Activists utilizing offensive language or imagery in order to report on or criticize their use may not anticipate automated filtering, but a user dedicated to using the language or imagery in order to harass others may be more likely to devise workarounds, or use language less likely to be caught in a filter.

Third, the use of blacklists almost certainly fails First Amendment scrutiny under either a designated or limited public forum analysis. Facebook’s profanity filter is expressly premised on feedback identifying which words or phrases users find offensive. A limitation on this basis is not viewpoint-neutral, rendering its use fatal in any forum.[53] Further, it is unlikely that a public institution will be able to bear its burden—to demonstrate that its restriction is reasonable—if it cannot identify the words that it is restricting. Similarly, an undisclosed list of prohibited terms necessarily means that members of the public do not have notice as to what speech is permitted or prohibited, ensuring that the restriction will “trap the innocent by not providing fair warning” about what is prohibited.[54]

An automated blacklist also fails First Amendment scrutiny because it automatically hides speech unless it is reviewed by an administrator of the page, effectuating a system of prior restraints on speech, “the most serious and the least tolerable infringement on” freedom of expression.[55] The risk prior restraints present to freedom of expression is so great that the “chief purpose” in adopting the First Amendment was to prevent their use.[56] Prior restraints are permissible only where they are bound by “narrow, objective, and definite standards”[57] and where the authorities’ review and determination must be made within a specific period of time.[58] Absent these procedural safeguards, which FIRE has not observed at any surveyed institution, a prior restraint is unconstitutional.

An automated system that bars specific words does not differentiate between offensive uses of a word or uses of profound public importance. While it may serve a college or university’s public relations goals to police public-facing expression by students and faculty for civility, it undermines the institution’s commitment to freedom of expression and obligations under the First Amendment.

Facebook’s customizable blacklists are used to restrain speech critical of government institutions and their corporate partners, or speech on matters of public concern.

Nearly a third of the public colleges and universities FIRE surveyed (60, or 30.3%) use a customized blacklist on their Facebook pages, collectively censoring 1,825 unique words and phrases. These range from the words populating George Carlin’s famous list of “Filthy Words”[59] to words and phrases pertaining to political and social matters of local and national concern. Together, these customized lists evidence use of Facebook’s technology to automatically censor criticism of institutions, corporate partners, campus controversies, and even sports-related chest-thumping.

These filters are not likely to pass First Amendment scrutiny. Moreover, many of these filters evidence viewpoint discrimination, as they are specifically designed to hide terms closely associated with particular criticisms. For example, restrictions on posting about “chickens” at the University of Kentucky are inarguably intended to suppress criticism by animal rights activists, even if a supporter of the university’s vendor would also face restrictions because of a post containing the same term.

Even where a filter is not likely to amount to viewpoint discrimination, its use is not likely to be sufficiently tailored to meet the institution’s obligations under the First Amendment.

For example, a student at the University of North Carolina at Chapel Hill might want to respond to a university post about Black History Month[60] to raise questions about “Silent Sam,” a Confederate monument that until recently stood on UNC’s campus. But they could not have done so if their comment used the monument’s name, because that phrase was on UNC’s Facebook blacklist.[61] Or, similarly, when the university posted a message announcing the removal of the monument’s pedestal,[62] comments containing the words “Silent Sam” would unquestionably be relevant. Yet none of the hundreds of visible comments contain those words—almost certainly because of the blacklist. Similarly, students who used the phrases “9/11” or “terrorism” in responding to the university’s post memorializing the September 11 attacks[63] would not reach their intended audience—again, almost certainly because of the blacklist. Yet each of these comments would be on-topic and not otherwise fall within a categorical exception to the First Amendment.

Similarly, students at schools like Oklahoma State University, where the names of political candidates are automatically scrubbed, would encounter difficulty if they mentioned their favored (or disfavored) candidates’ names in response to the schools’ posts about getting out the vote.[64]

These restrictions, even where they are viewpoint neutral, are not “reasonable in light of the purpose served by the forum”[65] because they burden relevant, on-topic expression in a medium intended to operate in real-time. Instead, they effectuate a system of prior review on particular terms, and institutions are not likely to provide the procedural protections required by the First Amendment.

FIRE’s survey revealed that public institutions are using the customized blacklist feature to limit a wide range of criticism, or to blunt discussion of local controversies:

  • The University of Kentucky blocks the words “birds,” “chicken,” “chickens,” and “filthy.” The university explained to FIRE that it instituted this restriction “around the time that Aramark came on campus” and activists began posting “highly graphic videos of chicken slaughter.” In 2014, the university announced it was entering into a 15-year contract worth $250 million with Aramark.[66] Similarly, Mississippi State University blocks mentions of “Aramark” and terms related to its dining facilities.
  • Texas A&M University blocked terms in an effort to bar criticism by animal rights activists, including People for Ethical Treatment of Animals, over research conducted on dogs. These terms include “peta,” “abuse,” “abusers,” and “lab.” PETA and the Electronic Frontier Foundation sued Texas A&M in 2018 over the censorship.[67] Santa Monica College likewise bars “cats,” “dissecting,” “torture,” and “killing” following a PETA campaign critical of cat dissection in an anatomy course.[68]
  • During protests over the “Silent Sam” Confederate monument, the University of North Carolina at Chapel Hill blocked posts containing the phrase “Silent Sam,” as well as mentions of “Nazis.” The university also blocked mentions of the terms “9/11,” “terrorist,” “terrorism,” and the name of a professor whose teaching about the September 11, 2001, terrorist attacks drew national media coverage when a student, who had not read any of the materials,[69] publicly complained about the materials read in the course.[70] The university also barred terms relating to sexual assault.
  • The University of Arizona automatically removes posts containing the word “rape” or the name of an itinerant preacher known for holding signs reading “You Deserve Rape,” which has drawn media criticism.[71] This restriction presumably removes complaints about the preacher.
  • Suffolk County Community College (NY) blocks posts concerning inclement weather closures, and includes terms that demonstrate that the purpose is to inhibit criticism, barring the words “blizzard,” “snow,” “dangerous,” “slip,” “scared,” “irresponsible,” “tragedy,” and “accident.” They also block “apologize,” “resign,” and “disgrace.”
  • Clemson University blocked mentions of Harambe, a meme about a gorilla that had previously generated controversy on campus when a residential employee sought to bar students from referencing the meme on their dormitory doors. Clemson also blocked phrases referring to other controversies on campus, including the name of a professor criticized for calling Republicans “racist scum,” the word “turtle” (apparently because of a student’s poll of his classmates, which was reported in the press, about purposefully driving over turtles). Clemson also blocks the words “boycott” and “Nike”—a company with which the university has entered into a $58 million contract.[72] Similarly, the university blocked the words “senators,” “pledge,” and “anthem,” likely relating to students who protested by sitting during the pledge of allegiance during a student government meeting.[73] Clemson also blocked the word “balloonsblow,” referring to environmental activists who called on the university to end its practice of releasing balloons during football games.[74] Ominously, Clemson also blocked the words “rape” and “rapeculture.” In response to FIRE’s records request, Clemson removed the blocked terms and implemented “measures so that blocks of protected speech do not occur” in the future.[75]
  • The once-popular awareness campaign “Kony[76] cannot be found on Auburn University’s Facebook page, because comments containing that word are automatically deleted.
  • The University of Mississippi prohibits mentions of its former mascot, “Colonel Reb,” a “caricature of an antebellum Southern plantation owner” no longer used by the university.[77]
  • The University of Washington blocks a pejorative pun on its team name, the Huskies (“Fuskies”), as well as references to its football rivals: “Cougars,” “Ducks,” “Bruins,” and “Beavers.” Texas A&M University blocks references to the “hook em” horn gesture of their University of Texas at Austin rivals, the Longhorns. Oklahoma State University blocks mentions of its rival football team, including the phrases “boomer sooner” (OU’s fight song), “university of oklahoma,” and “sooners.”
  • Not to be outdone by its rival, the University of Oklahoma blocks an emoji: ?.[78]
  • The University of Central Florida blocks the words “terrorist” and “sexual.”
  • Oakland Community College in Michigan blocks mentions of the name of a disgruntled former professor.
  • Mississippi State University blocks the term “fail state,” a play on its “Hail State” motto.[79]
  • A number of institutions, including Portland State University, Oklahoma State University, the University of Arizona, the University of North Carolina at Chapel Hill, blocked the names of political candidates, such as “Trump,” “Bernie,” or “Hillary.” Others, such as Arizona State University, restrict pejorative versions of their names or supporters, such as “Trumptards” and “NObama.”
  • West Virginia University blocks mentions of “couch burning,” a tradition long disfavored by the university’s administration and Morgantown residents.[80]

Not just words: Public universities block thousands of Facebook and Twitter users.

FIRE’s records requests reveal that of the 198 public colleges and universities that responded to FIRE’s survey, 173 (or 87%) blocked users on Facebook or Twitter. Together, these institutions blocked 13,197 Facebook users and 4,065 Twitter users from interacting with their posts, pages, or tweets. Accordingly, an institution that blocks at least one user blocks an average of 76 Facebook users and 23.5 Twitter users.

Because Facebook and Twitter do not keep records of the reason for blocking a particular account, and because institutions’ records reveal only the name of the blocked user (not what they posted), it’s unclear how many of these users were blocked for viewpoint-discriminatory reasons. It’s possible many were blocked for violating constitutionally-defensible regulations.

However, government actors cannot restrict access to a public forum based on an objection to the viewpoint of the speaker. Many blocked accounts relate to political, social, or religious movements, including many critical of the institution, raising the possibility that they were blocked for advocating a particular viewpoint.

For example:

  • The University of Kansas blocks “Boycott Koch Industries,” a Facebook account referencing America’s second-largest privately-held company,[81] which is based in Kansas and is a major donor to the university.[82] On Twitter, the university blocks accounts related to the Westboro Baptist Church, which had pledged to picket the university’s graduation.
  • Georgia State University blocked a “Georgia for Bernie” Twitter account. The university also complained to Twitter about a tweet from the account criticizing the university’s police because the tweet listed the public contact information of the university’s president and police chief.
  • The University of New Hampshire blocked a Twitter account belonging to “UNH Students for Gary Johnson,” a Libertarian Party presidential candidate, and blocked the Twitter account of Breitbart News—and other Twitter critics—after the outlet wrote about professors who called for the expulsion of counter-protesters who dressed as Harambe and Richard Nixon.[83] Other accounts appear to have been blocked because they criticized the university for employing Seth Abramson, whose Twitter criticisms of President Trump have gone viral.
  • The University of Montana blocked a Twitter account, @UMRape, which was being used to follow “the #UMRape crisis that has been plaguing” the university.
  • Mississippi State University blocks “Legalize Marijuana in Mississippi.”
  • A number of universities block accounts supporting the “Occupy” movement. For example, Ball State University blocks “Occupy Indiana,” Ivy Tech Community College in Indiana blocks “Occupy Richmond Indiana,” the University of Maryland-College Park blocks “Occupy Baltimore,” and the University of North Carolina blocks “Occupy Raleigh” on Facebook.
  • Idaho State University blocked a Facebook page critical of a university employee whose photographs of big game hunting in South Africa went viral.[84]
  • The University of Mississippi, which blocks mentions of its former “Colonel Reb” mascot on its Facebook page, also blocks a “Colonel Reb” Twitter account operated by an organization (which also has a registered student chapter at the university) that advocates for the mascot’s return.[85]
  • The University of Utah blocks animal rights activists, including PETA, former University of Utah student Jeremy Beckham, and “Animal Welfare News.” Activists, including PETA and Beckham, have criticized research conducted at the university’s labs.[86]
  • The University of Alaska Anchorage blocked an “Alaskans4Trump” account that had disparaged activist Shaun King in response to a tweet announcing that King would be speaking at the university.[87]

The reason users have been blocked is not always clear from the records produced to FIRE. (The University of North Dakota, for example, blocks a Facebook group entitled “Dog Enthusiasts of North Dakota, no posers, only real dog lovers.”) Most institutions block only a handful of users, blocking an average of 76 Facebook users and 23.5 Twitter users. Others wield the tool more liberally. Indiana University blocks the most Facebook users (1488), trailed closely by the University of Washington (1162), with the University of California, Los Angeles in a distant third (691). When it comes to blocked Twitter users, Portland State University blocks the most users (883), more than double that of Central Connecticut State University (307) and Arizona State University (253).

The extent of manual removal of posts and comments is unknown.

FIRE’s survey did not reveal information about the extent to which institutions utilize their ability to manually hide individual Facebook posts or comments after they’ve been posted. However, if institutions are utilizing tools to prevent content from being posted, it is certain that they are also removing content after it has been posted. Recall, for example, that at Ohio’s Wright State University, a public records request revealed that administrators hid hundreds of posts concerning an ongoing faculty union strike.[88] Unfortunately, this type of content removal is difficult to detect through broad public records requests.

How private and government actors can mitigate the risk of censoring speech on social media

Government actors and social media companies navigating the application of the First Amendment to the digital realm can both take steps to avoid infringing on fundamental civil liberties.

Public actors and private colleges must avoid censoring speech on social media.

For administrators and staff at public universities and colleges—as well as at private institutions that promise freedom of expression to students and faculty—there are a variety of ways to reduce the risk of censorship.

For example:

  • Don’t filter. Turn off Facebook’s filters.
  • Don’t block users. Don’t block users based on their viewpoint. If they repeatedly violate a policy, document it and give them an opportunity to contest any limits on their access.
  • Craft clear, publicly available policies. Be clear about the purpose of your social media pages. Is your page open for comments only from students or faculty, or is it open to the public? If you place limits on the content of speech, make sure they’re published, reasonable, narrow, and objective. A vague policy that leaves your staff with a wide range of discretion is a recipe for trouble. Some universities have begun to formulate new policies.[89]
  • Consistently enforce policies once in place. A failure to enforce policies consistently may lead to First Amendment liability even if the policy itself is clear, objective, and reasonable. Selective enforcement will lead to the appearance of bias, and may demonstrate unlawful viewpoint discrimination.[90] Further, at least one court has determined that, because policies were inconsistently enforced, a Facebook page was a designated public forum, not a limited public forum, meaning that it was subject to a stricter First Amendment analysis.[91]
  • Train staff on First Amendment principles. Make sure your staff members tasked with regulating and responding to online speech know what they can and can’t do, and make sure they ask for help when they’re not sure.[92]
  • Keep records. If you block a user for posting the same comment repeatedly, or for using unprotected speech, keep a record of it. This will facilitate supervision and correction if a user is blocked for an improper reason.

Social media companies can discourage government actors from abusing tools to censor online speech.

For example:

  • Distinguish between government and non-government accounts. If a user identifies themselves as a government actor, make that information public, and require government officials to designate themselves as such in order to create verified pages or accounts.
  • Require government actors to establish public-facing policies. Restrict government actors’ moderation tools unless they have created a public policy concerning user content.
  • Keep filters “off” by default, and only permit their use if the government actor has established a public-facing content policy.
  • Keep filters “off” by default, and only permit their use if the government actor has established a public-facing content policy.
  • Alert users when their comment has been filtered on a government actor’s page. If a user does not know that their speech has been hidden, they cannot remedy that censorship.
  • Make public the words contained on the profanity filters. Government actors should not be able to make use of a list of forbidden words hidden from the public.
  • Provide tools and reminders for government accounts. If an account is designated as belonging to a government actor, provide warnings alongside filtering or blocking tools that their use may have ramifications under the First Amendment. If a government actor blocks a user, require them to set forth their reason for doing so.

Conclusion

Freedom of expression is often one of the few tools available to people to defend themselves. As more of our public discourse takes place online, and not on the sidewalk or in the pages of newspapers, it is critical that public and private institutions alike vigorously defend expressive rights, and remain vigilant to the impact censorship will have on individuals, groups, and communities.

FIRE stands ready to assist students, student organizations, faculty members, and faculty organizations in defending their rights—online or off. If you believe your rights may be jeopardized, contact FIRE at https://www.thefire.org/alarm.


Footnotes

[1] Mark Zuckerberg, Standing for Voice and Free Expression, Georgetown University (Oct. 17, 2019) (transcript available at https://www.facebook.com/notes/mark-zuckerberg/standing-for-voice-and-free-expression/10157267502546634).
[2] This limitation has been upheld by the Supreme Court. McBurney v. Young, 569 U.S. 221, 224 (2013).
[3] Facebook, Moderate Your Facebook Page, Dec. 7, 2015, https://www.facebook.com/facebookmedia/blog/moderating-your-facebook-page; see also Facebook, How can I proactively moderate content published by visitors on my Page?, https://www.facebook.com/help/131671940241729 (last visited Apr. 15, 2020).
[4] Id.
[5] Id.
[6] Facebook, How do I ban or unban someone from my Page?, https://www.facebook.com/help/185897171460026 (last visited Apr. 15, 2020).
[7] Twitter, How to control your Twitter experience, https://help.twitter.com/en/safety-and-security/control-your-twitter-experience (last visited Apr. 15, 2020).
[8] This function does not infringe on a user’s First Amendment rights: One has a right to speak at or about the government, but the government is not obligated to pay attention to that speech. Accordingly, FIRE’s survey did not ask about muted users or content.
[9] Packingham v. North Carolina, 137 S. Ct. 1730, 1735 (2017); see also Knight First Amendment Inst. at Columbia Univ. v. Trump, 928 F.3d 226, 237 (2d Cir. 2019) (“Knight”) (“[S]ocial media is entitled to the same First Amendment protections as other forms of media.”).
[10] To be sure, the First Amendment does not restrict the use of Facebook or Twitter by private actors, such as businesses or private colleges. Nor do Facebook or Twitter themselves violate the First Amendment by providing these tools.
[11] See, e.g., Robinson v. Hunt Cty., 921 F. 3d 440, 447–49 (5th Cir. 2019) (assuming a sheriff’s Facebook page is a public forum and holding that a policy of deleting “inappropriate” comments was viewpoint discriminatory); Davison v. Randall, 912 F.3d 666, 681–87 (4th Cir. 2019) (county official’s Facebook page was a public forum); Lloyd v. City of Streetsboro, No. 18-3485, 2018 U.S. App. LEXIS 36090, at *9–14 (6th Cir. 2018) (unpublished opinion reversing sua sponte dismissal of complaint alleging viewpoint discrimination on city’s official Facebook page); Report and Recommendations, Clark v. Kolkhorst, No. A-19-cv-0198-LY-SH, *11–12 (W.D. Tex. Feb. 5, 2020), ECF No. 36 (recommending denial of motion to dismiss First Amendment claim); American Atheists, Inc. v. Rapert, No. 4:19-cv-17, at *43–44 (W.D. Ark. Sept. 30, 2019), ECF No. 27 (plaintiffs had “fair chance” of showing state legislator’s Twitter and Facebook blocks were viewpoint discrimination); Garnier v. Poway Unified Sch. Dist., No. 17-cv-2215-W (JLB), U.S. Dist. LEXIS 167247, at *14–21 (S.D. Cal. Sept. 26, 2019), ECF No. 42 (denying motion for summary judgment and finding that the interactive portions of school board members’ Facebook and Twitter accounts were designated, not limited, public forums); Order of Dismissal, Landman v. Scott, No. 19-cv-01367 (D. Colo. Sept. 10, 2019), ECF No. 27 (state legislator agrees to unblock Facebook and Twitter critic, pay $25,000); Findings of Fact and Conclusions of Law, Campbell v. Reisch, No. 2:18-cv-4129-BCW, 2019 U.S. Dist. LEXIS 138881, at *9–15 (W.D. Mo. Aug. 16, 2019), ECF No. 55 (interactive space created by state representative’s tweets was a designated public forum); Windom v. Harshbarger, No. 1:19-cv-24, 2019 U.S. Dist. LEXIS 95080, at *13–18 (N.D.W. Va. June 6, 2019), ECF No. 18 (First Amendment challenge survives motion to dismiss where constituent blocked from legislator’s “politician” Facebook page); Hyman v. City of Walnut Ridge, No. 2:18-cv-02138, 2019 U.S. Dist. LEXIS 90509, at *4–5 (E. D. Ark. May 30, 2019), ECF No. 17 (interactive portion of police department’s Facebook page was not government speech); One Wisconsin Now v. Kremer, 354 F. Supp. 3d 940 (W.D. Wis. Jan. 18, 2019) (legislators blocking critic on Twitter); People for the Ethical Treatment of Animals, Inc. v. Young, No. 4:18-cv-01547 (S.D. Tex. Sept. 10, 2018), ECF No. 31 (summary order denying motion to dismiss First Amendment claims premised on keyword-based content filters on public university’s Facebook page); Leuthy v. LePage, No. 17-cv-00296, 2018 U.S. Dist. LEXIS 146894, *36–43 (D. Me. Aug. 29, 2018) (governor’s Facebook page was limited public forum); Price v. City of New York, No. 15-cv-5871, 2018 U.S. Dist. LEXIS 105815, *25–46 (S.D.N.Y. June 25, 2018) (where NYPD precinct blocked Twitter user, whether the forum was a “public, designated, or nonpublic forum” was immaterial, as “viewpoint discrimination that results in the intentional, targeted expulsion of individuals . . . is unlawful in any forum,” including nonpublic forums); Dingwell v. Cossette, No. 3:17-cv-01531, 2018 U.S. Dist. LEXIS 95832 (D. Conn. June 7, 2018) (critic blocked from police Facebook page); Davison v. Plowman, No. 1:16-cv-180, 2017 U.S. Dist. LEXIS 4348, at *10 (E.D. Va. Jan. 10, 2017) (county’s policy and practice of encouraging comments on its Facebook page created a limited public forum).
[12] Knight First Amendment Inst. at Columbia Univ. v. Trump, 302 F. Supp. 3d 541, 549 (S.D.N.Y. May 23, 2018) (upheld on appeal by the Second Circuit in Knight).
[13] See, e.g., Prager Univ. v. Google LLC, No. 18-15712, 2020 U.S. App. LEXIS 5903, at *7 (9th Cir. Feb. 26, 2020) (no First Amendment claim that YouTube “censored” speech because the free speech clause “prohibits the government—not a private party—from abridging speech.”).
[14] Aaron H. Caplan, Invasion of the Public Forum Doctrine, 46 Willamette L. Rev. 647, 647 (2009-2010).
[15] ACLU of Nev. v. City of Las Vegas, 333 F.3d 1092, 1099 (9th Cir. 2003) (the “quintessential traditional public forums are side-walks, streets, and parks.”).
[16] See, e.g., Lowery v. Jefferson Cty. Bd. of Educ., 586 F.3d 427, 432 (6th Cir. 2009) (school board meetings give “citizens a chance to express their views to the board, [but] cannot accommodate the sort of uninhibited, unstructured speech that characterizes a public park.”).
[17] Se. Promotions, Ltd. v. Conrad, 420 U.S. 546, 555 (1975) (government-leased theater “under [the] control” of public officials); Denv. Area Educ. Telecomm. Consortium, Inc. v. F.C.C., 518 U.S. 727, 749 (1996).
[18] Government actors have sought to frame visitors’ Facebook or Twitter comments as government speech, arguing that because the government hasn’t removed those comments, it is effectively publishing the comments as the government’s own speech. Most people, however, would not view a Facebook comment or tweet as government-endorsed simply because it hasn’t been removed.
[19] United States v. Alvarez, 567 U.S. 709, 709 (2012).
[20] Rodriguez v. Maricopa Cty. Cmty. College Dist., 605 F.3d 703, 710 (9th Cir. 2009).
[21] Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 829–30 (1995). This is true even if the interactive space of the social media page is deemed a non-public forum, as access even to a non-public forum may only be “restricted as long as the restrictions are ‘reasonable and [are] not an effort to suppress expression merely because public officials oppose the speaker’s view.’” Cornelius v. NAACP Legal Def. & Educ. Fund, Inc., 473 U.S. 788, 800 (1985) (quoting Perry Educ. Ass’n v. Perry Local Educators’ Ass’n, 450 U.S. 37, 46 (1983)).
[22] Good News Club v. Milford Cent. Sch., 533 U.S. 98, 106 (2001) (distinguishing “traditional or open” public forums from “limited” public forums); Garnier v. Poway Unified Sch. Dist., No. 17-cv-2215-W (JLB), U.S. Dist. LEXIS 167247, at *14–21 (S.D. Cal. Sept. 26, 2019), ECF No. 42 (interactive portions of school board members’ Facebook and Twitter accounts were designated, not limited, public forums); One Wisconsin Now v. Kremer, 354 F. Supp. 3d 940, 953–55 (W.D. Wis. Jan. 18, 2019) (interactive portions of legislators’ Twitter accounts were designated public forums).
[23] Perry Educ. Ass’n, 450 U.S. at 45.
[24] Id.
[25] Garnier, U.S. Dist. LEXIS 167247 at *20–21.
[26] Faith Center Church Evangelistic Ministries v. Glover, 480 F. 3d 891, 908 n.8 (9th Cir. 2006) (A “limited public forum is a sub-category of the designated public forum, where the government opens a nonpublic forum but reserves access to it for only certain groups or categories of speech.”).
[27] Good News Club, 533 U.S. at 106–07 (quoting, in part, Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 829 (1995)).
[28] OSU Student All. v. Ray, 699 F.3d 1053, 1062 (9th Cir. 2012).
[29] See, e.g., Lister v. Def. Logistics Agency, 482 F.Supp.2d 1003, 1009¬¬–11 (S.D. Ohio 2007) (government agency’s bulletin board had “some aspects of a limited public forum” because it was “open to all employees” to “post matters addressing a broad range of topics or meetings,” rendering removal of religious materials unconstitutional).
[30] Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 830 (1995).
[31] Id.
[32] Perry Educ. Ass’n v. Perry Local Educators’ Ass’n, 450 U.S. 37, 46¬–49 (1983).
[33] Rodriguez v. Maricopa Cty. Cmty. College Dist., 605 F.3d 703, 710 (9th Cir. 2009).
[34] Compare Putnam Pit, Inc. v. City of Cookeville, 221 F.3d 834, 844 (6th Cir. 2000) (city’s website listing links to other sites was a non-public forum, not a public forum, because it was not structured to allow “dialogue between users,” but primarily “to convey information to the reader”) with Page v. Lexington Cty. Sch. Dist. One, 531 F.3d 275, 284 (4th Cir. 2008) (school district’s website was not a public forum, but “the issue would, of course, be different” if the website were a “type of ‘chat room’ or ‘bulletin board’ in which private viewers could express opinions or post information”).
[35] Knight First Amendment Inst. at Columbia Univ. v. Trump, 302 F. Supp. 3d 541, 572–73 (S.D.N.Y. May 23, 2018).
[36] Davison v. Randall, 912 F.3d 666, 687–88 (4th Cir. 2019) (official who blocked constituent because of his critical viewpoint was viewpoint discrimination, which is “prohibited in all forums.”).
[37] Robinson v. Hunt Cty., 921 F. 3d 440, 447 (5th Cir. 2019).
[38] Knight at 238–39.
[39] Davison v. Plowman, 247 F.Supp.3d 767, 777 (E.D. Va. 2017).
[40] See, e.g., Crowder v. Hous. Auth. of Atlanta, 990 F.2d 586, 591 (11th Cir. 1993) (a restriction vesting unfettered discretion in a government actor “opens the way to arbitrary suppression of particular points of view.”).
[41] Greg Harold Greubel, Public records reveal Wright State used unconstitutional Facebook page policy to censor pro-union speech during historic faculty strike, Found. for Indiv. Rights in Educ., Jan. 24, 2020, https://www.thefire.org/public-records-reveal-wright-state-used-unconstitutional-facebook-page-policy-to-censor-pro-union-speech-during-historic-faculty-strike; see also, Letter from Greg Harold Greubel, Staff Attorney, Foundation for Individual Rights in Education to Larry Y. Chan, General Counsel, Wright State University, Jan. 23, 2020, available at https://www.thefire.org/fire-letter-to-wright-state-university-january-23-2020.
[42] Healy v. James, 408 U.S. 169, 180 (1972) (there is “no room for the view that . . . First Amendment protections should apply with less force on college campuses than in the community at large”).
[43] As discussed in the methodology section of this report and in the appendices, some states or institutions could not be surveyed, failed to respond at all, sought exorbitant fees, or cited exceptions to open records laws, ranging from arguments that they cannot be required to “create” records to claims that revealing Facebook blacklists creates a risk to public safety.
[44] See, Miami Herald Pub. Co., Div. of Knight Newspapers Inc. v. Tornillo, 418 U.S. 241, 256–57 (1974) (a “[g]overnment-enforced right of access” in the form of a statute compelling newspapers to publish rebuttals “operates as a command in the same sense as” a rule prohibiting speech, violating the First Amendment). This right is fortified by statute shielding providers of an “interactive computer service” from civil liability for removing content they deem “objectionable.” 47 U.S.C. § 230(c)(1).
[45] Cohen v. California, 403 U.S. 15, 16 (1971).
[46] Id. at 26.
[47] Texas v. Johnson, 491 U.S. 397, 414 (1989).
[48] Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith, 2019, The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics at 1668–78, available at https://homes.cs.washington.edu/~msap/pdfs/sap2019risk.pdf.
[49] Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society at 67, available at https://dl.acm.org/doi/pdf/10.1145/3278721.3278729.
[50] Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber, 2019, Racial Bias in Hate Speech and Abusive Language Detection Datasets, available at https://arxiv.org/pdf/1905.12516.pdf.
[51] Mike Masnick, Masnick’s Impossibility Theorem: Content Moderation At Scale Is Impossible To Do Well, Techdirt, Nov. 20, 2019, https://www.techdirt.com/articles/20191111/23032743367/masnicks-impossibility-theorem-content-moderation-scale-is-impossible-to-do-well.shtml.
[52] Shirin Ghaffary, The algorithms that detect hate speech online are biased against black people, Vox, Aug. 15, 2019, https://www.vox.com/recode/2019/8/15/20806384/social-media-hate-speech-bias-black-african-american-facebook-twitter.
[53] See, e.g., Matal v. Tam, 137 S. Ct. 1744, 1763 (2017) (striking down regulation barring trademark registration for “any mark that is offensive to a substantial percentage of the members of any group” because it amounted to viewpoint discrimination, and observing that “[g]iving offense is a viewpoint”).
[54] Grayned v. City of Rockford, 408 U.S. 104, 108 (1972).
[55] Neb. Press Ass’n v. Stuart, 427 U.S. 539, 559 (1976).
[56] Near v. Minnesota, 283 U.S. 697, 713 (1931).
[57] Shuttlesworth v. Birmingham, 394 U.S. 147, 151 (1969).
[58] See, e.g., Pan Am v. Municipality of San Juan, No. 3:18-cv-1017 (PAD), 2018 U.S. Dist. LEXIS 208014, at *21–24 (D.P.R. Dec. 10, 2018) (surveying the procedural and substantive strands of the doctrine of prior restraint); see also, Covenant Media of S.C., LLC v. City of N. Charleston, 493 F.3d 421, 431 (4th Cir. 2007) (“A prior restraint on speech that imposes no time limitations on the decision-making process plainly fails” First Amendment scrutiny when the prior restraint is based on content.).
[59] In the Matter of a Citizen’s Complaint Against Pacifica Foundation Station WBAI (FM), New York, New York, 56 F.C.C.2d 94, 109, Declaratory Order (Feb. 21, 1975) (Transcript of George Carlin’s “Filthy Words” monologue, prepared by the Federal Communications Commission), archived at https://web.archive.org/web/20110123114427/http://www.law.umkc.edu/faculty/projects/ftrials/conlaw/filthywords.html.
[60] See, e.g., Univ. of N.C. at Chapel Hill, Recognizing Carolina’s black pioneers, Facebook (Feb. 23, 2018), https://www.facebook.com/uncchapelhill/videos/10155430896997709.
[61] After a letter from FIRE, UNC removed some of the terms on its customized list, but it is not clear what terms remain. Letter from Kara E. Simmons, Associate Vice Chancellor and Senior Univ. Counsel, Univ. N.C., to Adam Steinbaugh, Director, Individual Rights Defense Program, Found. for Indiv. Rights in Educ. (Sept. 13, 2019) (on file with author).
[62] Univ. of N.C. at Chapel Hill, Chancellor Folt announces resignation. . ., Facebook (Jan. 15, 2019), https://www.facebook.com/uncchapelhill/posts/10156120737537709.
[63] Univ. of N.C. at Chapel Hill, Carolina community gathers in remembrance of 9/11, Facebook (Sept. 11, 2018), https://www.facebook.com/uncchapelhill/videos/1925944624166629.
[64] See, e.g., Oklahoma State Univ., It’s Election Day, Cowboys! Get out and VOTE!, Facebook (Nov. 8, 2016), https://www.facebook.com/okstate/photos/a.145735572306/10153859326792307.
[65] Good News Club v. Milford Cent. Sch., 533 U.S. 98, 106–07 (2001) (quoting, in part, Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 829 (1995)).
[66] Jay Blanton, Dining Partnership Will Transform Vital Service to UK Campus, Univ. of Ky., June 13, 2014, https://uknow.uky.edu/campus-news/dining-partnership-will-transform-vital-service-uk-campus.
[67] Adam Schwartz, EFF Lawsuit Ends Censorship Against PETA on Public University’s Facebook Page, Electronic Frontier Found., Feb. 4, 2020, https://www.eff.org/deeplinks/2019/12/eff-ends-censorship-against-peta-public-universitys-facebook-page; see also Adam Steinbaugh, Texas A&M: Our secret list of naughty words you can’t say on our Facebook page doesn’t offend the First Amendment, Found. for Indiv. Rights in Educ., Aug. 23, 2018, https://www.thefire.org/texas-am-our-secret-list-of-naughty-words-you-cant-say-on-our-facebook-page-doesnt-offend-the-first-amendment.
[68] Press Release, Secret Student Video Exposes Dissection of Pregnant Cat at Santa Monica College, People for Ethical Treatment of Animals, Sept. 7, 2016, https://www.peta.org/media/news-releases/secret-student-video-exposes-dissection-pregnant-cat-santa-monica-college.
[69] Neel Ahuja, Distorting the study of 9/11 at UNC, News & Observer, Sept. 19, 2015, https://www.newsobserver.com/opinion/op-ed/article35735889.html.
[70] Jane Stancill, UNC course on 9/11 criticized in conservative publications, Charlotte Observer, Sept. 1, 2015, https://www.charlotteobserver.com/news/local/education/article33282939.html.
[71] Aviva Shen, University Of Arizona Student Tells Women: ‘You Deserve Rape’, ThinkProgress, Apr. 25, 2013, https://thinkprogress.org/university-of-arizona-student-tells-women-you-deserve-rape-14039c6c7737/.
[72] Manie Robinson, Clemson extends Nike partnership with 10-year, $58 million deal, Greenville News, Aug. 3, 2018, https://www.greenvilleonline.com/story/sports/college/clemson/2018/08/03/clemson-nike-extend-partnership-10-year-58-million-deal/897267002.
[73] Georgie Silvarole, Clemson University student senators sit for Pledge of Allegiance, WCNC, Sept. 28, 2017, https://www.wcnc.com/article/news/local/clemson-university-student-senators-sit-for-pledge-of-allegiance/479410454.
[74] Matt Connolly, Clemson ending this football gameday tradition for 2018, The State, July 28, 2018, https://www.thestate.com/sports/college/acc/clemson-university/article215700140.html.
[75] E-mail from Jermaine D. Johnson, Assistant General Counsel, Clemson Univ., to Adam Steinbaugh, Director, Individual Rights Defense Program, Found. for Indiv. Rights in Educ. (Dec. 21, 2018, 10:12 PM), available at https://www.documentcloud.org/documents/6394391-Clemson-University-Social-Media-Survey-Response.html.
[76] Invisible Children, Kony 2012, https://invisiblechildren.com/kony-2012 (last visited Apr. 15, 2020).
[77] Robbie Brown, Ole Miss Shelves Mascot Fraught With Baggage, N.Y. Times, Sept. 19, 2019, https://www.nytimes.com/2010/09/20/us/20mascot.html.
[78] Extending the middle finger is expression protected by the First Amendment. Swartz v. Insogna, 704 F.3d 105, 111 (2d Cir. 2013).
[79] Jason Kirk, Some moron spray-painted ‘Fail State,’ ‘Go Rebs,’ and more on Mississippi State’s campus, SBNation, Dec. 17, 2016, https://www.sbnation.com/college-football/2016/12/17/13994290/mississippi-state-stadium-vandalized-spraypaint.
[80] Jesse Wright, With Hopes to Curb Couch Burning, Morgantown Passes Outdoor Furniture Ban, W.VA. Pub. Broad., Apr. 7, 2015, https://www.wvpublic.org/post/hopes-curb-couch-burning-morgantown-passes-outdoor-furniture-ban.
[81] America’s Largest Private Companies, Forbes, https://www.forbes.com/largest-private-companies/list (last visited Apr. 15, 2020).
[82] Press Release, Koch Industries Gift of $1M Establishes Business, Engineering Scholarship Funds, Univ. of Kan. (Sept. 8, 2016), https://news.ku.edu/2016/09/08/koch-industries-gift-1m-establishes-business-engineering-scholarship-funds. FIRE also receives funding from the Charles Koch Foundation. FIRE does not accept money from donors who seek to alter our mission or interfere with our case work.
[83] Katherine Rodriguez, University of New Hampshire Professors Call for Trump Supporters to Be Expelled, Breitbart, Nov. 21, 2016, https://www.breitbart.com/politics/2016/11/21/university-new-hampshire-professors-call-trump-supporters-expelled.
[84] Michael H. O’Donnell, ISU accountant on safari becomes social media target, Idaho St. J., Aug. 5, 2015, https://www.idahostatejournal.com/members/isu-accountant-on-safari-becomes-social-media-target/article_f46ff95c-3b87-11e5-9f24-93c2ba7f4656.html.
[85] Colonel Reb Found., About Us, http://www.colonelreb.org/saving-colonel-reb (last visited Apr. 15, 2020); see also Univ. of Miss., Colonel Reb Found., https://olemiss.campuslabs.com/engage/organization/colonel-reb-foundation (recognized student organization webpage) (last visited Apr. 15, 2020).
[86] Courtney Tanner, University of Utah can’t charge PETA a $5K ‘prepayment’ for records on lab animal deaths, committee rules, Salt Lake Trib.., Nov. 14, 2018, https://www.sltrib.com/news/education/2018/11/14/university-utah-cant.
[87] @Alaskans4Trump, Twitter (Feb. 7, 2017, 1:30 AM), https://twitter.com/Alaskans4Trump/status/828913792993988608.
[88] Greg Harold Greubel, Public records reveal Wright State used unconstitutional Facebook page policy to censor pro-union speech during historic faculty strike, Found. for Indiv. Rights in Educ., Jan. 24, 2020, https://www.thefire.org/public-records-reveal-wright-state-used-unconstitutional-facebook-page-policy-to-censor-pro-union-speech-during-historic-faculty-strike.
[89] See, e.g., Northern Arizona University and the University of North Carolina at Chapel Hill.
[90] Thomas v. Chi. Park Dist., 534 U.S. 316, 325 (2002) (granting “waivers to favored speakers” or “denying them to disfavored speakers” would “of course be unconstitutional”); see, e.g., Business Leaders in Christ v. Univ. of Iowa, 360 F.Supp.3d 885, 907 (S.D. Iowa 2019) (disparate enforcement of nondiscrimination policy violated the First Amendment).
[91] Garnier v. Poway Unified Sch. Dist., No. 17-cv-2215-W (JLB), U.S. Dist. LEXIS 167247, at *20 (S.D. Cal. Sept. 26, 2019), ECF No. 42.
[92] To its credit, Wayne State University started training its social media staff after receiving a records request from FIRE.


Appendices

Appendix A

Surveyed institutions and response data

You can review the survey results, including links to each institution's records, here: https://cutt.ly/firesms.

Institutions in blue provided adequate responses. Copies of the records are hyperlinked.

Institutions marked in red failed to produce sufficient documents for the reasons specified in the accompanying footnote or hyperlinked document.

Appendix B

Exceptions to the survey methodology

The survey sought to cover two sets of institutions within each state, plus the District of Columbia: the three four-year institutions with the highest enrollment and the two two-year institutions with the highest enrollment. This would amount to 255 institutions, but not every state has qualifying institutions, some states’ public records laws do not extend to those institutions, and some institutions cannot be surveyed due to a legal conflict. What follows is a list of states or institutions falling under an exception or conflict:

  • Arkansas: Public records law limited to citizens of Arkansas. (-5 institutions).
  • California: East Los Angeles College could not be contacted due to litigation conflict. Replaced by American River College.
  • Delaware:
    • Only two four-year institutions, and only one two-year institution qualified. (-2 institutions).
    • The qualifying four-year institutions (Delaware State University and the University of Delaware) are largely exempt from Delaware’s public records law. (-2 institutions).
  • Iowa: Iowa State University could not be contacted due to litigation conflict. Cannot be replaced by another four-year institution because there are only three public four-year institutions in Iowa. (-1 institution).
  • Louisiana: Louisiana State University could not be contacted due to litigation conflict. Replaced by Louisiana Tech.
  • New Jersey: Rutgers University could not be contacted due to possibility of litigation conflict. Replaced with Rowan University.
  • Pennsylvania: The Right to Know law largely exempts four-year institutions, but does not exempt two-year institutions. (-3 institutions).
  • Rhode Island:
    • The Naval War College was not surveyed because its relationship with the armed services may change the First Amendment calculus. It was not replaced because there are only three, four-year institutions in the state. (-1 institution).
    • Rhode Island only has one two-year institution. (-1 institution).
  • South Carolina: The University of South Carolina could not be surveyed due to litigation conflict. Replaced by Coastal Carolina University.
  • Tennessee: FOIA limited to citizens of Tennessee. (-5 institutions).
  • Virginia: FOIA limited to citizens of Virginia. (-5 institutions).
  • Washington, D.C.:
    • There’s only one public, four-year institution in Washington, D.C., that is not affiliated with the military. (-2 institutions).
    • There are no public two-year institutions. (-2 institutions).
  • Wyoming: There is only one public, four-year institution. (-2 institutions).

Appendix C

Example public records request

The following is a lightly edited version of the request sent to Michigan State University:

This is a request for records pursuant to the Michigan Freedom of Information Act (Mich. Comp. Laws Ann. §§ 15.231 et seq.). If you are not the records custodian for MSU, please identify the correct person to contact.

This request seeks records relating to restrictions and settings concerning the official Facebook and Twitter accounts for MSU, and should be directed to the person responsible for operating those accounts.

RECORDS REQUESTED

I request the following records:

  1. A copy of the settings for the Facebook page maintained by Michigan State University (available at https://www.facebook.com/spartans.msu). This list is accessible by (A) logging into the Facebook page as an administrator, and then (B) clicking “Settings” at the top of the official page. The URL should look like: https://www.facebook.com/spartans.msu/settings/?tab=settings.
  2. A copy of the list of people or pages banned from the Facebook page referenced above. This list is accessible by: (A) logging into the Facebook page as an administrator, (B) clicking “Settings” at the top of the official page, (C) clicking “People and Other Pages” in the left column, and (D) selecting "Banned People and Pages" from the drop-down menu. The final URL should look like: https://www.facebook.com/spartans.msu/settings/?tab=people_and_other_pages.
  3. A list of the "blocked accounts" by the Twitter account maintained by Michigan State University (available at https://twitter.com/michiganstateu). This list is accessible by navigating to this URL while logged into the account: https://twitter.com/settings/blocked.

Fee waiver request: This request concerns a matter of public interest. The social media restrictions imposed by public institutions and officials — including the president, governors, and public universities — have been challenged on First Amendment grounds. This survey seeks to explore the extent to which public colleges and universities have similar restrictions.

The public interest would be well-served by granting a fee waiver. The request is not being sought for a commercial purpose, but is instead sought by a nonprofit organization to provide the public with information concerning the conduct of government actors as that conduct pertains to civil liberties in higher education.

If a fee waiver is not granted, please apprise me if the estimated costs will exceed $10.

Request for expedited processing: Completion of this survey depends on the institution with the slowest response time. We request that Michigan State University produce responsive records on an expedited basis. As you may be aware, a public body has five business days to respond. (Mich. Comp. Laws. Ann § 15.235(2)).

Appeal information: If you deny any portion, or all, of this request, please provide me with a written explanation of the reason(s) for your denial, including a citation to each specific statutory exemption you believe justifies the refusal to release the information and notify me of the appeal procedures available to me under the law. If you conclude that portions of the records that I request are exempt from disclosure, please release the remainder of such records for inspection and copying, redacting only the portion or portions that you claim are exempt.

Appendix D

Customized blocked words

The surveyed institutions blocked the following unique words using Facebook’s customizable blacklist. Duplicates were removed.

100
9/11
9-11
:))
@$$
#cocksoutforharambe
#dicksoutforharambe
<--- ♥ us ♥
?
$
$hit
2's!!!
2004toyota 4runner
5g
5t
a$$
a55
a55hole
abcreports
abuse
abusers
accident
aeolus
africant
aggy
agsaf
ahole
ahuja
alert
amazing
amcik
ame law
americanadvantagelawgroup@lawyer.com
anal
analprobe
andskota
anilingus
annoying
anthem
anus
apartment
apologize
apple
appleguru
applelovers
applemusiclist
apps
aramark
areola
areole
arian
arrrrr
arschloch
arse
artificially generated stampede awareness foundation
aryan
ash0le
asholes
asl
asla
ass
ass hole
ass monkey
assault
assbang
assbanged
assbangs
asses
assface
assfuck
assfucker
assh0le
assh0lez
asshat
assho1e
asshoel
asshole
assholery
assholes
assholz
asskisser
asslick
asslicker
asslickers
asslicks
assmaster
assmunch
assrammer
asswipe
asswipes
attention
auto
awaay
awayy
ayir
azazel
azz
azzhole
b-s
b!+ch
b!tch
b*tch
b00bs
b17ch
b1tch
b1tches
babe
babes
baboon
bad
balloon
balloonsblow
balls
ballsack
baloons
bammer
bammers
ban
bang
banger
banned
barf
bart
bassterds
bastard
bastards
bastardz
basterds
basterdz
bawdy
bdsm
beaner
bearded clam
beardedclam
beastial
beastiality
beastility
beat-off
beatch
beater
beating beat sucking
beating-off
beatoff
beaver
beavers
beavs
beecher
beer
beeyotch
before
believee
belly whacker
beotch
bernie
bert
bestial
bestiality
bfd
bi+ch
bi7ch
biatch
biche
big beoyotch
big tits
bigleaguepolitics.com
bigtits
bimbo
bing
bings
birds
birt
bitcard
bitch
bitchass
bitched
bitcher
bitchers
bitches
bitchin
bitching
bitchs
bitchy
blizzard
blocked
blondehot videos
bloody
blow
blow job
blowjob
blowjobs
blue waffle
bod
bodily
boffing
boink
boiolas
bollock
bollocks
bollok
bone
boned
bonehead
boner
boners
bong
boob
boobies
boobs
booby
booger
bookie
boomer
boomer sooner
bootee
bootie
booty
booze
boozer
boozy
bort
bosom
bosomy
boss
bowel
bowels
boycott
bra
brassiere
breast
breasts
britney sprays video
broken
brokenn
brokeon
brokken
brother dean
brown eye
browneye
browntown
bruins
bs
buceta
bucket bull bullshit
bug
bugger
bukkake
bull shit
bull-shit
bullsh*t
bullshit
bullshits
bullshitted
bullturds
bullying
bum
bung
bung hole
burt
business
busty
butch
butt
butt breath
butt fuck
butt-pirate
buttface
buttfuck
buttfucker
butthead
butthole
butthurt
button
buttpicker
buttplug
buttwipe
buy online
byob
bytch
c-0-c-k
c-o-c-k
c-u-n-t
c.0.c.k
c.o.c.k.
c.u.n.t
c0ck
c0cks
c0k
ca-ca
cabron
caca
cahone
call
camel jockey
camelfucker
cameljockey
cameltoe
cancel
canceled
cancelled
cannooot
canoe
carpet muncher
carpetmuncher
cashcars
cawk
cawks
cazzo
centsy
cervix
change org
cheap
check this out
chegg
chicken
chickens
chilango
chinc
chincs
chink
chode
chodes
chola
cholo
chraa
chuj
cialis
cipa
circle jerk
citylife
cl1t
class
cleared
click on advertisement
climax
clinton
clit
clitoris
clitorus
clits
clittie
clitty
close
closed
cnts
cntz
cobia
cocain
cocaine
cock
cock sucker
cock-head
cock-sucker
cock-sucking
cockblock
cockhead
cockholster
cockknocker
cocks
cocksauce
cocksmoker
cocksoutforharambe
cocksuck
cocksucked
cocksucker
cocksucking
cocksucks
coital
col
col.
col. reb
colonel mascot
commie
condom
coochi
coon
coons
cooter
corksucker
cost-free
couch burning
coug
cougars
cougs
cowfuck
cowfucker
cowfuckers
crabs
crack
cracker
crackwhore
crap
crappy
crash
crs
cruel
csc
cuck
cuckservative
cum
cum stain
cumdumpster
cummer
cummin
cumming
cums
cumshot
cumshots
cumslut
cumstain
cunilingus
cunillingus
cunnilingus
cunny
cunt
cuntface
cunthunter
cuntlick
cuntlicker
cuntlicking
cuntpunch
cuntpuncher
cunts
cuntsucker
cunty
cuntz
cya
cyberfuc
cyberfuck
cyberfucked
cyberfucker
cyberfuckers
cyberfucking
d's
d0ng
d0uch3
d0uche
d1ck
d1ld0
d1ldo
d4mn
dago
dagos
damage
dammit
damn
damned
damnit
dangerous
dawgie-style
daygo
death
deez
defriended
dego
delay
derka
deserve youtube
deviant
diaper double d
dic
dick
dick-ish
dickbag
dickdipper
dickface
dickflipper
dickhead
dickheads
dickish
dickless
dickripper
dicks
dicksipper
dickwad
dickweed
dickwhipper
dickzipper
diddle
die
dies
dike
dild0
dild0s
dildo
dildos
diligaf
dilld0
dilld0s
dillweed
dilweed
dimwit
dingle
dingleberry
dining services
dink
dinks
dipship
dipshit
dirsa
disckwad
discount
disgrace
diversity room
dkendrick
doggie dirty sanchez
doggie-style
doggy style
doggy-style
doingg
doinng
dominatricks
dominatrics
dominatrix
dong
doofus
doosh
dope
dopey
dotard
douch3
douche
douche bag
douchebag
douchebags
douchecanoe
douches
douchey
drive
drumpf
drunk
ducks
dumass
dumbass
dumbasses
dummy
dupa
durka
dyke
dykes
dyko
dziwka
eff
effin
effs
ejackulate
ejactulated
ejaculate
ejaculates
ejaculating
ejaculatings
ejaculation
ejakulate
ekrem
ekto
embarassing
embarrassing
emergency
enculer
enema
enlargement
enter to win
erect
erection
erotic
error
esad
essohbee
extacy
extasy
f u c k
f-u-c-k
f.u.c.k
f**cking
f**k
f**king
f*cking< f*ck
facebook.com
fack
faen
fag
fag1t
faget
fagg
fagg1t
fagged
fagget
fagging
faggit
faggot
faggots
faggott
faggotty
faggoty
faggs
faggy
fagit
fagot
fagots
fags
fagz
faig
faigs
faigt
fail state
failstate
fairy
fanculo
fanny
fannybandit
fart
farted
farting
fartknocker
farts
farty
fat
fat fatass
fatso
fcuk
feces
feck
feg
felatio
felch
felcher
felching
fellate
fellatio
feltch
feltcher
feltching
feminazi
ficken
filthy
fingerfuck
fingerfucked
fingerfucker
fingerfuckers
fingerfucking
fingerfucks
fire
fishbucket
fisted
fistfuck
fistfucked
fistfucker
fistfuckers
fistfucking
fistfuckings
fistfucks
fisting
fisty
fitt
flange
flikker
flipping bird
floozy
foad
foag
fondle
foobar
food
forefreedelivered
foreskin
fotze
fournier
free
freee
freeed
freeee
freex
friedcatz
frigg
frigga
fu
fu(
fu*k
fubar
fuck
fuck-tard
fuckass
fucked
fucker
fuckers
fuckface
fuckin
fucking
fuckings
fuckme
fucknugget
fucknut
fucko
fuckoff
fuckos
fucks
fucktard
fuckup
fuckwad
fuckwit
fudge packer
fudge-packer
fudge-packing
fudgepacker
fudgepacking
fuk
fukah
fuken
fuker
fukin
fukk
fukkah
fukken
fukker
fukkin
fuks
furburger
fuskies
futkretzn
fux0r
fvck
fxck
g-spot
g00k
gae
gag reflex
gai
gamesplaystoplay.co.cc
gamesplaytoplay
gangbang
gangbanged
gangbanger
gangbangers
gangbangs
ganja
gay
gay sex
gayboy
gayest
gaygirl
gays
gaysex
gayz
gazongers
gey
gfy
ghay
ghey
giftcard
gigolo
gina
gink
giving
givingg
givng
givving
glans
glitch
glitching
glory god goddamn
goatse
God damn
God Goddamn
god-damned
godamn
godamnit
goddam
goddammit
goddamn
goldenshower
gonad
gonads
goo.gl
gook
gooks
gooo
gop
gorilla
gorillalivesmatter
gorunner
gov
govgrantmoney
goyfa
grades
greaser
greee
grinder
gringo
grooniu
grooniu?zcgshpndpzruqjcyurch
groonsiu
gspot
gtfo
guido
guiena
guinne
gypsies
gypsy
h.c.c.
h00r
h0ar
h0m0
h0mo
h0r
h0re
h4x0r
haaha
haha
hahahahha
hahatodayfbs.net.tc
hair
hand fudge packer
hand-job
handjob
harambe
harambe!
harass
hard on
hardcoresex
hardee
harmbe
haughwout
haunt
hcc
he11
hebe
heeb
heil
heimey
hell
hells
helme
helvete
hemp
heree
heroin
herp
herpes
herpy
hershey highway
hey
hey there university
hick
hicks
hiddenpicz
hillary
himey
hitler
hiv
hoar
hobag
hoer
holocaust
hom0
homey
homo
homoey
homophobic
homos
honkey
honkie
honky
hooch
hood rat
hoodrat
hook
hookah
hookem
hooker
hoor
hoore
hootch
hooter
hooters
hore
horniest
horny
horrendous
horseshit
hotsex
housatonic
houston community college
http
http://apps.facebook.com
https
https://soundcloud.com/allnentertainmentt
huevon
hui
hump
humped
humping
hurry
hussy
hymen
i padds
i-paaaad
i-paaaadd
i-paaadd
ice
icrazy
idiocy
idiot
idiots
ifree
igiveaway
iipaaaad
iipaaaadd
iipaaaaddd
iipaaaadddtwos
iipaaaaddtwos
iipaaad
iipaaadd
iipaaadd2s
iipaaaddtwo's
iipaad
iipad
iipad 2's
iipad2
iipad2s
iipad2ss
iipads
iipads2
iippaaaaad2
ike likeeeeee
illuminati
im
inbred
incest
injun
insurance
intercourse
ipaaaad
ipaaad
ipaaad twoooosss
ipaaadd
ipaaadd2
ipaaaddd
ipaad
ipaad2
ipaad2's
ipaads
ipad
ipad2
ipad2entry
ipad2s
ipadd
ipads
ipadss
iphon5
iphone
iphone 5
iphone5
iphones
ippaaaadtwoo
ippaad
ippad
ippads
iq
iq score
irresponsible
isis
j3rk0ff
jack
jack off
jack-off
jackass
jackhole
jacking jackoff
jackoff
jambo
jap
japs
jerk
jerk jerkoff
jerk-off
jerk0ff
jerked
jerkoff
jersey
jersey store
jetblue
jewpidily
jigaboo
jihad
jisim
jism
jiss
jiz
jizm
jizz
jizzed
junkie
junky
k e r
kaffer
kaffir
kanker
kawk
kido
kike
kikes
kill
killary
kills
kinky
kkk
klan
klootzak
kma
kmia
knijnenburg
knob
knob end
knobend
knobs
knobz
knulle
Kock
kondum
kondums
kony
kooch
kooches
kootch
kracker
kraut
kuk
kuksuger
kum
kummer
kumming
kums
kunilingus
kunt
kunts
kuntz
kurac
kurwa
kusi
kyke
kyrpa
l3i+ch
l3itch
laate
lab
labia
lame
lampshade
late
latee
lech
leper
leppo
lesbian
lesbians
lesbo
lesbos
lez
lezbian
lezbians
lezbo
lezbos
lezzian
lezzie
lezzies
lezzy
libtard
libtards
life
like
like pls. :)
linthead
lipshits
lipshitz
listofapples
lmao
lmbo
lmfao
loin
loins
lube
lucky
lusty
m-fucking
Mafucka
mamhoon
mams
manigger
marijuana
masochits
masokist
massa
massterbait
masstrbait
masstrbate
masterbaiter
masterbat
masterbat3
masterbate
masterbates
masterbating
masterbation
masturbat
masturbate
masturbating
masturbation
maxi
md
mears
menses
menstruate
menstruation
merd
merde
messing
messng
messsin
messsing
meth
mexcrement
mexican't
mexicant
mibun
mick
middlesex
migger
minee
mofo
molest
money
monkleigh
moolie
moron
morons
moshky
moss
motehrfucking
motha mother-fucker
mothafuck
mothafucka
mothafuckas
mothafuckaz
mothafucked
mothafucker
mothafuckers
mothafuckin
mothafucking
mothafuckings
mothafucks
mother fucker
motherfuck
motherfucka
motherfucked
motherfucker
motherfuckers
motherfuckin
motherfucking
motherfuckings
motherfucks
motos
motss
mouliewop
mountainqueer
mountainqueers
ms
msu dining
mtherfucker
mthrfucker
mthrfucking
muff
muffdiver
muie
mulkku
murder
muschi
mutha n1gr
muthafuckaz
muthafucker
mutherfucker
mutherfucking
muthrfucking
my erotic videoss
myn
myne
n'er
nad
nads
nagur
naked
napalm
nappy
nastt
naugatuck
nazi
nazis
nazism
neel
negro
negroes
nepesaurio
ner
newinbox
news
newsfeedz
newzfeed
nfw
nice share
nig
nigga
niggah
niggas
niggaz
nigger
niggers
niggle
niggs
night
niglet
nigs
nigur
niiger
niigr
nike
nimrod
ninny
nipple
nobama
nooky
notmanyleft
nowayjoesez
nutsack
nutz
nvcc
nympho
odumbo
offer
omg
oout
open
opiate
opium
orafis
oral
orally
orangeman
organ
orgasim
orgasims
orgasm
orgasmic
orgasms
orgasum
orgies
orgy
oriface
orifice
orifiss
orospu
ou
ovary
ovum
ovums
p.u.s.s.y.
p0rn
packi
packie
packing
packy
pad
paddy
page
pagee
paki
pakie
paky
pantie
panties
panty
park
parking
partyvid
paska
pastie
pasty
pcp
pecker
pedo
pedobear
pedophile
pedophilia
pedophiliac
pedos
pee
peeenus
peeenuss
peenus
peepee
peinus
pen1s
penas
penetrate
penetration
penial
penile
penis
penis-breath
penises
penus
penuus
perse
perversion
peta
petition
peyote
phalli
phallic
phone phonesex
phuc
phuck
phuk
phuked
phuker
phuking
phukked
phukker
phukking
phuks
phuq
picka
picture
pierdol
piicture
pillow biter
pillow-biter
pillowbiter
pillu
pimmel
pimp
pimpis
pinko
piss
piss-off
pissed
pissers
pisses
pissin
pissing
pissoff
pissrr
pita
pizda
pjaavpietrab
pledge
plow
plssss
plug
pmp
pms
polac
polack
polak
pollock
poon
poonani
poontang
poontsee
poop
poor
porn
porno
pornography
pos
pot
potty
ppictureeee
ppiicctureeee
pr0n
pr1c
pr1ck
pr1k
preteen
prick
pricks
prig
profile
prostitute
prude
pube
pubic
pubis
pula
pule
punkass
punky
puss
pusse
pussee
pussies
pussy
pussypounder
pussys
puta
puto
puuke
puuker
qahbeh
queaf
queef
queefs
queer
queero
queers
queerz
quicky
quim
qweers
qweerz
qweir
r-tard
racy
rag head
raghead
ragheads
rape
rapeculture
raped
raper
rapist
rapists
raunch
rautenberg
really
rebel
recieved
recktum
rectal
rectum
rectus
redneck
rednecks
reefer
reetard
reich
republican
resign
retard
retarded
retards
revue
rim-job
rimjob
rimming
ritard
roads
rofll
roflmao
rotflmaotid
rotflolapmp
rtard
rtfm
rtwfq
rum
rump
rumprammer
ruski
s hit
s-h-1-t
s-h-i-t
s-o-b
s.h.i.t.
s.o.b
s.o.b.
s0b
sadism
sadist
sale
sales
sandnigger
scag
scank
scantily
scared
scat
schaffer
scheiss
schizo
schlampe
schlong
schmuck
screw
screwed
screwin
screwing
scrog
scrot
scrote
scrotum
scrud
scrwing
scum
seaman
seamen
seduce
seks
self
sell
selling
semen
senators
sendng
sex
sexual
sexy
sh!+
sh!t
sh*t
sh1t
sh1ter
sh1ts
sh1tter
sh1tz
shamedame
sharmuta
sharmute
sheister
shemale
shi+
shipal
shit
shit face
shite
shiteater
shited
shitface
shitfucker
shitfull
shithead
shithold
shithole
shithouse
shiting
shitings
shits
shitshow
shitt
shitted
shitter
shitters
shittin
shitting
shittings
shitty
shity
shitz
shiz
shot
shyt
shyte
shytty
shyty
silent sam
sissy
site
sitee
skag
skanck
skank
skankee
skankey
skanks
skanky
skeet
skittle
skittles
skrib
skype
skypes
slags
slave
sleaze
sleazy
slip
slut
slutdumper
slutkiss
sluts
slutty
smart
smegma
smut
smuts
smutty
snatch
snigger
sniper
snow
snowflake
snowflakes
snuff
sodom
sol
son-of-a-bitch
sooners
sorry
souse
soused
sperm
spic
spick
spik
spiks
splooge
spooge
sprays
spraysvideo!!!!..
spunk
stalker
steamy
stfu
stiffy
stoned
street
strip
stroke
stupid
suck
sucked
sucking
sucks
sucky
sumofabiatch
sumshots
sutz
t1t
taint
tampon
tard
tards
tawdry
teabagging
teat
teets
teez
terd
terrible
terroists
terrorism
terrorist
teste
testee
testes
testical
testicle
testis
the colonel
theyre
thisi
thiss
thisss
thrust
thug
tinkle
tit
titfuck
titi
tits
titslut
titt
tittiefucker
titties
titty
tittyfuck
tittyfucker
toe
toke
tomorrow
toots
top10 unviersities
tosser
towel-head
towelhead
towhead
tragedy
tramp
trannsy
tranny
transsexual
trashy
trevillian
truck
truebloodx
trump
trumptards
tubgirl
turd
turtle
turtles
tush
twat
twats
twink
two
two's
twoo's
twoos.
twos!!
ugly
um
umm
ummm
ummmm
undies
undrgrndsound
university of oklahoma
unwed
up
update
upp
urinal
urine
uterus
uyuo
uzi
v1agra
va1jina
vag
vag1na
vagiina
vagina
vaginas
vaj1na
vajina
valium
viagra
victoria's secret
video watch
video!!!!
videoo
videooo
videoooo
videooooo
videos pls
virgin
vixen
vodka
vomit
voyeur
vulgar
vullva
vulva
vvideoo
vviideoo
w00se
w0p
wad
walkways
wang
wank
wanker
watt
wazoo
weathe
web
webpage
webpagee
website is messing up right now
websitee
wedgie
weeabo
weed
weenie
weewee
weiner
weirdo
wench
weree
wetback
wh00r
wh0re
wh0reface
whack titties
whack-off
whatt
whipping
white boy
whitey
whiz
whoar
whoralicious
whore
whorealicious
whored
whoreface
whorehopper
whorehouse
whores
whoring
wigger
womb
woody
wop
wowreallly
wtf
wtfiyp
wth
wut
www.bruinslist.com
www.bruinspokerclub.com
x-rated
xrated
xxx
xxxx
yabbo
yabbos
yahoo
yahoos
yaun
ybya
ybysa
yeasty
ykyarw
yobbo
you
zcgshpndpzruqjcyurch
zeeb
zigaboo
zoophile
 
Share