Define inclusion criteria 

Just as clear decisions need to be made up front about geographic boundaries and scope, it’s also important to decide what will count as “news”—that is, deciding on criteria for which types of outlets you will include in and which (if any) outlets should not be included. 

In this section we discuss considerations around what medium types to include/exclude; what types of organizations beyond traditional newsrooms you may or may not want to include; and what types of content you may want to document as you create your inclusion criteria.

Types of Media

An obvious place to begin mapping local news is with newspapers, which are the anchor institutions for local news production in many communities. The well-known State of Local News reports by Penny Abernathy and the Medill School at Northwestern University have attempted to map all newspapers across the country since 2016, providing an important baseline for measuring the rapid shrinking of local news.

Focusing on newspapers is a critical place to start because they tend to provide much of the original news reporting in any locale and to set the agenda for other local area media–which is why some scholars refer to newspapers as “keystone media.” But they are not the only species in the local news ecosystem. Local television still draws comparatively large audiences and many stations are well-positioned to help fill gaps in local news; public radio also fills local news gaps (though commercial radio is far less likely to do so); and local podcasts, magazines, low-power TV and smaller affiliate stations and community newsletters may offer important (even if less frequent) local public affairs coverage. And even as print newspapers decline, more digital-first and hyper-local publications have emerged to fill local news and information gaps. 

Meanwhile, virtually all traditional news outlets have developed some kind of digital presence, and in many areas of the country those websites are key, even dominant, components of the local news ecosystem. And many news consumers today—even in so-called “news deserts”—encounter news via social media, including Facebook groups and other social media sites that have emerged in the void left by the closure of local newspapers. Some are even using other technologies, like text messaging, to reach their audiences. Researchers will therefore need to decide which kinds of media to document in their local news ecosystem. 

At a minimum, we suggest researchers include all identifiable newspapers, television and radio broadcast stations, and digital journalism sites that are physically located within a particular geographic region. We suggest being as comprehensive as possible, while recognizing that some types of media—particularly small print-only community newspapers and community social media sites—can be challenging to document.

Another basic and important decision is whether to require that a news outlet have some kind of website in order to be included in your census, or whether you will also try to identify small print publications and other outlets that do not have a digital presence. 

Especially if time and resources are limited, we recommend starting only with outlets that have a digital presence/URL. This method is likely to capture all of the most significant outlets in your geographic area along with most smaller outlets, and can be added to later through focus groups, audience surveys, reader surveys, and other methods.

Of course, it may also be important – depending on the purposes of your study – to include very small non-digital outlets that may be harder to discover. Are you more interested in documenting all news producing outlets that exist in your geographic locale, no matter how small, or in identifying the most widely used and potentially influential outlets (which, almost by definition, will have a digital presence)? From one perspective, small hyper local news outlets, whether in print or online, are important if one wants to thoroughly document all organizations contributing news and information in a local news ecosystem. On the other hand, small hyper-local outlets are unlikely to gain much audience reach or web traffic. 

Generally speaking, we recommend that researchers aim to be as inclusive as possible in documenting local news providers given the time and resources available, and also try to incorporate measures of audience size/reach when possible. 

Another important decision is whether your census will include news outlets that publish or broadcast in languages other than English. A truly complete local newsroom census should make every effort to include non-English news outlets. 

We discuss in the next section some strategies for identifying such outlets in your geographic area. Similarly, a truly complete census of local news outlets should include outlets owned by and/or serving racial,ethnic, gender, religious, and  communities of identity. As discussed below, producing a newsroom census that is fully inclusive is illuminating and well worth the effort.

Types of Newsrooms/Organizations

While it may seem self-evident that a “news” census will focus on traditional news media, the information landscape has changed so dramatically in recent years that it is worthwhile to think carefully about the types of organizations you will include in your census. For example, as resources for trained reporting staff shrink, more newsrooms have turned to interns and citizen volunteers to help them cover local news. In some communities, the closure of the local newspaper has prompted local residents without journalism training or experience to start their own social media-based community news service; in many communities, local news is being provided by student journalists at local student newspapers or working for traditional newsrooms. Moreover, in many places, local civic organizations and mutual aid organizations have become critical information resources—particularly for immigrant and linguistic minority communities.

 In today’s complex information environment, a significant amount of civically-relevant information available to the public is not produced by journalists, but rather by community and civic organizations, government agencies, public relations strategic communicators, social media users, etc. 

So, a relevant consideration is whether your census will include non-journalistic organizations. A study of local news in New Jersey (Stonbely 2021) discusses this challenge:

“Under some definitions of local news, a chamber of commerce, for example, might qualify, as some provide local economic information on their websites. Likewise, a local lifestyle magazine will often include personal health or local cultural news, though often based on a press release or in advertorial format. We chose not to include information-producing civic institutions or lifestyle outlets because the primary sources of information feeding them (the government in the former case, businesses and press releases in the latter) have an incentive to provide only flattering information; with the acknowledgement that they may be important information providers for some members of a community.”

In subsequent work, however, Stonbely has argued for thinking more expansively about where people get their news and information, noting that “people’s critical information needs are now met by all manner of local organizations, largely through social media.” From this perspective, mapping local new ecosystems should focus “on all civic information providers: local news outlets, yes, but also the communications of schools, chambers of commerce, health institutions, and small businesses. In other words, the public-information-producing organizations that constitute the infrastructure of community information and news landscapes.” A key consideration here is whether your project will have the time and resources required to identify all such organizations, and how you will go about it (see Overview of Approaches to Analyzing Local News above for ideas).

We suggest being as inclusive as possible, while recognizing the need to draw boundaries between journalistically produced news and other forms of local information. 

In particular, researchers may want to distinguish between news that adheres to standard journalistic practices like multi-sourcing and fact-checking and other kinds of more selective or opinionated content (a step which will require some content analysis in addition to the news census research described in this toolkit), or between content produced for a public audience versus information shared among more selective or closed groups, such as on social media sites.

Researchers may opt to also include in their analysis other types of local media, such as Facebook groups, Instagram pages, and other social media sites. For example, a news mapping project in Colorado included these sources by interviewing residents in each county to see what media they turn to for local news. Similarly, the authors of the Montana news census say they “cast a wide net when considering what is meant by ‘local news and information outlets.’ We included print, digital, radio, and broadcast outlets in our inventory—as well as innovative and nontraditional models like longform podcasts and curated newsletters.”

Yet another consideration is whether your census will include partisan news sources. 

On this question, approaches have varied. Based on interviews with members of the public that indicated the wide range of sources residents turn to for news and information, the local news census produced by the Colorado News Collaborative includes some outlets that define themselves in partisan terms, “because doing so reflects the reality of where people are getting their news and information in the area.” In another example, the author of a recent news census report out of Minnesota notes that their census includes some outlets that are “to some extent even beyond the boundaries of conventional journalism:

“We take a deliberately broad definition for what we include in our data knowing as we do that many people rely on a range of sources outside of professional journalism to stay informed about local information. We have therefore included in our news outlet census many sources, including commercial radio or single author digital creators, who may not consider themselves journalists but who nonetheless are an important conduit for local information in some communities.”

A particularly important and challenging consideration is how to treat the digital non-journalistic, often partisan outlets colloquially known as “pink slime.” 

These are “organizations that mimic local news outlets but push partisan politics or corporate communications without disclaimers or adherence to other good journalistic practices” (Moore et al 2024), outlets people may end up visiting when they search the Internet or turn to social media for local news. (Many of these sites are named in ways that suggest they are locally-based, even if they are not). One estimate suggests there are as many such outlets as there are legitimate newspapers in the United States; as of 2020, the largest operator of these sites, Metric Media, had at least 1,200 of these online outlets across multiple states. While some of these sites do offer some legitimate information, it is typically not written or vetted by journalists (and in fact much of their content is AI-generated and auto-populated across sites) and is mixed in with opinion pieces, press releases, and other non-journalistic material that is often not clearly labeled as such.

There is no clear consensus about how to treat these kinds of outlets. From one perspective, if these sites exist in your geographic area, perhaps it is important to document them as one potential source of local “news”–even though that news may be of dubious value. On the other hand, including pink slime sites in a report on “local news” could further blur the boundaries between journalistically-produced news and other kinds. One approach is to identify all outlets operated by known pink slime purveyors within the state or region by consulting sources like Bengani (2022) and either a) exclude them outright from the master list; or b) visit those sites to verify whether humanly-written, locally-relevant information appears prominently and often enough to justify inclusion in the master list; or c) simply document all such sites, being sure to clearly label them. In either case, best practice calls for clear categorization of all outlets and an explanation for readers to understand why various outlets have or have not been included in your master list.

Whatever the researcher ultimately determines the census should include, we stress the importance of being transparent about inclusion and exclusion criteria. 

If the rationale underlying those decisions is clear, readers will have the necessary information to determine how to make sense of your findings.  

Types of Content

A third consideration is whether you will examine the content of a news outlet before deciding whether to include it in your master list. There are two general approaches here: You can make a priori decisions about what kinds of media to include in your master list without looking at the content they produce; or, you can incorporate at least some minimal examination of content before deciding whether an outlet should be included.

In the first approach, you decide up front which types of outlets will be included or excluded, without examining the types of content they actually produce. Some studies (e.g., Oregon’s 2022 statewide ecosystem assessment) exclude commercial radio stations, for example, based on research suggesting that few carry much local news. A recent report on Minnesota’s journalism ecosystem includes a good example of a clear, concise explanation of how that research team decided what outlets to include without looking at the content outlets produce.

In the second approach, you commit the time to examine each outlet before deciding whether to include it. For example, except for its a priori exclusion of commercial radio stations, the Oregon news census is based on a simple content analysis that determined whether an outlet “a) appears to publish at least some local news content at least monthly, and b) produces original local journalism c) covering issues of local civic relevance.” (A valuable resource in this regard is the “critical information needs” criteria developed by Friedland et al [2012].) A 2021 report on New Jersey’s local news describes its inclusion criteria this way:

The local news providers studied here include those that would be recognized as “traditional” journalism outlets — newspapers, local television, and radio stations airing local news at the top and bottom of the hour. Also included were digital-native online newspapers, many of which are owned and/or staffed by former legacy-outlet reporters, as well as qualifying magazines serving ethnic, LGBTQ, and religious communities, if they provide regular local news as defined above about these communities or institutions.

To the extent that you decide not to exclude certain outlets a priori, but rather to examine them first before deciding whether to include them in your census, there are several things to look for. 

In general, we recommend that researchers invest time in visiting the website for each outlet to determine if the outlet carries at least some minimal level of civically relevant, locally focused news. Outlets that do not, can either be excluded from the master list, or included in the list but coded in a way that will allow you to distinguish between them and the other outlets in your list.

Again, the recent Washington state report provides a clear example of one approach to drawing boundaries around what kinds of news outlets and news content to include in a local news census:

“To meet the content quality threshold for inclusion in the database, the outlet must meet four main criteria, including that it must a) regularly publish substantial content on a website or social media page that is b) original, c) journalistic, and d) locally relevant. Substantial content that is published regularly means that the outlet has published content in the past 30 days that is substantial in length (e.g., an article of at least 300 words, video or audio of at least one minute). Original content means that the outlet has published content that was created by the outlet’s staff rather than by another outlet located outside of the community or by another organization. Journalistic content means that, in the past month, the outlet has published information that is “a ‘reported’ story or when a content creator gathered information from spokespersons, experts, documents and other sources and attributed information in the story to those sources.” Locally relevant content refers to, in the past month, the outlet has published content that is about or of specific interest to the community in which the outlet is located or to which it is oriented.”