A Shared Vocabulary for Local News Ecosystem Research

The Local News Impact Consortium (LNIC) is a journalism support organization that seeks to bolster the growing research community dedicated to studying the health of local information ecosystems. This research community is vast but often siloed in newsrooms, across support organizations, as well as within academic institutions, which have historically focused on communicating within their own narrow networks.

As Press Forward and other initiatives have sought to mobilize the philanthropic community around reinvigorating local news, we see a parallel need for coordination among this fragmented community of scholars, practitioners, and consultants conducting ecosystem research often so critical to the success of these endeavors. Last year, with seed funding from the Knight Foundation, the LNIC came together around a shared mission: to advance and improve standardization in measurement, data collection, and analytic approaches; facilitate wider adoption of methods using best practices and open-source tools; and grow capacity by enlisting and training more local experts to contribute to these efforts.

We believe local ecosystem research can become more rigorous, more efficient, and more responsive to the vast array of present needs—enabling an improved real-time understanding of how information ecosystems are changing within communities, across geographies and over time, thereby better serving the variety of stakeholders who depend on this research.

As we have discussed the Consortium in different venues over the past year, we have often encountered a number of questions about who we are, what we are trying to achieve, and even what it means to research local news and information ecosystems. After all, there are a variety of terms used to describe similar efforts: landscape scans, needs assessments, ecosystem analyses, asset mapping, etc. This document is intended to serve as a guide to answer basic questions about collective efforts to study local news ecosystems and suggest a shared vocabulary and framework for advancing research on these subjects.

Local news ecosystem research, and information ecosystem research more generally, refers simply to the study of how news and information flow through communities. Because ecosystems are complex and dynamic, a single study is rarely able to capture these information flows in their entirety. Instead, local news ecosystem research often involves studies of several distinct, but intertwined, subjects. These include audiences, journalists, institutions, content, and contexts. Each involve, at their core, a variety of important research questions with differing implications for different stakeholders:

Depending on who and what researchers are studying, they will often go about studying local news ecosystems in different ways. Most often, for example, ecosystem research involving the study of individuals (audiences or journalists) requires some form of interviewing. These can be done informally through listening sessions or more formally though either qualitative approaches (e.g., one-on-one in-depth interviews or focus groups) or quantitative methods (e.g., surveys).

Qualitative studies can be particularly useful for ecosystem research since researchers often want to surface important sources or topics of information within particular communities and may not know in advance precisely what to ask or how to ask about it. Quantitative studies, on the other hand, enable more systematic comparisons between profiles of survey respondents, but the value of such analyses depends heavily on the quality, rigor, and scale of the methods employed. Sophisticated sampling methods are typically critical for ensuring that all relevant subgroups within a given population are adequately included or else surveys will overrepresent the views of those who are easiest to reach and most enthusiastic to respond. Costs associated with collecting such data can also vary widely but can easily range from tens of thousands of dollars to more than $100,000 depending on the size and difficulty of the study.

Studies of institutions, content, and contexts can also involve interviews and surveys but research on these subjects more typically involves collecting a variety of different kinds of data. For example, studies of institutions often involves establishing databases cataloguing organizations and the communities in which they are situated or ostensibly serving; studies of content typically involve collecting specific samples of articles (or content from alternative sources) to perform discrete analyses of coverage; studies of contexts involve amassing indicators of institutional health tracked by third-parties (e.g., revenue, employment) or the U.S. Census (e.g., demographic profiles of communities) and organizing these data for statistical analyses.

In other words, knowing who and what researchers seek to study has significant implications for how they are most likely to go about collecting and analyzing data.

 In addition to the above questions, the why and when of local news ecosystem research tends to depend heavily on who is conducting the research. Different stakeholders often care about distinct aspects of local news ecosystem research, and a lack of access to standardized data often makes studying system-level changes more challenging.

Journalists and publishers, for example, have arguably the most at stake in understanding the health of their own ecosystems in order to identify opportunities for investments, inform priorities for innovation, and assess their own strengths and weaknesses. Better understanding their existing (and aspirational) audiences is critical for these purposes as well as how the broader context in which they operate compares to other communities. But newsrooms must also have an accurate picture of where they stand relative to other institutions of journalism and other sources of information in a given community—e.g., how crowded of a market it is?—as well as how their content compares and their ability to recruit or retain journalists given existing market conditions. These questions depend on understanding relative differences across communities.

Funders and investors in local news have similar concerns but often take a wider vantage point. More standardized data can be helpful for these stakeholders to be able to track changes over time across a range of places. At the same time, some funders and investors treat research less as a tool for guiding decision-making and more as a means for achieving other aims—either for marketing purposes to gin up excitement about potential investments or as a way to prod grantees to engage in more community-centered listening and engagement in the hopes that the practices themselves encourage more responsive journalism. Both may be valuable reasons for conducting research, but so long as the end users of the research are more focused on these alternative objectives, methodological rigor and researcher independence can be relegated as secondary concerns.

Members of the public often care deeply about ecosystem research for their own reasons. Concerned citizens may want to better understand what outlets are serving their own communities, who funds them, or how well they are meeting the information needs of their fellow community members—especially given increasingly fragmented and contentious information environments. Therefore, the public is likely to care more about access to specific, actionable information about their own communities whereas academic researchers are often more focused on advancing generalizable knowledge about the relationships between, for example, a healthy local news ecosystem and levels of political and civic engagement in general. Academic researchers therefore tend to value data collection approaches that facilitate such analyses but rarely focus on translating these findings for audiences other than each other. Support organizations and consultants on the other hand often tailor their insights to individual clients to help them guide their own strategic decision-making and external communications and tend, therefore, to be less committed to sharing their data widely.

Last but not least are the array of other stakeholders who also want to understand local information ecosystems. These include those who regularly disseminate their own information in communities directly or indirectly (via news media) including government agencies, civic organizations and advocacy groups; advertisers and small businesses; as well as technology companies and public relation firms who play important roles in shaping information ecosystems to advance their own interests.

In short, groups often differ in what they value most about local news ecosystem research, reflecting their own varied aims and objectives. Understanding these differences is important since it often shapes both research questions and methodologies in consequential ways.

Lastly, in addition to offering this shared vocabulary, below are several of the most frequently asked questions we have heard while conversing about ecosystem research with various stakeholders this past year. We lay out our typical response below, but we also want to stress that we do not have all the answers. The spirit of our consortium is collaborative by design: We are not seeking to dictate the single way that local news research should be conducted but rather create the conditions for this community to arrive at shared standards and approaches.

My newsroom or my community is unique; why bother conducting research using methods and standards designed to facilitate comparisons to other markets when I just want to understand our own circumstances?

Every news organization and every community can point to features of their own ecosystem that make their situations unique, and it is true that one of the downsides of comparative research is that measures used to compare across organizations and communities tend to flatten out many of these particularities and nuances. That said, using shared methods and standardized approaches—at least as part of a research strategy, even if not its entirety—can be hugely beneficial for any organization or community conducting ecosystem research.

There are at least three reasons why.

First, comparisons to other organizations and communities enables more effective interpretation around what are (or may not be) significant or meaningful deviations from established benchmarks.  For example, when evaluating levels of interest among local news consumers, preferences for particular digital platforms, or frequency or depth with which certain topics get covered locally, it is rare that these absolute measures speak for themselves. It is the relative difference in these levels that is often most meaningful, which allows stakeholders to assess precisely what may be distinct about the organizations or markets they are studying.

Second, even without making comparisons to other places, it is often the case that researchers seek to understand changes over time in their own community or the performance of a particular organization that has taken steps to alter its editorial or engagement strategies. Through the use of consistent measurement approaches, researchers gain a more accurate understanding of these changes and assess in a more compelling way what has been most effective.

Third, this work is costly and complicated; individual newsrooms or community foundations or researchers should not have to reinvent the wheel each time they seek to answer similar questions. The LNIC is developing shared tools designed to make it easier and more cost-effective to conduct local news research to solve that challenge, not further complicate this work.

Why is local ecosystem research so expensive?

There are many reasons why research can be costly. Local information ecosystem research is often time-consuming to conduct since much of the data we are interested in analyzing has not (yet) been assembled in any consistent or standardized format. Fielding surveys especially has also become increasingly expensive over the last few decades, even as costs associated with distributing surveys online has fallen considerably, because it is more difficult than in the past to reach a representative sample. This is especially the case for researchers committed to including younger, less educated, less politically engaged publics who are more difficult to reach.

The LNIC seeks to bring down these costs by improving access to shared resources such as common codebooks for assembling databases of local information providers and libraries of interview protocols and survey questionnaires. We are even exploring mechanisms for pooling resources around conducting local ecosystem surveys jointly. (Let us know if you’re interested!) But realistically, every ecosystem study is subject to limitations and every method comes with tradeoffs. The key is to be thoughtful about what those tradeoffs may be and where it makes sense to commit scarce resources. After all, insights drawn from non-representatives surveys or datasets that contain important omissions can easily lead decision-makers to draw the wrong conclusions.

Researchers (academics especially) never agree with each other; how will they ever come to a consensus around methods and best practices?

Research communities are often criticized for being too slow, too siloed, and even too pitted against each other in zero-sum competitions for funding, which makes it difficult to advance knowledge in a coordinated way. These critiques are not entirely unfair. And while it is true that research communities are built on cultures of critique and peer review, scientific advances depend on open interrogation of methodological approaches and data quality.

To that end, the LNIC is built around a shared commitment to the values of open science (including transparency in methodology and data-sharing) and the use of open source technology (including through the tools we are developing and validating). This shared commitment serves to unite our research community even as we engage in fierce debate. And while the scale of the challenges around standardization cannot be underestimated, there are numerous successful examples of similar research partnerships that have become essential tools across the social sciences, including projects such as the American National Election Studies, the Cooperative Election Study, the General Social Survey, CHIP50, TESS, etc.

Does it matter where I start?

There is no single right or wrong way to conduct this work, but there are advantages to starting first by assembling a database of news outlets and other important information providers within a given ecosystem before proceeding with other data collection efforts.

There are many reasons to start here.

First, gathering data on existing outlets and assembling an initial database is simply easier and cheaper to do than collecting a survey of audiences in a local community.

Second, these data on institutions can help inform interviews or surveys conducted with members of the public and other data collection efforts involving individuals at local media and community organizations. These subsequent steps of data collection can, in turn, be particularly critical for ensuring the data collected in the larger database is more comprehensive by identifying sources that researchers themselves may not be aware of. Third, publishing interactive maps of the outlets and providers in a given ecosystem can serve as a valuable resource for stakeholders and members of the community as additional research efforts unfold. These maps can be thought of as starting points—not the end points—for ecosystem research. Many of us who have published interactive maps of outlets in our ecosystems have found that by making these resources available to members of the public, journalists, and other stakeholders, it has helped to identify omissions in the map, ongoing changes in our ecosystem, and areas for future data collection (e.g., about ownership, coverage area, etc.).

Third, the underlying databases powering these maps are often needed in order to conduct additional kinds of data collection including news content analyses and surveys of publishers and owners, both of which depend on having a comprehensive portrait of the relevant information providers across a given ecosystem.

How do I get involved with the LNIC?

Drop us a line here, join our Slack community, or sign up to receive our newsletter.