The National Academies of Science has come out with a new report on citizen science. Citizen science is the practice of using large numbers of volunteers, rather than professional researchers, to collect data for scientific research. This is not a new idea: the Audubon Christmas bird count, for example, has been going on for more than a century. But only recently have scientists started to examine these projects’ efficacy.

A 2010 article in Annual Reviews by Janis L. Dickinson, Benjamin Zuckerberg, and David N. Bonter examined some of the issues surrounding citizen science projects. There are huge advantages, like the ability to accumulate a dataset lasting years and covering multiple continents. Data of that breadth can be a game changer when studying global processes such as climate change. Additionally, the ubiquity of smart phones has vastly improved the public’s ability to gather and share data. So much data can be gathered, in fact, that it has forced the development of new analytical techniques.

Monitoring and surveillance are good applications for citizen science. Experiments fare less well. It is far easier to update a database of observations than it is to monitor half a million control groups. The participants benefit in terms of learning and engagement with scientific process. In fact, Dickinson notes, many projects have been designed mostly with education in mind, and would benefit from more involvement by research specialists.

There are cons. For one thing, citizen science is best for studies that do not require specialized equipment; it’s not coming soon to a particle accelerator near you. Additionally, it’s impossible to closely monitor every volunteer, so the quality of data collection, plus any systemic flaws or biases, are difficult to assess. In 2017, a large team led by Erendira Aceves-Bueno tried to compare volunteer-sourced data to professionally-collected data. Their conclusion was that citizen scientists were reasonably accurate, producing quality work about 73% of the time. However, depending on study type, data collected by citizens differed from professional data almost 50% of the time. In many other cases agreement was only 80% or so. That might not be precise enough for actionable science.

Aceves-Bueno recommends that professionals collect a sample of data first for reference, making it possible to identify where volunteer data differs. Results are better when volunteers receive training and work on projects for extended periods of time. Additionally, volunteers seem to collect better data when they have a vested interest in the project. This jibes with the National Academies study finding that efforts should be made to recruit volunteers who more closely resemble the makeup of society. As it stands, citizen scientists are mostly white and relatively affluent.

When the issues of accuracy and representation are fully addressed, citizen science will be an excellent source of high quality research.