We need to talk about nonprobability samples

• Published in: Trends in ecology & evolution (Amsterdam) Vol. 38; no. 6; pp. 521 - 531
• Main Authors: Boyd, Robin J., Powney, Gary D., Pescott, Oliver L.
• Format: Journal Article
• Language: English
• Published: England Elsevier Ltd 01-06-2023
• Subjects: Biodiversity; biodiversity monitoring; citizen science; convenience sample; risk-of-bias; sample representativeness; Sampling Studies; selection bias; sample representativeness; selection bias; risk-of-bias; citizen science; convenience sample; biodiversity monitoring
• ISSN: 0169-5347; 1872-8383
• DOI: 10.1016/j.tree.2023.01.001

As the data revolution gathers pace, researchers are increasingly relying on nonprobability samples from meta-databases, citizen science and other sources to monitor biodiversity.The use of nonprobability samples can lead to biased inference, and seemingly large nonprobability samples can actually have very low information content.A number of recent high-profile disagreements in the biodiversity literature stem from the use of such samples, and the inadequate communication of their potential weaknesses.Nonprobability samples can be useful for the purpose of monitoring biodiversity, provided that their limitations are assessed, mitigated where possible, and the almost inevitable remaining issues clearly communicated.
In most circumstances, probability sampling is the only way to ensure unbiased inference about population quantities where a complete census is not possible. As we enter the era of ‘big data’, however, nonprobability samples, whose sampling mechanisms are unknown, are undergoing a renaissance. We explain why the use of nonprobability samples can lead to spurious conclusions, and why seemingly large nonprobability samples can be (effectively) very small. We also review some recent controversies surrounding the use of nonprobability samples in biodiversity monitoring. These points notwithstanding, we argue that nonprobability samples can be useful, provided that their limitations are assessed, mitigated where possible and clearly communicated. Ecologists can learn much from other disciplines on each of these fronts.