This week researchers from the Harvard Digital Lab for the Social Sciences (DLABSS) released a working paper analyzing the use of digital volunteers in social science research. A full version of the working paper is available here (PDF).
The central finding of the study is that digital volunteer labs such as DLABSS can generally perform as well as better known contemporary and traditional survey platforms. This includes cheap, but generally reliable online marketplaces such as Amazon Mechanical Turk (MTurk) and Google Consumer Surveys (GCS), which have gained immense popularity and widespread use across the social sciences in the past decade. As the below table illustrates, respondents in DLABSS look generally comparable to those in these other digital platforms, and also appear broadly similar to respondents in more nationally representative social science samples.
The study also finds that, like other platforms mentioned above, DLABSS is able to replicate a wide range of classic and contemporary social science findings. Table 2 illustrates a selection of experimental studies successfully replicated by DLABSS, meaning that the study was re-run using DLABSS and the results were similar both in terms of expected effects and confidence levels around those effects.
With roughly 9,000 volunteers at present, DLABSS is an example of how researchers and academic institutions can build on the innovation of digital crowdsourcing and further lower barriers to social science research. DLABSS has already hosted over 70 studies by Harvard faculty, graduate, and undergraduate students. We look forward to the continued growth and expansion of DLABSS as a free resource for academics at Harvard and beyond.