Conference site » Proceedings

NEXT: A system to easily connect crowdsourcing and adaptive data collection

Scott Sievert
University of Wisconsin–Madison

Daniel Ross
University of Wisconsin–Madison

Lalit Jain
University of Michigan, Ann Arbor

Kevin Jamieson
University of California, Berkeley

Rob Nowak
University of Wisconsin–Madison

Robert Mankoff
The New Yorker


Obtaining useful crowdsourcing results often requires more responses than can be easily collected. Reducing the number of responses required can be done by adapting to previous responses with \textquotedbl{}adaptive\textquotedbl{} sampling algorithms, but these algorithms present a fundamental challenge when paired with crowdsourcing. At UW–Madison, we have built a powerful crowdsourcing data collection tool called NEXT ( that can be used with arbitrary adaptive algorithms. Each week, our system is used by The New Yorker to run their Cartoon Caption contest ( In this paper, we will explain what NEXT is and it's applications, architecture and experimentalist use.


crowdsourcing, adaptive sampling, system

Bibtex entry

Full text PDF