Who would toil away for pennies, completing trivial tasks and answering surveys? A quarter-million users registered with Amazon’s Mechanical Turk service. Just who exactly are the people behind the Mechanical Turk and can we rely on the data we get from the nameless faceless masses ? In this blog posted on Oracle.com I review two recent studies published and presented at CHI 2010. In one study we see a shift in the demographics of the crowd from stay-at-home moms in the US to an increasingly growing younger male worker from India.
Are people really taking a task seriously when they only get paid a dime for their time? Another study set out to find out what percent of the crowd was in it just for the money and not taking the tasks seriously or just rushing through to get paid. This study found most (88%) completed a simple task that required only a modicum of attention (presumably the 12% were rushing through or making careless errors). Sixty-one percent conscientiously answered both an easy and more involved question, suggesting the majority (but far from all) of this global workforce are providing an honest effort. Whether this effort will provide representative and reliable results for all studies is a subject of future crowdsouricng research.
|UX Measurement Boot Camp : Three Days of Intensive Training on UX Methods, Metrics and Measurement Aug. 7th-9th 2019|