Christoph Hube

Understanding and Mitigating Worker Biases in the Crowdsourced Collection of Subjective Judgments

Crowdsourced data acquired from tasks that comprise a subjective component (e.g. opinion detection, sentiment analysis) is potentially affected by the inherent bias of crowd workers who contribute to the tasks. This can lead to biased and noisy ground-truth data, propagating the undesirable bias and noise when used in…