Mitchell L Gordon

The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality

Machine learning classifiers for human-facing tasks such as comment toxicity and misinformation often score highly on metrics such as ROC AUC but are received poorly in practice. Why this gap? Today, metrics such as ROC AUC, precision, and recall are used to measure technical performance; however, human-computer inter…