With Few Eyes, All Hoaxes are Deep

Quality control is critical to open production communities like Wikipedia. Editors enact quality control on the borders of Wikipedia to review edits (counter-vandalism) and new article creations (new page patrolling) shortly after they are saved. In this paper, we describe a long-standing set of inefficiencies that have plagued new page patrolling by drawing a contrast to the more efficient, distributed processes for counter-vandalism. To effect better page review distribution, we develop an effective automated topic model based on a labeling strategy that leverages a folksonomy developed by subject specific working groups in Wikipedia (WikiProject tags) and a flexible ontology (WikiProjects Directory) to arrive at a hierarchical and uniform label set. We are able to attain very high fitness measures (macro ROC-AUC: 95.2%, macro PR-AUC: 74.5%) and real-time performance using word2vec-based features on the intial draft versions of articles. Finally, we present a proposal for how incorporating this model into current tools will shift the dynamics of new article review positively.