MechanicalHeart: A Human-Machine Framework for the Classification of Phonocardiograms

Listening to heart sounds is an important first step in evaluating the cardiovascular system and is important in the early detection of cardiovascular disease. We present and evaluate a framework for combining machine learning algorithms, crowd workers, and experts in the classification of heart sound recordings. The development of a hybrid human-machine framework is motivated by the past success in utilizing human computation to solve problems in medicine and the use of human-machine frameworks in other domains. We describe the methods that decide when and how to escalate the analysis of heart sounds to different resources and incorporate their decision into a final classification. Our framework was tested with a combination of machine classifiers and crowd workers from Amazon's Mechanical Turk. The results indicate a hybrid approach achieves greater performance than a baseline classifier alone, utilizing less expert resources while achieving similar performance, compared to a framework without the crowd.