Impact Assessment Card: Communicating Risks and Benefits of AI Uses

Communicating the risks and benefits of AI uses is crucial for regulatory compliance and increasing public awareness, but its effectiveness is currently limited to individuals with technical expertise and often presented in highly specialized impact assessment reports. Drawing upon the HCI and CSCW literature on making complex concepts broadly accessible, we propose an impact assessment card for communicating the risks and benefits of AI uses in a way that is accessible to individuals without technical expertise. Through an iterative design process, we conducted three focus groups with a total of 12 participants who identified design requirements for an impact assessment card and designed a set of speculative cards. We then reviewed these speculative cards and iteratively produced the final version of the card. We evaluated this card's effectiveness for conducting a real-world task, that is, to write an email to either recommend the implementation of a hypothetical AI system or advising against it, and compared the task's outcomes (i.e., email quality, efficiency, usability, and preference) against a baseline fully-fledged report in an online study with 235 participants grouped in three cohorts: AI developers; compliance experts; and ordinary individuals who reflect US census in terms of age, sex, and race. After controlling for the type of cohort and task, as well as our participants' expertise in AI and technology more broadly, we found that the most significant difference in the task's outcomes was attributed to the use of card or report. In fact, across all three cohorts, the card was found to be more usable and effective; participants spent less time on executing the task at hand and wrote emails of higher quality. Surprisingly, the card not only helped ordinary individuals but also proved useful to developers and compliance experts---two cohorts that are already attuned to the impact assessment process and frequently use reports as part of it. We reflect on the role of HCI in further refining the card through the use of color, language, and metaphorical representations, aiming to break down barriers to understanding the risks and benefits of AI uses and, ultimately, transforming impact assessment cards into a standardized tool for AI governance.