2022-afst-designs.html

---
layout: publication
year: 2022
title: '"Why Do I Care What''s Similar?" Probing Challenges in AI-Assisted Child Welfare Decision-Making through Worker-AI Interface Design Concepts'
authors:
  - Anna Kawakami
  - Venkat Sivaraman
  - Logan Stapleton
  - Hao-Fei Cheng
  - Adam Perer
  - Zhiwei Steven Wu
  - Haiyi Zhu
  - Kenneth Holstein
type:
  - Conference
venue: DIS
venue_location: Virtual
venue_tags:
  - DIS
venue_url: https://dis.acm.org/2022/
tags:
  - AI-Assisted Decision Making
  - Social Work
  - Design
  - Visualization
---

Data-driven AI systems are increasingly used to augment human decision-making in
complex, social contexts, such as social work or legal practice. Yet, most
existing design knowledge regarding how to best support AI-augmented
decision-making comes from studies in comparatively well-defined settings. In
this paper, we present findings from design interviews with 12 social workers
who use an algorithmic decision support tool (ADS) to assist their day-to-day
child maltreatment screening decisions. We generated a range of design concepts,
each envisioning different ways of redesigning or augmenting the ADS interface.
Overall, workers desired ways to understand the risk score and incorporate
contextual knowledge, which move beyond existing notions of AI interpretability.
Conversations around our design concepts also surfaced more fundamental concerns
around the assumptions underlying statistical prediction, such as inference
based on similar historical cases and statistical notions of uncertainty. Based
on our findings, we discuss how ADS may be better designed to support the roles
of human decision-makers in social decision-making contexts.

<p><em>Two co-first authors contributed equally to this work.</em></p>