Suicide Prevention Using Language Model Assistants

To identify both novel suicide risk factors and potential bystanders from suicide death narratives

We facilitate data-driven insights from the NVDRS data to support the development of novel suicide interventions by leveraging language models as assistants to both NVDRS data annotators and prevention research experts. First, we design an algorithm that uses language models (LMs) to predict existing structured variables in the NVDRS from the law enforcement and coroner’s medical reports, to ensure that language models can do this task effectively. Indeed, we find that LM predictions match existing data annotations with high accuracy across multiple NVDRS variables. Next, we introduce a human-in-the-loop algorithm that helps experts efficiently build and refine guidelines for predicting novel variables by having them only focus on providing feedback for incorrect LM predictions. We apply our algorithm to a real-world case study to find evidence of victim interactions with legal professionals, which surfaces a substantial opportunity for upstream intervention that is not captured in the original structured data.