loading page

The Heads Up Checkup Digital Mental Health & Behavioral Risk Screening System: Clinical Inter-Rater Reliability for Identifying Youth in Crisis
  • +1
  • Nancy P. Genero,
  • Alyssa Lozano,
  • Mariana Sanchez ,
  • Miguel Cano
Nancy P. Genero
Wellesley College Department of Psychology

Corresponding Author:[email protected]

Author Profile
Alyssa Lozano
University of Miami School of Nursing and Health Studies
Author Profile
Mariana Sanchez
Florida International University Robert Stempel College of Public Health and Social Work
Author Profile
Miguel Cano
The University of Texas Southwestern Medical Center School of Health Professions
Author Profile


ABSTRACT The present study evaluated the inter-rater reliability of the Heads Up Checkup (HCU), a brief digital mental health and behavioral adaptive screening system designed for use in primary care and diverse school settings. Two independent licensed clinical psychologists reviewed a random sample of thirty (N=30) HCU clinical screening reports (CSRs) of 13-14 year old adolescents drawn from a larger sample (N=846) enrolled in a predominantly Hispanic public middle school in California. Results showed strong inter-rater agreement (Fleiss kappa=.93) between clinician ratings and the screener’s priority risk index (HPI) in identifying students “in crisis.” In addition, clinicians’ ratings of confidence in their priority judgments were found to be significantly higher for the high-risk “in crisis” cases. Reasonable evidence of convergent validity emerged due to the significant correlations between clinician ratings of psychological distress and the HPI. Overall findings suggest that as an online universal school-based screener, the HCU has valid utility for identifying high-risk “in crisis” cases which can translate into timely and pragmatic real-world therapeutic solutions for diverse populations of adolescents. Future research directions with respect to the refinement of measurement issues and feasibility of implementation of population-level online screeners in schools are discussed. Keywords: Adolescent psychological distress, youth mental health risk assessment, inter-rater reliability, convergent validity.