AI as a Second Opinion: Evaluating Workflow Timing in Clinical Psychology - Roxhage,André; Henriksson,Erik (2026)
This study investigated how AI support timing impacts clinicians’ diagnostic accuracy and efficiency during simulated psychiatric assessment using TalkToAlba, a clinical conversational AI platform. Professionals with experience in mental health (N = 51) were randomized into three conditions: control/delayed (no initial AI), retroactive (AI post-interview), or proactive (AI pre- and post-interview). Participants interviewed interactive AI-simulated patients in a chat interface, made a diagnostic assessment, and completed documentation. Final diagnostic accuracy was higher in the retroactive than in the proactive condition (72.2% vs. 38.5%), however, this difference did not reach statistical significance (Fisher’s exact test, p = .079) and should be interpreted cautiously given the uneven distribution of patients across conditions. Clinician accuracy was strongly associated with whether the AI system’s suggested diagnosis was correct (p = .003), indicating vulnerability when AI output was inaccurate. Total task duration did not differ across conditions, but AI-assisted journal note generation reduced documentation time (p = .048). Contrary to expectations, clinician-AI agreement was highest when clinicians worked independently (κ = .62). Clinicians reported a strong will to adopt AI clinical practices, as well as a significant preference for receiving support after the interview rather than before (p = .029). Findings suggest that a human-first approach where the AI acts as second opinion instead of using AI as an investigative lead, has potential to mitigate cognitive risks such as anchoring and automation bias
Keywords: Generative AI; clinical decision support; diagnostic accuracy; workflow efficiency; generativ AI; kliniskt beslutsstöd; diagnostisk träffsäkerhet; arbetseffektivitet