What we know:

Attempts to create a personalized treatment rule (PTR) for enhancing the treatment of major depressive disorder (MDD) with antidepressant medication (ADM), psychotherapy, or their combination have been hindered by suboptimal analytical techniques, modest sample sizes, and limited predictor sets [1-7]. Examinations of extensive electronic health records (EHRs) aimed at simulating experiments, followed by randomized controlled trials (RCTs), offer potential solutions to these challenges [8].

Our aim:

Our aim in this paper was to advance previous studies on PTR development by investigating the feasibility of creating an initial PTR based on the extensive EHR administrative database available in the Veterans Health Administration (VHA) system. The report introduced a proof-of-concept investigation based on 43,470 outpatients initiating treatment for MDD in VHA Primary Care Mental Health Integration (PC-MHI) clinics. These clinics provide access not only to ADMs but also to psychotherapy and combined ADM-psychotherapy. The outcome was an integrated assessment of one or more negative outcomes occurring to the patient within 365 days following MDD treatment initiation: either a suicide attempt, a visit to a psychiatric emergency departments or urgent care facility, a psychiatric hospitalization, or a suicide death.

What we did:

Since this was an observational rather than an RCT study, the findings have the potential to be influenced by the fact that the type of treatment received was nonrandom with respect to potential confounding factors [9]. An extensive EHR and geospatial database, consisting of 5,865 variables, was used to address this issue by using state-of-the-art techniques to account for nonrandom treatment assignment with respect to these variables. We then derived an initial PTR using a 70% training subset and evaluated the strength of the PTR in the remaining 30% test subset.

Doubly robust techniques were employed to adjust for nonrandom treatment assignment, enhancing both overall model accuracy and fairness [10]. The investigation of heterogeneity of treatment effects (HTEs) involved two main steps: (1) We began by predicting an expected outcome for every patient across the three treatment options by establishing treatment-specific models within patient subgroups. We then imputed expected treatment-specific outcomes for all patients, irrespective of their actual treatment, based on these models [11, 12]. (2) We then used within-patient estimates of comparative treatment effects across interventions to generate within-patient scores that represent patient-specific expected differences in outcomes across treatment options. We then used a series of random forest models [13] to predict these expected difference scores. Predictor importance was assessed utilizing the kernel Shapley additive explanations (SHAP) technique [14].

What we found:

8.6% (standard error [S.E.] = 0.2%) of patients experienced a negative outcome. Significant statistical variability was observed in the overall likelihood of a negative outcome associated with baseline predictors (area under the receiver operating characteristic curve [AU-ROC]=0.68, S.E.=0.01). After adjusting for this observed variability, the average treatment-specific likelihood of a negative outcome was found not to differ significantly across the three treatment options: 9.1% (S.E.= 0.3) for ADM-only, 8.5% (S.E.= 0.3) for psychotherapy-only, and 8.8% (S.E.=0.4) for combined ADM-psychotherapy. The primary predictor categories contributing to the overall risk of negative outcomes were physical ailments (proportional SHAP=53.3%), patient-level social determinants of health (SDoH; proportional SHAP=35.6%), and the frequency of visits for mental or substance-related disorders (proportional SHAP=22.9%). Among the top 5% of patients with the highest anticipated negative outcome risk, the prevalence of a negative outcome in the test sample was 32.6%, relative to 7.1% in the rest of the test sample.

The PTR analyses then determined that psychotherapy-only was the best treatment option for 56.0% of patients, among whom risk of a negative outcome was about 20% lower relative to receiving one of the other treatments. Treatment types did not affect negative outcome risk for the remaining patients.

If this PTR was used in practice, overall treatment costs would change only modestly, as there would be a reduction of 16.1% in MDD patients prescribed ADMs and an increase in 2.9% of patients receiving psychotherapy.

The overwhelmingly important predictors of optimization with psychotherapy-only were geocode-level SDoH variables (proportional SHAP=93.5%). That is, patient neighborhood characteristics rather than individual-level patient characteristics were the key predictors of the extent to which psychotherapy-only would be the preferred treatment.

Conclusion:

Our finding that the 5% of patients with the highest predicted risk of a negative outcome based on information known about the patient prior to beginning treatment was 32.6% might lead some clinicians to conclude that these patients are likely treatment-resistant, in which case both clinical guidelines and health-economic evaluations would call for more intensive treatments than those considered here – possibly involving aggressive medication dosing, increased psychotherapy frequency, or more advanced treatment options, such as antipsychotic augmentation, electroconvulsive therapy, ketamine, or transcranial magnetic stimulation [15-18].

Leaving aside this highest-risk segment of the patient population, we found that psychotherapy-only was associated with significantly reduced risk of a negative outcome relative to other treatment options for one segment of the patient population. It is noteworthy that a recent meta-analysis of RCTs found a similar aggregate result for a similar composite negative outcome [19]. Although our sample size was sufficient to detect a significant difference of this sort in our total sample, we failed to do so. This indicates either that differences in average treatment effects across treatment types are less pronounced in the VHA compared to the broader population examined in the meta-analysis, or that there were remaining biases in our nonexperimental analysis that led to an underestimation of the advantages of psychotherapy-only in the total population. A pragmatic trial would be necessary to resolve these two competing potential scenarios.

At the same time, we observed significant HTE in our sample, with 56% of patients deemed to benefit more from psychotherapy-only than the other treatments. The anticipated advantage of optimal versus suboptimal treatment assignment linked in this segment of the sample was approximately 20% proportional decrease in risk of the negative outcome. This effect size would normally be clinically significant but small.

Whether such a small effect size warrants a pragmatic trial is unclear. An alternative, given absence of evidence that ADM-only is superior to the other options in any segment of the patient population, would be to advocate for psychotherapy among all VHA PC-MHI clinic patients considered as high risk of the negative outcome considered here based on our aggregate risk model. As immediate access to psychotherapy is not consistently available, this type of general rule might be useful in PC-MHI settings when prioritizing patients for psychotherapy.

References:

1. Cohen ZD, DeRubeis RJ. Treatment selection in depression. Annu Rev Clin Psychol 2018; 14: 209-236.
2. Driessen E, Dekker JJM, Peen J, Van HL, Maina G, Rosso G et al. The efficacy of adding short-term psychodynamic psychotherapy to antidepressants in the treatment of depression: A systematic review and meta-analysis of individual participant data. Clin Psychol Rev 2020; 80: 101886.
3. Driessen E, Fokkema M, Dekker JJM, Peen J, Van HL, Maina G et al. Which patients benefit from adding short-term psychodynamic psychotherapy to antidepressants in the treatment of depression? A systematic review and meta-analysis of individual participant data. Psychol Med 2023; 53: 6090-6101.
4. Kessler RC, van Loo HM, Wardenaar KJ, Bossarte RM, Brenner LA, Ebert DD et al. Using patient self-reports to study heterogeneity of treatment effects in major depressive disorder. Epidemiol Psychiatr Sci 2017; 26: 22-36.
5. Kraus C, Kadriu B, Lanzenberger R, Zarate CA, Jr., Kasper S. Prognosis and improved outcomes in major depression: A review. Transl Psychiatry 2019; 9: 127.
6. Maj M, Stein DJ, Parker G, Zimmerman M, Fava GA, De Hert M et al. The clinical characterization of the adult patient with depression aimed at personalization of management. World Psychiatry 2020; 19: 269-293.
7. Perna G, Alciati A, Daccò S, Grassi M, Caldirola D. Personalized psychiatry and depression: The role of sociodemographic and clinical variables. Psychiatry Investig 2020; 17: 193-206.
8. Kessler RC, Luedtke A. Pragmatic precision psychiatry—A new direction for optimizing treatment selection. JAMA Psychiatry 2021; 78: 1384-1390.
9. Chen RJ, Wang JJ, Williamson DFK, Chen TY, Lipkova J, Lu MY et al. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nat Biomed Eng 2023; 7: 719-742.
10. Funk MJ, Westreich D, Wiesen C, Stürmer T, Brookhart MA, Davidian M. Doubly robust estimation of causal effects. Am J Epidemiol 2011; 173: 761-767.
11. Desai RJ, Franklin JM. Alternative approaches for confounding adjustment in observational studies using weighting based on the propensity score: A primer for practitioners. BMJ 2019; 367: l5657.
12. Breiman L. Random forests. Mach Learn 2001; 45: 5-32.
13. van der Laan MJ, Gruber S. Working Paper 290: Targeted minimum loss based estimation of an intervention specific mean outcome. 2011. https://biostats.bepress.com/ucbbiostat/paper290/. Accessed March 5, 2024
14. Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. 31st Conference on Neural Information Processing Systems (NIPS 2017): Long Beach, CA, December 4-9, 2017.
15. Malhi GS, Bell E, Boyce P, Bassett D, Berk M, Bryant R et al. The 2020 Royal Australian and New Zealand College of psychiatrists clinical practice guidelines for mood disorders: Bipolar disorder summary. Bipolar Disord 2020; 22: 805-821.
16. McIntyre RS, Rosenblat JD, Nemeroff CB, Sanacora G, Murrough JW, Berk M et al. Synthesizing the evidence for ketamine and esketamine in treatment-resistant depression: An international expert opinion on the available evidence and implementation. Am J Psychiatry 2021; 178: 383-399.
17. McIntyre RS, Suppes T, Tandon R, Ostacher M. Florida Best Practice Psychotherapeutic Medication Guidelines for adults with major depressive disorder. J Clin Psychiatry 2017; 78: 703-713.
18. Ross EL, Zivin K, Maixner DF. Cost-effectiveness of electroconvulsive therapy vs pharmacotherapy/psychotherapy for treatment-resistant depression in the United States. JAMA Psychiatry 2018; 75: 713-722.
19. Zainal NH. Is combined antidepressant medication (ADM) and psychotherapy better than either monotherapy at preventing suicide attempts and other psychiatric serious adverse events for depressed patients? A rare events meta-analysis. Psychol Med 2024; 54: 457-472.