Thursday, 6 February 2014

My Bloody Valentine: CBT for unmedicated psychosis

When I critiqued Morrison et als exploratory CBT trial with people who stop taking anti-psychotic medication, I promised to write a post on the final study
Well it appeared in the Lancet today and a free copy is here. I am not going to describe the study in detail as it is excellently covered in the Mental Elf blog today. Contrary to the fanfare of glowing comments by highly respected schizophrenia/psychosis researchers, I think the paper has so many issues that I may need to write a second post. But I'm keeping it simple here to concentrate on the primary outcome data - symptom change scores on the PANSS.

'Soon' by My Bloody Valentine (Andy Weatherall mix)

The study examines schizophrenia patients who have decided not to take anti-psychotic medications; 37 were randomly assigned to 9 months CBT and 37 assigned to - what the authors call TAU (but is obviously quite an important manner that will become clear below)

What do the primary outcome PANSS scores (total, positive and negative symptoms) reveal?

Table 1. PANSS scores during the intervention (up to 9 months) and follow ups to 18 months

The key questions are:
Do the CBT and TAU groups differ in PANSS scores at the end of the intervention (9 months) and at the end of the study (18 months)? One simple way to address both questions is to calculate the Effect Sizes at 9 months and at 18 months.

9 months
PANSS total       =  -0.37   (95 CI -0.96 to 0.22)
PANSS positive  =  -0.18  (95 CI -0.77 to 0.40)
PANSS negative =  -0.45  (95 CI -1.04 to 0.14)
Examination of effect sizes at the end of the intervention (9 months) reveals that CBT and TAU groups do not differ significantly on any of the three primary outcome measures at the end of intervention (i.e. all CIs cross zero)

18 months
PANSS positive is nonsignificant, while PANSS total and PANSS  negative effect sizes are moderately sized, the lower end CIs are very close to zero (at -0.05 and -0.08) suggesting marginal significance
18 months
PANSS total = -0.75 (95 CI -1.44 to -0.05)
PANSS positive = -0.61 (95 CI -1.27 to 0.05)
PANSS negative = -0.45 (95 CI -1.47 to -0.08)

A closer inspection of the means shows that the significant differences at 18 months almost certainly reflects an increase in symptom scores for the TAU group rather than a decrease for the CBT group (compare CBT at 9 and 18 months and TAU at 9 and 18 months)

My final and crucial point concerns within group symptom reduction
Table 2 shows the baseline PANSS scores on primary outcome measures and its informative to compare change from baseline within each group (CBT and control)
Table 2. PANSS scores at baseline

If we compare baseline and the end of the intervention 9 months:

PANSS total
CBT group show a reduction from 70.24 to 57.95 =12.29 
TAU group show a reduction from 73.27 to 63.26 =10.01

PANSS positive
CBT group show a reduction from 20.30 to 16.0 =4.30
TAU group show a reduction from 21.65 to 17.0 = 4.65

PANSS negative
CBT group show a reduction from 13.54 to 12.50 = 1.04
TAU group show a reduction from 15.49 to 14.26 = 1.23

So, after 9 months of intensive CBT intervention, controls - who don't even receive a placebo - show a greater reduction in positive and negative symptoms !

Moreover, the 'natural' reduction shown at 9 months by TAU is as large as the reduction shown by the CBT group at the very end of the trial (18 months: PANSS total =13.77; PANSS pos 5.67 and PANSS neg 1.01) - no significant difference exists between TAU reduction at 9 months and CBT reduction at 9 or 18 months

What then have Morrison et al shown?
I would argue that their data show, for the first time, how patients who choose to be unmedicated display fluctuations in symptomatology (as we might expect given they are unmedicated) ...but crucially, these fluctuations are as large as the changes seen in the CBT group. Hence, it is reasonable to ask...have Morrison et al simply documented 'normal fluctuation' in the symptomatology of unmedicated patients ...and nothing to do with CBT


  1. "No significant difference exists between TAU reduction at 9 months and CBT reduction at 9 or 18 months". Raw numbers are slightly in favour of CBT group, what makes you conclude that this is not significant? A very similar small difference in favour of TAU at 9 months leads you to conclude "So, after 9 months of intensive CBT intervention, controls - who don't even receive a placebo - show a greater reduction in positive and negative symptoms"

    1. In terms of 'change', the scores favour TAU for both positive and negative symptoms but CBT for overall PANSS score.

      I am clear - CBT & TAU show no significant differences at end of the intervention - but the direction goes against CBT even after 9 months of intensive treatment (compared to absolutely nothing, not even a placebo in the TAU group)

  2. In this study there is a conflict between the kind of analysis typically carried out in relatively small treatment trials (ie comparison of means at key outcome points such as end of trial and end of follow-up) and the more sophisticated analysis employed by the investigators (‘Primary analysis was by intention to treat. Changes in all primary and secondary outcomes were analysed with STATA’s xtreg command to fit random effects regression models (essentially, repeated measures ANCOVAs) with summed scores as dependent variables, allowing for attrition and the variable follow-up times introduced by the trial design.’).

    Is the latter approach appropriate for a smallish trial of this type, especially given the high attrition rate? Is there scope for generating false positive or misleading results? Or is it actually a better choice, for example when you have a lot of data points?

    I don’t know the answer to these questions – my statistical expertise is limited – but I wondered if anyone out there can comment.