In a Manner of Speaking

In a Manner of speaking
I just want to say
That I could never forget the way
You told me everything
By saying nothing

In a Manner of Speaking (by Tuxedo Moon)

 

I was recently asked to write an article on some of the problems associated with psychological research (e.g. publication bias, authors 'spinning' their results and so on). We often hear complaints from scientists that journalists put a false spin on their findings; however, I am interested in how authors present or spin their own findings.

Sometimes this spinning in a paper seems to starkly contrast with results of the paper. I can only conclude that some reviewers or editors suffer from what I would call the 'Daylight Robbery Syndrome' - from the adage that some burlgars are unlikey to be caught because they rob you so blatantly.  Here are my current top five quotes from papers on CBT for psychosis published in high impact journals ...For me, the quotes are the academic equivalent of someone kicking down your front door in broad daylight and then carrying off your 42 inch plasma TV screen under their arm past all of your friends and neighbours...

Therapeutic work is rife with vague statements, bias and misrepresentations; however, I am sure comparable examples exist in other areas and that readers may readily be able quote examples from their own research areas.

 

In a Manner of Speaking (covered by Nouvelle Vague)

1. " Interestingly, psychosocial treatments—such as cognitive behavioural therapy (CBT) and, more recently, arts therapies (music therapy, art therapy, and body movement or dance therapy)—have shown more promise than drug treatments in reducing negative symptoms and their impact, and the National Institute for Health and Clinical Excellence (NICE) has recently recommended these treatments."
 

The editorial goes on to discuss the largest, best and most recent study of art therapy for addressing the symptoms of people with schizophrenia (the so-called Matisse trial by Crawford et al 2012)

"The findings of the Matisse trial unfortunately suggest that art therapy, as currently practised in the UK, is unlikely to be of clinical benefit for people with negative or other symptoms of schizophrenia—a conclusion that the profession of art therapy will no doubt find unsettling. However, arts therapies, because they rely on creative expression rather than verbal communication, and some cognitive behavioural approaches, still have the greatest potential for success in the treatment of negative symptoms."

Kendall, T. (2012). Treating negative symptoms of schizophrenia. BMJ-British Medical Journal, 344(7847), 8.

In additon to the elephantine self-contradiction of the latter paragraph - two other things are wrong here. First, CBT does not significantly reduce negative symptoms - Wykes et al (2008) meta-analysis shows that studies with "acceptable" levels of quality find no significant impact of CBT on negative symptoms. Second arts therapy does not appear to reduce psychotic symptoms nor does it outperform medication in reducing negative symptoms. The fact is that no study has compared medication versus any of these therapies ...and so, no data even exist on the issue. 

 

Perhaps Editorials - unlike this one in the BMJ - really ought to be peer-reviewed!

 

2. Although we failed to show a statistically significant effect of the intervention we cannot rule out a beneficial effect of the cognitive therapy on transition rate (although it could be argued that the sample size required to show such an effect, and the small effect sizes reported here, would make such an endeavour unfeasible in practical terms and unwarranted in clinical terms).

Morrison AP, Stewart SL, French P, Bentall RP, Birchwood M, Byrne R, Davies LM, Fowler D, Gumley AI, Jones PB, Lewis SW, Murray GK, Patterson P, Dunn G. Early detection and intervention evaluation for people at risk of psychosis: multisite randomised controlled trial. British Medical Journal, 344

The failure to find a significant effect does not eliminate possibility of an effect! I have to ask, what could then eliminate the possibility of an effect...especially with a very small effect size and a huge sample required? Extremely torturous admission of CBT failure...or is it?

 

3. Patients receiving either CBT or supportive counselling in combination with usual treatment demonstrated better symptomatic recovery but no significant reduction in relapse compared with those receiving usual treatment alone... We suggest that the optimum psychosocial management of early schizophrenia would include a combination of CBT and family intervention.

Tarrier, N., Lewis, S., Haddock, G., Bentall, R., Drake, R., Kinderman, P., ... & Dunn, G. (2004). Cognitive-behavioural therapy in first-episode and early schizophrenia 18-month follow-up of a randomised controlled trial. The British Journal of Psychiatry, 184(3), 231-239.

How to ignore one whole strand of the main findings and also to make a nonsignificant result sound as if it were significant! CBT was no better than Supportive Counselling and yet the authors advocate CBT alongside family intervention - and the latter was not examined at all in this study...a strange conclusion-cocktail! 

4. We did not find significant between-group differences on symptom reduction, indicating no significant benefit of CBT over PE.” [psycho-education]...The use of an active rather than passive control intervention created a more stringent comparison for CBT, which may have further reduced power to detect the hypothesized changes”

Cather, C., Penn, D., Otto, M. W., Yovel, I., Mueser, K. T., & Goff, D. C. (2005). A pilot study of functional cognitive behavioral therapy (fCBT) for schizophrenia. Schizophrenia Research, 74(2), 201-209.

The finding of no significant benefit of CBT is blamed on a pesky control group - those cheeky controls reducing power! If only we hadn't used controls, our results, we would have had a significant result .. hey diddle diddle...
 

 

5. In the linear regression, faster resolution of symptoms in the groups allocated to either psychological treatment condition was seen, compared with routine care alone, but not at statistically significant levels...In summary, for auditory hallucinations, CBT is an improvement on routine care (but the effect is not statistically significant at the α =0.05 level)

Lewis, S., Tarrier, N., Haddock, G., Bentall, R., Kinderman, P., Kingdon, D., ... & Dunn, G. (2002). Randomised controlled trial of cognitive—behavioural therapy in early schizophrenia: acute-phase outcomes. The British Journal of Psychiatry, 181(43), s91-s97.

It doesn't mater how many times one says 'faster', 'better', improvement' - 'No'significant effect' means no significant effect!

Blind commitment to a theory is not an intellectual virtue: it is an intellectual crime.

Imre Lakatos (1970)