Thursday, 28 February 2013

Scientists are Artists: From Brian Eno to the Amygdala


Dipping swords in metaphors yeah but what does he know?
We're always one step behind him he's Brian Eno
Brian Eno!
He promised pretty worlds and all the silence
I could dream of

Brian Peter George St John Le Baptiste De La Salle Eno
Brian Eno by MGMT   

 
Scientists are geeks and artists and musicians are cool ....aren't they?

Science and scientists are viewed as somewhat controlled, controlling, rule-bound, formulaic (the Scientific Method) and emotionally bereft Mr Spocks. By contrast, artists are seen as inspired, more random, often troubled and free souls who free us and our imaginations

These stereotypes are a view of the seemingly quiescent surface of science. Scrape beneath the tranquil surface though ...and you often find a festering, turbulent and creative underbelly.

It is important to acknowledge and embrace the art of science,  embrace the random....but being scientists and perhaps self-defeating...we might like some rules to help us control this random (ha!) creative process!

In this context, let me introduce Brian Peter George St John Le Baptiste De La Salle Eno or as he is more commonly known - Brian Eno. Eno is, of course, best know for producing many famous bands (U2, Bowie, Talking Heads) and was a founder member of Roxy Music. His influence on electronica has featured in my blog previously.

Here I mention his Oblique Strategies (Over One Hundred Worthwhile Dilemmas) - a deck of simple printed cards created by Brian Eno and artist Peter Schmidt and first published in 1975. Each card contains an aphorism intended to help artists break creative blocks.
 

 "By This River" (Eno, Roedelius, Moebius) from Before & After Science
 




The idea of Oblique Strategies is to select cards at random - here is a program that does this for you



 
The cards contain a phrase or cryptic remark that may be used to overcome a creative impasse or dilemma. Some are specific to music composition and Eno has used them throughout his own recordings and his production of others - the cards were a key part of the production of David Bowie's seminal Berlin trilogy of Low, Heroes and Lodger.


"We used oblique strategies a lot. Sense of Doubt was done almost entirely using the cards, and we did talk about work methods. But no, I don't think we [Eno & Bowie] have that much in common. But that's fine, so long as there's give and take"
Working on Sense Of Doubt, Bowie and Eno each pulled out an Oblique Strategy card and kept it a secret from the other. As Eno described it:

It was like a game. We took turns working on it; he'd do one overdub and I'd do the next. The idea was that each was to observe his Oblique Strategy as closely as he could. And as it turned out they were entirely opposed to one another. Effectively mine said, 'Try to make everything as similar as possible.' … and his said, 'Emphasize differences.



 
 "Sense of Doubt" by David Bowie - incipient alien hand syndrome
 
Many of the cards are however more general and useful in many situations - I have often found them to act as catalysts for creative ideas both in a previous more artistic career and now as a scientist. 

Science and art are not mutually exclusive and most, if not all, of us engage with both ...the struggle between structure and serendipity and not a reduction of one to the other
As Eno says
Of course, the chances of you getting a great piece of music are quite remote....But the chances of you getting a seed for something are quite strong.
 
Oblique Strategies will not give you a high H-Index or guarantee a successful scientific career, but may provide that lacking creative spark at the many overly-analytical stages of the scientific process



 
 

In this context, it is worth noting that the Oblique Strategy cards are a simple stimulus open to massive interpretation - not unlike inkblot cards from the Rorschach test. Research shows that the number of unique responses to random figures (like the one above) is linked to having larger amygdalae. Asari et al (2010) asked healthy participants to report what the figure looked like the number of responses correlated positively with amygdala size.
 

The authors suggest of course that amygdalar enlargement in the normal population might be related to creative mental activity. Indeed, unique Rorschach responses are observed at higher frequency in the artistic population (e.g. Ramachandra, 1994; Rorschach, 1921) as well as more frequently in those with psychosis (e.g. Exner, 2003)....So, perhaps Oblique Strategies will enlarge your amygdalae
 
 
 
"Through Hollow Lands" For Harold Budd (arr. Fred Frith, Eno)
from Before & After Science
 
To finish with a random, creative amygdala-Brain Eno connection - here is 'Through Hollow Lands' from Eno's Before and After Science album. The psychology link? This piece was co-arranged by and features Fred Frith - one of the founding members of the illustrious and wonderfully monickered Henry Cow - an art group that came out of Cambridge University in the late 1960s/early 1970s. Fred Frith is, of course, the brother of the psychologist Chris Frith... who has written extensively on the Social Brain? (mentioning the amygdala a lot) and here is Henry Cow with their track Amygdala
 
I wonder if scientists have large amygdalae?

'Amygdala' by Henry Cow
 


Intellect catching up with intuition









 

 


Saturday, 9 February 2013

Satellites of Love: uninterpretable results




In llama land there's a one-man band
And he'll toot his flute for you
Come on fly with me, let's take off in the blue
Frank Sinatra (Come Fly with Me)
 

Cognitive Behavioral Therapy for negative symptoms in psychotic disorders: a pilot study

Staring, Hurne & van der Gaag (2013 in press)

 
Another month and another CBT for psychosis study that is uninterpretable. In previous months, I have commented here and on Twitter about CBT for psychosis studies often failing to use a control group and/or blinding of outcome assessments. As an example, I recently blogged on: Cognitive therapy for people with a schizophrenia spectrum diagnosis not taking antipsychotic medication: an exploratory trial by Morrison et al

Here we have another, in press at Journal of Behavior Therapy and Experimental Psychiatry. While Morrison et al referred to theirs as 'exploratory', Staring et al provide us with the ubiquitous .... 'pilot'


Satellite of Love by Lou Reed
 
 
What did Staring et al do?
In an open trial 21 adult outpatients with a schizophrenia spectrum disorder with negative symptoms received an average of 17,5 sessions of CBT-n (CBT for negative symptoms). At baseline and end-of-treatment, they assessed negative symptoms. They claim that "Intention-to-treat analyses showed a within group effect size of 1.26 on negative symptoms"

An effect size of 1.26 is, of course, massive - if only it meant something. As noted previously nonblind (open) studies are prone to large bias and without a control group, who knows why they changed. To be fair to the authors, they do have a limitations section - and it is quite large! Here is part of it:
"This was an uncontrolled study. Therefore the efficacy findings are biased. Patients may have improved over the course of the study by self-initiated change or because other treatments were helpful. The lack of a control group means that this effect was not controlled for."

"Second, also as a consequence of the uncontrolled design, measurements were not blind. We mostly used self-report measurements, and patients were fully aware that they had received an active treatment for their negative symptoms. This may have caused efficacy to be overestimated"

 
One thing we may be sure of is that the effect is overestimated...why? Because studies meeting acceptable quality i.e. RCTs that are blind and with controls suggest an effect size of at most .20 and nonsignificant (Wykes et al 2008) - Seems an overestimation is likely - by about 6 fold!

Anyway, my point here is that we need to have some monitoring of  studies that are essentially uninterpretable. To borrow a phrase from Lou Reed, I see them as "Satellites of Love" - autopilot research - probably done out of love - but ultimately clutter-up the science atmosphere.

The authors finish with:
"Randomized controlled trials with sufficient statistical power will need to be performed in order to confirm or refute our results"

This is somewhat odd because, as noted, many RCTs have already shown CBT to be ineffective for negative symptoms - as such... the study was refuted at conception

...All such satellites should engage auto-pilot and return to 'Llama land'

p.s. ask youself - would the study have been published if the results were negative?




Tuesday, 29 January 2013

Its Just a Story: Transition to Psychosis & CBT

 
 
 

I've been living through changes...And I could swing for you
I can see the veins in my hands...Are showing through
But if you disguise what...These things are doing to me
If you criticize them...I'll know that you can see...
Until you realise
It's just a story
The Teardrop Explodes (Treason)

Stafford MR, Jackson H, Mayo-Wilson E, Morrison AP, & Kendall T. (2013). Early interventions to prevent psychosis: systematic review and meta-analysis. BMJ, 346, f185


Oh the irony ....just as many (often in the UK) were clamouring for the DSM-5 to dispense with so-called Attenuated Psychosis Syndrome (which they did reject), the Brits sneaked it in through the back door as (Ultra) High Risk for Psychosis in the new NICE guidelines for Psychosis and Schizophrenia in Children and Young People. In their 485 page guide, NICE advise on 'treatment' for those aged between under 18 (estimates suggest a 0.4% prevalence of psychotic disorders in children aged between 5 and 18 years)

The cliche states that 'An ounce of prevention is worth a pound of cure' and a recent meta-analysis (Open Access in BMJ) examines whether various interventions might prevent or delay transition to psychotic disorders for people at 'High Risk'. The paper was written by some members of the NICE committee who prepared the guideline for children and young people; and underpins some of their decision making process...so, is worthy of closer inspection

I want to focus on their recommendation - CBT - as their conclusion implies CBT is the best course of 'action' for preventing transition to psychosis

Stafford et al declare that "Among people at “ultra high risk” of psychosis, about 22% to 40% transition within 12 months [and that] Interventions that delay or prevent transition to psychosis from this prodromal syndrome could be clinically and economically important." Unquestionably, if up to 40% of those who would go on to develop psychosis (schizophrenia) could receive an intervention to delay or even prevent such a transition, this might be viewed as a worthy ambition.

What is (Ultra) High Risk and what did Stafford et al do?

Stafford et al meta-analysed 5 studies examining people "judged to be at risk of developing psychosis on the basis on a clinical assessment identifying prodromal features."

and they conclude that Cognitive Behavioural Therapy has the effect of "reducing transition to  psychosis at 12 months (risk ratio 0.54 (95% confidence interval 0.34 to 0.86); risk difference −0.07 (−0.14 to −0.01)."

Teardrop Explodes (Treason 1980) - Julian Cope as popstar and unfortunate dancing


A few of my observations:

1) The authors report that the Risk Difference is -0.07. This means that CBT produces a 7% reduced risk for transition to psychosis compared to control. To translate this into another common metric, the Number Needed to Treat (NNT), an average of 14 people need to be treated with CBT to prevent one person developing psychosis - this NNT is somewhat larger than typically claimed by individual studies!


Figure 1. Risk Ratios for Number of transitions to psychosis in CBT vs supportive counselling

2) As can be seen in Fig 1, the small significant effect reflects the pooling of 5 studies; however, none of the five studies were themselves significant! The wonderful world of meta analysis (I have referred to this phenomenon previously in my blogs on using LSD to treat alcholism and how to deal with negative findings

Figure 1 also shows that the transition rate is extremely small in these studies - nowhere near the heights of 40% but closer to 10% on average. In other words, 90% of the identified High Risk individuals do not develop psychosis at all! In this context, we need to seriously consider the risks of labelling 9/10 people as High Risk when they are not! The stigma of being labelled High Risk was one major complaint about adding Attenuated Psychosis Syndrome to the DSM-5. As McGorry et al (2003) rightly state "most cases of first episode psychosis still come from the low-risk and undetected groups" (p.784) - not from the ironically entitled 'Ultra High Risk'!

 
3) Although the NNT and Risk Difference figures are not overly impressive, they do appear to record a significant effect. Nonetheless, the difference between CBT and control in conversion rates disappears beyond 12 months i.e. the point at which this meta-analysis terminates. As the authors state:
"At 18 months, there was low quality evidence that CBT is associated with fewer transitions (0.63 (0.40 to 0.99)), and the effect did not remain significant in sensitivity analysis (0.55 (0.25 to 1.19))."
 
If a small effect of CBT disappears beyond 12 month comparisons - how useful is it? And why stop at 12 months?

4) Analysis of the effects of CBT on symptoms of psychosis is also worth considering in transition studies and the authors did examine this.
"Combined effects for positive symptoms of psychosis, depression, and quality of life were not significant at any time point."
To reiterate, CBT did not impact symptoms at all!

5) Finally, the authors claim that all of the 5 included studies were high quality (having excluded others for example, for lack of blinding for raters at outcome). Their Figure 2 (see below) indicates that the 5 CBT studies (Addington 2011; Morrison 2004; Morrison 2011; Phillips 2009 and van der Gaag 2011), were all blind at outcome assessment (see blinding of outcome assessment column)

However, the Morrison et al (2004) study was not blinded at outcome - as they say in that 2004 paper
"Assessors were intended to be masked to the condition to which the patient was allocated; however, this proved difficult in practice because the participants often divulged information about their therapist, or used language that suggested they were receiving cognitive therapy...It proved impossible fully to maintain masking to treatment allocation for assessment of the primary outcome" Morrison et al (2004)
It seems somewhat odd to make such a mistake- as Morrison one of the authors of this current meta-analysis! Removing Morrison et al (2004) from the analysis reduces the Risk Ratio (RR=.60 [.37 to 0.99) which now becomes marginally significant (p=.044) and suggests that the NNT with CBT would be 22 to stop one psychosis transition.

To conclude, it seems that Ultra High Risk for Psychosis, which has been sneaked into NICE guidelines for children and the young, has a limited application - probably less than 1 in 10 individuals actually convert. Moreover, the claim that CBT provides the best form of 'treatment' also seems unsubstantiated. As usual, I am sure we will hear claims for further, better, larger trials - to justify the conclusion already made!


 

Saturday, 22 December 2012

Santa Dog

 
 
Santa Dog's a Jesus Fetus
Santa Dog's a Jesus Fetus
Santa Dog's a Jesus Fetus
Has no presents,
Has no presence
In the future...
...In the future
 
 
I started blogging mid March 2012 and until now, made 18 posts. As we approach Christmas and the end of the year, here are my Top 5 in terms of views
 
Santa Dog by the Residents (my favourite xmas song)
 
 

Number 5: Whats Your Poison - LSD vs Alcohol


 
 
 

At number 5 was my very first post on the meta-analysis by Krebs & Johansen suggesting using LSD as a treatment for alcoholism. The study attrated a lot of media attention and the following striking comment from Prof David Nutt that: "Overall there is a big effect, show me another treatment with results as good; we've missed a trick here…This is probably as good as anything we've got [for treating alcoholism]." Needless to say, I was not as convinced and my blog drew some ire from one of the authors, who e-mailed me asking me to refrain from any more public comments and then posted their own response on the Nature wesbite (strangely through an intermediary!)

 

Number 4: Strange Fruit: Is Racism a Mental Illness? 

 
 
 
At Number 4 was my blog on whether racism could be a form of mental illness. Originally spurred by a newspaper article about a man who tried to use his schizophrenia diagnosis to mitigate his violent racist behaviour in court...and failed; and a subsequent Twitter interaction with @JonesNev, who argued that schizophrenia "often does lead otherwise liberal, kind, non-racist people to become glaringly, bluntly racist, sexist, phobic, nymphomanic, hostile". Anyway, it led me to investigate past claims from psychiatrists that racism is an abnormal belief that could qualify as a delusion.
 
 

Number 3: CBT: She's Lost Control(s) Again

 
 
 
Number 3 was one of my several blogs about CBT. In this case, the first study to examine using CBT to alleviate symptoms in cases of psychosis where the individuals have chosen to be unmedicated. Of course, the study attracted wide media attention and for me, premature attention for their unfinished trial of CBT in unmedicated psychosis (e.g on BBC Radio 4 All in the Mind link in the blog). The study contains major methodological flaws, nobody should view it as other than fatally flawed and I await the results of their properly controlled trial for some interpretable data.
 
 

Number 2: CBT: You Spin me Round

 
 
At number 2, another CBT study and with the same main author as the unmedicated trial at number 3. This time it was using CBT to prevent transition to psychosis. The main upshot here is that it failed to show any effect - yet the authors went to extraordinary lengths to spin the results positively. We should not blame journalists when results are unreasonably/falsely represented in the media - however in this case: authors, media and even the journal (British Medical Journal) were at fault for spinning unanimously negative results
 
 

Number 1: Negativland: What to do about negative findings?

 
 
 
By a clear margin, my Negativland blog received the most hits. This post concerned the issue of how we deal with negative findings in science (psychology). Since this post, a great deal of much-needed discussion has occurred around this issue and about how psychology in particular might get its house in order. I have subsequently much expanded the ideas in this blog and hopefully, an article will appear in the Open Access journal BMC_Psychology in the New Year.
 
I have enjoyed my 10-month foray into blogging and much appreciate all of the feedback and interactions that have stemmed from this - thanks for your interest
Merry Xmas and a Happy New Year
 
 
 
 
 
 
 
 
 
 
 
 

Tuesday, 11 December 2012

Significantly nonsignificant




In the morning I'd awake and I couldn't remember
What is love and what is hate? - the calculations error

Flaming Lips (Morning of the Magicians)


Cognitive behaviour therapy for psychosis can be adapted for minority ethnic groups: A randomised controlled trial
Shanaya Rathod, Peter Phiri, Scott Harris, Charlotte Underwood, Mahesh Thagadur, Uma Padmanabi & David Kingdon
 

Recently I have commented on what I see as methodologically poor, biased and spun studies of CBT for psychosis - every time I want to blog on someting else, out comes another questionable study - this time 'in press' at Schizophrenia Research, which I think requires comment - in particular, the gap between the reality and presentation of findings

This is an RCT looking at the use of CBT in UK minority groups with a diagnosis of schizophrenia
A total, n = 33 participants, who were randomly allocated to CBT for psychosis (CBTp n = 16) and treatment as usual (TAU n = 17). Although a relatively small study, the authors did a power analysis based on previous pilot studies and suggest that a minimum of 12 per arm of the trial would be sufficiently powered - obviously they exceed this expectation.

Did the CBT group show any benefit (i.e. reduction of symptoms) over the TAU group?

Well, the authors say yes in the abstract:
Results: Post-treatment, the intervention group showed statistically significant reductions in symptomatology on overall CPRS scores, CaCBTp Mean (SD) = 16.23 (10.77), TAU = 18.60 (14.84); p = 0.047,with a difference in change of 11.31 (95% CI:0. 14 to 22.49); Schizophrenia change: CaCBTp = 3.46 (3.37); TAU = 4.78 (5.33) diff 4.62 (95% CI: 0.68 to 9.17); p = 0.047 and positive symptoms (delusions; p = 0.035, and hallucinations; p = 0.056). At 6 months follow-up, MADRAS change = 5.6 (95% CI: 2.92 to 7.60); p < 0.001. Adjustment was made for age, gender and antipsychotic medication.  
Conclusion: Participants in the CaCBTp group achieved statistically significant results post-treatment compared to those in the TAU group with some gains maintained at follow-up. High levels of satisfaction with the CaCBTp were reported

Flaming Lips - In the Morning of the Magicians

The key aspect here though is the bolded statement, added casually at the end of the results about 'adjustment'.  Table 1 from the paper presented below indicates (as the authors rightly admit) that the TAU group were older, had longer duration of illness and greater medication and they duly adjusted analyses for these variables


 
However, the abstract results all refer to unadjusted scores - see the Table 2 below - all are taken from the non-adjusted column. If you glance to the final column (Adjusted reduction from baseline) - not a single comparison is significant!

Table 2 Results

Maybe someone can explain to me, what is happening here? Is it blatant author spinning to get further funding (as they mention this in the discussion). Is it collusion from reviewers, who let this through? Failure to spot the obvious by reviewers?
Perhaps I am overly critical - letters on a postcard (in the comments section please)

Monday, 3 December 2012

Who Watches the Watchmen? Bias in Studying Bias

 
 
The coins are often very old by the time they reach the jeweller
With his hands and ashes he will try the best he can
He knows that he can only shine them
Cannot repair the scratches
The Jeweller (by Pearls Before Swine)
 
 
Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for schizophrenia.
Niemeyer & Musch & Pietrowsky (2012)
 
Dan Dreiberg: I'm not the one still hiding behind a mask; Rorschach: No. You're hiding in plain sight. Interaction from The Watchmen 

 
Hiding things in plain sight is often the best place to hide them! In my last blog, I referred to Daylight Robbery Syndrome where researchers say things so boldly that readers may be convinced of their validity even when they are not consistent with the data. Here I would like to refer to how researchers may intentionally or unintentionally hide something in front of the reader. In particular, how the application of methods used for detecting bias in meta-analyses may themselves be prone to their own biases.
 
This current paper, published recently in Schizophrenia Research, examines publication bias in studies of "psychotherapeutic interventions for schizophrenia".
 
 Opening Scene from the Watchmen (Unforgettable by Nat King Cole)

As the authors Niemeyer et al rightly state:
"Meta-analyses are prone to publication bias, the problem of selective publication of studies with positive results. It is unclear whether the efficacy of psychotherapeutic interventions for schizophrenia is overestimated due to this problem. This study aims at enhancing the validity of the results of meta-analyses by investigating the degree and impact of publication bias."

This is certainly true for trials of psychological interventions, where the decision to submit a paper for publication is related to the outcome of the trial. For example, Coursol and Wagner (1986) found that when therapeutic studies had positive outcomes (i.e. clients improved) 82% submitted their paper, but with negative outcomes (client did not improve) only 43% submitted their articles (for similar conclusions from a recent meta analysis, see Hopewell et al 2009).
 
Returning to the current paper, Niemeyer  et al used current standard meta-analytic methods to estimate bias, including: Begg and Mazumdar's adjusted rank correlation test, Egger's regression analysis and the trim and fill procedure. They applied these techniques to data sets derived from systematic reviews up to September 2010. I have remarked on these bias methods briefly in my previous post 'Negativland'.

Following their analyses Niemeyer  et al concluded:
"Overall, we found only moderate evidence for the presence of publication bias. With one notable exception, the pattern of efficacy of psychotherapy for schizophrenia was not changed in the data sets in which publication bias was found. Several efficacious therapies exist, and their efficacy does not seem to be the result of publication bias."
 
This apparent lack of bias in this paper might be contrasted with the large bias documented by Cuipjers et al (2010) in studies examining CBT for depression. Cuipjers and colleagues found an effect size of .67 in 175 comparisons comparing CBT to a control condition; however adjustment for publication bias according to Duval & Tweedie’s trim and fill procedure reduced the mean effect size to 0.42 with 51 studies assumed to be missing i.e residing in file drawers because they were negative.

 

The Jeweler (by Pearls Before Swine) 


I intend to concentrate briefly on the 10 data sets from meta-analyses for studies of CBT for schizophrenia (2 data sets from Lynch et al 2010; 1 from Lincoln et al 2008; 1 from Wykes et al 2008; 5 from Zimmerman et al 2005; and 1 from Jones et al 2010: see Table 1)
 
Table 1. Bias analysis of CBT for schizophrenia meta analyses (from Niemeyer & Musch 2012)


 
A few notable features about Table 1 and the analysis of bias

1) Our meta analysis (Lynch, Laws & McKenna 2010) criticised for being overly selective by some (because we analysed high quality studies using an active control group!) produced a data set with the fewest imputed (i.e. missing studies)

2) The Wykes et al (2008) analysis is curious. First the authors state that the effect size was Cohen's d - when in fact Glass' delta was used (a quite different effect size). This could of course be a simple error. The choice of outcome variable, however, is not an error - the authors chose to analyse Low Quality studies from the Wykes paper. Why would a study of bias select only low quality studies and not the high quality or at least both? 

Table 2 shows where these (positive symptom) effect sizes were derived from the Wykes et al paper. What is clear is that the low quality studies are not significantly heterogeneous, while the high quality studies show significant heterogeneity - indeed this is partly borne out by the far broader spread of scores for the 95% confidence intervals in high quality studies (even though they constitute almost half the number of low quality studies)

Table 2 Effect sizes from Wykes et al (2008)
 

By selecting low quality studies, it would seem that the probability of finding bias may be diminished and at the very least, the estimate of bias is unreliable

3) From the Lincoln et al (2008) meta-analysis, the authors selected data for 9 studies comparing CBT vs TAU. Omitted, however, was an additional comparison of 10 studies of CBT vs active control (see Table 3 below)

Table 3. Data from Lincoln et al (2008) Meta-analysis

The notable thing again is that the authors chose to exclude one analysis that has: a) a larger sample b) a non-significant effect and c) far greater 95% Confidence Intervals i.e. variance. Again, these factors could obviously conspire against finding bias and leave us uncertain about bias in this meta-analysis.

4) From the Zimmerman et al meta analysis, the authors included 5 analyses and all bar the last had proportionally large numbers of imputed studies. The one comparison that produced no imputed studies was the comparison with an 'active' control (like Lynch et al - which also produced no imputed studies)

5) Finally Niemeyer & Musch selected one comparison from the Jones et al (2010) Cochrane meta-analysis - a somewhat odd choice to be included - measuring 'relative risk for leaving a study early'.

These decisions are made somewhat odder and less reliable by the fact that Niemeyer & Musch failed to include any data from meta-analyses by the same Cochrane group examining symptoms (Jones et al 2004) or indeed other meta-analyses such as that by Rector and Beck (2002) or the UK NICE Committee (2009)

Anyway, to conclude, the bias analysis of CBT for Psychosis by Niemeyer et al is itself biased by unexplained choices made by the reviewers themselves. Not being psychic, I have no idea why they made these choices or what differences it would make to include different measures and additional meta-analyses. One thing I do know, however, is that any claim that bias does not exist in studies examining CBT for psychosis...is a biased and unreliable! Researchers are familiar with the idea of GIGO (Garbage In Garbage Out), which has often been levied at meta-analysis - perhaps we now need to consider METAGIGO!
 

Tuesday, 27 November 2012

In a Manner of Speaking


In a Manner of speaking
I just want to say
That I could never forget the way
You told me everything
By saying nothing
In a Manner of Speaking (by Tuxedo Moon)
 

I was recently asked to write an article on some of the problems associated with psychological research (e.g. publication bias, authors 'spinning' their results and so on). We often hear complaints from scientists that journalists put a false spin on their findings; however, I am interested in how authors present or spin their own findings.

Sometimes this spinning in a paper seems to starkly contrast with results of the paper. I can only conclude that some reviewers or editors suffer from what I would call the 'Daylight Robbery Syndrome' - from the adage that some burlgars are unlikey to be caught because they rob you so blatantly.  Here are my current top five quotes from papers on CBT for psychosis published in high impact journals ...For me, the quotes are the academic equivalent of someone kicking down your front door in broad daylight and then carrying off your 42 inch plasma TV screen under their arm past all of your friends and neighbours...

Therapeutic work is rife with vague statements, bias and misrepresentations; however, I am sure comparable examples exist in other areas and that readers may readily be able quote examples from their own research areas.


 
In a Manner of Speaking (covered by Nouvelle Vague)

1. " Interestingly, psychosocial treatments—such as cognitive behavioural therapy (CBT) and, more recently, arts therapies (music therapy, art therapy, and body movement or dance therapy)—have shown more promise than drug treatments in reducing negative symptoms and their impact, and the National Institute for Health and Clinical Excellence (NICE) has recently recommended these treatments."
 
The editorial goes on to discuss the largest, best and most recent study of art therapy for addressing the symptoms of people with schizophrenia (the so-called Matisse trial by Crawford et al 2012)
"The findings of the Matisse trial unfortunately suggest that art therapy, as currently practised in the UK, is unlikely to be of clinical benefit for people with negative or other symptoms of schizophrenia—a conclusion that the profession of art therapy will no doubt find unsettling. However, arts therapies, because they rely on creative expression rather than verbal communication, and some cognitive behavioural approaches, still have the greatest potential for success in the treatment of negative symptoms."
Kendall, T. (2012). Treating negative symptoms of schizophrenia. BMJ-British Medical Journal, 344(7847), 8.


In additon to the elephantine self-contradiction of the latter paragraph - two other things are wrong here. First, CBT does not significantly reduce negative symptoms - Wykes et al (2008) meta-analysis shows that studies with "acceptable" levels of quality find no significant impact of CBT on negative symptoms. Second arts therapy does not appear to reduce psychotic symptoms nor does it outperform medication in reducing negative symptoms. The fact is that no study has compared medication versus any of these therapies ...and so, no data even exist on the issue. 
 
Perhaps Editorials - unlike this one in the BMJ - really ought to be peer-reviewed!

 
2. Although we failed to show a statistically significant effect of the intervention we cannot rule out a beneficial effect of the cognitive therapy on transition rate (although it could be argued that the sample size required to show such an effect, and the small effect sizes reported here, would make such an endeavour unfeasible in practical terms and unwarranted in clinical terms).
Morrison AP, Stewart SL, French P, Bentall RP, Birchwood M, Byrne R, Davies LM, Fowler D, Gumley AI, Jones PB, Lewis SW, Murray GK, Patterson P, Dunn G. Early detection and intervention evaluation for people at risk of psychosis: multisite randomised controlled trial. British Medical Journal, 344

The failure to find a significant effect does not eliminate possibility of an effect! I have to ask, what could then eliminate the possibility of an effect...especially with a very small effect size and a huge sample required? Extremely torturous admission of CBT failure...or is it?
 
3. Patients receiving either CBT or supportive counselling in combination with usual treatment demonstrated better symptomatic recovery but no significant reduction in relapse compared with those receiving usual treatment alone... We suggest that the optimum psychosocial management of early schizophrenia would include a combination of CBT and family intervention.

Tarrier, N., Lewis, S., Haddock, G., Bentall, R., Drake, R., Kinderman, P., ... & Dunn, G. (2004). Cognitive-behavioural therapy in first-episode and early schizophrenia 18-month follow-up of a randomised controlled trial. The British Journal of Psychiatry, 184(3), 231-239.

How to ignore one whole strand of the main findings and also to make a nonsignificant result sound as if it were significant! CBT was no better than Supportive Counselling and yet the authors advocate CBT alongside family intervention - and the latter was not examined at all in this study...a strange conclusion-cocktail! 

4. We did not find significant between-group differences on symptom reduction, indicating no significant benefit of CBT over PE.” [psycho-education]...The use of an active rather than passive control intervention created a more stringent comparison for CBT, which may have further reduced power to detect the hypothesized changes”


Cather, C., Penn, D., Otto, M. W., Yovel, I., Mueser, K. T., & Goff, D. C. (2005). A pilot study of functional cognitive behavioral therapy (fCBT) for schizophrenia. Schizophrenia Research, 74(2), 201-209.

The finding of no significant benefit of CBT is blamed on a pesky control group - those cheeky controls reducing power! If only we hadn't used controls, our results, we would have had a significant result .. hey diddle diddle...
 
 

5. In the linear regression, faster resolution of symptoms in the groups allocated to either psychological treatment condition was seen, compared with routine care alone, but not at statistically significant levels...In summary, for auditory hallucinations, CBT is an improvement on routine care (but the effect is not statistically significant at the α =0.05 level)

Lewis, S., Tarrier, N., Haddock, G., Bentall, R., Kinderman, P., Kingdon, D., ... & Dunn, G. (2002). Randomised controlled trial of cognitive—behavioural therapy in early schizophrenia: acute-phase outcomes. The British Journal of Psychiatry, 181(43), s91-s97.

It doesn't mater how many times one says 'faster', 'better', improvement' - 'No'significant effect' means no significant effect!


Blind commitment to a theory is not an intellectual virtue: it is an intellectual crime.
Imre Lakatos (1970)