'Representative' - its too "too too" to put a finger on

Prove it... just the facts... the confidential

This case, this case, this case that I...

I've been workin' on so long...

Chirpchirp...the birds
they're giving you the words
The world is just a feeling you undertook.
Now the rose, it slows

you in such colorless clothes

Fantastic! You lose your sense of human.

Project, Protect

It's warm and it's calm and it's perfect

It's too "too too" to put a finger on

This case is closed

Prove it [Television]

I recently tweeted that a new review of meta-analyses of CBT for various disorders was biased - at least as far as the section on schizophrenia is concerned. The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses (Hofmann, Asnaani, Vonk, Sawyer & Fang), which is published in Cognitive Therapy and Research is undoubtedly an ambitious work born of some fortitude. And to be fair, it deserves more detail - than my original 140 characters - about why I think the review is biased.

Here is the verbatim section from the paper:

Schizophrenia and Other Psychotic Disorders

Meta-analyses examining the efficacy of psychological treatments for schizophrenia revealed a beneficial effect of CBT on positive symptoms (i.e., delusions and/or hallucinations) of schizophrenia (e.g., Gould et al. 2001; Rector and Beck 2001). There was also evidence (e.g., Zimmermann et al.2005) that CBT is a particularly promising adjunct to pharmacotherapy for schizophrenia patients who suffer from an acute episode of psychosis rather than a more chronic condition.

CBT appeared to have little effect on relapse or hospital admission compared to other interventions, such as early intervention services or family intervention (e.g., Bird et al. 2010 ; Alvarez-Jimenez et al. 2011). However, CBT had a beneficial effect on secondary outcomes. For example, a more recent meta-analysis by Wykes et al. (2008) examined controlled trials of CBT for schizophrenia and confirmed findings from previous meta-analyses (e.g., Gould et al. 2001; Rector and Beck 2001), suggesting that CBT had a small to medium effect size as compared to control conditions on both positive and negative symptoms. In addition, this meta-analysis revealed medium effect sizes for improvements in secondary outcomes that were not the direct targets of treatment, including general functioning, mood, and social anxiety.

Why do I think its biased? Largely because it is unrepresentative. In the paper, Hofmann et al state "We identified 269 meta-analytic studies and reviewed of those a representative sample of 106 meta-analyses examining CBT for the following problems:...".
I suppose its conceivable that a review may be 'representative' if it references a minority (<40%) of the meta-analyses? Presumably the 60% majority were somehow unrepresentative?

So, how were the 'representative' meta-analyses selected? The paper is surprisingly remiss on this issue - telling us nothing about how selection occured. Leaving aside problems justifying their choices, it is still possible that Hoffman et al managed to formulate a representative account of the meta analyses examining CBT for schizophrenia? They included reference to meta analyses by: Gould et al. 2001; Rector and Beck 2001; Zimmermann et al (2005); Wykes et al (2008); Bird et al. (2010); and Alvarez-Jimenez et al. (2011) - so. lets take a look at what they did select

Rector & Beck (2001) and Gould et al (2001) are the earliest and hence, the smallest published meta-analyses; and almost completely overlap (with 5 of 7 studies being the same). The early studies in these meta analyses were of poor quality and thus inflated the effect sizes e.g. Rector & Beck refer to:

"...the absence of representative sampling (Drury et al., 1996a, 1996b), the absence of blind ratings (Kuipers et al., 1997; Pinto et al., 1999), the absence of reliable and valid multidimensional symptoms measures (Drury et al., 1996a, 1996b; Tarrier et al., 1993a), the failure to control for potential between-group differences in the use of medications (Kuipers et al., 1997; Tarrier et al., 1993a), and the failure to ensure treatment adherence and competence of CBT therapists (Drury et al., 1996a, 1996b; Kuipers et al., 1997)."

Regarding the next, larger meta-analysis of Zimmerman et al (2005), which contained better studies and therefore revealed a much smaller effect size (two and half times smaller than Rector & Beck), Hoffman et al write "There was also evidence (e.g., Zimmermann et al.2005) that CBT is a particularly promising adjunct to pharmacotherapy for schizophrenia patients who suffer from an acute episode of psychosis rather than a more chronic condition." This was not true in the Zimmerman et al meta-analysis, which states that for acute and chronic patients "...this difference was not statistically significant"(p.7)

Wykes et al (2008) - Hofmann et al say "In addition, this meta-analysis (Wykes et al 2008) revealed medium effect sizes for improvements in secondary outcomes that were not the direct targets of treatment, including general functioning, mood, and social anxiety."

Wykes et al calculated effect sizes for these variables according to whether they rated studies as having high or low methodological quality - Social anxiety was calculated from 2 studies, both of low quality; and for studies with acceptable levels of methodological quality, the Effect sizes for Functioning were .147 and for Mood just .084 - not medium (not even small), not representative and not significant (see Table below) - Any effect of CBT on these outcomes is a direct consequence of running low quality studies.

Hoffman et al also say "Wykes et al. (2008) examined controlled trials of CBT for schizophrenia and confirmed findings from previous meta-analyses (e.g., Gould et al. 2001; Rector and Beck 2001), suggesting that CBT had a small to medium effect size as compared to control conditions on both positive and negative symptoms" Again, as the table shows, the effect size for negative symptoms was only significant for low quality studies and not for high quality studies; and true, the effect size for positive symptoms was significant for high quality studies albeit small and barely significant at .22 (with a low end 95% CI of .016 meaning it could become nonsignificant with 1 or 2 more negative studies being reported)

Turning to Bird et al. (2010) and Alvarez-Jimenez et al. (2011), it is not explicitly stated in the article, but these two meta-analyses assess CBT for first episode patients - although they contrast with Wykes et al (2008) which did not examine first episode. Anyway, Alvarez-Himinez found "no statistically significant advantages in favor of CBT [and] CBT showed no clinical benefits on relapse rates compared with either supportive counseling or TAU." And Bird et al reported that "the present review failed to find any evidence that CBT reduced relapse rates in early psychosis, which suggests that the main benefits of this intervention are likely to be a reduction in symptom" Bird et al found no significant reduction in positive and negative symptoms at the end of the trials, but surprisingly found significant effects occur at up to 2 years after the trials finish - but the sample here is just 3 studies

Finally, returning to representativeness -Hoffman et al failed to mention at least three meta analyses published since Wykes (2008), which have been more explicitly negative about the effects of CBT on psychotic symptoms and relapse, including:

Lynch, Laws McKenna (2010) who said "CBT is no better than non-specific control interventions in the treatment of schizophrenia and does not reduce relapse rates."

Newton-Howes & Woods (2011) who said "Theoretically based CBT therapies, although proving effective, may not out perform more accessible and simpler forms of therapy for patients with schizophrenia in reducing psychopathology. Consideration of supportive therapy should be made for patients with psychotic mental disorder"

Jones et al (2012) most recent Cochrane meta analysis concludes "Trial-based evidence suggests no clear and convincing advantage for cognitive behavioural therapy over other - and sometime much less sophisticated - therapies for people with schizophrenia."

To conclude, I stand by my tweet that the review by Dr Hofmann and colleagues represents a biased view of the findings. Given the structured guidelines on how to write meta-analyses, the attention to publication bias, detailing the search process and providing reasons for excluding studies and so on - the raison d'etre for meta analysis would seem to be undermined by reviews of meta-analyses that do not adhere to the same or similar principles.

Autism - A gender defiant disorder

Everything with a downside may have an upside, bear with me ...I am no expert on autism, but the following paper struck me so much, I wanted to dash something off on it here (while listening to the peerless Suede...for reasons that are obvious...to me!)

Bejerot et al (2012) have just published an intriguing paper entitled 'The extreme male brain revisited: gender coherence in adults with autistic spectrum disorder" in The British Journal of Psychiatry.

In their paper, they claim that "...ASD, rather than being characterised by masculinisation in both genders, may constitute a gender defiant disorder"

Professor Simon Baron-Cohen is probably the most well known advocate of the ‘Extreme Male Brain’ theory to have suggested that an increased androgen exposure in utero may contribute to the development of ASD in both genders (e.g. Baron-Cohen 2002). The extreme male brain theory of autism was, however, first suggested by Hans Asperger (1944) when he wrote:

‘The autistic personality is an extreme variant of male intelligence. Even within the normal variation, we find typical sex differences in intelligence… In the autistic individual, the male pattern is exaggerated to the extreme’ (Uta Frith’s translation).

Despite the ‘extreme male brain’ theory suggesting that autism spectrum disorder (ASD) is an extreme variant of male intelligence, Bejerot et al noted in the clinic that paradoxically individuals with ASD often display androgynous physical features regardless of gender

The Beautiful Ones (by Suede)

What did they do?

In this paper, Bejerot et al examined serum hormone levels, anthropometry, the ratio of 2nd to 4th digit length (2D:4D) and psychiatric symptomatology in 50 adults with high-functioning ASD and age and gender-matched neurotypical controls. They took photographs of face and body, as well as voice recordings, which were assessed with respect to gender coherence, blindly and independently, by eight assessors (see below)

2D:4D ratio - it has been argued (e.g. by John T Manning and many others) that the ratio of two digits in particular, the 2nd (index finger) and 4th (ring finger), is affected by exposure to androgens e.g. testosterone while in the uterus and that this 2D:4D ratio can be considered a crude measure for prenatal androgen exposure, with lower 2D:4D ratios pointing to higher androgen exposure. The 2D:4D ratio has been linked to many health e.g. heart disease, cancer etc as well as psychological factors

A selected assessor group including four men and four women (aged 18–47 years and from divergent socioeconomic backgrounds), individually assessed the digital photographs and voice samples presented on a computer. They were carefully informed to estimate whether the participants had more or less gender-typical features, but not to judge beauty. Femininity in the women and masculinity in the men were assessed, face and body unconnected, applying a 5-point Likert scale: very gender typical; gender-typical; average; weak gender coherence; very weak gender coherence. Coded photographs and voice samples of the control and ASD groups, men and women separately, were shown in random order. First, front and profile photographs of each participant’s face were shown, and the assessors were to estimate the age and gender coherence of the faces. Next, assessors estimated gender coherence from the body photographs, showing the shoulders and downwards. Finally, the assessors evaluated the voices, coded with a random number, on the computer according to the above-described 5-point Likert scale.

Poor Lemmy

What did they find?

Women with ASD had higher total and bioactive testosterone levels, less feminine facial features and a larger head circumference than female controls.

Men in the ASD group were assessed as having less masculine body characteristics and voice quality, and displayed higher (i.e. less masculine) 2D:4D ratios, but similar testosterone levels to controls. Androgynous facial features correlated strongly and positively with autistic traits measured with the Autism-Spectrum Quotient in the total sample.

and look younger too...In males and females with ASD, dehydroepiandrosterone sulfate (DHEAS) did not decrease with age (compared to controls). This suggests an abnormal hypothalamic–pituitary–adrenal axis. And the authors speculate "Whether the clinically observed ‘young look’ in middle-aged people with ASD is associated with a reduced decline in DHEAS or not is worth exploring in future studies."

So Young (by Suede)

Table 1. Anthropometric measures

The authors some bold and intriguing conclusions:

"Rather than being a disorder characterised by masculinisation in both genders, ASD thus seems to be a gender defiant disorder. Our results strongly suggest that gender incoherence in individuals with ASD is to be expected and should be regarded as one reflection of the wide autism phenotype. The prevalence of ASD appears to be rising and so does gender identity disorder. The comorbidity between these two disorders is striking"

References

Bejerot S, Eriksson JM, Bonde S, Carlstrom K, Humble MB, Eriksson E (2012) The extreme male brain revisited: gender coherence in adults with autism spectrum disorder. British Journal of Psychiatry. Doi:10.1192/bjp.bp.111.097899.

Hug drug or thug drug?

 H-A-T-E
Is that how you spell love in your dictionary
K-I-C-K
Pronounced as kind
F-U-C-K
Is that how you spell friend in your dictionary
Black on black
A guidebook for the blind

Dictionary by XTC

Ecstasy is widely known as a drug that produces feelings of emotional warmth, and empathy toward others (even total strangers, as well as a general sense of well being)

Beyond these enactogenic properties, however, a far lesser known consequence of taking E is....increased aggression

Reid et al (2007) describe data collected from 260 ecstasy users. After controlling for key predictors of aggression (that are independent of ecstasy use), a higher prevalence of lifetime ecstasy use was linked to higher levels of aggressive and violent behaviour. Particularly those who exhibit low self-control were more affected by ecstasy use than those who do not in terms of aggression.

Verheyden et al (2002) looked at the sub-acute effects of MDMA. Forty participants were examined on the night of taking the E and then 4 days later. E users rated lower levels of aggression than controls on the night of drug use, but significantly higher levels of aggression mid-week (both for men and women); and in men change in aggression correlated with the amount of MDMA taken on the weekend

Fig 1. Verheyden et al (2002)
Lower aggression when taking E and higher aggression later

In a rat study, Wallinga et al (2009) assessed (behavioural) aggression in rats 5 days before and 23 days after MDMA administration (by introducing an 'unknown' male rat into cage). They found that MDMA treatment increased aggressiveness in only low aggressive rats and not in medium and high aggressive animals. This suggests that "the vulnerability for increased aggression long after a single MDMA treatment is dependent on the individual's trait aggressiveness and not on the degree of MDMA induced serotonergic neurotoxicity."

Wan et al (2009) examined abstinent Ecstasy users, free of drugs for a minimum of 21 days. They found that aggressive behaviour history correctly identified 73% of those who had been regular ecstasy users and 78.3% of those who had not. The abstinent Ecstasy users showed moderate to large effects for aggression and anti-social behaviour (on lifetime history of aggression rating scale) compared to controls who were mostly cocaine and alcohol dependent.

Table 1. Wan et al (2009)

Curran et al (2004) found that ecstasy users show increased aggression 4 days after taking E both on self-ratings and crucially on objective measures of interpretative bias. Their ecstasy and control groups were well matched for age, education, trait aggression, and trait depression. Importantly, the two groups did not differ significantly in their reported use of drugs other than MDMA. On the interpretive bias task, Ecstasy users were faster in completing sentences that could be interpreted aggressively compared with those which had only neutral interpretations. The control group showed the exact opposite pattern of being slower to complete aggressive than neutral sentences - this suggests that ecstasy users are sensitised to aggressive material


Gerra et al (2001) examined participants with a history of MDMA use and nonuser controls were tested on a laboratory measure of aggression (receiving monetary reward for repeated button pressing and provoked by having it taken away by a fictional other). The authors found that the user group was considerably more aggressive than nonusers and that there was a significant correlation found between the extent to which participants had been exposed to MDMA and aggressive response (r=.78). They also found that aggressiveness in the MDMA-using subjects seemed to be associated more with MDMA pharmacological effects than with personality traits. Finally, they noted that the aggressive responses of Ecstasy users were higher than those seen in heroin users in the same experimental scenario.

Finally, the issue of self-harm with ecstasy...

Kim et al (2011) examined the relationship between ecstasy use and suicidal behaviour among adolescents in the United States. Data from the adolescent subsample (ages 12–17, N = 19,301) of the 2000 National Household Survey on Drug Abuse were used in the analyses. They found that the rate of past year suicide attempt among adolescents with lifetime ecstasy use was almost double that of adolescents who had used other drugs only, and nine times that of adolescents with no history of illicit drug use. After controlling for other related factors, the effect of ecstasy use remained significant.

References

Curran, H.V.; Rees, H.; Hoare, T.; Hoshi, R.; Bond, A. Empathy and aggression: Two faces of Ecstasy? A study of interpretative cognitive bias and mood change in Ecstasy users. Psychopharmacology 2004, 173, 425-433

Gerra, G., Zaimovic, A., Ampollini, R., Giusti, F., Delsignore, R., Raggi, M. A., Laviola, G., Macchia, T., & Brambilla, F. Experimentally induced aggressive behavior in subjects with 3,4-methylenedioxymethamphetamine (‘‘ecstasy’’) use history: Psychobiological correlates.
Journal of Substance Abuse, 2001, 13, 471–491.

Kim J, Fan B, Liu X, Kerner N and Wu PEcstasy use and suicidal behavior among adolescents: findings from a national survey. Suicide and Life Threatening Behaviour, 2001 41, 435-444

Reid, L.W.; Elifson, K.W.; Sterk, C.E. Hug drug or thug drug? Ecstasy use and aggressive behavior. Violence Vict. 2007, 22, 104-119

Verheyden SL, Hadfield J, Calin T, Curran HV . Sub-acute effects of MDMA (3,4-methylenedioxymethamphetamine, "ecstasy") on mood: evidence of gender differences. Psychopharmacology (Berl), 2002, 161: 23-31

Wallinga AE, ten Voorde AM, de Boer SF, Koolhaas JM, Buwalda B. MDMA-induced serotonergic neurotoxicity enhances aggressiveness in low- but not high-aggressive rats. Eur J Pharmacol 2009;618(1-3):22-27.

Wan, L.; Baldridge, R.M.; Colby, A.M.; Stanford, M.S. Enhanced intensity dependence and aggression history indicate previous regular Ecstasy use in abstinent polydrug users. Prog. Neuropsychopharmacol. Biol. Psychiatry 2009, 33, 1484-1490.

The Drugs (ratings) Dont Work

The British Medical Journal has just published a paper "Quantifying the RR of harm to self and others from substance misuse: results from a survey of clinical experts across Scotland" by Taylor et al (2012)

Abstract

Objective: To produce an expert consensus hierarchy of harm to self and others from legal and illegal substance use.

Design: Structured questionnaire with nine scored categories of harm for 19 different commonly used substances.

Setting/participants: 292 clinical experts from across Scotland.
Results: There was no stepped categorical distinction in harm between the different legal and illegal substances. Heroin was viewed as the most harmful, and cannabis the least harmful of the substances studied. Alcohol was ranked as the fourth most harmful substance, with alcohol, nicotine and volatile solvents being viewed as more harmful than some class A drugs.

Conclusions: The harm rankings of 19 commonly used substances did not match the A, B, C classification under the Misuse of Drugs Act. The legality of a substance of misuse is not correlated with its perceived harm. These results could inform any legal review of drug misuse and help shape public health policy and practice

Harm to self ratings (ranked from top to bottom)

Sample

The authors state

"One of the strengths of this study is the large number of experts involved. Two hundred and ninety-two addiction multidisciplinary experts across Scotland were involved making it the largest national panel to be involved in this type of study.

How representative is the 'large' sample of 'experts'? Well, 50% of the respondents worked in Glasgow and 55.5% of the sample were Addiction Community Psychiatric Nurses - I am not sure how representative that might make any opinion re the relative ranking of drugs. It would have been useful to see a breakdown by occupation - the data are available. I would also liked to have seen a breakdown of responses from face-to-face versus e-mail questionnaires (as responses were collected using both methods and demand characteristics are well known to change between these two approaches (especially with contentious material).

What constitutes expertise here?

Do each of the respondents have experience with all 19 of the drugs (personal or professional)? Is there a relationship between the experience of the raters and their ratings?

Obviously the raters work in addiction services and it is perhaps no surprise that the top ranked drugs are the most addictive and the bottom ranked drugs are all regarded as non-addictive - in this respect are addiction workers simply rating the addictive character of different drugs?

Ratings

Adapting a rating scale developed by Prof Nutt and colleagues (2007), Taylor et al used 9 parameters

"(a) physical harm caused by acute, chronic and parenteral use; (b) psychological harm; physical harm and intensity of pleasure linked to dependence and (c) social harm from intoxication; other social harms and associated healthcare costs"

The method section of the paper states "Participants were asked to score each substance for each of the nine parameters, using a 4-point scale, with 0 being no risk, 1 some risk, 2 moderate risk and 3 extreme risk." This doesn't reflect what is said in the Appendix i.e. that respondents could also say NA (Not Applicable) - they give no information about the number of NA responses or how they were dealt with.

It also unclear how the 'total' harm metric was derived - if it was from the average of all 9 scores then why is it different from the average of the personal and social harm ratings (which presumably reflects those 9 scores)? This leads to anomalies in the paper. For example, if Nicotine rates 2.42 and 2.33 for Personal and Social Harm respectively - why does it appear below Inhaled Solvents 2.38 and 2.18 (when both are lower!)

What in fact is the purpose of the separate ratings for personal and social harm - the assumption is they are measuring different things...but are they? Are the raters using the same information to rate both? I suspect so - the correlation between the two sets of ratings is .964 - so they share 93% of their variance. So in the minds of the raters ...personal harm is the same as social harm.

Finally, what does any metric mean without knowledge of the variance? The authors present mean ratings and without something like the standard deviations for the ratings, we cannot know how one drug differs from another in rating!

The Verve- The drugs don't work 

What about harm?

Even if we ignore the rating issues, it should be noted that 16/19 drugs were rated as producing moderate to extreme Personal Harm.

Finally why rate the drugs at all? In a Horse Race, the Horse always wins

As remarked by @StuartJRitchie on Twitter - "Weird to ask 'experts' rather than, y'know, look at the published evidence..." Stuart makes a good point here. Why not address the empirical data derived from studies of the drugs themselves?

My own bete noire is that the ratings, whatever their outcome, fail to incorporate the consequences for cognition. How is cognitive impairment captured in such ratings and how could it be captured?