'Representative' - its too "too too" to put a finger on

Prove it... just the facts... the confidential

This case, this case, this case that I...

I've been workin' on so long...

Chirpchirp...the birds
they're giving you the words
The world is just a feeling you undertook.
Now the rose, it slows

you in such colorless clothes

Fantastic! You lose your sense of human.

Project, Protect

It's warm and it's calm and it's perfect

It's too "too too" to put a finger on

This case is closed

Prove it [Television]

I recently tweeted that a new review of meta-analyses of CBT for various disorders was biased - at least as far as the section on schizophrenia is concerned. The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses (Hofmann, Asnaani, Vonk, Sawyer & Fang), which is published in Cognitive Therapy and Research is undoubtedly an ambitious work born of some fortitude. And to be fair, it deserves more detail - than my original 140 characters - about why I think the review is biased.

Here is the verbatim section from the paper:

Schizophrenia and Other Psychotic Disorders

Meta-analyses examining the efficacy of psychological treatments for schizophrenia revealed a beneficial effect of CBT on positive symptoms (i.e., delusions and/or hallucinations) of schizophrenia (e.g., Gould et al. 2001; Rector and Beck 2001). There was also evidence (e.g., Zimmermann et al.2005) that CBT is a particularly promising adjunct to pharmacotherapy for schizophrenia patients who suffer from an acute episode of psychosis rather than a more chronic condition.

CBT appeared to have little effect on relapse or hospital admission compared to other interventions, such as early intervention services or family intervention (e.g., Bird et al. 2010 ; Alvarez-Jimenez et al. 2011). However, CBT had a beneficial effect on secondary outcomes. For example, a more recent meta-analysis by Wykes et al. (2008) examined controlled trials of CBT for schizophrenia and confirmed findings from previous meta-analyses (e.g., Gould et al. 2001; Rector and Beck 2001), suggesting that CBT had a small to medium effect size as compared to control conditions on both positive and negative symptoms. In addition, this meta-analysis revealed medium effect sizes for improvements in secondary outcomes that were not the direct targets of treatment, including general functioning, mood, and social anxiety.

Why do I think its biased? Largely because it is unrepresentative. In the paper, Hofmann et al state "We identified 269 meta-analytic studies and reviewed of those a representative sample of 106 meta-analyses examining CBT for the following problems:...".
I suppose its conceivable that a review may be 'representative' if it references a minority (<40%) of the meta-analyses? Presumably the 60% majority were somehow unrepresentative?

So, how were the 'representative' meta-analyses selected? The paper is surprisingly remiss on this issue - telling us nothing about how selection occured. Leaving aside problems justifying their choices, it is still possible that Hoffman et al managed to formulate a representative account of the meta analyses examining CBT for schizophrenia? They included reference to meta analyses by: Gould et al. 2001; Rector and Beck 2001; Zimmermann et al (2005); Wykes et al (2008); Bird et al. (2010); and Alvarez-Jimenez et al. (2011) - so. lets take a look at what they did select

Rector & Beck (2001) and Gould et al (2001) are the earliest and hence, the smallest published meta-analyses; and almost completely overlap (with 5 of 7 studies being the same). The early studies in these meta analyses were of poor quality and thus inflated the effect sizes e.g. Rector & Beck refer to:

"...the absence of representative sampling (Drury et al., 1996a, 1996b), the absence of blind ratings (Kuipers et al., 1997; Pinto et al., 1999), the absence of reliable and valid multidimensional symptoms measures (Drury et al., 1996a, 1996b; Tarrier et al., 1993a), the failure to control for potential between-group differences in the use of medications (Kuipers et al., 1997; Tarrier et al., 1993a), and the failure to ensure treatment adherence and competence of CBT therapists (Drury et al., 1996a, 1996b; Kuipers et al., 1997)."

Regarding the next, larger meta-analysis of Zimmerman et al (2005), which contained better studies and therefore revealed a much smaller effect size (two and half times smaller than Rector & Beck), Hoffman et al write "There was also evidence (e.g., Zimmermann et al.2005) that CBT is a particularly promising adjunct to pharmacotherapy for schizophrenia patients who suffer from an acute episode of psychosis rather than a more chronic condition." This was not true in the Zimmerman et al meta-analysis, which states that for acute and chronic patients "...this difference was not statistically significant"(p.7)

Wykes et al (2008) - Hofmann et al say "In addition, this meta-analysis (Wykes et al 2008) revealed medium effect sizes for improvements in secondary outcomes that were not the direct targets of treatment, including general functioning, mood, and social anxiety."

Wykes et al calculated effect sizes for these variables according to whether they rated studies as having high or low methodological quality - Social anxiety was calculated from 2 studies, both of low quality; and for studies with acceptable levels of methodological quality, the Effect sizes for Functioning were .147 and for Mood just .084 - not medium (not even small), not representative and not significant (see Table below) - Any effect of CBT on these outcomes is a direct consequence of running low quality studies.

Hoffman et al also say "Wykes et al. (2008) examined controlled trials of CBT for schizophrenia and confirmed findings from previous meta-analyses (e.g., Gould et al. 2001; Rector and Beck 2001), suggesting that CBT had a small to medium effect size as compared to control conditions on both positive and negative symptoms" Again, as the table shows, the effect size for negative symptoms was only significant for low quality studies and not for high quality studies; and true, the effect size for positive symptoms was significant for high quality studies albeit small and barely significant at .22 (with a low end 95% CI of .016 meaning it could become nonsignificant with 1 or 2 more negative studies being reported)

Turning to Bird et al. (2010) and Alvarez-Jimenez et al. (2011), it is not explicitly stated in the article, but these two meta-analyses assess CBT for first episode patients - although they contrast with Wykes et al (2008) which did not examine first episode. Anyway, Alvarez-Himinez found "no statistically significant advantages in favor of CBT [and] CBT showed no clinical benefits on relapse rates compared with either supportive counseling or TAU." And Bird et al reported that "the present review failed to find any evidence that CBT reduced relapse rates in early psychosis, which suggests that the main benefits of this intervention are likely to be a reduction in symptom" Bird et al found no significant reduction in positive and negative symptoms at the end of the trials, but surprisingly found significant effects occur at up to 2 years after the trials finish - but the sample here is just 3 studies

Finally, returning to representativeness -Hoffman et al failed to mention at least three meta analyses published since Wykes (2008), which have been more explicitly negative about the effects of CBT on psychotic symptoms and relapse, including:

Lynch, Laws McKenna (2010) who said "CBT is no better than non-specific control interventions in the treatment of schizophrenia and does not reduce relapse rates."

Newton-Howes & Woods (2011) who said "Theoretically based CBT therapies, although proving effective, may not out perform more accessible and simpler forms of therapy for patients with schizophrenia in reducing psychopathology. Consideration of supportive therapy should be made for patients with psychotic mental disorder"

Jones et al (2012) most recent Cochrane meta analysis concludes "Trial-based evidence suggests no clear and convincing advantage for cognitive behavioural therapy over other - and sometime much less sophisticated - therapies for people with schizophrenia."

To conclude, I stand by my tweet that the review by Dr Hofmann and colleagues represents a biased view of the findings. Given the structured guidelines on how to write meta-analyses, the attention to publication bias, detailing the search process and providing reasons for excluding studies and so on - the raison d'etre for meta analysis would seem to be undermined by reviews of meta-analyses that do not adhere to the same or similar principles.

Autism - A gender defiant disorder

Everything with a downside may have an upside, bear with me ...I am no expert on autism, but the following paper struck me so much, I wanted to dash something off on it here (while listening to the peerless Suede...for reasons that are obvious...to me!)

Bejerot et al (2012) have just published an intriguing paper entitled 'The extreme male brain revisited: gender coherence in adults with autistic spectrum disorder" in The British Journal of Psychiatry.

In their paper, they claim that "...ASD, rather than being characterised by masculinisation in both genders, may constitute a gender defiant disorder"

Professor Simon Baron-Cohen is probably the most well known advocate of the ‘Extreme Male Brain’ theory to have suggested that an increased androgen exposure in utero may contribute to the development of ASD in both genders (e.g. Baron-Cohen 2002). The extreme male brain theory of autism was, however, first suggested by Hans Asperger (1944) when he wrote:

‘The autistic personality is an extreme variant of male intelligence. Even within the normal variation, we find typical sex differences in intelligence… In the autistic individual, the male pattern is exaggerated to the extreme’ (Uta Frith’s translation).

Despite the ‘extreme male brain’ theory suggesting that autism spectrum disorder (ASD) is an extreme variant of male intelligence, Bejerot et al noted in the clinic that paradoxically individuals with ASD often display androgynous physical features regardless of gender

The Beautiful Ones (by Suede)

What did they do?

In this paper, Bejerot et al examined serum hormone levels, anthropometry, the ratio of 2nd to 4th digit length (2D:4D) and psychiatric symptomatology in 50 adults with high-functioning ASD and age and gender-matched neurotypical controls. They took photographs of face and body, as well as voice recordings, which were assessed with respect to gender coherence, blindly and independently, by eight assessors (see below)

2D:4D ratio - it has been argued (e.g. by John T Manning and many others) that the ratio of two digits in particular, the 2nd (index finger) and 4th (ring finger), is affected by exposure to androgens e.g. testosterone while in the uterus and that this 2D:4D ratio can be considered a crude measure for prenatal androgen exposure, with lower 2D:4D ratios pointing to higher androgen exposure. The 2D:4D ratio has been linked to many health e.g. heart disease, cancer etc as well as psychological factors

A selected assessor group including four men and four women (aged 18–47 years and from divergent socioeconomic backgrounds), individually assessed the digital photographs and voice samples presented on a computer. They were carefully informed to estimate whether the participants had more or less gender-typical features, but not to judge beauty. Femininity in the women and masculinity in the men were assessed, face and body unconnected, applying a 5-point Likert scale: very gender typical; gender-typical; average; weak gender coherence; very weak gender coherence. Coded photographs and voice samples of the control and ASD groups, men and women separately, were shown in random order. First, front and profile photographs of each participant’s face were shown, and the assessors were to estimate the age and gender coherence of the faces. Next, assessors estimated gender coherence from the body photographs, showing the shoulders and downwards. Finally, the assessors evaluated the voices, coded with a random number, on the computer according to the above-described 5-point Likert scale.

Poor Lemmy

What did they find?

Women with ASD had higher total and bioactive testosterone levels, less feminine facial features and a larger head circumference than female controls.

Men in the ASD group were assessed as having less masculine body characteristics and voice quality, and displayed higher (i.e. less masculine) 2D:4D ratios, but similar testosterone levels to controls. Androgynous facial features correlated strongly and positively with autistic traits measured with the Autism-Spectrum Quotient in the total sample.

and look younger too...In males and females with ASD, dehydroepiandrosterone sulfate (DHEAS) did not decrease with age (compared to controls). This suggests an abnormal hypothalamic–pituitary–adrenal axis. And the authors speculate "Whether the clinically observed ‘young look’ in middle-aged people with ASD is associated with a reduced decline in DHEAS or not is worth exploring in future studies."

So Young (by Suede)

Table 1. Anthropometric measures

The authors some bold and intriguing conclusions:

"Rather than being a disorder characterised by masculinisation in both genders, ASD thus seems to be a gender defiant disorder. Our results strongly suggest that gender incoherence in individuals with ASD is to be expected and should be regarded as one reflection of the wide autism phenotype. The prevalence of ASD appears to be rising and so does gender identity disorder. The comorbidity between these two disorders is striking"

References

Bejerot S, Eriksson JM, Bonde S, Carlstrom K, Humble MB, Eriksson E (2012) The extreme male brain revisited: gender coherence in adults with autism spectrum disorder. British Journal of Psychiatry. Doi:10.1192/bjp.bp.111.097899.

Hug drug or thug drug?

 H-A-T-E
Is that how you spell love in your dictionary
K-I-C-K
Pronounced as kind
F-U-C-K
Is that how you spell friend in your dictionary
Black on black
A guidebook for the blind

Dictionary by XTC

Ecstasy is widely known as a drug that produces feelings of emotional warmth, and empathy toward others (even total strangers, as well as a general sense of well being)

Beyond these enactogenic properties, however, a far lesser known consequence of taking E is....increased aggression

Reid et al (2007) describe data collected from 260 ecstasy users. After controlling for key predictors of aggression (that are independent of ecstasy use), a higher prevalence of lifetime ecstasy use was linked to higher levels of aggressive and violent behaviour. Particularly those who exhibit low self-control were more affected by ecstasy use than those who do not in terms of aggression.

Verheyden et al (2002) looked at the sub-acute effects of MDMA. Forty participants were examined on the night of taking the E and then 4 days later. E users rated lower levels of aggression than controls on the night of drug use, but significantly higher levels of aggression mid-week (both for men and women); and in men change in aggression correlated with the amount of MDMA taken on the weekend

Fig 1. Verheyden et al (2002)
Lower aggression when taking E and higher aggression later

In a rat study, Wallinga et al (2009) assessed (behavioural) aggression in rats 5 days before and 23 days after MDMA administration (by introducing an 'unknown' male rat into cage). They found that MDMA treatment increased aggressiveness in only low aggressive rats and not in medium and high aggressive animals. This suggests that "the vulnerability for increased aggression long after a single MDMA treatment is dependent on the individual's trait aggressiveness and not on the degree of MDMA induced serotonergic neurotoxicity."

Wan et al (2009) examined abstinent Ecstasy users, free of drugs for a minimum of 21 days. They found that aggressive behaviour history correctly identified 73% of those who had been regular ecstasy users and 78.3% of those who had not. The abstinent Ecstasy users showed moderate to large effects for aggression and anti-social behaviour (on lifetime history of aggression rating scale) compared to controls who were mostly cocaine and alcohol dependent.

Table 1. Wan et al (2009)

Curran et al (2004) found that ecstasy users show increased aggression 4 days after taking E both on self-ratings and crucially on objective measures of interpretative bias. Their ecstasy and control groups were well matched for age, education, trait aggression, and trait depression. Importantly, the two groups did not differ significantly in their reported use of drugs other than MDMA. On the interpretive bias task, Ecstasy users were faster in completing sentences that could be interpreted aggressively compared with those which had only neutral interpretations. The control group showed the exact opposite pattern of being slower to complete aggressive than neutral sentences - this suggests that ecstasy users are sensitised to aggressive material


Gerra et al (2001) examined participants with a history of MDMA use and nonuser controls were tested on a laboratory measure of aggression (receiving monetary reward for repeated button pressing and provoked by having it taken away by a fictional other). The authors found that the user group was considerably more aggressive than nonusers and that there was a significant correlation found between the extent to which participants had been exposed to MDMA and aggressive response (r=.78). They also found that aggressiveness in the MDMA-using subjects seemed to be associated more with MDMA pharmacological effects than with personality traits. Finally, they noted that the aggressive responses of Ecstasy users were higher than those seen in heroin users in the same experimental scenario.

Finally, the issue of self-harm with ecstasy...

Kim et al (2011) examined the relationship between ecstasy use and suicidal behaviour among adolescents in the United States. Data from the adolescent subsample (ages 12–17, N = 19,301) of the 2000 National Household Survey on Drug Abuse were used in the analyses. They found that the rate of past year suicide attempt among adolescents with lifetime ecstasy use was almost double that of adolescents who had used other drugs only, and nine times that of adolescents with no history of illicit drug use. After controlling for other related factors, the effect of ecstasy use remained significant.

References

Curran, H.V.; Rees, H.; Hoare, T.; Hoshi, R.; Bond, A. Empathy and aggression: Two faces of Ecstasy? A study of interpretative cognitive bias and mood change in Ecstasy users. Psychopharmacology 2004, 173, 425-433

Gerra, G., Zaimovic, A., Ampollini, R., Giusti, F., Delsignore, R., Raggi, M. A., Laviola, G., Macchia, T., & Brambilla, F. Experimentally induced aggressive behavior in subjects with 3,4-methylenedioxymethamphetamine (‘‘ecstasy’’) use history: Psychobiological correlates.
Journal of Substance Abuse, 2001, 13, 471–491.

Kim J, Fan B, Liu X, Kerner N and Wu PEcstasy use and suicidal behavior among adolescents: findings from a national survey. Suicide and Life Threatening Behaviour, 2001 41, 435-444

Reid, L.W.; Elifson, K.W.; Sterk, C.E. Hug drug or thug drug? Ecstasy use and aggressive behavior. Violence Vict. 2007, 22, 104-119

Verheyden SL, Hadfield J, Calin T, Curran HV . Sub-acute effects of MDMA (3,4-methylenedioxymethamphetamine, "ecstasy") on mood: evidence of gender differences. Psychopharmacology (Berl), 2002, 161: 23-31

Wallinga AE, ten Voorde AM, de Boer SF, Koolhaas JM, Buwalda B. MDMA-induced serotonergic neurotoxicity enhances aggressiveness in low- but not high-aggressive rats. Eur J Pharmacol 2009;618(1-3):22-27.

Wan, L.; Baldridge, R.M.; Colby, A.M.; Stanford, M.S. Enhanced intensity dependence and aggression history indicate previous regular Ecstasy use in abstinent polydrug users. Prog. Neuropsychopharmacol. Biol. Psychiatry 2009, 33, 1484-1490.

The Drugs (ratings) Dont Work

The British Medical Journal has just published a paper "Quantifying the RR of harm to self and others from substance misuse: results from a survey of clinical experts across Scotland" by Taylor et al (2012)

Abstract

Objective: To produce an expert consensus hierarchy of harm to self and others from legal and illegal substance use.

Design: Structured questionnaire with nine scored categories of harm for 19 different commonly used substances.

Setting/participants: 292 clinical experts from across Scotland.
Results: There was no stepped categorical distinction in harm between the different legal and illegal substances. Heroin was viewed as the most harmful, and cannabis the least harmful of the substances studied. Alcohol was ranked as the fourth most harmful substance, with alcohol, nicotine and volatile solvents being viewed as more harmful than some class A drugs.

Conclusions: The harm rankings of 19 commonly used substances did not match the A, B, C classification under the Misuse of Drugs Act. The legality of a substance of misuse is not correlated with its perceived harm. These results could inform any legal review of drug misuse and help shape public health policy and practice

Harm to self ratings (ranked from top to bottom)

Sample

The authors state

"One of the strengths of this study is the large number of experts involved. Two hundred and ninety-two addiction multidisciplinary experts across Scotland were involved making it the largest national panel to be involved in this type of study.

How representative is the 'large' sample of 'experts'? Well, 50% of the respondents worked in Glasgow and 55.5% of the sample were Addiction Community Psychiatric Nurses - I am not sure how representative that might make any opinion re the relative ranking of drugs. It would have been useful to see a breakdown by occupation - the data are available. I would also liked to have seen a breakdown of responses from face-to-face versus e-mail questionnaires (as responses were collected using both methods and demand characteristics are well known to change between these two approaches (especially with contentious material).

What constitutes expertise here?

Do each of the respondents have experience with all 19 of the drugs (personal or professional)? Is there a relationship between the experience of the raters and their ratings?

Obviously the raters work in addiction services and it is perhaps no surprise that the top ranked drugs are the most addictive and the bottom ranked drugs are all regarded as non-addictive - in this respect are addiction workers simply rating the addictive character of different drugs?

Ratings

Adapting a rating scale developed by Prof Nutt and colleagues (2007), Taylor et al used 9 parameters

"(a) physical harm caused by acute, chronic and parenteral use; (b) psychological harm; physical harm and intensity of pleasure linked to dependence and (c) social harm from intoxication; other social harms and associated healthcare costs"

The method section of the paper states "Participants were asked to score each substance for each of the nine parameters, using a 4-point scale, with 0 being no risk, 1 some risk, 2 moderate risk and 3 extreme risk." This doesn't reflect what is said in the Appendix i.e. that respondents could also say NA (Not Applicable) - they give no information about the number of NA responses or how they were dealt with.

It also unclear how the 'total' harm metric was derived - if it was from the average of all 9 scores then why is it different from the average of the personal and social harm ratings (which presumably reflects those 9 scores)? This leads to anomalies in the paper. For example, if Nicotine rates 2.42 and 2.33 for Personal and Social Harm respectively - why does it appear below Inhaled Solvents 2.38 and 2.18 (when both are lower!)

What in fact is the purpose of the separate ratings for personal and social harm - the assumption is they are measuring different things...but are they? Are the raters using the same information to rate both? I suspect so - the correlation between the two sets of ratings is .964 - so they share 93% of their variance. So in the minds of the raters ...personal harm is the same as social harm.

Finally, what does any metric mean without knowledge of the variance? The authors present mean ratings and without something like the standard deviations for the ratings, we cannot know how one drug differs from another in rating!

The Verve- The drugs don't work 

What about harm?

Even if we ignore the rating issues, it should be noted that 16/19 drugs were rated as producing moderate to extreme Personal Harm.

Finally why rate the drugs at all? In a Horse Race, the Horse always wins

As remarked by @StuartJRitchie on Twitter - "Weird to ask 'experts' rather than, y'know, look at the published evidence..." Stuart makes a good point here. Why not address the empirical data derived from studies of the drugs themselves?

My own bete noire is that the ratings, whatever their outcome, fail to incorporate the consequences for cognition. How is cognitive impairment captured in such ratings and how could it be captured?

Ecstasy ~ Russian Roulette with your Memory

‘The general agreement that is emerging about ecstasy is that while you are using the drug, you might expect a very subtle memory impairment but it’s probably not significant in the real world ...[and] When you stop using it, as most people do, things go back to the way they were.’
-Professor V Curran, Daily Mail 26/7/2012

To E or not to E - that is the question. I felt compelled to say something triggered by an article in Today's Daily Mail Ecstasy tablets are far more harmful than previously thought - and taking just ten can cause brain damage. The article is based on a unique and important paper by Wegner and colleagues entitled "A prospective study of learning, memory, and executive function in new MDMA users" (to be published shortly in Addiction).

In the context of the obvious methodological limitations associated with studying recreational drug use, this research is unique insofar as it is a prospective cohort study and so, examines the effects of E on non-users who said they intended to use E during the next 12 months. The users and non-users were well matched and drug use was confirmed by hair testing in a random sample (one-third) for MDMA, 3,4-methylenedioxyamphetamine (MDA), 3,4-methylenedioxy-N-ethylamphetamine (MDEA), amphetamine, methamphetamine and cannabinoids.

This is a more intriguing study of E than most because the participants were "almost MDMA naive" (a lifetime maximum of 5 Ecstasy tablets) and had no history of

"...ingestion of any other illicit psychotropic substances besides cannabis on more than five occasions before the day of the first examination; a history of alcohol misuse (according to DSM-IV criteria, APA 1994); and regular medication (except for contraceptives)...Main inclusion criterion at baseline was a high probability of future ecstasy use"

- So, a remarkable sample of pretty clean-living individuals until now!

Over the 12 months, 23 participants did use more than 10 Ecstasy tablets (mean= 33.6, range 10-62; mean occasions 13.5, range 4-36). The remaining individuals used Ecstasy once or more, but less than 10 times and were excluded from the analyses. The controls consisted of 43 participants who had not indulged in any recreational drug use during the year or previously.

The groups (E users and non-users) did not differ in gender distribution, age, years of education, duration of cannabis use before the 1st assessment, duration of cannabis use between the 1st and the 2nd assessments, mean days since last cannabis use at 2nd assessment, or Ravens Matrices (a measure of general intellectual reasoning)

Neurocogitive testing

Wagner et al gave participants a range of executive and memory tests. The main findings were no significant differences between E users and non-users for a measure of verbal learning and memory (auditory verbal learning task), but moderate-large effects on a measure of visual paired associate learning and memory (LGT3) both for immediate and delayed testing (-.667 & -.746: see Table 1).

Table 1. Memory in Ecstasy users and non-users

Conclusions

Wagner and colleagues conclude there were:

"Significant effects of immediate and delayed recall of a paired associates learning task between subjects who used 10 or more ecstasy pills and subjects who did not use any illicit substance apart from cannabis during the course of the year (non-users) were found. No significant differences were found on any of the other neuropsychological tests."

Now, it doesn't surprise me that Wagner et al found moderate to large effects of E on memory. In 2007, along with Joy Kokkalis, I wrote Ecstasy (MDMA) and memory function: a meta-analytic update. We meta-analysed 28 published studies assessing the effects of E on memory in over 600 users and 400 non-users. We reported large impairments of verbal STM (d=.63) and LTM (d=.87) memory - equivalent to performance in over three-quarters of ecstasy users being worse than in non-ecstasy using controls. Crucially, the degree of memory impairment was unrelated to the lifetime number of tablets consumed (mean range was 16-902 in the studies). By contrast, we found no effect of E on visual memory, but did find that concurrent cannabis use impacted specifically on visual memory

Robert E Niro: the Deer Hunter

Our finding that lifetime dose did not predict degree of memory impairment suggests a non-linear relationship and that taking E may be akin to playing Russian Roulette with your memory - for some a large number of tablets may be required before memory is impaired, but for others very few may be necessary - the reasons for the individual differences remain unknown to scientists and most importantly, unknown to anyone using E.

and Final...E

I started by saying that the Daily Mail article prompted me to blog this - not because the journalist got anything wrong necessarily, but because I couldn't see how Professor Val Curran (from UCL) could justifiably make the assertions outlined at the top of this post. In addition to our meta-analysis, at least five other meta-analyses (Verbaten, 2003; Nulsen et al, 2010; Zakzanis et al 2007; Kalechstein et al 2007; and Murphy et al 2012) - have reported moderate to large memory impairment associated with recreational Ecstasy use.

When talking about recreational drugs, it is easy to be misrepresented in the press (as I have discovered). Nevertheless, in our minds, quotes from well-respected scientists are assumed to be reliable...and in the area of recreational drug use, it is all to easy to see how pithy quotes regularly shape thinking in the general public - even though the empirical evidence runs in the contrary direction.

I am beginning to wonder if the recent problems in psychology (false data, retractions and so on) reflect a problem that is all too peculiar to psychologists - too many experts offering too many opinions. As scientists who work with people, we are often asked to comment on surprising findings concerning human behaviour, the brain... life and the universe. Sometimes its fine because we know the literature and have read the paper, but at other times, experts clearly have not read the paper, dont seem to know the literature but still offer an opinion - I am convinced that giving - what are nothing nore than personal or lay opinions essentially - would be far less likely in other sciences...another possibility of course is that the expert studiously avoids citing evidence because of a preferred personal conviction - science?

So, is there general agreement that the memory problems associated with E are subtle, do not impact real world and return to normal?

You used to slide down the carpeted stairs

Or down the banister

You stuttered like a kaleidoscope

'Cause you knew too many words

The Magnetic Fields - 'Take Ecstasy with me'

References

Kalechstein AD, De La Garza R, 2nd, Mahoney JJ, 3rd, Fantegrossi WE, Newton TF. (2007). MDMA use and neurocognition: a meta-analytic review. Psychopharmacology; 189:531–537

Laws KR, Kokkalis J. (2007) Ecstasy (MDMA) and memory function: a meta-analytic update. Hum Psychopharmacol ;22:381–388

Murphy, P.N., Bruno, R., Wareing, M., Ryland, I., Fisk, J.E., and Montgomery, C. (2012). The effects of ecstasy (MDMA) on visuospatial memory performance: Findings from a systematic review with meta-analysis. Human Psychopharmacology: Clinical and Experimental, 27, 113-138

Nulsen CE, Fox AM, Hammond GR (2010). Differential effects of ecstasy on short-term and working memory: a meta-analysis. Neuropsychol Rev. 20:21-32.

Wagner D, Becker B, Koester P, Gouzoulis-Mayfrank E & Daumann J (in press) A prospective study of learning, memory, and executive function in new MDMA users. Addiction

Zakzanis K. K., Campbell Z., Jovanovski D. (2007) The neuropsychology of ecstasy (MDMA) use: a quantitative review. Human Psychopharmacology: Clinical and Experimental , 22, 427-435

.

Negativland: What to do about negative findings?

Elephant in the Room (by Banksy) 

This violent bias of classical procedures [against the null hypothesis] is not an unmitigated disaster. Many null hypotheses tested by classical procedures are scientifically preposterous, not worthy of a moment's credence even as approximations. If a hypothesis is preposterous to start with, no amount of bias against it can be too great. On the other hand, if it is preposterous to start with, why test it? Ward Edwards, Psychological Bulletin (1965)

Several interesting blog posts, discussions on Twitter and regular media articles (e.g. Times Higher Education) have recently focused on the role of negative (so-called null) findings and the publish or perish culture. Pete Etchells' nice recent blog discusses the pressure to publish in science and how this may lead to a bias against Quality over Quantity. This discussion was in the context of the recent reports concerning the faking of data by Diederik Stapel and others.

 Reading Pete's blog, I thought to write a few lines about a related, but an issue that receives less attention - what can we do to correct for negative findings not being reported and how do we deal with negative findings when they are reported?

In his blog, Pete gives the following description of how we generally think that null findings influence the scientific process

...if you run a perfectly good, well-designed experiment, but your analysis comes up with a null result, you're much less likely to get it published, or even actually submit it for publication. This is bad, because it means that the total body of research that does get published on a particular topic might be completely unrepresentative of what's actually going on. It can be a particular issue for medical science - say, for example, I run a trial for a new behavioural therapy that's supposed to completely cure anxiety. My design is perfectly robust, but my results suggest that the therapy doesn't work. That's a bit boring, and I don't think it will get published anywhere that's considered prestigious, so I don't bother writing it up; the results just get stashed away in my lab, and maybe I'll come back to it in a few years. But what if labs in other institutions run the same experiment? They don't know I've already done it, so they just carry on with it. Most of them find what I found, and again don't bother to publish their results - it's a waste of time. Except a couple of labs did find that the therapy works. They report their experiments, and now it looks like we have good evidence for a new and effective anxiety therapy, despite the large body of (unpublished) evidence to the contrary.

The hand-wringing about negative or null findings is not new...and worryingly, psychology fares worse than most other disciplines, has done for a long time and (aside from hand-wringing) does little to change this situation. For example, see Greenwald's 'Consequences of the Prejudice against the Null Hypothesis' published in Psychological Bulletin in 1975. The table below comes from Sterling et al (1995) showing that <0.2% of papers in this sample accepted the null hypothesis (compare to a sample of medical journals below)

 

 Table 1. From Sterling et al 1995

More recently Fanelli (2010) confirmed the earlier reports about psychology/psychiatry being especially prone to the bias of publishing positive findings. Table 2 below outlines the probability for a paper to report positive results in various disciplines. It is evident that, compared to every other discipline, Psychology fares the worst - being five times more likely as the baseline (space science) to publish positive results! We might ask 'why' psychology? and what effect does it have?

Table 2 Psychology/Psychiatry bottom of the league

Issues from Meta-Analysis

It is certainly my experience that negative findings are more commonly published in more medically oriented journals. In this context, the use of meta analysis becomes very interesting.


The File Drawer effect and Fail-Safes
Obviously meta-analysis is based on quantitatively summarising the findings that are accessible (tending to be those published of course). This raises the so-called file-drawer effect, whereby negative studies may be tucked away in a file drawer because they are viewed as less publishable. It is possible in meta analysis to statistically estimate the file-drawer effect - the original and still widely used method is the Fail-Safe statistic devised by Orwin, which essentially estimates how many unpublished negative studies would be need to overturn a significant effect size in a meta analysis. A marginal effect size may require just one or two unpublished negative studies to overturn it, while a strong effect may require thousands of unpublished negative studies to eliminate the effect.

So, at least we have a method for estimating the potential influence of negative unpublished studies.

Where wild psychologists roam - Negativland by Neu!

 

Funnel Plots: imputing missing negative findings

Related to the first point, we may also estimate the number of missing effect sizes and even how large they might be. Crucially, we can then impute the missing values to see how it changes the overall effect size in a meta-analysis. This was recently spotlighted by Cuijpers et al (2010) in their timely meta-analysis of psychological treatments for depression, which highlighted a strong bias toward positive reporting.

A standard way to look for bias in a meta analysis is to examine funnel plots of the individual study effect sizes plotted against their sample sizes or the standard errors. When no bias exists, studies with smaller error and larger sample sizes cluster around the mean effect size. By contrast, smaller samples and greater error variance produce far more variable effect sizes (in the tails). Ideally, we should observe a nicely symmetrical inverted funnel shape.

Turning to Cuijpers paper, the first figure below is clearly asymmetrical, showing a lack of negative findings (left side). Statistical techniques now exist for imputing or filling in these assumed missing values (see figure below where this has been done). The lower funnel plot gives a more realistic picture and adjusts the overall effect size downwards as a consequence (Cuijpers imputed 51 missing negative findings - the dark circles) - which reduced the effect size considerably from 0.67 to 0.42.

 

Figure 1. Before and After Science: Funnel Plots from Cuipjers et al (2010)

No One Receiving (Brian Eno - from Before & After Science)

Question "What do you call a group of negative (null) findings?" Answer: "A positive effect"

As noted already, some more medically-oriented disciplines seem happier to publish null findings - but what precisely may be some of the implications of this - especially in meta analyses? Just publishing negative findings is not the end of the questioning!


Although some may disagree, one area that I think I know a fair bit about is the use of CBT for psychosis. The following forest plot in Figure 2 is taken from a meta analysis of CBT for psychosis from Wykes et al (2008).

Figure 2. Forest plot displaying Effect sizes for CBT as treatment for Psychosis

(from Wykes et al 2008)

In meta analysis, forest plots are often one of the most informative sources of information - because they reveal much about the individuals studies. This example shows 24 published studies. The crucial information here, however, concerns not the magnitude of individual effect sizes (where the rectangle sits on the X axis), but
...the confidence intervals - these tell us everything!


When the confidence intervals pass through zero, we know the effect size was nonsigificant. So, looking at this forest plot only 6/24 (25%) studies show clear evidence of being significant trials(Trower 2004; Kuipers 1997; Drury 1997; Milton 1978; Guadiano 2006; Pinto 1999). Although only one quarter of all trials were clearly significant, the overall effect is significant (around 0.4 as indicated by the diamond at the foot of the figure)


In other words, it is quite possible for a vast majority of negative (null) findings to produce an overall significant effect size - surprising? Other examples exist (e.g. streptokinase: Lau et al 1992; for a recent example, see Rerkasem & Rothwell (2010) and indeed, I referred to one in my recent blog "What's your Poison?" on the meta analysis assessing LSD as a treatment for alcoholism (where no individual study was significant!).

Some argue that the negative studies are only negative because they are underpowered - however, this only seems likely with a moderate-large effect size that produces a nonsignificant statistical result. And further speculate that a large trial will prove the effectiveness of the treatment' however, when treatments have subsequently been evaluated in definitive large trials, they have often failed to reach significance. Egger and colleagues have written extensively on the unreliability of conclusions in meta-analyses where small numbers of nonsignificant trials are pooled to produce significant effects (Egger & Davey Smith. 1995)

So, negative or null findings are perhaps more and less worrisome than we may think. Its not just an issue of not publishing negative results in psychology, its also an issue of what to do with them when we have them

Keith R Laws (2012). Negativland: what to do about negative findings http://keithsneuroblog.blogspot.co.uk/2012/07/negativland-what-to-do-about-negative.html

Revox, Xerox, Redux: Thomas Leer & Robert Rental

Punk was never a parasite gnawing at the inner fabric of society or even the music establishment, but it did lay some unexpected eggs on its welcoming host. The anti-establishment DIY ethos of making noncommercial music, and then pressing and distributing your own records clearly emerged with post-punk and electronic music...

After all, most punk bands were signed by majors, their commercial appeal to young people was obvious and blindly exploited by major labels. Its easy to overlook how swiftly punk came and went - all under a Labour government, well before Maggie Thatcher was elected as Prime Minister. Rather, it was a peculiarly British revolution from the lo-fi British electronic pioneers, Thomas Leer, Robert Rental, the Normal, who initiated this crucial DIY sea-change in UK (and world) music.



Daniel Miller (aka the Normal) went on to develop his label Mute into the important artist stable it remains today, starting with Depeche Mode. And his first single Warm Leatherette and TVOD are well-known. His JG Ballard Crash inspired Warm Leatherette has been covered by the likes of Grace Jones as well as Trent Razor, Geordie White and Pete Murphy. 

Although I knew Daniel Miller at that time, the evolution of Mute is well-documented. Rather I wanted to talk about two friends of mine at that time: Thomas Leer (born Wishart) and Robert Rental (born Donnachie). Thomas and Robert came from the same Scottish hometown of Port Glasgow and in the mid 70s they moved to London together, with their respective partners (and sisters) Liz and Hilary Farrow (I recall Robert and Hilary also had a least one, possibly two children then) .All of Tom's home-made experiments were 'tested' on the world by Liz, who worked at the original Virgin Records store in Oxford Street - where she regularly played cassettes of his prolific outputs to those perusing the singles-section in the shop basement.

Necessity is the Mother of Invention

In 1978 Thomas Leer pressed 650 copies of his double A side Private Plane and International on his own Oblique records - possibly a nod to the Oblique Strategies of Peter Schmidt and Brian Eno - 100 worthwhile dilemmas - used by Eno in recording sessions to overcome artistic blocks or to seek inspiration - an online version is here - be inspired!

Robert Rental similarly pressed 650 of his double A side single ACC and Paralysis on his own alliteratively consistent Regular records. When they rapidly sold out, both singles were re-released by 'Company' in somewhat different sleeves (slightly less DIY)

Private Plane/International were recorded in Tom's small Finsbury Park flat in 3 days using a TEAC A3440 4-track recorder and an ALICE mixing board. "The only FX used were a Watkins Copicat tape echo unit, Electo Harmonix DrQ filter, an old Roland drum machine and a Stylophone 350S. The process was simply a case of laying the tracks down one at a time, applying FX as I went along, and then mixing them all down onto a REVOX A77 mastering machine" They then moved everything across the Thames to Robert's Battersea flat to record ACC/Paralysis.

The DIY nature is evident above in my original time-jaundiced copies of the 45s: the xeroxed covers, even some felt tip colouring on Robert's (by him), John Bull printing on the white labels, handwritten cut-out details, even Robert's home address (where their album 'the Bridge' was recorded - see below).

 

Tom's single made a big impact - being made NME single of the week by Tony Parsons (now social commentator and author of Man & Boy), which normally assured fame and fortune in those days. The two 45s have a special feel that I believe have stood the test of time partly ‘because’ of the way they were recorded – at home on 4-track, in the same room where Liz was sleeping (hence Tom's vocals are delivered so softly), guitar, rhythm from a cheap drum machine and a bubbling bass that sounds too fast (in the style of Neu on Fur Immer or Hallo Gallo) and finally, Rolf Harris stylophone lead melodies! Without doubt, this 45 would be on my desert Island disc selection. Robert's two songs, by contrast, were less immediately accessible and at the time, overshadowed by his friends release; however, to my ear, Robert's ACC/Paralysis have matured more with age - on ACC sounding like a 'tired and emotional' Barry White on acid or on Paralysis like a terribly sad Tim Buckley being played at 16 RPM.

I heard Private Plane by Thomas Leer which he did all by himself, all the playing and writing, everything. That was the big turning point for me because it introduced me to a whole new form of music...Leer had all these drum machines and loops and totally different instrumentation, and this whole new world opened up, listening to his experimentation with atmospheres. I realised then that I didn`t have to make songs that sounded like everybody else...People would buy a stylophone and a small tape player and put out a record they had made in their bedroom. That was really inspiring.- Matt Johnson,The The

I don`t have to go into a studio, the fact that Thomas Leer made his record at home with a Revox and on mostly hired equipment, I thought was marvellous and I like his record a lot. In fact it`s one of my favourite records at the moment.– John Foxx, Ultravox

The Bridge: "I thought the idea of the song was dead"

Following their successful singles, Thomas and Robert produced 'The Bridge' for Industrial Records (Throbbing Gristle's label). One month after Margaret Thatcher was first elected as UK Prime Minister, and just as Joy Division released Unknown Pleasures, Tom and Rob spent two weeks (18 June to 2nd July 1979) recording the album using TG's 8-track recorder holed-up at Robert's high rise council flat (identified above). In the same vein as David Bowie's Low, released the previous year, the Bridge consists of two discreet sides - one with regular song structures, the other more fluid  instrumental pieces.


Thomas said "Robert wanted songs and I wanted to do a pure ambient album...the idea of the song was dead and the future belonged to instrumental music. Voices could be used, but not in a structured way" - which I suppose is what he had been doing even in his poppier songs - the lyrics are largely indecipherable and add a melodic line for the listener.

As an example of the structured minimal electro pop of side one, here is Day Breaks, Night Heals - a dirty, grungy Kraftwerk as filtered through a more apologetic British mentality. The distinctive sound is of the Wasp synthesizer made by the British Electronic Dream Plant with its monophonic touch-sensitive keyboard, dual oscillators, it could be run on batteries, had a built in speaker, and connected to other Wasps or its sister sequencer -the Spider. It was the first affordable synthesiser to use digital technology- I bought mine over 30 years ago and it still works!

Although repetition is a major force in music it was never used in this way before - Terry Riley
From side 2, here is Interferon , which is a paradigmatic example of a looping technique pioneered by Brian Eno on Discreet Music. Althoiugh it was used earlier by people like Terry Riley, who referred to it as the 'Time Lag Accumulator' -listen to his Ecstasy  from 1968; and then more explicitly by Eno's collaborator Robert Fripp (formerly of King Crimson), who referred to it as Frippertronics. Fripp introduced this to Eno on their album No Pussyfooting (listen to Heavenly Music Corporation).

Interferon uses a mixture of tape loops and synthesiser sounds played backward to develop a dense though dark orchestral sound - closer to the experimentation of Riley than the quietness of Eno. As noted on The Bridge sleeve (see below), you can actually hear interfering electrical items click (a fridge or other domestic device) echoing away infinitely during Interferon. At other points on side 2, you can hear their soft Scottish voices chatting in the background, coughing, attempting repeatedly to light cigarettes, excerpts of the TV - all encompassed in a womb of hypnotic loops - side 2 as a whole contains some of my favourite electronic insrumental pieces - music made in life - truly ambient!

This simple looping technique is described here (with Eno's Discreet Music in the background). It consists of two Revox tape machines linked together with one recording input and then the tape being fed and played through the second machine; the output of Revox 2 is then fed back via leads to Revox 1 and an infinite echo loop occurs with gradual decay. This technique could not be simpler or more analogue - as the length of the delay echo is purely dependent upon the distance between the two Revox machines.

The back cover of The Bridge shows the dominating presence of the tape-loop decks
Robert (left) and Thomas (right)

Robert Rental and the Normal live

Robert did play live a few times with Daniel Miller (as Robert Rental and the Normal) during a typically perverse Rough Trade tour in which they were support act for Stiff Little Fingers in 1979.

Fortunately one gig was recorded and released as a single-sided LP by Rough Trade: Live at the West Runton Pavilion - a smallish venue on the North Norfolk coast (which now no longer exists but did host everyone from Chuck Berry to T-Rex, Sex Pistols, the Clash and Joy Division). Some great footage of Robert and Daniel playing live to the somewhat bemused would-be-punk audience also exists (see below)

Before setting off on the tour Robert and Daniel played a single night at the Cryptic One Club under Trinity Church, Bishops Bridge Road, Paddington, along with Metabolist, Cabaret Voltaire and Throbbing Gristle. This was one of many great nights organised in unusual venues by the gig promoters 'Final Solution' - or Colin Faver (who worked in Small Wonder Records in Walthamstow, London) and Kevin Millins - who deserve credit for the role they played. Colin subsequently became a DJ on the rave scene. By the way Colin, if you ever read this, you still have my copy of the 1st Annual Report by Throbbing Gristle (on cassette) - 30 years is a long loan!

NME journalist, and now culture commentator, Paul Morley's description of the Cryptic Club evening resonates closely with my memories:

A youth club under a church. A cramped cellar, lots of arches. The "stage" is 15 feet deep and six feet wide. Twenty at most of the audience can see it. A few more can see part of it. In a corner lager is being sold for 50 pence a can. The writing on the wall doesn't say "a culture never falls to pieces, it just gives birth", but that's only 'cos I'd forgotten my chalk.It was hot, crowded, murky, NOISY now and then, silent but mostly decadent. Paul Morley

Also Cosey Fanni Tutti of Throbbing Gristle recalled that night:

"It was an electric night, very alive and thumping energy from  wall to wall and from roof to well below ground. A great shift and a great surge forward happened that night.... Suffice  it to say it was special in MANY many ways to all the people who attended" - Cosey 

Cryptic One Club 11th November 1978 (the remainder of my Ticket stub)

Later

Robert produced a solo demo cassette of Wasp-based instrumentals 'Mental Detentions' in 1979 (with titles A1, A2 etc B1, B2...an example is loaded here), but only released one more musical offering - the propitious Double Heart for Mute Records, then retired from music to spend time with his family. Very sadly, Robert died of lung cancer in 2000 (aged just 48)

Thomas was always a musical chameleon and I vividly recall certain new albums making a strong impression on him (the forboding contents of PIL's Metal Box, Peter Hammil's A Black Box, Joy Division's Unknown Pleasures to name a few) and then these following through into his musical experiments. After The Bridge, he signed for Cherry Red and released a funky well-acclaimed album called Contradictions. He later signed for Arista, for whom he produced the Fairlight-heavy 'The Scale of Ten', which contains two wonderfully subversive Eastern-flavoured pop-tunes about heroin

International (a completely new song about the mundane life of an international heroin-trader)

Came in on flight 83, making a drop here for the company, seems to be travelling light, a secret compartment holds the Chinese white.

He lives his life on the move, making connections in cold hotel rooms, but in a far away place, poppies are harvested in the morning haze

...carrying across the world and selling it to boys and girls, delivered from the company, packaged by international

and Chasing the Dragon (about using Heroin for the first time)

lay your money down, a bigger price you have to pay for what you are about.. to try...clear a space, strike a light, I need to satisfy, watch the crystals come to life and feel your senses die...and fly

I had not mentioned Thomas' use of words until now - largely because his voice was often so subtle that the words were indecipherable (he once said to me that he didn't try to create linear stories usually, but used phrases for their sound and atmosphere more than anything else)

International (top) and Chasing the Dragon (bottom)

"You can hardly separate music from technology" Trevor Horn

Thomas went on to form ACT with Claudia Brücken (formerly of Propaganda who had produced hits in the mid 80s - here they oddly but nicely cover Throbbing Gristle's Discipline). ACT released a few singles and an album for the ZTT label with famed producer Trevor Horn, who co-owned ZTT - coincidentally with Paul Morley (quoted above). Here is ACT with Snobbery and Decay - in receipt of the full Trevor Horn 'wall-of-sound' production treatment (that many benefited from including ABC,  Frankie Goes to Hollywood,  and the Pet Shop Boys)

Following this, Thomas more or less retired from the music industry for many years until a few sporadic releases more recently on small labels - largely low budget productions inspired by modern offerings from bands such as Future Sound of London. Tom's latest offering From Sci Fi to Barfly is available only on the web as far as I can determine but here are a couple of beautiful tracks: Blood of a Poet and Paths of Least Resistance

I am still astonished that no major artist has covered the perfect pop songs that lurk behind the lo-fi production of Thomas Leer's Private Plane and International (or many of his later songs).

My strongest personal recollection of Thomas and Robert is how charming and inspiring they were and our rehearsing together in a short-lived attempt to form a band (provisionally called Acid)consisting of Robert, Thomas, Liz, Matt Johnson and myself ...but that's another story...