Cass Report

[angrysoba]

This arrived in my inbox this morning:​

Yale’s “Integrity Project” Is Spreading Misinformation About The Cass Review And Youth Gender Medicine: Part 3​

Yeah, I saw that too. I began reading it before realizing how long it was (something like 8000 words) which also reference a number of other papers both pro- and con- the Cass Review and some in-between, such as Gideon Meyorowitz-Katz (who I tend to think of as a pretty good science writer and who did much of the work to unmask the massive fraud in some of the ivermectin studies). In this case, however, Singal is pointing to some of Gideon M-K's critiques of the Cass Review critics.

 

[angrysoba]

AJ Eckert is the guest blogger who managed a gender-affirming clinic and wrote the previous SBM pieces on gender-affirming care for minors, attacking critics as transphobic and frequently linking to papers that failed to support the claims made (e.g. a claim that puberty blockers do not affect brain development was supported by a link to a paper discussing the need to conduct long-term studies on whether puberty blockers have negative effects on brain development).
I wasn't suggesting that Eckert's departure from Anchor gender clinic is related to the Cass review but it might be related to SBM going quiet on this topic. I don't know if Eckert is still working in the gender care field at a different clinic since nothing I can find online seems to be updated including the SBM bio.

Ah, I see. So has SBM actually addressed it at all since it came out? I know Steven Novella has in a way that suggests he hadn't really looked at it properly and which Gideon M-K (mentioned above) and the Studies Show (also mentioned above) and Jesse Singal (mentioned above) could have corrected him on such as his assertion that the Cass Review just invented an ad hoc exclusion/inclusion criteria, which is not true. Gideon M-K writes about the scale used (but points out that it did change from the one it pre-registered which GM-K pulls them up on saying there should be a clearly given rationale for it even though it is unlikely to have changed the outcome)...

I don't know if his series has been covered here, but as I said, I think he's usually quite good on these sorts of things and does offer some criticisms of the review...

The Cass Review Into Gender Identity Services for Children - Part 2

What the review got right

gidmk.substack.com

 

[Elaedith]

Ah, I see. So has SBM actually addressed it at all since it came out? I know Steven Novella has in a way that suggests he hadn't really looked at it properly and which Gideon M-K (mentioned above) and the Studies Show (also mentioned above) and Jesse Singal (mentioned above) could have corrected him on such as his assertion that the Cass Review just invented an ad hoc exclusion/inclusion criteria, which is not true. Gideon M-K writes about the scale used (but points out that it did change from the one it pre-registered which GM-K pulls them up on saying there should be a clearly given rationale for it even though it is unlikely to have changed the outcome)...

I don't know if his series has been covered here, but as I said, I think he's usually quite good on these sorts of things and does offer some criticisms of the review...

The Cass Review Into Gender Identity Services for Children - Part 2

What the review got right

gidmk.substack.com

I think it's a valid criticism of the Cass Review that they did not explain the change from using MMAT stated in their pre-registration to using NOS for some of the systematic evixence reviews. There are sometimes valid reasons to change protocol from pre-reg but it should be explained clearly. In this case, they used NOS for the studies of hormone treatments. All the studies in these reviews involved non-RCT quantitative designs, for which NOS is designed. They used MMAT for the review on psychosocial transition which included qualitative, quantitative and mixed-method designs. MMAT is specifically for systematic reviews that contain both qualitative and quantitative and/or mixed methods studies, so in both cases the appraisal tool seems appropriate for the type of studies evaluated within each review. I don't think there is anything sinister about deciding to use NOS for the quantitative-only reviews; they probably just decided that there was no reason to use a tool intended for reviews with diverse study designs for reviews that had only quantitative studies. They may have intended originally to include all treatments in one review (which would then involve diverse designs) which would explain why they originally proposed only MMAT.
I have seen activists claim that the switch was due to MMAT recommending against excluding low quality studies (implying this was part of a sinister plot to brand all studies with positive results as low quality and exclude them). However, I have not been able find anything to support this and several papers discussing the MMAT list excluding low quality studies as an accepted use of the tool.

 

[Elaedith]

Ah, I see. So has SBM actually addressed it at all since it came out? I know Steven Novella has in a way that suggests he hadn't really looked at it properly and which Gideon M-K (mentioned above) and the Studies Show (also mentioned above) and Jesse Singal (mentioned above) could have corrected him on such as his assertion that the Cass Review just invented an ad hoc exclusion/inclusion criteria, which is not true. Gideon M-K writes about the scale used (but points out that it did change from the one it pre-registered which GM-K pulls them up on saying there should be a clearly given rationale for it even though it is unlikely to have changed the outcome)...

I don't know if his series has been covered here, but as I said, I think he's usually quite good on these sorts of things and does offer some criticisms of the review...

The Cass Review Into Gender Identity Services for Children - Part 2

What the review got right

gidmk.substack.com

Novella repeated misinformation about it on the SGU podcast where there is no opportunity to respond directly (it appears any emails about it were ignored). It hasn't been addressed directly on SBM where it is possible for anyone to post comments. I know Gorski has also repeated misinformation on social media and he blocks anyone who points out errors.

 

[angrysoba]

I think it's a valid criticism of the Cass Review that they did not explain the change from using MMAT stated in their pre-registration to using NOS for some of the systematic evixence reviews. There are sometimes valid reasons to change protocol from pre-reg but it should be explained clearly. In this case, they used NOS for the studies of hormone treatments. All the studies in these reviews involved non-RCT quantitative designs, for which NOS is designed. They used MMAT for the review on psychosocial transition which included both qualitative, quantitative and mixed-method designs. MMAT is specifically for systematic reviews that contain both qualitative and quantitative and/or mixed methods studies, so in both cases the appraisal tool seems appropriate for the type of studies evaluated within each review. I don't think there is anything sinister about deciding to use NOS for the quantitative-only reviews; they probably just decided that there was no reason to use a tool intended for reviews with diverse study designs for reviews that had only quantitative studies. They may have intended originally to include all treatments in one review (which would then involve diverse designs) which would explain why the originally proposed only MMAT.
I have seen activists claim that the switch was due to MMAT recommending against excluding low quality studies (implying this was part of a sinister plot to brand all studies with positive results as low quality and exclude them). However, I have not been able find anything to support this and several papers discussing the MMAT list excluding low quality studies as an accepted use of the tool.

Yeah Gideon M-K himself does not see anything sinister in it either, and seems to think as you do, only that it would have been better off being explained...

At some point, the reviewers switched to the Newcastle-Ottawa Scale for rating the literature included in their reviews. In the reviews, there is no reason given for this that I can see, which is certainly not best practice. There’s a belief going around online that the reason that the reviewers switched the scales is because the MMAT recommends against excluding low quality work, while the NOS has no such recommendation.

As someone who does systematic reviews professionally, this argument makes no sense to me. All rating for bias is to some extent subjective. While both the MMAT and NOS attempt to create some measure of objective ranking for research, they are both ultimately up to the judgement of the reviewers who are using the tools. Changing your rating scale isn’t going to magically change the conclusions of a systematic review, especially when the review uses a narrative (i.e. subjective) synthesis method anyway.

In addition, as I noted above, including low quality studies in these reviews probably wouldn’t change much, because the low quality of the papers reduces their usefulness anyway. I very much doubt that anyone cared enough about the rating scale to switch it for nefarious reasons - the most likely explanation is that they didn’t find many qualitative studies in their searches.

 

[angrysoba]

Novella repeated misinformation about it on the SGU podcast where there is no opportunity to respond directly (it appears any emails about it were ignored). It hasn't been addressed directly on SBM where it is possible for anyone to post comments. I know Gorski has also repeated misinformation on social media and he blocks anyone who points out errors.

This is a shame since Novella has also been giving talks (such as at CSI-Con) where he says the important thing for skeptics to do is to share disagreements but has, as you say, appeared to avoid them except in venues where there cannot be a meaningful dialogue.

Then again, maybe he will respond on his blog, Neurologica...

A Discussion about Biological Sex - NeuroLogica Blog

At CSICON this year I gave talk about topics over which skeptics have and continue to disagree with each other. My core theme was that these are the topics we absolutely should be discussing with each other, especially at skeptical conferences. Nothing should be taboo or too controversial. We...

theness.com

 

[pipelineaudio]

Well, for proper skeptics, admitting you're wrong shouldn't be a problem.

When you are talking about SGU who actually circled the wagons around Rebecca Watson, good luck there

 

[d4m10n]

Then again, maybe he will respond on his blog, Neurologica...

Novella writes:

Coyne’s position is that we should categorize biological sex by gametes and gametes only. Why? He only said during his talk that this is how it is done, so an appeal to tradition, I guess. He seems to be engaged in a bit of circular logic – biological sex is binary because of gametes, and we use gametes to define biological sex because they are essentially binary (with rare exceptions).

I really don't get the sense Novella spent much time interrogating where Coyne is coming from here. The gamete binary exists in nature because of selection pressures which took place aeons ago, sexual dimorphism is layered atop of it. The argument for going back to gametes isn't about tradition, it's about why we are observing a binary in the first place. All the variables which measurably differ between male and female placental mammals go back to adaptations around the gamete binary which preexists class Mammalia in general and human tradition in particular.

Novella also writes:

Most people who identify as trans knew their gender identity from a very young age, and their identity is remarkably persistent over their lives.

This right here is sort of a "tell me you didn't read the Cass Report without telling me" moment; Cass takes pains to point out that a large and increasing fraction of patients at Tavistock no longer fit this description, since they present with gender issues for the first time around adolescence rather than at a "very young age" as in the original Dutch study.

 

[Elaedith]

A petition has been started calling for 'An Independent Evaluation of the Cass Review on Child Gender Services'. It states that 'We believe that trans healthcare should be based on unbiased research that is peer reviewed.' I wonder if they are aware that all the systematic evidence reviews conducted for the Cass Research programme by the University of York were peer reviewed before publication? It would be funny if they reach 10,000 signatures and get a government response pointing this out.

The petition did reach over 10,000 and the government has responded.

 

[Elaedith]

A new systematic review and meta-analysis from Canada supports the findings of Cass and other independent reviews regarding low certainty of evidence for puberty blockers. This used GRADE for evaluating evidence quality and modified ROBINS-I to assess risk of bias in comparative observational and before-after studies. It is also the first to use meta-analysis rather than narrative review. There is another one on hormone treatments that I haven't look at yet.

 

[Elaedith]

A second peer-reviewed critique of the critiques of Cass review has just been published. It focuses on the Yale Integrity Project paper and three other sources that were referenced within this, and lists the errors and misrepresentations in detail (also summarising the claims and rebuttals in Tables).

 

[Matthew Best]

A response to the New England Journal of Medicine's article about the Cass Review says:

"The foundational claim of the NEJM article is the following:

The Review calls for evidentiary standards for GAC [gender-affirming care] that are not applied elsewhere in pediatric medicine. Embracing RCTs [randomized-controlled trials] as the standard, it finds only 2 of 51 puberty-blocker and 1 of 53 hormone studies to be high-quality.

Unfortunately, this is all completely false....

In this short quote, the authors get the total number of studies wrong, the number of high quality studies wrong, the standard of evidence wrong, and the requirement for randomized-controlled trials wrong.

This is quite a lot of wrong in a short passage, but without this claim, much of the rest of the article falls apart."

As Jesse Singal says in his piece:

"Complete this sentence: “The New England Journal of Medicine won’t correct an objectively false claim it published, but I still trust it in general because _________.” I’m sorry, but there is no version of this sentence that would not mark you as a rube."

 

[Elaedith]

I would like to say I'm surprised by these lies being repeated in a peer-reviewed piece in the New England Journal of Medicine, but unfortunately this is not surprising anymore.
And nearly a year after the review was released, an open 'letter from academics' containing the lie that the review 'does not include a proper systematic literature review since it disregards most research evidence because it fails to reach the impossibly high bar of a double-blind trial' remains published without correction. Apparently there are no consequences for publicly lying about this.

 

[Matthew Best]

The New England Journal of Medicine has responded!

What they wrote, in case you've forgotten was this:

The Review calls for evidentiary standards for GAC [gender-affirming care] that are not applied elsewhere in pediatric medicine. Embracing RCTs [randomized-controlled trials] as the standard, it finds only 2 of 51 puberty-blocker and 1 of 53 hormone studies to be high-quality.

What they now say is this:

We do not say that the Cass Review calls for randomized controlled trials of puberty blockers. We note that the review applies a higher standard to gender affirming care than is applied to pediatric care in general, and that it “[e]mbrac[es] RCTs as the standard.” Both statements are correct. The Cass Review states that RCTs are the “gold standard.” At various points, the Review suggests deficiencies in evidence because RCTs have not been conducted.

This approach is arguably even more problematic than simply outright stating that RCTs are required (again, a claim we never make about the Review). It creates the impression that RCTs are a desirable standard in this context, without outright saying that is the case, thus allowing the authors to seek to avoid criticism when the matter is raised (as this exchange reflects).

Thus, many authors and readers have concluded that the Review demands RCTs.

Two Unsurprising Updates On The Ongoing Self-Immolation Of Expert Authority

Plus: an April event New Yorkers will want to come to

jessesingal.substack.com

Needless to say, Singal is not impressed.

 

[Emily's Cat]

<Grumble> That's not a whole lot better. Cass Review notes that RCT are generally considered the gold standard for research, but it also notes that in some cases RCT are not feasible nor ethical - it says this as part of the justification for being far more generous in the research it considered for review than would normally be accepted.

 

[Elaedith]

The New England Journal of Medicine has responded!

What they wrote, in case you've forgotten was this:



What they now say is this:

Two Unsurprising Updates On The Ongoing Self-Immolation Of Expert Authority

Plus: an April event New Yorkers will want to come to

jessesingal.substack.com

Needless to say, Singal is not impressed.

It's a ridiculous and disingenuous response.
When I read that original statement 'Embracing RCTs [randomized-controlled trials] as the standard, it finds only 2 of 51 puberty-blocker and 1 of 53 hormone studies to be high-quality.', I did notice that they did not directly say the studies were rated low quality because they were not RCTs, but clearly the statement implies that. I wondered if called out they would try to weasel out of it by pointing to the fact that Cass does in fact mention RCTs being the gold standard within the report. This suggests that the statement above is intentionally misleading and not just due to credulity and lack of fact checking.

Cass rightly refers to RCTs being the gold standard and the lack of RCTs within the report, which needs to be considered when assessing the overall quality of the evidence. This has nothing to do with the lack of high quality ratings for studies, which were rated on scale for non-RCT designs.

 

[d4m10n]

Out of curiosity, has anyone here ever been in an RCT as a subject?

I was, a while ago, back when Moderna was still in phase III trials for the COVID vaccine. There was a particular night when I came to believe (with something like > 95% subjective confidence) that I wasn't in the control group because I had several symptoms consistent with early flu or a vaccine reaction. Turns out I was correct, as it happens.

Anyhow, my point is that even with fairly mundane interventions, subjects will tend to notice when the effects really hit. With cross-sex hormones, these effects might be more subtle than vaccinations at first, then more and more undeniable later on.

 

[Elaedith]

My understanding is that lots of blinded RCTs become unblinded at some point due to side effects. However, Cass pointed out that RCTs can be conducted without blinding and still be preferable to non-randomised designs.

 

[zooterkin]

Out of curiosity, has anyone here ever been in an RCT as a subject?

I was, a while ago, back when Moderna was still in phase III trials for the COVID vaccine. There was a particular night when I came to believe (with something like > 95% subjective confidence) that I wasn't in the control group because I had several symptoms consistent with early flu or a vaccine reaction. Turns out I was correct, as it happens.

Anyhow, my point is that even with fairly mundane interventions, subjects will tend to notice when the effects really hit. With cross-sex hormones, these effects might be more subtle than vaccinations at first, then more and more undeniable later on.

I was in one supposedly looking at the effects of SSRIs on reward processing. I took something which may or may not have been citalopram for 7 days, then did some tests in an MRI scanner. I had no obvious effects from taking the tablets, so I don't know if they were placebo or I was just not affected by them.

I did ask if they could tell me after the test was complete which arm of the test I was on, as it might be useful to know in future, but I've heard nothing so far.

 

[d4m10n]

My understanding is that lots of blinded RCTs become unblinded at some point due to side effects. However, Cass pointed out that RCTs can be conducted without blinding and still be preferable to non-randomised designs.

Precisely this. The act of randomly assigning patients with a specific condition into groups which are not self-selected is part of what makes RCTs much more powerful than other forms of studies.