Unscientific Method
Unscientific Method
An astronomer’s peer-reviewed work is passed under the “equity” lens and found wanting.
Another day, another retraction of a scientific paper for violating the code of diversity. On November 1, astronomer John Kormendy withdrew an article from the Proceedings of the National Academy of Sciences (PNAS), after a preprint version that he had just posted on the web drew sharp criticism for threatening the conduct of “inclusive” science. Three days later, the preprint version was scrubbed as well (though a PDF can still be found here.) The paper had passed the journal’s three-person peer-review system and was awaiting publication. Kormendy’s forthcoming book on the same topic had also passed peer review and had been printed for distribution. Now distribution of the book has been put on hold, likely permanently.
Kormendy, an expert on supermassive black holes and professor emeritus at the University of Texas at Austin, acknowledges no errors in his research. “I didn’t do anything [methodologically] wrong,” he told me. “I trust my techniques; I trust the results. I checked for bias in great detail.” Nevertheless, he issued an apology on November 1: “I now see that my work has hurt people. I apologize to you all for the stress and the pain that I have caused. Nothing could be further from my hopes. I fully support all efforts to promote fairness, inclusivity, and a nurturing environment for all.”
What was so hurtful in his article? Kormendy had aimed to reduce the role of individual subjectivity in scientific hiring and tenure decisions. He created a model that predicted a scientist’s long-term research impact from the citation history of his early publications. He tested the results of his model against a panel of 22 prestigious astronomers, many of whom had advised the federal government on scientific research priorities and had served as jurors on high-profile astronomy prizes. That panel rated the research impact of the 512 astronomers whom Kormendy had run through his model; the panel’s conclusions closely matched the model’s results. Kormendy’s paper stressed that hiring decisions should be made “holistically.” Scientific influence was only one factor to consider; achieving gender and racial balance in a department was also a legitimate concern, he wrote.
Formulas for quantifying scientific influence on the basis of a citation record are hardly new. PNAS itself published the proposal for one such well-known measure, known as the “h-index.” But that was in 2005. In 2021, a different standard for evaluating ideas applies: Do they help or hinder females and underrepresented minorities in STEM? Kormendy’s model, tweeted an astrophysicist at the City University of New York, “JUST TOOK ANY TINY STEPS WE ARE MAKING TOWARDS EQUITY AND THREW THEM OUT OF THE WINDOW” (capitalization in the original). An astronomer in Budapest objected that Kormendy had failed to consult with “relevant humanities experts” about cumulative bias against females and minorities. Equally damningly, Kormendy had suggested that the profession should overcome its underrepresentation problem by hiring female and minority scientists, who, in the words of the Budapest astronomer, “match the success rate of the majority (i.e., men).”
After Kormendy withdrew the paper, a University of Texas colleague tweeted of her hopes that the work of doing “science inclusively” could now continue. Others directed pot shots at the panel of 22 raters for being a “bunch of old people” from Western universities who were not representative of the “astronomy community.” But that non-representation was exactly the point—scientific expertise is not democratic. These were scientists at the top of their field, whose accomplishments would in earlier times have been a source of authority.
Naturally, the fact that 19 of the panelists were men was a red flag. But Kormendy had tried to get more female raters; they turned down his offer to join the project in higher proportion than the males he solicited. (The three female panelists rated female astronomers higher than the male panelists did. Kormendy’s attribution of this discrepancy to bias on the part of the males won him no credit.)
None of the paper’s critics spelled out how publication metrics (known as “bibliometrics”) conflict with equity. Many have rebuffed or ignored attempts to seek clarification. Presumably, the critics intuit (correctly) that quantitative measures of scientific influence will show that white males have had the greatest impact on science to date. That finding would not be inequitable on its face, however, unless we define equity as equality of outcome.
The paper’s methodology came under desultory challenge as well. Bryan Gaensler, an astronomy professor at the University of Toronto, told Inside Higher Ed: “I don’t think the premise that motivated this work is correct, and I don’t think the actual work done should have passed peer review.” The work did pass peer review, however, and no one has claimed that the oversight process was manipulated. The solution to disagreements over premises or method is to publish a rebuttal, not to disappear the allegedly incorrect paper, absent a showing of fraud or belatedly discovered errors so great as to undercut the entire enterprise.
Even some of Kormendy’s rating panelists issued apologies after the fact for their participation. Brian Schmidt, chancellor of the Australian National University and a Nobel prize-winner, wrote on Twitter: “As an unintended consequence Of this article, I hope our field can be more Reflective of our hiring practices, and the inequitable gatekeeping that occurs into astronomy to this day. I am sorry for my involvement” [capitalization in the original].
The Kormendy retraction is now the fifth in recent years cancelling a scientific paper deemed to bear negatively on equity in STEM. Previous cancellations include a mathematical model to explain why evolution would select for greater variability in inherited traits among males of a species and an empirical study comparing the benefits of male and female mentorship in STEM (male mentorship proved more advantageous). The authors of the latter retracted article expressed “deep regret” for having “caused pain.”
And now, in addition to the inhibitions on publishing, the cancellation machine is explicitly wiping out judgments of scientific merit if they fail to meet a diversity quota. In October, a few days before the Kormendy retraction, a committee that awards fellowships for the American Geophysical Union cancelled the slate of finalists that peer scientists had forwarded to it because the three finalists were all white men. Better not to award a fellowship at all than to give it to a white male. The leader in the cancellation effort admitted that the finalists, who specialize in the study of snow and ice, were “truly, amazingly deserving.” But the cancellation would result in a “fairer process,” she told E & E News.
The cancelling committee presented no evidence of unfairness in the nomination process, apart from the unacceptable result. Indeed, the entire American Geophysical Union fellowship process was decidedly pro-female: female finalists overall had a nearly 50 percent greater chance of being selected for a fellowship than male finalists. That disparity is not regarded as unfair, just as the higher ratings given to female scientists by female raters in the Kormendy study were not regarded as biased.
From here on, no STEM job or honor awarded to a female or an underrepresented minority will be free from the justified suspicion that the selection was the result of “equity” concerns. The pressure on STEM laboratories and academic faculties to hire by quota rather than by scientific merit grows by the day. And the judgment of scientific research now hinges not on its validity but on whether it allegedly causes “hurt” or impedes the achievement of proportional representation in STEM studies.
The “only thing on anyone’s mind now is redressing inequities,” Kormendy told me, adding that he supports that “honorable” aim. But science is not about social justice. It’s about the advancement of knowledge via the free exchange of ideas and the careful testing of results. Step by step, we are shutting down the very processes of open inquiry and the cultivation of excellence that have freed humanity from so much unnecessary suffering.
Post a Comment