Guest Essay by Kip Hansen — 3 March 2024 — 2100 words
Omnia in mensura et numero et pondere disposuisti — Thou hast ordered all things in measure, and number, and weight.
– credited to Solomon’s Book of Wisdom
The basis of physical science is measurement. Measurement is just another word of quantification. Quantification is another word for counting.
In addition to quantification, science entails the qualification of things.
quantify means to find or calculate the quantity or amount of (something).
qualify means to characterize by naming an attribute, basically it means to state any property or characteristic of something. [ reference ]
Taxonomy is a qualitive science – it classifies life forms according to types, characteristics, etc.
The so-called “hard sciences” depend on quantification: measurement and counting. [ reference ]
There is little controversy about the importance of measurement and counting in the enterprise of the sciences, despite the occasional objections from philosophers.
In our modern Mass and Social Media world, numbers are presented to give a sense to “factualness” to ideas. It has been known that numbers have been used to tell lies probably since the beginning of the general use of numbers. “How to Lie with Statistics” by Darrell Huff was published in 1954 to explain this phenomenon and became a classic.
Forbes published an interesting piece by Christopher Kim in 2013 “6 Ways Numbers Can Lie To Us”. Kim’s list includes:
1. Small sample size: Drawing conclusions from small samples
2. Using Big Meaningless Numbers: 14,097,321 . . .
3. Correlation, not causation: Numbers stated in such as way as to imply causation, when they only show a correlation
4. Selection bias: using numbers imply that data came from a random sample when in actuality, the sample has been carefully (or carelessly) chosen
5. Visual trickery: Think Global Temperature graphs with a top-to-bottom range of 3 degrees, to make the increase look huge and alarming or this example:
6. Arbitrary cutoffs: “This is another form of selection bias. Setting arbitrary start-and-end points that impact the meaning of data.”
Great list, but certainly not exhaustive.
And the Biggest Omission? Failing to admit that Numbers are Just Numbers. Numbers are not the things they quantify. Sounds so obvious, doesn’t it? Of course, just telling you the number “687” isn’t useful or informative if I don’t also tell you “687 whats” – 687 apples, 687 inches of string — 687 degrees Celsius – 687 touchdowns.
Similarly, telling you “627 then 687!” has the same problem – nonsensical without the “whats” and “whens”.
And the #2 Biggest Omission? Failing to make a clear statement of exactly what was counted/measured and exactly how the counting/measurement was done. (this could and often does extend to exactly whens.)
In order to prevent the Biggest Omission – How many whats? – every number needs to be accompanied by (even if just implicitly) a clear statement of what has been counted and how it has been counted.
If we want any number to be considered scientific, the rules for this specification [“describing or identifying something precisely”] become stronger and stronger – we should consider this requirement paramount. In a scientific journal paper, this information is sketched out in the ‘Methods’ section and hopefully is more fully specified in the Supplemental Materials.
When this is not done properly, we end up with numbers and graphs like those we see in the Climate Change field: Global Sea Level Rise and Global Surface Temperature. The general public, encouraged by the activists, climate change crisis advocates and complicit journalists, are led to believe that these “numbers” are something real and are, in fact, the thing they are labelled: that the numbers on the graphs are something that could be found in the physical world. This is not true – and I have exhausted myself explaining this here in the past.
Today I will share an example that has created a controversy in the field of medical statistics. An unlikely topic for discussion here at WUWT, but it is a near perfect example and will avoid all the food-fighting over Climate Change.
Maternal Mortality Rate
The topic pops up in the journal Science in an article that is the push-back to another study: “Have U.S. deaths from pregnancy complications tripled? CDC pushes back on study claiming overestimates”
The media has been reporting that maternal deaths [Maternal Mortality Rates] have “spiked”, “climbed dramatically”, “are getting worse”, and that we have an “unacceptably high U.S. maternal mortality rate”.
These stories are reporting the CDC’s announcement, released in March 2023, titled “Maternal Mortality Rates in the United States, 2021”. [ or as a .pdf here ]
The report starts with this:
“A maternal death is defined by the World Health Organization as “the death of a woman while pregnant or within 42 days of termination of pregnancy, irrespective of the duration and the site of the pregnancy, from any cause related to or aggravated by the pregnancy or its management, but not from accidental or incidental causes. Maternal mortality rates, which are the number of maternal deaths per 100,000 live births, are shown in this report by age group and race and Hispanic origin.”
The news carried by the media is based on the simple statement:
“In 2021, 1,205 women died of maternal causes in the United States compared with 861 in 2020 and 754 in 2019 (2). The maternal mortality rate for 2021 was 32.9 deaths per 100,000 live births, compared with a rate of 23.8 in 2020 and 20.1 in 2019.” And this graph:
Clearly, as shown the total U.S. MMR [Maternal Mortality Rate] nearly doubled from 2018 through 2021.
That is a shocking statistic. To round out the picture of MMR, here’s two international views:
Our World In Data supplied the above charts on MMR around the world. The little inset in the left panel shows the slight uptrend in the US MMR over the period reported by the CDC. The good news, that MMRs have dramatically fallen, almost everywhere, to near zero since 1950 (invention of antibitoics, I suggest) is not mentioned. But the barely visible uptick in U.S. MMR is shouted from the rooftops and front pages.
The reported increase in U.S. MMR was so shocking that a group of maternal health researchers, headed by K. S. Joseph, School of Population and Public Health, University of British Columbia, decided to re-evaluate the CDC data. Earlier this month, on 12 March 2024, their paper was published in the American Journal of Obstetrics and Gynecology and titled: ”Maternal mortality in the United States: are the high and rising rates due to changes in obstetrical factors, maternal medical conditions, or maternal mortality surveillance?” [The study is Open Access and available for download as a .pdf from this page ]
Their title gives away their suspicions:
“National Vital Statistics System reports show that maternal mortality rates in the United States have nearly doubled, from 17.4 in 2018 to 32.9 per 100,000 live births in 2021. However, these high and rising rates could reflect issues unrelated to obstetrical factors, such as changes in maternal medical conditions or maternal mortality surveillance (eg, due to introduction of the pregnancy checkbox).”
The big story is this: “But controversy broke out last week over just how bad the situation is, when a paper by academic epidemiologists published in the American Journal of Obstetrics and Gynecology (AJOG) provoked unusual pushback from the U.S. Centers for Disease Control and Prevention (CDC). The paper suggested a widely reported tripling in the U.S. maternal mortality rate (MMR) over the past 2 decades was in fact largely due to a CDC-led recording change on death certificates, the addition of a “pregnancy checkbox.””
“The “pregnancy checkbox” was inserted on death certificates starting in 2003 to address what was at the time a widely acknowledged, substantial underreporting of maternal mortality: At the time, as many as 50% of physicians completing death certificates failed to report that a woman was, or was recently, pregnant. On death certificates, physicians now are asked to check a box indicating a person was pregnant when they died, or within 42 days of the end of the pregnancy. Doctors are not to check the box if a person died of accidental or incidental causes unrelated to pregnancy, for instance, in a car crash or from a gunshot wound. Although the agency rolled out the feature in 2003, it took 14 years before all 50 states adopted the surveillance tool. After that happened in 2017, the agency began to compute the nationwide rate using the checkbox.” [ as reported in Science]
Now we see the issue here. There was a change in what was being counted. When did this change? 2017. When did MMR start “skyrocketing”? At the end of 2017 (years 2018 onward). Once all 50 U.S. states had a checkbox covering possible pregnancy, the CDC started using the checkboxes (counting checkboxes as opposed to counting maternal deaths) to determine Maternity Mortality Rate.
The Joseph et al. study concludes:
“The high and rising rates of maternal mortality in the United States are a consequence of changes in maternal mortality surveillance, with reliance on the pregnancy checkbox leading to an increase in misclassified maternal deaths. Identifying maternal deaths by requiring mention of pregnancy among the multiple causes of death shows lower, stable maternal mortality rates and declines in maternal deaths from direct obstetrical causes.”
The Joseph study looked at death certificates and only counted deaths as Maternal Mortality if the death certificate actually listed pregnancy as one of the contributing causes of death. “Cause of Death” is seldom simple short of something as obvious as a bullet to the head — I covered this during the Covid days in Cause of Death: A Primer – that essay has this image of a death certificate (which also shows the pregnancy checkbox, labelled “If Female”, in the center):
Joseph et al. maintain that the death be counted as a Maternal Mortality only if pregnancy or a related issue of childbirth is specifically mentioned in Parts I or II. When the counting is re-done that way, Joseph et al. found “stable maternal mortality rates and declines in maternal deaths from direct obstetrical causes.”
It is that conclusion that has resulted in a broadside attack on Joseph et al. from the other stakeholders in maternal health, including the CDC itself [quoted] and the American College of Obstetricians and Gynecologists (ACOG). Many mass media outlets covered the story slamming Joseph et al. as threatening to “… reduce the U.S. maternal mortality crisis to an ‘overestimation’ is irresponsible and minimizes the many lives lost and the families that have been deeply affected.” [ source ]
This counting controversy isn’t restricted to the Joseph et al. paper. The CDC’s very own National Center for Health Statistics (NCHS) in a report dated January 30, 2020, had previously reached the exact same conclusions as Joseph: “NCHS found that the increase in maternal mortality in the United States is not likely due to a true increase in the underlying extent of maternal mortality. Rather, the majority of the observed increase in the MMR is attributed to changes in data collection methods (i.e., the gradual adoption of the checkbox). Based on the pre-2003 coding method, the MMR was 8.9 in 2002 and 8.7 in 2018.”
Bottom Lines:
1. Nothing about this controversy changes the real-world number of women who died. Those women died, from whatever cause. Their families, their children, their husbands, their parents suffered their loss.
2. But the over-count has made a big difference in health politics. If Maternal Health stakeholders can point to alarming statistics and create a National Health Crisis from them, more sympathy and money will pour into their cause. More attention and money may actually be a good thing if it leads to more research and actions that can reduce maternal deaths.
3. However, it is never a good thing to create a crisis out of the miscounting, mis-measurement, and mis-reporting of numerical facts.
4. The numbers you see reported (and hyped) in the media are probably false, mis-counted, mis-measured, mis-labelled and do not factually represent the thing they claim to show. [follows from John P. A. Ioannidis, “Why Most Published Research Findings Are False”.]
5. Always, if it is important to you, carefully dig in to find out exactly what was counted/measured and exactly how it was counted or measured.
# # # # #
Author’s Comment:
I shouldn’t have to point out the parallels between this controversy and Climate Science. Alarming numbers are created, labelled as something shocking, public is alarmed and politicians react. Cooler heads reexamine the numbers and point out that “it isn’t really that bad”. Cooler heads are attacked and vilified (even though the official IPCC science agrees with them).
We can all be fooled by numbers – this seems to be a human trait. Personally, I think it stems from a deep innumeracy. This tendency can be overcome with doing due diligence and applying critical thinking skills.
At minimum, we need to ask: What exactly did they actually count? Exactly how did they count it? Does the number really represent the thing (idea, physical fact, actuality) they say it does?
Thanks for reading.
# # # # #
Related