CDC’s epidemic predictor makes estimates, courts controversy

ATLANTA – Last fall, when Martin Meltzer calculated that 1.4 million people might contract Ebola in West Africa, the world paid attention.

This was, he said, a worst-case scenario. But Meltzer is the most famous disease modeler for the nation’s pre-eminent public health agency, the Centers for Disease Control and Prevention. His estimate was promoted at high-level international meetings. It rallied nations to step up their efforts to fight the disease.

But the estimate proved to be off.

Way, way off.

Like, 65 times worse than what ended up happening.

Some were not surprised. Meltzer has a lot of critics who say he and his CDC colleagues have a habit of willfully ignoring the complexities of disease outbreaks, resulting in estimates that over-dramatize how bad an outbreak could get – estimates that may be skewed by politics. They say Meltzer and company also overestimate how much vaccine is needed and how beneficial it has been.

Overblown estimates can result in unnecessary government spending, they say, and may further erode trust in an agency that recently has seen its sterling reputation decline.

“Once we cry wolf, and our dire predictions turn out not to be the case, people lose confidence in public health,” said Aaron King, a University of Michigan researcher who in a recent journal article took Meltzer and others to task for making what he called avoidable mistakes.

Meltzer, 56, is unbowed. “I am not sorry,” he said.

He dismisses his peers’ more complicated calculations as out of touch with political necessities, telling a story about President Lyndon Johnson in the 1960s. Johnson was listening to an economist talk about the uncertainty in his forecast and the reason a range of estimates made more sense than one specific figure. Johnson was unconvinced.

“Ranges are for cattle,” Johnson said, according to legend. “Give me a number.”

What Meltzer does is not particularly glamorous. He and others use mathematical calculations to try to provide a more precise picture of a certain situation, or to predict how the situation will change. They write equations on chalkboards, have small meetings to debate which data to use, and sit at computers. Meltzer spends a lot of time with Excel spreadsheets.

But modelers have become critical in the world of infectious diseases.

Top CDC officials came to Meltzer last summer, when the epidemic was spiraling out of control and international health officials were quickly trying to build a response. Meltzer was asked to project how bad things could get if nothing was done, as well as to estimate how stepped-up aid could bend the curve of the epidemic.

Meltzer and his colleagues created a spreadsheet tool that projected uninterrupted exponential growth in two countries, Liberia and Sierra Leone.

His prediction – published last September – warned that West Africa could be on track to see 500,000 to 1.4 million Ebola cases within a few months if the world sat on its hands and let the epidemic blaze.

About 21,000 cases materialized by mid-January – a terrible toll, to be sure, but also just a tiny fraction of the caseload Meltzer and his CDC colleagues warned about.

Did Meltzer blow it? Many say no. He and his colleagues clearly stated they were providing a worst-case scenario of how bad things could get. They also predicted a far lower number of cases if more help was sent – which already was happening when the model estimates were released.

But the worst-case figures got the most attention. The media focused on them in headlines. Health officials highlighted them in their push to get more money and manpower devoted to the epidemic. And interestingly, those are the numbers health officials describe as the most successful part of Meltzer’s prediction paper.

“I think it galvanized countries – and people – to put in more effort” into fighting the epidemic, said Dr. Keiji Fukuda, formerly a colleague of Meltzer’s at CDC who is now assistant director-general of the World Health Organization.

CDC is supposed to prepare the America for the worst, so it makes sense for CDC modelers to explore extreme scenarios. If Meltzer’s estimates push policymakers to bolster public health defenses, it’s all to the greater good, some say.

But there are others who feel that the result corrupts both science and politics.

“Public health officials are well aware that their statistics get used – and misused – to justify an increase in their funding” or to bolster vaccination campaigns and other efforts, said Peter Doshi, assistant professor at the University of Maryland School of Pharmacy.

Modeling – so poorly understood by the public, the media, and even many people in public health – provides an opportunity to bend numbers to support goals, he argued.

Said David Ozonoff, a Boston University environmental health professor: “The way risk assessment is done in this country is the policy makers shoot the arrow and the risk assessors paint a target around it. There’s a flavor of this with modeling, too. If you say the purpose (of a modeling estimate) is motivational, that’s another way of saying it’s not scientific.”

Some say more of a separation between CDC administrators and the modelers might engender more trust in the numbers the agency uses. Perhaps an outside agency – an NIH institute on public health, if one were ever created – could do the modeling and report their findings to CDC, said Lone Simonsen, a research professor at George Washington University who formerly worked at the CDC and at the National Institutes of Health.

More immediately, CDC could increase its collaboration with top academic modelers, she added.

Meltzer is not interested. He is wary of proposals for greater collaboration or reliance on non-agency modelers, especially during emergency situations. And more sophisticated models do not interest him.

“Accuracy for the sake of accuracy is merely interesting,” he said. “And interesting is not good enough.”