A new study conducted at the University of Washington concludes that the Montana Meth Project, having spent millions of dollars on a now-famous advertising campaign, “has had no discernable impact on meth use.”
The study was conducted by Lewistown native D. Mark Anderson and was published in the September issue of the Journal of Health Economics. Anderson is a UW doctoral student in economics who said that to date, the Meth Project “has not been empirically scrutinized in a rigorous fashion.”
His conclusion, based on a study of some of the same data used by the Montana Meth Project, is that after accounting for a downward trend in meth use that was already under way before the project was launched in 2005, “effects on meth use are statistically indistinguishable from zero.”
“From a policy perspective,” Anderson said in the study’s conclusion, “this research is important because it highlights the ineffectiveness of a campaign that is widely regarded as successful.”
Bill Slaughter, executive director of the Montana Meth Project, issued a rebuttal in which he said Anderson’s con-clusions were “unfounded” and that the study ignored other statistics that would have given “a more complete picture of impact.”
Slaughter criticized Anderson for using “percentage point declines” rather than comparing the rates of decline in usage between Montana and other states. Slaughter said the rate of decline in meth use in Montana went from 39 percent between 1999 and 2005, when the Meth Project was launched, to 63 percent from 2005 to 2009.
Although the rates of decline were similar in North Dakota and Wyoming during the earlier period, he said, their rates of decline were only 37 percent and 42 percent, respectively, from 2005 to 2009.
Anderson said Slaughter is simply wrong in saying he didn’t use rates of decline, and he questioned whether Slaugh-ter had read the study.
“The whole study is based on comparing rates of change over time compared to other states,” Anderson said.
Slaughter said the fact remains that usage rates declined much more steeply in the years after the Meth Project be-gan. He also said Anderson “disregarded all this other third-party data.”
Among them, Slaughter said, were reports showing that meth-related crime dropped 63 percent in Montana between 2005 and 2007, having increased sharply in the years leading up to 2005. There were also big drops in the number of children in foster care because their parents were using methamphetamine, he said.
Anderson’s study was mostly based on data from Youth Risk Behavior Surveys, conducted nationally by the Centers for Disease Control and on the state level by education and health agencies. It measures all sorts of risky behaviors by teenagers.
Anderson said he chose to use YRBS data because the Meth Project makes heavy use of the same numbers and because the YRBS is filled out by teens, the same audience targeted by the project’s gritty, graphic ad campaign.
The Montana Meth Project was founded by software billionaire Tom Seibel, a part-time Montana resident who started the project with his own money. According to Anderson, the Thomas and Stacey Siebel Foundation invested more than $25 million in starting the program between 2005 and 2007.
The project was given $2 million in funding by the Montana Legislature in 2007, followed by nearly $1.5 million from the federal government later that year. Since 2007, the project has been expanded into Arizona, Idaho, Illinois, Wyoming, Colorado, Hawaii and Georgia.
Between 2005 and 2007, the Montana Meth Project saturated the state with 45,000 television ads, 35,000 radio ads, 10,000 print impressions and 1,000 billboards. The ads starkly portrayed the consequences of meth use, depicting rape, robbery, prostitution and various disfigurements, including rotten teeth, severe weight loss, scars and scabs.
Anderson said he became interested in the Meth Project during a visit to Billings in 2009. He read a story about the success of the program, and at first he was impressed, he said.
As he points out in a footnote in his study, the annual operating budget of the project was only $2.5 million, while the economic costs of meth in Montana had been estimated at $200 million to $300 million a year.
If the success rate claimed by the project was valid, Anderson said, the cost-effectiveness would be appealing to people making public policy.
But he also read about another study of the Meth Project, this one by David Erceg-Hurn, a doctoral student in clinical psychology at the University of Western Australia. In his study, published in Prevention Science, Erceg-Hurn concluded that “there is no compelling evidence that teenage meth use has declined as a result of the ads.”
Prevention Science, like the Journal of Health Economics, is a peer-reviewed, professional journal.
Anderson said Erceg-Hurn also criticized the Meth Project for failing to account for pre-existing trends in meth use, but didn’t use “any type of formal statistical analysis to support his criticisms.”
In addition to examining YRBS statistics, Anderson looked at drug treatment admission reports from the U.S. Department of Health and Human Services.
Those reports confirmed the analysis of the YRBS results, he said, showing that the Meth Project “had no discernible impact on meth use.”
Slaughter said there is just too much other evidence to ignore that testifies to the success of the project, and it was unfair of Anderson to say the whole project was a bust after looking at a couple of sets of data.
“I think that’s painting it with a really broad brush,” he said.