Health for life

Measuring ROI

In yesterdays posting on Al Lewis book Why No One Believes the Numbers, the Disease Management Care Blog pointed out that the measurement of population health management (PHM) outcomes remains an inexact and still evolving science. While that can be a source of endless fascination for the DMCB, the inability of the industry to rustle up credible "return on investment" numbers has prompted some observers to condemn PHM as a waste of money.

The search for simple answers explains much of the appeal of this book.

According to author, one important solution is the "dummy year analysis" (DYA). This relies on repeated year-over-year measurements of utilization that use multiple comparison pairings of all patients with the condition of interest. When thats combined with a "plausibility" check list, Mr. Lewis says purchasers of the Patient Centered Medical Home (PCMH), disease management or wellness programs should be able to get a better fix on whether they saved any money. You can a sense of that perspective here.

The DMCB isnt too sure about that because a) other factors that have nothing to do with population health management can also impact utilization during and after the dummy years, making it difficult to assign an attributable ROI and b) entire health plan populations can likewise regress toward a regional or national mean.

The DMCB also sees three additional reasons why there may be less to this books methodology than meets the eye:

1. When employers, health plans, accountable care organizations or other buyers have a list of names that have been through a care program, they typically want to understand the outcomes for the individuals on that list. If thats the case, the challenge is to find an adequate comparator that portrays what would have happened in the absence of the care program. Multiple options for identifying a parallel comparator have been used in published science for decades. Thats difficult, imperfect, but not broken.  It remains an option.

2. While the book is replete with examples of "actuaries behaving badly," it is impossible to underestimate the influence of actuarial science and trending on premium rate setting, statutory accounting, and the regulation of insurance. As a result, if the actuaries say money is - or is not - being saved, health system leaders ignore their insights at their peril.

3. Isolating the impact of PCMH, disease management or wellness program out of all the other "noise" of a changing economy, evolving consumerism, benefit changes, electronic health record databases, medical advances, inflation and the news media is a function of an increasingly sophisticated and changing statistical sciences and computational technology. Its ironic, but one outcome has been a better description and measurement of the uncertainty surrounding a result.
 
To the authors credit, Why No One Believes the Numbers is not being promoted as the single best methodology that will lead PCMH, disease management and wellness programs to outcomes certainty. Rather, it is one option among many in asking whether a program had any financial impact.

Ultimately, theore, thats why the DMCB advises that measuring outcomes in PHM - absent an ironclad methodology - comes down to using multiple approaches to triangulate on the truth. After reading Why No One Believes the Numbers, some readers may choose it as one of those approaches.

Changing opinion from right to left

Has marriage made the Disease Management Care Blog a better person?

While it and the spouse heartily agree on the ultimate answer, using hard numbers to prove it to a skeptical mother-in-law is a different matter. To definitively answer the question, the affable DMCB came up with some proposed measurement approaches, such as:

1. Pre vs. Post: comparing past beer guzzling to present-day chardonnay sipping;

2. Actuarial: actual vs. projected appreciation for the leather-clad vampire vixens of Underworld;

3. Comparison to a Control: the DMCBs willingness to take direction on shrubbery trimming, versus that of more docile hubbies;

4. Randomized and Prospective: (the DMCB has wisely opted to not go there).

While the DMCB continues to work on the complex methodology of marital outcomes, it is reminded of a key paradox: while we live in an "Information Age," other pressing questions - such as the extent of the Eurozones influence on GDP, the merits of vouchers in public education and the link between the Presidents approval rating and his governing by remote memo - likewise defy conclusive measurement.

Whats more, frustrated by our worlds complexity, we ironically want fewer answers. The DMCB suggests this search for simplicity partially explains the luster of a balanced budget amendment, laws on minimum medical loss ratios, Newts bombast, Obamas rhetoric, blanket coverage of birth control and, last but not least, single approaches to assessment of population-based programs.

Which brings the DMCB to Al Lewis $10,000 challenge, in which he dares anyone to come up with a more accurate approach to measuring disease management return on investment. 

Al is a luminary in the disease management firmament who leads the Disease Management Purchasing Consortium. He was there at the founding of the Disease Management Association of America, led the attack on the vendors past lazy outcomes reporting and has been instrumental in questioning the conclusions about North Carolinas Medical Home Program. He now claims to offer the only approach to accurately measuring the financial impact of disease and population health management.

Maybe, but the DMCB would like to humbly offer up an alternate perspective.

Check out this DMCB paper that simultaneously deployed three uncomplicated methodologies to assess the claims expense impact of a chronic heart failure disease management program. While all three gave different answers, they all pointed in the same direction.

That was enough for the DMCB boss to continue the programs funding.

This same overlapping and multi-layed approach also underlies the Care Continuum Alliances Outcomes Guidelines Reports, which recommend a suite of measurement approaches that pivot on important determinants such as population characteristics, the influence of confounders as well as bias and the resources available to answer the question.

None of this should be any surprise to seasoned and prudent health administrators, physician leaders, clinical program architects or DMCB readers. They know that good actuaries use complimentary and overlapping approaches to come up with the right premium. They understand that good medical researchers demand caution, skepticism and multiple research studies before reaching any conclusions.

In other words, there is no one-time and one-size fits all approach in outcomes assessment.

All this adds up to the fact that optimum outcomes measurement triangulates on the truth. The measurement approaches advocated by the DMCB, Care Continuum Alliance members and health system leaders have been around for years, are within the reach of standard statistical software, are familiar to researchers and are highly adaptable to the circumstances of 99.99% of disease management programs, not to mention the medical home and accountable care organizations.

The DMCB says that when multiple, competing, overlapping, repeated and adaptable measurement methodologies point in the same direction, thats when regulators, consumers, purchasers, buyers, providers and patients and mothers-in-law can be really confident that they have the answer they need.

Image from Wikipedia

But wheres the money?

The Disease Management Care Blog has always been leery of the "return on investment" (ROI) metric in health care. It knows that there are precious few health care interventions that actually "save" money. Many prevention, wellness and disease management programs     - depending on the analysis -  can cost money since they a) add additional resources to a system with already high fixed costs with b) a short one-year time horizon.

Yet, the good news is even if a program isnt successful in slowing the rate of cost inflation (or "bending the curve," which represents the savings), it can still represent a great value.  Thats because the additional benefit represents significant benefit for each additional dollar of spending.

Thats the message in this recent JAMA Viewpoint editorial Assessing Value in Health Care Programs authored by Kevin Volpp, George Loewenstein and David Asch. They offer up a thought experiment. Consider, they say, a state-of-the-art medication compliance campaign for heart attack victims that avoids a number of costly hospitalizations.  The price tag at $2000 has a positive "ROI" because the investment is less than the avoided cost of the hospitalizations.  However, if the price tag is $3000 and the investment is now greater than the cost of the hospitalizations, the ROI is "negative" even though the same number of patients didnt have to be hospitalized.

The DMCB recommends readers keep this manuscript/link handy the next time some Finance weenie demands an "ROI calculation."

Speaking of readers, the DMCB is happy to announce that it just hit 500 Twitter followers.  Thats in addition to more than 500 "RSS" subscribers, 461 Google Reader subscribers and thousands of return visitors per month.  The DMCB knows each was earned one person at a time.