Health for life

Measuring ROI

In yesterdays posting on Al Lewis book Why No One Believes the Numbers, the Disease Management Care Blog pointed out that the measurement of population health management (PHM) outcomes remains an inexact and still evolving science. While that can be a source of endless fascination for the DMCB, the inability of the industry to rustle up credible "return on investment" numbers has prompted some observers to condemn PHM as a waste of money.

The search for simple answers explains much of the appeal of this book.

According to author, one important solution is the "dummy year analysis" (DYA). This relies on repeated year-over-year measurements of utilization that use multiple comparison pairings of all patients with the condition of interest. When thats combined with a "plausibility" check list, Mr. Lewis says purchasers of the Patient Centered Medical Home (PCMH), disease management or wellness programs should be able to get a better fix on whether they saved any money. You can a sense of that perspective here.

The DMCB isnt too sure about that because a) other factors that have nothing to do with population health management can also impact utilization during and after the dummy years, making it difficult to assign an attributable ROI and b) entire health plan populations can likewise regress toward a regional or national mean.

The DMCB also sees three additional reasons why there may be less to this books methodology than meets the eye:

1. When employers, health plans, accountable care organizations or other buyers have a list of names that have been through a care program, they typically want to understand the outcomes for the individuals on that list. If thats the case, the challenge is to find an adequate comparator that portrays what would have happened in the absence of the care program. Multiple options for identifying a parallel comparator have been used in published science for decades. Thats difficult, imperfect, but not broken.  It remains an option.

2. While the book is replete with examples of "actuaries behaving badly," it is impossible to underestimate the influence of actuarial science and trending on premium rate setting, statutory accounting, and the regulation of insurance. As a result, if the actuaries say money is - or is not - being saved, health system leaders ignore their insights at their peril.

3. Isolating the impact of PCMH, disease management or wellness program out of all the other "noise" of a changing economy, evolving consumerism, benefit changes, electronic health record databases, medical advances, inflation and the news media is a function of an increasingly sophisticated and changing statistical sciences and computational technology. Its ironic, but one outcome has been a better description and measurement of the uncertainty surrounding a result.
 
To the authors credit, Why No One Believes the Numbers is not being promoted as the single best methodology that will lead PCMH, disease management and wellness programs to outcomes certainty. Rather, it is one option among many in asking whether a program had any financial impact.

Ultimately, theore, thats why the DMCB advises that measuring outcomes in PHM - absent an ironclad methodology - comes down to using multiple approaches to triangulate on the truth. After reading Why No One Believes the Numbers, some readers may choose it as one of those approaches.

Changing opinion from right to left

Has marriage made the Disease Management Care Blog a better person?

While it and the spouse heartily agree on the ultimate answer, using hard numbers to prove it to a skeptical mother-in-law is a different matter. To definitively answer the question, the affable DMCB came up with some proposed measurement approaches, such as:

1. Pre vs. Post: comparing past beer guzzling to present-day chardonnay sipping;

2. Actuarial: actual vs. projected appreciation for the leather-clad vampire vixens of Underworld;

3. Comparison to a Control: the DMCBs willingness to take direction on shrubbery trimming, versus that of more docile hubbies;

4. Randomized and Prospective: (the DMCB has wisely opted to not go there).

While the DMCB continues to work on the complex methodology of marital outcomes, it is reminded of a key paradox: while we live in an "Information Age," other pressing questions - such as the extent of the Eurozones influence on GDP, the merits of vouchers in public education and the link between the Presidents approval rating and his governing by remote memo - likewise defy conclusive measurement.

Whats more, frustrated by our worlds complexity, we ironically want fewer answers. The DMCB suggests this search for simplicity partially explains the luster of a balanced budget amendment, laws on minimum medical loss ratios, Newts bombast, Obamas rhetoric, blanket coverage of birth control and, last but not least, single approaches to assessment of population-based programs.

Which brings the DMCB to Al Lewis $10,000 challenge, in which he dares anyone to come up with a more accurate approach to measuring disease management return on investment. 

Al is a luminary in the disease management firmament who leads the Disease Management Purchasing Consortium. He was there at the founding of the Disease Management Association of America, led the attack on the vendors past lazy outcomes reporting and has been instrumental in questioning the conclusions about North Carolinas Medical Home Program. He now claims to offer the only approach to accurately measuring the financial impact of disease and population health management.

Maybe, but the DMCB would like to humbly offer up an alternate perspective.

Check out this DMCB paper that simultaneously deployed three uncomplicated methodologies to assess the claims expense impact of a chronic heart failure disease management program. While all three gave different answers, they all pointed in the same direction.

That was enough for the DMCB boss to continue the programs funding.

This same overlapping and multi-layed approach also underlies the Care Continuum Alliances Outcomes Guidelines Reports, which recommend a suite of measurement approaches that pivot on important determinants such as population characteristics, the influence of confounders as well as bias and the resources available to answer the question.

None of this should be any surprise to seasoned and prudent health administrators, physician leaders, clinical program architects or DMCB readers. They know that good actuaries use complimentary and overlapping approaches to come up with the right premium. They understand that good medical researchers demand caution, skepticism and multiple research studies before reaching any conclusions.

In other words, there is no one-time and one-size fits all approach in outcomes assessment.

All this adds up to the fact that optimum outcomes measurement triangulates on the truth. The measurement approaches advocated by the DMCB, Care Continuum Alliance members and health system leaders have been around for years, are within the reach of standard statistical software, are familiar to researchers and are highly adaptable to the circumstances of 99.99% of disease management programs, not to mention the medical home and accountable care organizations.

The DMCB says that when multiple, competing, overlapping, repeated and adaptable measurement methodologies point in the same direction, thats when regulators, consumers, purchasers, buyers, providers and patients and mothers-in-law can be really confident that they have the answer they need.

Image from Wikipedia

But wheres the money?

The Disease Management Care Blog has always been leery of the "return on investment" (ROI) metric in health care. It knows that there are precious few health care interventions that actually "save" money. Many prevention, wellness and disease management programs     - depending on the analysis -  can cost money since they a) add additional resources to a system with already high fixed costs with b) a short one-year time horizon.

Yet, the good news is even if a program isnt successful in slowing the rate of cost inflation (or "bending the curve," which represents the savings), it can still represent a great value.  Thats because the additional benefit represents significant benefit for each additional dollar of spending.

Thats the message in this recent JAMA Viewpoint editorial Assessing Value in Health Care Programs authored by Kevin Volpp, George Loewenstein and David Asch. They offer up a thought experiment. Consider, they say, a state-of-the-art medication compliance campaign for heart attack victims that avoids a number of costly hospitalizations.  The price tag at $2000 has a positive "ROI" because the investment is less than the avoided cost of the hospitalizations.  However, if the price tag is $3000 and the investment is now greater than the cost of the hospitalizations, the ROI is "negative" even though the same number of patients didnt have to be hospitalized.

The DMCB recommends readers keep this manuscript/link handy the next time some Finance weenie demands an "ROI calculation."

Speaking of readers, the DMCB is happy to announce that it just hit 500 Twitter followers.  Thats in addition to more than 500 "RSS" subscribers, 461 Google Reader subscribers and thousands of return visitors per month.  The DMCB knows each was earned one person at a time.

Read this lead American Journal of Managed Care article on Geisingers approach to the patient centered medical home (PCMH), and you may agree that the major findings are:

1) the longer patients with chronic conditions are exposed to care coordination, the greater the impact on claims expense, and

2) while insurance claims went down, the savings werent enough to generate a return on investment, i.e., the program itself cost too much.

The DMCB also agreed but mined the article to find out more.

Geisinger has a ""Proven Health Navigator" system of primary care sites with "embedded" nurse case managers who serve medically complex patients. As more primary care sites were recruited into the system and as more Medicare Advantage patients were enrolled by the nurses, it became possible to contrast the duration of exposure to Navigator with the amount of savings. Based on over one million member-months in 43 primary care sites over four years, the authors found that from one to twelve months of exposure, patients claims expenses were not statistically significantly less than expected. However, once more than twelve months elapsed, the percent savings ranged from 4.3% to 6.7%. Yet, while the savings per member per month ranged from approximately $70 to $120, that was still not enough to exceed "the actual dollar amount invested in implementing" Navigator.

What else can the DMCB conclude?

1) When it comes to reducing claims expense, itll take more than 12 months to see a reduction in claims expense, i.e., "to bend the curve."  According to these data, itll take 2 years or more.  That means starting a care coordination program is a two to three year commitment.

2) The authors point out that with more time or more patients, they may have been able to achieve enough observations to achieve a statistically significant return on investment. Unfortunately, close reading shows there is little information in the manuscript on the program costs which led to the authors conclusions.

Today on Serious Eats: Moroccan-Style Chickpea Soup. I give it three snaps up in Z formation.

Every few months, instead of highlighting a newly beloved edible, we at CHG will compile a list of dishes that totally bombed. Sometimes the recipes themselves are bad, but mostly, it’s my own fault, since I’m semi-literate and terrible at improvising. Seriously, there’s a French Onion Soup at the bottom of this post that didn’t just make me doubt my own skills, but the existence of god.

The reasons for the Hall of Shame pieces are threefold:

1. This way, you know we don’t post everything we make. Dishes can’t just be cheap and healthy. They have to taste like actual food. That people would eat. Without accompanying upchuck.
2. Bad food is funny.
3. I forgot reason #3.

So, without further ado, here’s the very best of the very worst. Please respect me in the morning.

Parsnip Fries
Okay, you know how one of the basic tenets of food prep is chopping ingredients into similar-sized pieces, so they cook evenly? I ignored that rule here. Some parsnips burned, others didn’t heat through. I think the one in the middle is giving me the finger.

Turkey Meatballs
Giada DeLaurentiis’ turkey meatballs are widely known and well reviewed. “Hey,” I thought, “Why not ignore her directions completely, add more egg, and chop the onions into something resembling hubcaps? Then, I’ll burn the meat beyond recognition and cover the carnage in red sauce.” Bad plan, me. Bad plan.

Lime Basil Sorbet
It turns out, when a recipe calls for an ice cream maker, you can’t substitute an 8x8 Corningware baking dish. Doy. This block of lime and basil ice haunted our freezer for a month before I finally sacked up and tossed it. Apologies to Cooking Light, which deserves better from its readers.

Parmesan Steak Fries
This one wasn’t me! At least, I don’t think it was. No, it was probably me. Either way, this Everyday Food recipe tasted like nothing. They’re usually infallible, so I shoulder the blame. Martha, please forgive me. Don’t hit me with your ornamental loom.

Watermelon & Feta Salad
When an excellent, competent food blogger asks you to cut up a block of feta, and you use cheap pre-crumbled cheese complete with anti-caking agent, you deserve whatever weird Watermelon-cheese soup you end up with. Which is what I did.

Pureed Broccoli Pasta
Sigh. Again with the feta. I don’t think I added enough liquid, either, meaning the only thing grainier than my picture was the sauce itself.

French Onion Soup
Once upon a time, there was a culinary nitwit who wanted French Onion Soup, but didn’t have the time to make one from scratch. So, she opened a can of Progresso, crumbled up an old roll, and melted a hunk of about-to-go-bad mozzarella on top. This all occurred in the microwave, which actually yelled at her for attempting such a travesty. Then she died of bad food. The end.

The cookies. THE COOKIES.
A few months before Christmas, I received an e-mail from a major food publication asking me to enter a baking contest. Ecstatic and newly motivated, I devised a recipe for Double Chocolate Cherry Cookies, which was supposed to whet editors’ appetites, revolutionize cookie making, and bring about world peace. Eight slightly different batches later, they still sucked and I gained seven pounds. We’ll get ‘em next year, Buttermaker.

And that’s the whole shameful bunch. Readers, what dishes have you mucked up lately? Do tell.

~~~
If you like this piece, you might also dig:

• Alas and Alack: The CHG Hall of Shame
• CHG Hall of Shame II: Even Shamier
• Chicken Noodle Soup for the Failure’s Soul