CMS Innovation Center Tackles Implicit Bias

The disparate impact of the COVID-19 pandemic on beneficiaries based on factors such as race, ethnicity, geography, and income, as well as a review of lessons from its first decade of work, led the Centers for Medicare and Medicaid Services (CMS) Innovation Center to articulate a new vision: “Achieve equitable outcomes through high quality, affordable, person-centered care.” To realize this vision, the Innovation Center developed a strategic plan organized around five objectives, one of which is to advance health equity.

In support of its health equity work, the Innovation Center conducted a review of three existing experimental payment and service delivery models to determine whether implicit bias may be present and, if so, whether such bias has led to the unintentional exclusion of certain beneficiary groups from the models. The results of this analysis, discussed below, are informing broader efforts to address bias across the Innovation Center’s models.

CMS’s Objective Of Advancing Health Equity

CMS defines health equity as “the attainment of the highest level of health for all people, where everyone has a fair and just opportunity to attain their optimal health regardless of race, ethnicity, disability, sexual orientation, gender identity, socioeconomic status, geography, preferred language, and other factors that affect access to care and health outcomes.” The Innovation Center’s health equity objective fully aligns with CMS’s 2022 Strategic Plan, which seeks to address the myriad health disparities that underlie the US health system.

The Innovation Center’s “Advance Health Equity” objective includes four areas of focus: Develop new models that address health equity and social determinants of health; increase the number of beneficiaries from underserved communities that receive care through value-based payment models by increasing the participation of Medicare and Medicaid providers who serve them; evaluate models specifically for their impact on health equity and share data and “lessons learned” to inform future work; and strengthen data collection and intersectional analyses for populations defined by demographic factors such as race, ethnicity, language, geography, and disability—to identify gaps in care and develop interventions to address them (in a manner that Protected Health Information complies with HIPAA and other applicable laws).

Background On Implicit Bias And Innovation Center Models

The Innovation Center develops and tests health care payment and service delivery models to improve patient care, lower costs, and align systems to promote patient-centered practices. For the purpose of this exercise, “implicit bias” was defined as “a differential impact created or exacerbated, without intention, by an algorithm, set of sequential rules, or standard processes within a model, with a particular focus on racial and ethnic groups.” The assessment focused on three established models: the Kidney Care Choices Model, Comprehensive Care for Joint Replacement Model, and Million Hearts® Cardiovascular Risk Reduction Model. Together, these models represent a small but varied microcosm of the Innovation Center portfolio. Specifically, they include the voluntary, mandatory, and voluntary/mandatory hybrid models, and vary by financial methodology, beneficiary attribution, risk stratification, and degree to which providers assume financial risk. Although all of the models serve diverse beneficiary populations, none were specifically designed to reduce disparities in health care.

The goal of the Innovation Center’s review was to inventory potential biases in these three models to better understand how to detect potential implicit bias in existing and future models, a necessary precursor to mitigating or eliminating such bias. We began by first researching health disparities related to each model’s target beneficiary populations. We then systematically inventoried all algorithms, rules, processes, and policies within each model including: (1) criteria for provider and beneficiary eligibility and selection; (2) beneficiary attribution; (3) risk assessment and screening tools; (4) provider tools likely to be employed; (5) payment design and risk-adjustment algorithms; and (6) model and evaluation design. For each inventoried item, we carefully considered its rationale/purpose; whether it achieves that purpose for all populations; and, finally, whether there are any potential unintended consequences in light of our understanding of existing health disparities within the model’s target population. The key findings of our review are discussed below.

Key Findings

Kidney Care Choices Model

Black Americans are more than three times as likely to have end-stage renal disease (ESRD) as White Americans. They also spend more time on transplant waiting lists, have lower access to live donor transplants, and experience lower rates of graft survival.

The Kidney Care Choices (KCC) Model aims to improve quality and reduce cost through care coordination and payment incentives for providers—also referred to as model participants—serving beneficiaries with chronic kidney disease and ESRD, as well as those who have had transplants. Beneficiary alignment for the model is determined in part by level of kidney function, as typically measured by the estimated glomerular filtration rate (eGFR).

The eGFR is calculated from the amount of creatinine in the blood, which can vary by age, sex, and body weight. According to the National Kidney Foundation, clinical trials have demonstrated that people who identify as Black can have, on average, higher levels of creatinine in their blood; therefore, it was thought that a race adjustment for individuals who identify as Black would yield a more accurate eGFR.

Race-adjustment of eGFR was the standard screening practice by kidney care providers at the time of the KCC Model’s conceptualization; however, many experts have since warned that the race-adjustment artificially elevates kidney function in Blacks, leading to delayed referrals to specialists and transplant listing, and potentially worse health outcomes for Black patients. After reassessing the race-adjusted eGFR, a taskforce convened by the National Kidney Foundation and the American Society of Nephrology recommended in September 2021 that kidney care providers immediately adopt a revised eGFR equation that does not include race. The taskforce acknowledged that the revised estimating equation had its own limitations, but that it was preferable to the race-adjusted equation because the bias and inaccuracy is “minimal and of equal impact across all patient groups, and not concentrated within one group.”

The Kidney Care Choices Model performance period began on January 1, 2022. Beneficiaries who may have met the medical eligibility criteria for the model (that is, had chronic kidney disease or ESRD) on or around that date may have had their kidney function assessed by a provider using the race-adjusted eGFR before the taskforce’s guidance was released. The use of the race-adjusted eGFR may have erroneously elevated Black beneficiaries’ kidney function, and, as a result, they may have been incorrectly assessed as not meeting the medical eligibility criteria for the model. The number of Black beneficiaries who may have been excluded from the model cannot be reliably estimated. The Innovation Center communicated the new guidance to the model participants through its monthly newsletter and “office hour” sessions. Other components within CMS are also exploring policies to address use of the race-adjusted eGFR.

Comprehensive Care For Joint Replacement Model

Although Black Americans are more likely to experience arthritis-related work limitations and severe pain, they are less likely than White Americans to receive lower extremity joint replacements. This disparity can be explained partly by the fact that Black patients are approximately half as likely to receive an offer of total knee replacement and less likely to accept the offer.

The Comprehensive Care for Joint Replacement (CJR) model tests whether bundled payments for lower extremity joint replacements can improve quality of care and reduce spending by reducing fragmentation of care. The CJR model began as a mandatory model for its first two performance years, in 2016–17. The model became a hybrid mandatory and voluntary model for the 2018–20 performance years; however, it will be fully mandatory again through 2024. Mandatory models eliminate selection bias at the provider level but could lead to selection bias at the patient level.

In the CJR model, CMS provides participating hospitals with a target price for an episode of care—a period of time that begins with a patient’s hospitalization for the joint replacement and ends 90 days after hospital discharge. Hospitals are held financially accountable for all care costs incurred by a patient during the 90-day period; they generate savings by keeping spending below the target price and are required to repay Medicare if episode spending exceeds the target price. A common strategy to manage spending for lower extremity joint replacement episodes is reducing unnecessary institutional postacute care, a widely acknowledged area of potential waste in health care delivery.

The CJR target price is based on a blend of the hospital’s historical spending and regional averages, and was not initially adjusted for sociodemographic factors associated with higher costs, such as race or socioeconomic status. Notably, compared with White patients, Black and low-income patients are more likely to be discharged to a skilled nursing facility or rehab facility following surgery, both of which are associated with increased odds of 30-day hospital readmission and higher spending. This presents an opportunity for bias within the model as CJR providers could make fewer offers of joint replacement surgery to Black and low-income individuals in an effort to keep spending below the CJR target price and generate savings under the model.

An evaluation of the CJR Model found that beneficiaries receiving joint replacements at participating hospitals while the model was in effect were less medically complex than those receiving joint replacements at those same hospitals before model implementation began. Additionally, they were less likely to be dual eligible for both Medicare and Medicaid, an indicator of lower socioeconomic status. One year’s evaluation report showed that they were also less likely to be Black, but this finding was not consistent across model years as it was for medically complex patients and dual-eligible beneficiaries. The reasons for these shifts in patient mix are still being explored but may suggest that providers could be selecting patients that are less likely to require institutional postacute care.

Concerns about selection bias prompted CMS to revise the risk-adjustment formula used to set the target price to include dual-eligibility status for the three-year extension of CJR that began on January 1, 2022. CMS is collecting data to help understand the impact of the revised risk-adjustment formula, as well as other changes in patient volume that may indicate selection bias based on patient sociodemographic characteristics.

Million Hearts™ Cardiovascular Risk Reduction Model

In the US, Blacks are more likely than Whites to experience heart failure, stroke, and peripheral vascular disease, and to do so at an earlier age. Moreover, Black and Hispanic adults are more likely than their Asian and White counterparts to be obese and to have diabetes, which are both risk factors for heart disease.

Under the Million Hearts® Model, all providers were required to use the American College of Cardiology/American Heart Association’s Atherosclerotic Cardiovascular Disease (ASCVD) Risk Calculator to predict 10-year ASCVD risk. Beneficiaries whose risk score indicated elevated risk were eligible for the model and received targeted interventions to reduce their risk of heart attack and stroke.

The ASCVD Risk Calculator was developed specifically for Black and White populations. For anyone identified as “Other,” the calculator produces the same risk estimates as for White individuals. According to a 2018 report from the American Heart Association and American College of Cardiology, this approach “may systemically underestimate risk in patients from certain racial/ethnic groups and those with lower [socioeconomic status] (SES)…and overestimate risk in patients with higher SES or who have been closely engaged with preventive health care services.” In fact, individuals selecting the “Other” race category in the ASCVD Risk Calculator’s mobile application will see a pop-up notification stating that the choice of “Other” may underestimate risk in South Asians, Hispanics from Puerto Rico, and American Indians, and may overestimate risk in some Americans of East Asian or Mexican ancestry.

As such, the required use of this calculator in the Million Hearts® Model may have systematically underestimated 10-year ASCVD risk for certain racial and ethnic groups and people with low socioeconomic status, which may have excluded them from the model and the benefits associated with ASCVD prevention. The Million Hearts® Model ended in December 2021, before this analysis was conducted. Given that the ASCVD Risk Calculator was a central tool of the model, a risk calculator that does not carry risk of bias (if such a tool exists) could not be selected prior to the model’s end date.

Prior to the assessment, early model data pointed to a different source of potential bias in the Million Hearts® Model: Beneficiaries whose race data in the Million Hearts® Data Registry (that is, provider or patient self-reported data) did not match their race as recorded in CMS claims data were excluded from participation. This policy disproportionately impacted Black beneficiaries, resulting, according to CMS internal analysis, in the exclusion of at least 982 Black beneficiaries during the second year of the model. The Innovation Center subsequently eliminated this criterion for beneficiary eligibility. As a result, Black beneficiaries who were not previously enrolled due to the mismatch were enrolled in the next performance periods.

Moving Forward

The assessment examined three models to identify potential sources of bias and found that use of certain risk-assessment and screening tools, provider tools, and payment design and risk-adjustment algorithms has led to the exclusion of some beneficiaries from these models. These findings are troubling not only because of the limited access to the benefits of Innovation Center models but also because diverse model participation is critical for robust evaluation and confidence in generalizing results to all of the populations served through CMS programs.

As described above, the Innovation Center has taken initial actions to address these sources of bias for ongoing models; however, the findings underscore the need for a more systematic evaluation of implicit bias in current and new models. To this end, the Innovation Center has begun to develop a step-by-step guide to screen for and mitigate bias in Innovation Center models. This guide will be piloted for use in new models currently in development, with the intention of having all future models screened for implicit bias with this guide prior to launch. In the long term, this critical effort will support CMS’s broader commitment to providing equitable, high-quality care for beneficiaries in the Medicare, Medicaid, and CHIP programs.