By Peter Basch, MD, FACP and Michael Zaroukian, MD, PhD, FACP, FHIMSS
Incorrect Assumptions Lead to Incorrect Conclusions That Misinform Policy
While Democrats and Republicans may each have their own take on how best to move our country’s physicians and hospitals from paper to electronic records, there has been broad consensus that widespread adoption of electronic health records (EHRs) is a necessary step in the evolution to better, safer and more affordable care…at least until last week, when two reports were published that appear to question that conventional wisdom. The first, published as an op-ed piece in the Wall Street Journal, appears to center on a recent systematic review of research articles containing economic analyses of EHRs and health IT systems, specifically looking to see if study interventions were associated with lower costs for medication management. The editorialists concluded that there is no evidence from four to five decades of research that EHRs save money and that the current policy to use financial incentives to accelerate EHR adoption is misdirected.
The second report, from the Center for Public Integrity and published in the New York Times, does not question the rationale behind the policy but instead presents the startling and disturbing finding that whatever policy makers were hoping for, there is evidence that doctors and hospitals using EHRs are in fact billing for more and higher complexity services – and are thus increasing costs. Worse, the New York Times article suggests that much of the higher billing is the result of electronic ‘science fiction’ in which medical scientists (doctors) electronically record findings without actually ascertaining them. We will focus here on a critique of the Wall Street Journal op-ed piece. In Part 2 to follow, we will provide a perspective on the Center for Public Integrity report.
What do we know of the potential benefits of health IT and EHRs? They are now understood to be supportive infrastructure (i.e., tools) that can facilitate delivery, documentation and payment for health care services. As tools, they have no independent ability to make care better or worse, or make care more or less expensive. We also know that just like other tools, simply making EHRs better will not necessarily make care better, safer, or more affordable. However, there is evidence that well-designed and deployed EHRs and supporting health IT in the hands of trained professionals committed to using these tools wisely and well can facilitate care that is more effective, efficient, safe, timely, equitable and patient-centered.
How EHRs are used and whether they appropriately or inappropriately increase or decrease physician and hospital billing depends significantly on the healthcare delivery system in which they are used, the EHR training and proficiencies of users, the accuracy and completeness with which clinicians document their findings, and the alignment of financial incentives with specific provider actions. While time-limited financial incentives (defined as short term extra payments or penalties) are a small component of achieving financial alignment with policy goals, the most significant contributor to improved practices is the sustainable business case – namely the provider payment system.
Thus, where physicians and hospitals are paid only for the volume and complexity of services delivered, one could expect that EHRs would be used to support higher volumes and increased documentation of more complex care, particularly when payment is proportional to the extent of documentation. One would also expect to consistently see the converse – EHRs being used to reduce unnecessary and inappropriate services and complexity – when the payment system is closely aligned with these objectives. This is exactly what is increasingly seen in studies that analyze settings of care based on the payment model. Specifically, studies done at health systems such as Kaiser (which is both a provider and payer) more consistently and reliably show reductions in unnecessary volume of services and avoidable cost while demonstrating high levels of quality.
What problems did we have with the Wall Street Journal op-ed article? From our perspective, there were three major flaws. First, we believe the editorialists drew an inaccurate and misleading conclusion from the systematic review when they wrote that “a comprehensive evaluation of the scientific literature has confirmed what many researchers suspected: The savings claimed by government agencies and vendors of health IT are little more than hype.” As evidence, we offer our own summary of the systematic review summarized in the op-ed piece. In conducting this review, the investigators electronically screened over 35,000 potentially relevant citations, finding only 31 (<0.1%) that contained cost data and an evaluation of the costs. However, they found that the methods used in the various studies were so heterogeneous that they could not be synthesized and so they were forced to limit their analysis to a narrative review. Most studies (74%) included in the systematic review were conducted in U.S. hospital settings and most (61%) focused on computerized decision support systems (CDSS). Only five studies included a full economic analysis.
The authors concluded that the quality of the economic literature in this area is poor and that a few of the studies they analyzed may offer cost advantages despite high technology acquisition costs. They otherwise indicated that study designs and other factors made it difficult to reach any definitive conclusion regarding cost-effectiveness, cost-utility or cost-benefit of computerized medication management systems and that more rigorous studies were needed to address whether health IT interventions for medication management are cost-effective. The authors acknowledged that there was no attempt to look at costs associated with other types of orders, the effect on selection of more appropriate therapy, adherence to guidelines, or other economic factors.
So does this review support the op-ed conclusion that the “savings claimed by government agencies and vendors of health IT are little more than hype”? We don’t believe so. Instead, we gleaned the following take-away messages from the systematic review: 1) there have been very few economic studies of medication management in EHR systems; 2) the methodologies used to conduct such analyses have not been consistent, making it difficult to combine the data and draw definitive conclusions; 3) a significant fraction of the studies focused only on certain types of medication therapy (antibiotics), care settings (hospitals) or types of medication management tools (CDSS); 4) there was evidence of cost advantages in some of the studies; and 5) that additional, more rigorous studies are needed to determine the cost-effectiveness of currently deployed health IT for medication management.
Second, with regard to the systematic review upon which the op-ed piece was based, we would contend that the authors were looking where one could already predict something would be unlikely to be found, namely decades of research and tens of thousands of citations, only a tiny fraction of which related to the research question, used dissimilar methods and only a handful (5) of which included a full economic analysis that at best could only be applied to a fraction (hospitals) of care settings, with no more than two research studies exploring the same category of clinical decision support (CPOE = 2, computerized reminders = 2, and CDSS = 1). Giving significant weight to these predictably negative or inconclusive findings is scientifically inappropriate. For example, the op-ed authors emphasized the multi-decade span of citations screened rather than the more focused timespan of articles reviewed. Of the five full economic analyses that were reviewed, four were published between 2004 and 2008, and only one earlier (1992). None were conducted using systems whose features and functionalities were shaped by the specifications of the HITECH Act or CMS Meaningful Use incentive program objectives or measures. Both of the two CPOE studies showed significant cost benefits, one in savings from prevention of adverse drug events and the other in the costs related to medication prescribing errors. The study that examined computerized medication reminders involved an intervention to improve tetanus immunization with the primary outcome being improvement in immunization rates rather than costs. One would expect that this computerized reminder study would not show a positive cost-benefit outcome because the intervention was not very effective using either computerized or non-computerized methods (<30% net improvement in immunization verification or administration rates), the condition for which prevention of illness would have resulted in large savings (tetanus) was too rare (0.10 cases/million population) to be expected even in a population 1000 times the size of the one studied, and the intervention could have easily been one that has already been shown to be sensitive to the method by which a reminder is presented.
To us, claiming that negative studies such as the tetanus immunization computerized reminder study is conclusive evidence that EHRs do not save money and that all such claims are “hype” is no more logical than concluding that light bulbs on average do not light up a room because they were not demonstrated to do so in studies that included settings where the bulb was dead or the light switch was turned off. To imply that one’s conclusions reflect four or five decades of research, thereby lending additional validity and finality to the conclusions when even a cursory read of the systematic review indicates otherwise suggests at a minimum a lack of understanding of EHR history and design. With rare exceptions, EHRs were not designed to even consider cost, let alone cost reduction strategies (value decision support) until perhaps the last decade, and even then only in earnest after the HITECH Act was passed in 2009 and the CMS Meaningful Use Incentive Program goals, objectives, measures and payment criteria were finalized in 2010. This is underscored by the finding of economic data in less than 0.1% of the 35,510 citations screened for the systematic review. Further, if a physician has never been trained to consider cost reduction strategies (e.g., to consistently use lower cost medications when possible and not contraindicated) in her/his decision making, it is not reasonable to expect that merely changing the ordering process from paper to electronic would reliably change behavior.
Third and most important, even though the editorialists arrived at this conclusion for the wrong reason, most policy makers have matured their vision of health IT to arrive in nearly the same place – that health IT is at best an enabling infrastructure to the delivery of higher quality, higher value care. In fact, the policy underpinnings of the HITECH Act (which includes the EHR incentive legislation) are based on that kernel of truth in the Wall Street Journal op-ed – there is sufficient evidence to conclude that EHRs in-and-of-themselves will NOT reliably lead to better, safer, and more affordable care just by their mere presence in hospitals and physician practices. Thus, the HITECH Act is intentionally NOT an incentive program for the mere implementation of EHRs; it is a program that incents EHR adoption of particular capabilities that are used in meaningful ways that were deemed through the process of literature review and expert opinion (including findings from Institute of Medicine Reports) to enable better, safer and more efficient care.
We would like to offer a better and more constructively focused editorial headline related to this new systematic review analyzing EHR use for cost reduction in the areas of medication management: “Systematic Review Underscores Paucity of Research Data on the Costs, Effectiveness and Benefits of Computerized Medication Management Systems but Hints that in Some Settings Substantial Savings May be Possible and are Deserving of Additional Study”.
We encourage you to continue reading about this topic in Part 2, where we will tackle the recently published report by the Center for Public Integrity reporting that physicians and hospitals using EHRs show higher charges than their colleagues using paper records.