Pe Peer Re Review is not simply a hurdle to overcome: an example - PowerPoint PPT Presentation

Pe Peer Re Review is not simply a hurdle to overcome: an example on transforming a preterm birth cost analysis Norman J Waitzman, PhD Professor and Chair, Economics DeCART, Data Science for the Health Sciences, August 3, 2018

Order of Service o Overview o Brief review of Waitzman analysis for IOM Report on Preterm Birth o Stage 1: New Study, review in response to request o Stage 2: Digging in the heels. o Stage 3: Relent o Stage 4: Safe Landing or Crash? o Lessons

Setting the Stage for Launch: IOM Report on Preterm Birth Cost IOM Panel on Preterm Birth convened in 2005, “Causes, Consequences and Prevention” o New contribution of the Report is the Analysis of Societal Preterm Birth Costs o Direct Medical and Special Education Costs and Indirect Productivity Costs o Relied on Utah data (Select Health, Intermountain Health Care) for Medical Care Cost Estimates o Adjustments made to project to nation o

IOM Report on Preterm Birth Costs Final Estimates by Age and Cost Category

IOM Report on Preterm Birth Costs Inpatient Medical Cost by GA and Age

IOM Report on Preterm Birth Costs Validation Against Earlier Work

Review Request: Inpatient Costs in one state o Update of an earlier analysis that was cited in IOM report o Nearly comprehensive linked discharge records (including transfers) from birth hospitalization (Cost) to birth certificates (GA) o Cost-to-Charge ratios to determine incremental cost by GA category (extreme/moderate/mild preterm relative to term) o Maternal costs and professional fees included o Conclusion: “Real” cost has increased over time (more heroic attempts at reducing infant mortality)

Stage 1 Review: Liftoff (summary statement) o “The authors have basically updated their earlier estimates from about a decade ago with 2009- 2011 data. . . . While updates are perhaps always welcome, the justification and indeed, conclusion of the piece is that there have been dramatic increases in costs due to changes in technology.” o As it stands, however, the conclusion is not yet well substantiated. If the conclusion is to be maintained, methods for updating cost need to be applied with greater precision. Otherwise, a more nuanced conclusion is merited.”

Launch: 3 Pesky Issues o COMMENT #1: Statewide preterm rate is incorrect. . . differs substantially from published rate (CDC). o The overall preterm birth rate from the analyses is 12.9%. This is close to 30% higher than the approximate 10% average preterm birth rate for CA reported in Vital Statistics for 2009-2011 (see Martin et al., "Measuring Gestational Age in Vital Statistics Data, . . .", National Vital Statistics Report 64 (5), June 1, 2015,Table 4). This 30% is the difference from the LMP establishment of preterm birth on the birth certificate. OE, the preferred method, was significantly lower still, at about 8.7%. One assumes that LMP was adopted for this study, but the authors should state whether LMP or OE was used. o But, even if LMP was used, the critical question is how does one reconcile a so-called population estimate of 12.9% with a 10% official rate?

Pesky Issue #2 o COMMENT #2 Inflation adjustment is critical to the conclusion of REAL COST INCREASES, but a general rate is used (CPI) rather than one tailored to the services provided o First, . . . National rather than state specific CPI. Second, and of greatest import, is the rationale for using a general CPI rather than an index tailored to health care. The CPI is for all goods and services in a consumer basket. It also reflects transaction prices rather than costs. This study, on the other hand, specifically addresses inpatient hospital services, and its costs, not reimbursements . o The inpatient hospital producer price index (PPI) would appear to be the closest to the best possible index to adjust these costs (producer prices reflect actual prices paid for inputs, approximating costs). That PPI has increased dramatically faster over the past decade than has the CPI, and when deployed, the residual increase in cost dwindles to the extent that the uncovered increase could border on noise . . . o Aside from the stark difference in the overall preterm birth rate from national statistics and that uncovered in this analysis noted above, this is the major issue with the manuscript, as it leaves its primary justification/conclusion (major cost-driving changes over the past decade in technology) wanting.

Pesky Issue #3 o COMMENT #3 Assymetry in Methods for Estimating Facility Costs versus Professional Fees, which are absent from the Discharge Abstracts o why was the physician fee algorithm by DRG adjusted by payer type if the estimate is for cost and not reimbursement? This creates an asymmetry relative to inpatient hospital costs using cost-to-charge ratios where all like DRGs are accorded the same cost regardless of payer. If specialty mix are different by payer, then fine. o Otherwise, it would seem that there should be a single cost estimate for the physician portion regardless of payer so as to avoid inconsistency.

Stepping Back: Rudimentary Principles Underlying Comments An Expenditure or Cost is Price X Quantity (P x Q). If the claim is that intensity of services, “Q’s” have grown, have to adjust properly for “P’s”. Authors have not. “Charge” is not “Reimbursement” which departs from“Cost.” Cost is estimate of value of Real Resources; Reimbursement is amount paid and may reflect market power and conditions; Charges are simply an accounting mechanism with no definitive relationship to cost. Discharge abstracts permit estimates of “Cost” through “Cost-to-Charge” ratios. Methods applied to professional fees, however, rely on “reimbursement”, not cost. State or area costs vary and change differently from a national trajectory. Improper to apply national indices to adjust for local estimates.

Recommendation to Editor (“no such thing as a free launch”) REVISE, with Major Revisions

The Empire Strikes Back: Author Response o #1 Preterm Birth Rate: Minor modification to make OE estimate (10.2%) rather than LMP estimate o #2 Use of general price index: Respectfully disagree that the general CPI is inappropriate o #3 Asymmetry of Method: Peer-reviewed methodology was used, no alternative available. o In other words, “We choose to blow off the issues raised by the review.”

Return of the Jedi: Reviewer Comment Iteration #2 o Comment #1 Discrepancies with CDC published preterm rate o There seems to be a fundamental misreading of the reviewer's comment in this response. The referenced Vital Statistics report, as the reviewer noted in the initial review, showed an 8.7% estimate by the OE method and 10% for the LMP method. By switching to the OE method and arriving now at a 10.2% preterm rate, there is no consistency with current data at all as claimed by the authors. o A relevant extension of a different comment: This is a partial response. Why not a validation exercise as suggested?

Jedi Returns: Comment #2 o Comment #2 CDC Index for Cost Adjustment o If one gives the benefit of the doubt, the authors undertook a very cursory reading of the reviewer's comment. A less generous assessment is that the authors lack s rudimentary command of the extensive price index literature, particularly in the area of health care economics, concerning which index is most appropriate under what context. . . . [T]hey are comparing a specific type of health care service (inpatient delivery care) several years ago to the same type years later, and seek to express the change in real terms. That is how it is used in the referenced paragraph. The reviewer suggested an index tailored precisely to what the authors are assessing, the producer price index (PPI) for inpatient care. This index does not suffer the weaknesses of transaction prices that the authors appear to claim. In one recent review in a vast literature on the appropriate price index to apply in health care analysis, I quote: "To adjust estimates of costs of inpatient services from different years, the PPI for inpatient services appears currently to be the best option (Dunn, Grosse and Zuvekas. 2016. "Adjusting Health Expenditures for Inflation: A Review of Measures for Health Services Research in the United States" Health Services Research, 175-196. DOI: 10.1111/1475-6773.12612). o The use of the PPI challenges the fundamental conclusion about the extent of real increases in preterm delivery care over time.

Jedi Returns: Comment #3 o Comment #3 CDC Asymmetry of methods between facility and professional costs o As noted in the initial comment, the Peterson et al. (2015) did not provide the means to undertake a proper adjustment to cost, thereby yielding asymmetric methods on the professional versus inpatient facility components of estimates in this paper The authors, it would seem, would either 1) acknowledge the asymmetry in method , that one reflects "cost" whereas the other reflects "payment" and therefore there estimates are not uniformly either. Or, preferably, 2) they would, perhaps use literature on % that Medicaid fees to physicians for inpatient delivery services covers cost or Medicaid as % of Medicare payment to physicians (treating Medicare as cost-based numerare)., and then adjust Medicaid payment from Peterson et al. across the board accordingly.