One for Good Measure: Evaluating EMS

     I was working an overnight shift in a busy metropolitan EMS system when we were called to the home of Mae, an elderly female who was short of breath. She said she was having difficulty dealing with the loss of a loved one, and awoke 30 minutes ago feeling anxious and sad. She did better with a little oxygen and a lot of sympathy.

     Having learned long ago that there isn't enough equipment or expertise in the back of an ambulance to rule out a cardiac event, I suggested we proceed to the nearest ED for a more definitive assessment. En route Mae asked if I'd handled a lot of cases like hers. I caught myself parroting the usual platitudes about my agency's experience with all things medical, but then it occurred to me that I didn't know what she meant. A lot of cardiac calls? Anxiety cases? Geriatric patients? Middle-of-the-night runs? It didn't matter. I had no idea how our alarms were distributed by age, time of day, chief complaint or any other criteria.

     Next my inquisitive patient wanted to know what the charge for the ambulance ride would be. After another pensive pause, I replied that I wasn't sure, but her insurance probably would cover it. Truthfully, I didn't have a clue. I wondered what it cost us to do a call like this, or any other call. Surely someone in our company must keep track of those things.

     Not necessarily.

MINDING YOUR BUSINESS

     In Search of Excellence, by Tom Peters and Robert Waterman, was considered a must-read manifesto for entrepreneurs and corporate leaders in the mid 1980s. The premise of the book was that common indicators of management excellence prevailed in the most successful U.S. companies. By postulating and dissecting eight commonsense characteristics, they offered a blueprint for building new businesses and renovating struggling ones. The authors pointed out that "the companies that we have called excellent are among the best at getting the numbers, analyzing them, and solving problems with them."¹ Is there any reason EMS organizations should be exempt from such diligence? How well do we measure company and employee performance? Are there business tools we can adapt and apply in EMS to help us?

     This article focuses on five attributes worth quantifying in almost any professional setting. The methodology is tailored to EMS.

1. QUALITY

     Most of us have been horizontal in a hospital or ambulance at least once. When I'm a patient, my priorities are clear: I want to feel better, and I want someone with medical training to tell me, "I know what your problem is, and I'm going to fix it." When that happens, I'm so grateful, I could rename my children after my healthcare providers.

     As EMS managers, however, we confront the challenge of evaluating care delivered by third parties: our employees. There are many complicating variables: time, standards of care, patient cooperation, environment, etc. The key to grading prehospital care is to recognize practical limitations to data collection, quantify what we can, and treat spurious conclusions with skepticism.

     One method of measuring outcome is called PET, for prehospital evaluation technique. PET specializes in grading quality of care. It's not just a number-crunching tool; it's also a process, requiring planning and preparation before implementation.

     PET highlights three indicators of EMS performance: prehospital impression, prehospital care and hospital diagnosis. PET pundits compare prehospital care to the other two data elements by assigning common codes to each, based on local protocols. The goal is to determine whether prehospital care matched prehospital impression and hospital diagnosis. When that is not possible due to ambiguous information, matches are assigned null values so those cases aren't included in subsequent analysis.

     An example of the PET process could be a CHF patient treated in transit with a bronchodilator instead of nitrates. The prehospital impression might have been COPD. The prehospital care matched that assumption, but the hospital diagnosis was pulmonary edema. PET would show a match between the first two elements (the provider's intervention supported the prehospital impression) and a mismatch between the latter two (a hospital diagnosis that was inconsistent with prehospital care).

     As PET accumulates cases, users can view the proportion of matches by agency, provider, date range, patient age, presenting problem and receiving facility (Figure 1). Results are meaningful only as relative, rather than absolute, measures. To calculate, for example, that a particular provider's prehospital care matched the hospital's diagnosis 70% of the time means nothing until that figure is compared to the percentages achieved by other employees.

     Consider the value of the following hypothetical feedback from PET:

• Your agency's prehospital care was 20% more consistent with ED diagnoses through the first half of this year than it was last year.
• 30% of Medic A's calls show discrepancies between his working diagnosis and his treatment. (Could this be an educational issue?)
• Your staff nailed its narcotic calls but missed 15% of hypoglycemic presentations, according to the receiving hospitals. (How thoroughly are your people assessing AMS cases?)

     Once you can quantify results such as these, it's much easier to address performance issues and drive a QA/QI process.

     PET relies on a user-defined table of standing-order interventions and associated vital signs to determine which prehospital protocol applies to each of the three matched fields. Rather than make a futile effort to include all conceivable chief complaints, savvy PET users limit qualifying cases to the most common presenting problems.

     Predictors of a successful PET implementation include:

• A knowledgeable administrative staff. Coding of the three key fields usually is handled by clerical employees. It helps if personnel charged with that assignment have some basic training in medical terminology. EMT certification is even better.
• Cooperation between transporting and receiving agencies. Hospitals must provide unambiguous ED diagnoses so PET users can assign appropriate protocol codes. Respiratory difficulty isn't specific enough.
• Administrative access to patient care data. PET uses prehospital vital signs and interventions to determine qualifying treatment protocols. Prehospital reports should be circulated to the people responsible for coding PET data, in a manner consistent with HIPAA regulations.
• Ambition tempered by common sense. Are you wondering why PET doesn't compare prehospital impressions to hospital diagnoses? After prehospital providers consult with medical control or the receiving hospital, prehospital impressions begin to change. Early PET practitioners discovered it was more practical to disregard that match than to enforce documentation of the initial prehospital impression. Another example of quality trumping quantity was the decision to code only one presenting problem per patient.

     There are other ways to assess quality of care—recency of experience, for example. By way of illustration, ask yourself how prepared you are to deal with the more obscure elements of your standard of care. The newborn in respiratory distress? The obstructed airway? Frostbite? Snakebite? Everything else that bites?

     You can anticipate your staff's readiness to handle these and other low-volume calls. We begin by coding call types, as we did with PET. Our purpose is to quantify experience by provider over time. For example, shouldn't you know how often each medic intubated an infant during the past year? In many systems, that number would average less than one. We can't increase the supply of pediatric patients (nor would we want to), but we can mandate manikin practice and targeted didactic reviews. Your job, as manager, is to provide the tools and the rationale for your people.

2. CUSTOMER SERVICE

     There are "left-brained" and "right-brained" aspects to customer service.² Our left brain, or logical side, seeks hard evidence of performance. Our right brain, or subjective side, is more likely to conclude, as Justice Potter Stewart did when discussing obscenity in 1964, that "I know it when I see it."³

     Corporate America tries to capture feedback associated with both philosophies. A common example of right-brained customer service evaluation is the survey. Most of us have been subjected to these, usually via telephone during dinner or our favorite TV show. Is there ever a good time to bother someone at home? Probably not; hence the risk of corrupting customer service through measurement of customer service.

     Let's examine the left-brained approach. We begin by identifying ways that we satisfy our customers, such as timely delivery or availability of goods for manufacturing firms, and completion of work on time for the service sector.

     The S in EMS tells us all we need to know about which group we belong to. We sell treatment, transportation and time, not products. Treatment was covered in our discussion of quality. Transportation and time are measurable but controversial.

     A year ago I was a big proponent of tracking response times. A national standard of eight minutes or less, from receipt of call to arrival on scene, had emerged,⁴ and it seemed logical to conclude that outcome was inversely proportional to response time. Other research, including work by emergency physician Peter Pons, et al, and British researcher Linnie Price, disputes that assumption.^5–7

     In Price's work, experienced paramedics were asked to judge the impact of sub-eight-minute response times on prehospital care. The consensus was that response times were not always accurately determined, and that there was no discernible benefit associated with beating the eight-minute goal. In fact, some medics commented that expedited responses carried risks to both patients and providers.

     The Pons study concluded that a sub-eight-minute response was not associated with improved survival to discharge. Benefits were not noticed until the four-minute barrier was broken.

     Now let's approach this from a different perspective: perceived response time. Customer satisfaction specialists know that, in their world, perception supersedes reality. What we do for customers is important. What they think we do is even more important.

     A 1999 study revealed that patients tend to overestimate response times by 36% (12.4 minutes perceived vs. 9.1 minutes actual).⁸ This means that even if there's medical validity to the eight-minute goal, beating it might not impress our patients.

     On-scene times might be a better indicator of customer service and quality of care. Our goal is to deliver patients to definitive care as soon as possible. I can't think of any common scenarios where a patient would benefit if EMS purposely delayed transport.

     Elapsed time is calculated by subtracting arrival time on scene from arrival time at the hospital (be sure to allow for transports on either side of midnight). Results can be summarized by agency, vehicle, time frame, technician, driver and presenting problem (Figure 2).

3. PRODUCTIVITY

     Before I became a medic, I was an engineer. The two things engineers learn to do best are problem solving and productivity improvement. Successful businesses depend on those skills, which is why engineers are in such demand.

     Productivity is expressed as output over input, i.e., what you get back for what you put in. We can improve productivity by increasing output, decreasing input or both.

     Patient care, transportation and revenue are examples of EMS output. We provide manpower and vehicles as input. Our productivity (how efficiently we use inputs to create output) can be measured by calculating, for example:

• Revenue per employee-hour
• Revenue per transport-hour
• Revenue per transport-mile
• Transport hours per vehicle-day.

     Figure 3 shows a sample productivity summary. Notice how employee-hours and vehicle-days express resources used over a fixed period of time.

     We could expand this analysis to determine the profitability of our vehicles and our people for different types of transports, times of day, etc. Perhaps billing rates or crewing should be adjusted. The endpoint we seek isn't productivity measurement, it's productivity improvement.

4. COST ACCOUNTING

     If you want to increase the profitability of your business, cost reduction is even more important than revenue growth. Do the math: if it costs you $9 to earn $10 (a pretty impressive 10% profit margin), you'd either have to sell another $10 worth of goods or services to make one more dollar of profit, or just cut your expenses by $1. Each dollar of cost reduction is worth $10 of revenue increase.

     As we saw in our productivity section, labor and transportation are significant EMS cost elements. We can expand Figure 3 to incorporate two indicators of cost: labor variance and transport variance (Figure 4).

     Any outcome that's different from what you expected can be considered a variance. If you planned to work 40 hours last week but only clocked 36, that is a variance of four hours, or 10% of your schedule. Another example is paying $8,400 for a new roof after getting an estimate of $8,000. That's a $400 variance (5%).

     Variances can be favorable or unfavorable. Sometimes it depends on your point of view. If you're the homeowner in the latter example, spending an extra $400 for a roof definitely is unfavorable. The terms positive and negative also apply, but a positive variance is not necessarily favorable.

     Labor variance is the difference between what your company earned per employee-hour and what you have to earn to be profitable. Standard revenue per employee-hour combines wages, a prorated amount for benefits (medical coverage, insurance, vacation, etc.) and overhead (rent, vehicle maintenance, supplies, etc.). That sum is then marked up to include a profit.

     Transport variance shows how your agency performed per transport-hour. Standard revenue per transport-hour considers all direct and indirect expenses associated with operating an ambulance for an hour, an allowance for overhead and a percentage for profit.

     Positive variances on our sample report are favorable, meaning they represent more profit than expected. Negative variances (in parentheses) reveal shortfalls in earnings. Over time we can see which cases, employees, vehicles, routes and presenting problems contributed the most profit.

     There are variances associated with materials, too. Since supplies usually account for less than 10% of an EMS agency's budget, we don't pay as much attention to that area as labor, facilities and equipment. However, if your company spends $300,000 a year on materials, and you can reduce that by 10%, you've just paid for one or two new cardiac monitors.

     A good way to minimize material costs is to consider the trade-off between purchase price and holding cost. Here's how that works: Vendors who sell supplies want you, the buyer, to commit to the largest possible order. Everyone in the supply chain benefits from quantity. Manufacturers keep production lines running longer, and salespeople make heftier commissions. Vendors understand that buyers need inducements to load up on inventory, so they offer discounted unit prices at higher quantities. Sounds like a win-win situation, right? Perhaps not.

     Suppose your agency uses 3,000 nonrebreather masks annually. If you purchase four month's supply, the price might be $2 each. But before you can say "oxyhemoglobin," the sales rep offers to reduce the price to $1.95 if you agree to buy at least 3,000 masks—a year's supply—all at once. You would save five cents a piece, or $150 on the order—or would you?

     The total price of the larger order would be $5,850. That's $3,850 more you'd be spending today (3,000 masks at $1.95 each vs. 2,000 at $2). If your company can earn 1% per month, either by investing that sum or by not having to borrow as much money to keep the business afloat, then your agency makes $154 (not compounded, before taxes) just by leaving that $3,850 in the bank for four more months.

     Next month you would withdraw $2,000 from the $3,850 to buy another four month's supply of NRBs at $2 each. That leaves $1,850 to earn interest for four months ($74). The annual interest earned by making three smaller purchases instead of one large buy is approximately $230 or 53% more than the savings based solely on the purchase price! Time to go back to the bargaining table.

     That $230 figure is known as inventory holding cost. It is the price you pay for spending money on goods you don't need right away. Before you accept a quantity discount, make sure it's not costing you more to hold inventory than you're saving on the purchase.

     Another cost to consider is lost opportunity. Suppose your agency is asked to do an interfacility transport one hour from now, but you're short ambulances or personnel. Not doing that transport costs whatever your company would have earned on that job (revenue minus expenses). But there is also a possibility that the customer will be disappointed by your inability to respond and deny your agency future profitable business. Lost opportunity costs can help justify additional staff or equipment.

5. SAFETY

     We tell our employees that nothing in EMS is more important than safety. There are, however, limited opportunities to measure it.

     Consider the lost-time accident (LTA). When an employee is hurt on the job and misses work due to that injury, your company is the proud owner of an LTA. One way of quantifying LTAs is to consider the number of employee-hours lost due to work-related injuries.

     For example, if Medic A misses two eight-hour shifts and Medic B can't work three 12-hour tours, the total would be 52 employee-hours lost. You can calculate the cost of LTAs by considering sick time, disability payments and other benefits owed employees who can't work.

     LTAs also can be dollarized according to vehicle-days lost. Suppose an ambulance is damaged and taken out of service for a week. That's seven days of insurance and other fixed expenses that won't be offset by revenue. Lost opportunity costs might be a factor, too.

BEYOND BOOKKEEPING

     Not everything about enterprise can be squeezed into a spreadsheet. Peters addresses this by warning of "analysis that strives to be precise about the inherently unknowable."¹ An EMS example would be the proportion of cardiac arrest patients who regain consciousness after bystander-initiated CPR. Without knowing how many of these people truly were pulseless, there is no practical way to calculate the percentage who survived despite, rather than because of, chest compressions.

     The key is transforming data into information. "Data is power," says Wayne Zygowicz, EMS chief of Littleton (Colo.) Fire Rescue. "Using the right data for the right reasons at the right time is the power to get the job done."⁹

References

1. Peters T, Waterman R. In Search of Excellence: Lessons from America's Best Run Companies. Grand Central Publishing, 1982.

2. Silver J. Movie Day at the Supreme Court or "I Know It When I See It": A History of the Definition of Obscenity. https://library.findlaw.com/2003/May/15/132747.html.

3. Right Brain vs. Left Brain. www.funderstanding.com.

4. Pons P, Markovchick V. Eight minutes or less: Does the ambulance response time guideline impact trauma patient outcome? J Emerg Med 23(1):43–48, Jul 2002.

5. Pons P, Haukoos J, Bludworth W, et al. Paramedic Response Time: Does It Affect Patient Survival? www.defrance.org.

6. Price L. Treating the clock and not the patient: Ambulance response times and risk. Quality and Safety in Health Care 15:127–130, 2006.

7. Bledsoe B. Is EMS Response Time a Good Indicator of EMS System Performance? www.merginet.com.

8. Harvey A, Gerard W, Rice G, et al. Actual vs. perceived EMS response time. Preh Emerg Care 3(1):11–14, Jan–Mar 1999.

9. Zygowicz W. Simple reports for a perfect EMS world. J Emerg Med Serv 32(8):66–68, 2007.

Mike Rubin, BS, NREMT-P, is a paramedic in Nashville, TN, and a member of EMS World’s editorial advisory board. Contact him at mgr22@prodigy.net.