Introduction: Setting Damages is a Complex Matter

In civil litigation, few questions are as important as where to set damages. Beyond the basic issue of who prevails, litigants, juries and courts struggle with the issue of how loss should be compensated. What is an accurate estimation of the impact on the plaintiff of the event under dispute? In the corporate setting, that impact generally involves some combination of lost profits, lost enterprise value through reduced business valuation. To estimate these accurately and fairly is an extremely complex matter. Some of the elements – such as damage to reputation – are arguably subjective. Even the more objective elements, such as lost income, are not straightforward. They require the development of alternate histories – one of them concerning what happened in fact, the other representing what might have happened had the event in question not occurred or “but-for” the event in question.

Benchmarks Used by Expert Witnesses Are Increasingly the Basis for Successful Daubert Challenges

To arrive at claims for damages, litigants turn to expert witnesses – in particular, financial, economic, and accounting experts who, it is assumed, can apply established models and methodologies. The role of these experts is to set benchmarks that can be used to measure performance and determine – in ways that are authoritative and will be clear to the court – how much was lost. But for too many litigants and their attorneys, the participation of expert witnesses provides no assurance at all of success. The benchmarks they use are wildly inaccurate – in some instances, so much so that their testimony is excluded on Daubert challenge and never comes before the court at all. The result is that the case is put at risk. Daubert challenge is a significant issue for litigants and counsel. Its use and its effectiveness are on the rise. A recent study showed that in 2010, almost 50 percent of expert witnesses were excluded on Daubert. Many of these challenges focused on the assumptions behind benchmarks and their reliability. In the well-known case of Celebrity Cruises, Inc. v. Essef Corp., Judge Francis excluded five of seven Celebrity plaintiff experts and one of three defendant experts on Daubert. We will examine the Daubert aspects of this case in some detail.

The Problem with Benchmarks: They Make Too Many Assumptions, and Many of Those Assumptions Are Wrong

Why is benchmarking so inaccurate? The process of benchmarking involves making assumptions – about future business performance, economic conditions and the competitive landscape of the litigant’s industry. As we all know from well-worn bromides, making assumptions is a dangerous business. The particular problem here is that any assumption will ramify through years of projected business or financial performance. Even a small error in the assumption can lead to a wildly inaccurate conclusion. In addition to the general problem of making assumptions, there are specific problems as well. One is reliance on a “one-size-fits-all” approach. Many experts rely on a single benchmarking methodology that they apply across a wide range of situations. The methodology may or may not fit. If it does not, a successful Daubert challenge is likely to result. There is a closely-related problem: Many experts apply ex-ante benchmarks, the kind of forward-looking analysis appropriate to litigation for shareholder damages, to the calculation of lost profits and lost enterprise value, a setting where ex-post benchmarks ought to apply. Valuation experts and investment bankers use metrics that are designed to forecast future performance. They measure past performance, then look ahead in order to create a prediction, using data that might include the company’s own forecasts. In estimating damages, associated with events that lead to lost profits or business reputation (lost enterprise value), the task is different – the starting point is a given date in the past, the date when the alleged harmful event took place, and the requirement is to try to calculate two different scenarios. One scenario reflects what actually happened, and the other reflects what might have happened if the event had not occurred. There is more and different data available – including information about how the company, the industry and economy actually performed. None of that would apply in projecting a valuation for shareholder damages, but it is essential for calculating lost profits or lost enterprise value. Many experts have had their testimony excluded specifically because they used valuation-type metrics in cases involving damages from harmful acts. Benchmarking for damages to shareholders resulting from accounting or financial fraud does, however, need to be based on forward-looking valuation methods. It is critical that experts – and litigators – know the different requirements and apply the right benchmarking methods for the particular type of case.

Bad Benchmarking in Action: In the Celebrity Cruises Case, Daubert Challenges Derailed a Parade of Experts

What is the impact of questionable benchmarking practices on a case? For a dramatic example, one need look no further than the matter of Celebrity Cruises, Inc. v. Essef Corp., 434 F. Supp. 2d 169 - Dist. Court, SD New York 2006 No. 96 CIV. 3135(JCF). The background, in brief, is this: Celebrity Cruises is an upscale cruise line. It was an independent company until it was acquired by Royal Caribbean (RCCL, later RCL) in 1997. In the summer of 1994 there was an outbreak of Legionnaire’s Disease aboard one of Celebrity’s ships, the Horizon, during a New York to Bermuda cruise. A number of passengers sued Celebrity and also Essef, the manufacturer of the ship’s whirlpool spa, which according to the U.S. Centers for Disease Control and Prevention had been the source of the outbreak. In a bellwether procedure – where a verdict for one plaintiff determines liability for all parties – Celebrity was found to be 30 percent liable for the outbreak and Essef 70 percent liable. Damages were awarded to the bellwether plaintiff, a married couple. Celebrity then filed its claims against Essef, seeking compensation for direct costs (such as the damages it paid to the plaintiffs and the cost of decontaminating the ship) as well as lost profits during the period from the outbreak to its acquisition by RCCL, and lost business value. Celebrity claimed that because of the outbreak, its purchase price was lower than it would otherwise have been. Celebrity identified experts to testify concerning all categories of damages. Essef chose not to challenge the Celebrity experts who would address direct costs and damages already paid. But Essef did challenge the experts who would address the more abstract claims about Celebrity’s lost profits and lost business value. It also challenged a Celebrity expert who opposed Essef’s motion for summary judgment. Celebrity, in turn, challenged each of Essef’s three experts.

The Experts and Why They Failed

The challenges produced a sort of expert witness bloodbath. Here are some of the most critical challenges and how they turned out:

• Tracking lost profits – using a price-increase spike: James Winchester, a former Wall Street stock analyst who was president and sole employee of Quantified Value Partners, testified for Celebrity and developed an estimate of lost profits. He linked profits to pricing, then compared Celebrity’s pricing to a market proxy, using a comparison period from March 1993 to June 1994, the period before the outbreak. He then projected the pricing forward to show what profits would have been had there been no outbreak. But on examination, it turned out that Celebrity’s prices were significantly lower than the proxy’s in March 1993. Over the next 15 months, Celebrity’s prices rose dramatically, closing the gap. But there was nothing to indicate that Celebrity’s upward trend would have continued. If anything, prices might have fallen as Celebrity rolled out three massive new ships – adding capacity that might have reduced demand. In fact, Celebrity’s own management pointed to a weakening of demand in 1995. There was no evidence that Celebrity’s pricing could have continued to outpace the market. Mr. Winchester’s testimony was found to lack the reliability required under Daubert, and his testimony was excluded.

• Transaction benchmarks that compared apples to oranges: Mr. Winchester also analyzed Celebrity’s lost enterprise value, comparing the Celebrity acquisition to two other large industry transactions – the acquisition of Norwegian Cruise Line (NCL) by Star Cruises and the merger of P&O Princess with Carnival. But the others weren’t really comparable. The scales were radically different – Celebrity had 8,000 berths, NCL 12,000 and P&O Princess 30,000. The other two transactions were subject to competitive bidding, whereas the only other offer for Celebrity was a last-minute affair. Finally, the other transactions took place roughly two and a half years after Celebrity’s, and Winchester did not account for market changes over the intervening years that might have affected the relative value of the sales. His lost enterprise value analysis was also excluded.

• Projecting lost enterprise value using forecasts instead of peer or industry performance as the basis: Pamela M. O’Neill, a principle of XRoads Solutions Group, projected an expected growth rate for Celebrity’s revenue which in turn was based on a proxy consisting of RCCL and Carnival. She took projected growth rates for each of the two companies as established by analysts in 1994, then applied them to Celebrity. The problem is that the market proxy did not in fact display anything close to the projected growth rates. According to analysts, RCCL’s revenue was expected to grow at rates ranging from 2.5 to 4.9 percent between 1994 and 2000; Carnival was expected to grow at rates ranging from 0.6 to 2.1 percent. As it happened. RCCL’s growth in 1995, 1996 and 1997 was 0.21 percent, -1.08 percent and -1.25 percent. Carnival’s growth rate during the same three years was -2.6 percent, -3.7 percent and -1.7 percent. Ms. O’Neill admitted in her deposition that she did not know these actual figures were available, and later acknowledged she would have considered them had she known. Forward-looking methodology such as this can be appropriate for valuing an enterprise at a single point in time. But it does not adequately measure damages that occur after the point when the projection is made. In this case, all three companies lost ground. But Ms. O’Neill attributes Celebrity’s shortfall to the effects of Legionnaire’s Disease, while making no attempt to determine why the other two companies declined. Even worse: Ms. O’Neill’s lost enterprise value calculation began with Celebrity’s 1993 budget projections, which she then compared to 1997 actuals. As Judge Francis pointed out: “Indeed, to take 1993 as an example, Celebrity budgeted earnings before interest, taxes, depreciation, and amortization (EBITDA) of $61.9 million for the Horizon and its sister ships, the Zenith and the Meridian, but the actual EBITDA for those three vessels in that year was $55.4 million. Using Ms. O’Neill’s methodology, this would indicate damages of over $83 million for that year, even though the Legionnaires’ outbreak had not yet occurred.” Her evidence was excluded.

• Taking the company’s word for growth rates: Dr. David B. Lasater, a senior managing director of FTI Consulting Inc., did without industry proxies. Instead, he estimated projected profits based on Celebrity’s own five-year plan as formulated by management in January 1994. He compared anticipated profits with actual profits and concluded that Celebrity had lost approximately $101 million He then adjusted his projections in various ways, calculating higher or lower lost-profit figures, but always using the company’s own projections as the basis. Judge Francis rejected this approach stating: “Dr. Lasater’s lost profits analysis is flawed in at least one major respect: the projection of profits based on Celebrity’s five-year plan is wholly unreliable. [T]he entrepreneur’s ‘cheerful prognostications’ are not enough.” Schonfeld v. Hilliard, 218 F.3d 164, 173 (2d Cir.2000) (quoting Dobbs Law of Remedies § 3.4). Indeed, Robert P. Schweihs, another of Celebrity’s experts, explicitly rejected use of the five-year plan to project anticipated profits after December 31, 1994. To calculate the value of the company after the outbreak, Dr. Lasater began with the $1.312 billion purchase price paid by RCCL, then adjusted it for anticipated synergies. But he does not take into account that the purchase price – including the synergies – was negotiated between Celebrity and RCCL. Celebrity might have lost bargaining power because of the Legionnaire’s outbreak, or there might have been balanced negotiations. In either case, the synergy figure is the result of a negotiation – it is too subjective to be the basis for an enterprise value calculation. For his excessive reliance on the company’s own projections, Judge Francis noted that: “A methodology so sensitive to one highly subjective variable lacks the necessary reliability.” Thus, Dr. Lasater’s testimony was excluded.

• Using more sophisticated methodologies…that still rely on the company’s growth projections: The lost-profits analysis by Allan Pfeiffer, managing director of Standard & Poor’s Corporate Value Consulting, also relied on Celebrity’s five-year plan. He took a more conservative approach than Dr. Lasater – for example, excluding out-of-pocket and “brand repair” costs – but still used Celebrity’s projections as the basis for his calculations, projections that were not borne out in reality. Judge Francis noted that: “This analysis suffers from the same fatal flaw as Dr. Lasater’s methodology: reliance on projections that were not borne out in reality. This defect drives the entire calculation and is not repaired by identifying a lower bound using a methodology which, standing alone, might be more reliable. Mr. Pfeiffer’s lost profits analysis is therefore excluded.” This fatal flaw is the basis for the entire calculation, and using lower boundaries for the estimate does not offset the fundamental problem. His lost enterprise value analysis used six different sets of calculations to arrive at an average figure, but still relied either on Celebrity’s five-year plan or a 1995 projection prepared for the company by The Blackstone Group LLC, that was nearly identical. Mr. Pfeiffer developed a reasonable rate of return analysis, but did not justify it by comparing it to other companies. Mr. Pfeiffer’s testimony was thus excluded.

• Using benchmarks of company suffering alleged stigma does not control for stigma to other ships: Joseph E. Obermeyer, the principal of Obermeyer & Associates, Inc., a business consulting firm, testified for Essef, providing an expert opinion that criticizes the reports of Celebrity’s witnesses and also includes an independent evaluation of Celebrity’s damages. The centerpiece of Mr. Obermeyer’s report is his evaluation of Celebrity’s lost revenues. He began by identifying a relationship in revenue per passenger cruise day produced by the Horizon and its sister ships, the Zenith and the Meridian, before the Legionnaires’ outbreak. Mr. Obermeyer then observed that revenue declined sharply for the Horizon and the Meridian in the quarter immediately after the outbreak, but not for the Zenith. By the fourth quarter of 1994, the Meridian’s revenues had returned to their pre-incident pattern, and by the third quarter of 1995, the Horizon had as well. Mr. Obermeyer then compared the actual revenues for these three vessels to that which he projected based on their prior performance. He concluded that Celebrity experienced a revenue shortfall of approximately $5.7 million during the third quarter of 1994, $1.6 million in the fourth quarter of 1994, and $325,000 in the second quarter of 1995. Judge Francis rejected this benchmarking approach stating: “The fallacy of this analysis is that it assumes that the Zenith was unaffected by the Legionnaires’ incident.” Mr. Obermeyer’s testimony was thus excluded.

• Bad yardsticks…and ignoring your own methodology: The only Celebrity expert to testify at trial was Robert P. Schweihs, managing director of Willamette Management Associates. He presented a lost-profits analysis showing the impact of the outbreak on Celebrity’s EBITDA, comparing it to a yardstick made up of three other cruise lines. He also developed a lost enterprise value analysis based on discounted cash flows, using a formula to determine the weighted average cost of capital (WACC). Based heavily on his testimony, the jury awarded Celebrity damages of $190 million for lost profits and lost enterprise value.

Essef moved for a new trial, citing serious flaws in Mr. Schweihs’ presentation. There were several, Essef said. Essef contended that the companies that made up the yardstick were too dissimilar to be useful – one was industry giant Carnival, with billions in revenue and multiple brands; the other was a company that mainly operated riverboat cruises. Finally, and most significantly, Essef argued that in developing the lost enterprise value calculation, Mr. Schweihs had ignored a basic tenet of WAAC calculation – he failed to take into account the impact of debt on the capitalization of the business. He used a generalized cost-of-capital figure for the cruise industry, but did not “relever” it to apply a debt figure to the yardstick companies or to Celebrity. In so doing, he ignored the advice he laid out in his own textbook. As he later acknowledged, he misspoke and said that he had

Consequences: Deficient Expert Testimony Leads to a Drastic Reduction in the Award

As a result of the flaws in Mr. Schweihs’ testimony, the verdicts for lost profits and lost enterprise value were reversed by Judge Francis. At a new trial, this time seeking damages of roughly $60 million rather than the $190 million originally awarded, Celebrity and Schweihs again presented the yardstick analysis with the two comparable companies that had been criticized by the court. Schweihs added a third comparable company that was considered more reliable. Yet his analysis reached a similar valuation. This time the jury returned $15 million in damages to Celebrity. On appeal, the court found the evidence sufficient to support the award, but applied a thirty percent reduction for comparative fault. Celebrity Cruises, Inc. v. Essef Corp., 530 F. Supp. 2d 532 (S.D.N.Y. 2008). In February 2008 the District Court entered a $30.4 million judgment against Pentair, the company that had acquired Essef, for out-of-pocket costs, expenses and lost profits, including interest. Celebrity and Pentair both appealed. According to a 10-Q filing by Pentair on July 22, 2008, the parties agreed to a $35 million settlement – a sharp reduction from the $190 million jury verdict, and one attributable in large measure to the unreliable benchmarking produced by Celebrity’s experts. Essef also had some of its experts excluded on Daubert, and of the Essef experts who did testify, some of their testimony was excluded as well. But the impact of the challenges fell most directly on Celebrity and had the greatest effect – a negative one – on Celebrity’s case.

Consulting Experts – Who Help Develop the Case but Do Not Testify – Can Improve Expert Testimony and Reduce the Chances of a Successful Daubert Challenge

What can be done to improve the reliability of expert testimony? Clearly, when it comes to benchmarking, there is a great deal of sloppy, unreliable work being done. But how is a litigation team to know that the benchmarking is unreliable? While many litigators are conversant with valuation and the estimation of damages, it’s unlikely that most have the technical or academic background to recognize and correct a flawed methodology for, say, WAAC calculation. The answer is simple – set an expert to catch an expert. Typically, litigators hire experts late in the trial preparation process. The expert’s role is to prepare testimony that supports the case. The alternative – one that every litigator should consider – is to engage an expert at the beginning of the trial process, not to testify but to help develop the strategy of the case, vet the experts, and prepare Daubert challenges to the opposition’s experts. In the Celebrity case, it was the consulting expert that helped bring down all seven of Celebrity’s experts – five through exclusion, one through a fatal flaw in his own prescribed methodology, and one during depositions, before the expert’s testimony ever saw the light of day. The consulting expert performed an analysis of the expert reports, sat in on the depositions and trial, and ultimately used the experts’ own responses to impeach their credibility. The expert who was discredited during his deposition was excluded by Celebrity themselves even before Essef asked for his exclusion under Daubert. In general, a consulting expert can add several kinds of value – value that stretches beyond the question of benchmarking – to the law firm and the litigation team, including:

• Improved win rate. The consulting expert can help the litigation team and the firm’s management determine the odds of winning the case, so that they can decide whether it is worth the investment of time and resources. They might instead decide the costs are not justified and they should not pursue the case or, if on defense, settle it.

• Strategy. The consulting expert can inform the development of strategy, guiding the litigation team toward winning arguments and away from weak ones.

• Discovery. The consulting expert can work with the litigation team to determine what information should be requested and what is most critical to the success of the case.

• Summary judgment. The consulting expert might be able to identify a quick way to show that the case should be quickly dismissed, for example due to a flaw in the way damages are calculated.

• Mediation. The consulting expert can help show weaknesses in the case before the mediator that will bring about a quick settlement.

• Expert reports and rebuttal reports. The consulting expert allows full flexibility –with little or no risk of opposition discovery – to test alternative theories of damages and effectively rebut the opposition’s experts.

• Depositions. The consulting expert can prepare questions and in most cases sit in on the deposition in order to help with follow-up questions.

• Trial. The consulting expert can play a significant role in preparing for trial and can even sit in at the trial to help prepare follow-up questions to witnesses.

All of this is in addition to the consulting expert’s obvious role in preparing Daubert motions. The central responsibility here of the consulting expert is to show exactly why the opposing expert is unqualified or is using unscientific methods to reach conclusions. While it can be costly to engage an expert so early in the process, the cost is more than justified by the expert’s impact on win rate and on the quality of the case that is presented. The consulting expert can be a powerful factor and can make an enormous difference in the case – so much so that you may want to limit the expert to an exclusive consulting role and not require him or her to testify.

The Most Critical Need is a Better Approach to Benchmarking

The consulting witness can be an important contributor across the board. But as the Celebrity case shows, there is also the specific need for improved benchmarking. Until that can happen, litigation will be compromised, and courts will find it difficult if not impossible to arrive at damages awards that are fair and accurate. What the consulting expert needs to correct is the flawed tendency to use inappropriate benchmarks such as those used for ex-ante valuation modeling in the assessment of damages. To be defensible, benchmarks need to conform as closely as possible to business and economic reality. In practice that means applying ex-post benchmarks – measurements applied at the end of the period under scrutiny, that factor in such details as operating costs, fluctuations in business demand and changes in the economy. Ex-ante benchmarks – the kind of forward-looking assumptions used by investment bankers – are error-prone and are best avoided. Worse is using an ex-ante projection that is based solely on plaintiff’s own forecast, such as the management forecasts used to predict but-for performance in the Celebrity case. An expert who understands the operating realities of a business is an essential partner in arriving at an accurate benchmark. Litigators need to make sure that they have that hardcore, economic and operating experience at the table – as the consulting expert, guiding the benchmarking work of other experts, or as a testifying expert, but one with the right skills and the right viewpoint. The result will be expert witnesses and cases that have a much better chance of standing up in court.

About Don May, Ph.D.

Donald M. May, Ph.D., is a Director in the Litigation and Corporate Financial Advisory Services Group at Marks Paneth LLP. Dr. May has prepared expert reports and expert witness testimony related to business valuation, hedge fund valuation, lost profits, lost enterprise value, time-series forecast models, asset and investment portfolio valuation and statistical forecasting models and methodologies.