ABA Studies ‘Super,’ ‘Best,’ and Other Lawyer Rankings (Part II)

By Roy Simon

[Originally published in NYPRR October 2011]

In NYPRR June 2011 issue, I began discussing the controversial subject of lawyer and law firm rankings and ratings. I explored the history of lawyer rankings and ratings and the impact of Bates v. State Bar of Arizona, 433 U.S. 350 (1977). I described some active rankings and ratings companies, and I recounted the litigation over Super Lawyers and Best Lawyers in New Jersey. This month I will examine how the ABA got roped into studying lawyer rankings and ratings, and I will describe the valuable Informational Report that resulted from the ABA’s work.

The ABA Debates Law Firm Rankings and Ratings

About a month before the ABA’s February 2010 mid-year meeting, U.S. News & World Report (U.S. News) announced that it had joined forces with Best Lawyers and planned to begin ranking law firms in Fall 2010. The U.S. News/Best Lawyers team would rank law firms in over 100 metropolitan areas, and roll those local rankings into state and national rankings. The rankings would identify what U.S. News considered to be the top few law firms by practice area or by other criteria.

Given the questionable reputation of U.S. News in the law school world, this was troubling news for many practicing lawyers. As virtually every aspiring lawyer in America knows, U.S. News & World Report publishes an annual ranking of law schools. it is the dominant player in the field. (Can you name another entity or person that ranks law schools?) Yet the U.S. News rankings are of dubious accuracy. For example, a 2009 study by Louis H. Pollack in the Drexel Law Review found the U.S. News list of the “Top 100 Schools” to be “flawed;” and a study in the SMU Law Review found the results “unreliable” and subject to “significant random error.” In addition, many law school professors and deans believe that the U.S. News rankings have adversely affected legal education and increased law school tuition by distorting spending and policy priorities. Many lawyers were concerned that U.S. News rankings of law firms would similarly distort the world of law practice.

The New York State Bar Association responded forcefully to the U.S. News plans. It sent a report to the ABA saying that the ABA has an obligation to carefully review all issues related to attempts to rank law firms and urged further study of the issue. “If the rankings are not valid,” the report stated, “lower ranking law firms and lawyers will be unfairly penalized.” The New York State Bar also urged the ABA House of Delegates to adopt the following resolution:

RESOLVED, That the American Bar Association examine the efforts by U.S. News & World Report, working with Best Lawyers, to publish a national, state and local ranking of law firms.

FURTHER RESOLVED, That the American Bar Association examine the methodology being used to develop the rankings to determine the accuracy and validity of any such rankings, examine the ethical issues involved in lawyers and law firms responding to requests for information for the rankings, and examine all issues relevant to the rankings.

FURTHER RESOLVED, That a report of the examination be presented to the House of Delegates at the 2010 Annual meeting.

FURTHER RESOLVED, That pending completion of the final report and its recommendations, the American Bar Association urge lawyers and law firms to exercise caution in determining whether to provide information for the purpose of ranking law firms.

Vince Buzard, a Rochester lawyer who served as President of the New York State Bar Association in 2005–2006 and was the leader of the NYSBA group pressing the ABA to study the validity of law firm rankings and ratings, presented the NYSBA’s proposed resolution to the ABA House of Delegates at that ABA’s February 2010 mid-year meeting. (I do not think Mr. Buzard has any personal bone to pick with the rating companies, which give him high marks. As the opening paragraph of his web page at Harris Beach notes, Mr. Buzard has been “[s]elected as one of the Best Lawyers in America in appellate practice and by New York Super Lawyers in his appellate and business litigation practices…”) Mr. Buzard spoke passionately about the potential dangers of lawyer and law firm rankings, calling them random and flawed. Especially troubling, U.S. News was beginning to ask lawyers for detailed information about their law practices — the names of their clients, the types of matters they handled, how many matters of that kind they had handled, how those matters were resolved, etc. Before lawyers went to the trouble of providing that information, Mr. Buzard thought the ABA should provide lawyers with some guidance as to whether the rankings were valid.

Mr. Buzard identified three specific harms that could stem from the U.S. News rankings.

First, if the rankings are statistically or methodologically invalid, then they are unfair to lawyers who receive lower rankings than they deserve. And once the rankings become entrenched — as the U.S. News law school rankings have become entrenched — lawyers with unfairly low rankings will be marked, and their low rankings will haunt them for years. For these reasons, Mr. Buzard urged the ABA to hire expert consultants to assess the validity of the methodologies used by anyone presuming to publish rankings.

Second, as a December 2009 study by the Government Administrative office (GAO) demonstrated, U.S. News law school rankings are a major driver of law school cost increases, because law school deans cannot afford to ignore them. One ingredient of the U.S. News law school rankings is how much the law school spends per student in certain categories (e.g., faculty and library). That creates a disincentive for law schools to contain costs. Although Mr. Buzard did not expressly say so, he implied that if law firm rankings take into account expenditures per client or per lawyer on advertising and promotion (e.g., client newsletters or technology resources), then those factors will increase the cost of doing business for law firms. (Mr. Buzard did not predict whether legal fees would consequently increase or law firm profits would decrease, but presumably either one would be undesirable.)

Third, he said that if law firm rankings and ratings contained misinformation, then the information would harm consumers, not help them. The legal profession had an obligation to protect the public from misinformation.

Mr. Buzard recognized that it could be risky for firms to take steps to prevent or restrict the flow of information to consumers, but he said that law firm rankings and ratings could profoundly affect law firms and the ABA should act to control them. Most big firms did not have to worry about rankings and ratings, but small and medium firms did need to worry, because they would be the ones to get whacked by unfairly low rankings.

Taking issue with Mr. Buzard’s approach, former NYSBA President Steven Krane said it was not time for a resolution requiring the ABA to study particular issues regarding law firm rankings and ratings. Speaking in his capacity as a member of the ABA Board of Governors and as Liaison to the ABA Commission on Ethics 20/20, he noted that the Ethics 20/20 Commission was studying the information available to the public concerning legal services, so the issue of law firm rankings and ratings was already properly before the Commission, and the House of Delegates should let the Commission do its work. The House of Delegates should not attempt to dictate to the Commission which issues to study concerning law firm rankings and ratings. Mr. Krane urged the delegates to postpone a vote on the resolution indefinitely so that the Ethics 20/20 Commission could conduct a full and fair study, without the constraints imposed by the proposed NYSBA resolution.

In response Mr. Buzard expressed doubt that the Ethics 20/20 Commission would do enough. He said the Commission’s goals were limited to studying whether existing law adequately protects the public when the public uses websites to rate or evaluate lawyers. But ABA President Carolyn Lamm, who agreed with Krane, said the issue of ratings belonged with the Ethics 20/20 Commission. She joined Krane in recommending that the proposed resolution be postponed indefinitely.

Next, a lawyer named Anthony Palermo from the Senior Lawyers Division expressed great alarm over ratings by outside organizations that he said had no rational basis or methodology. The lawyers of America wanted guidance on what the ranking and rating criteria were, he said, and they needed that guidance right away, because the U.S. News rankings were due out in a matter of months. He said it would be a simple thing to find out what the methodology used would be, who would do the ratings, and what the criteria would be.

A law professor from Thomas Cooley School of Law (one of the bottom-ranked schools in the U.S. News law school rankings) noted that the U.S. News law school rankings had been criticized by nearly every law school dean. For example, U.S. News did not take diversity into account in its rankings, and law schools therefore often diverted few resources to increasing diversity, because raising diversity could not raise a law school’s rankings. He urged the profession to act so that law firm rankings and ratings would not cause law firms to divert their resources in order to move higher in the rankings and skew decisions at law firms in the same way that U.S. News law school rankings had skewed decisions at law schools.

A few other speakers, and the report that the New York State Bar Association had submitted in support of its resolution, suggested that lawyers might be breaching the duty of confidentiality by supplying information to rankings and ratings providers, and that an amendment to Rule 1.6 (“Confidentiality of Information”) might be necessary to deal with these breaches.

In the end, the arguments of Mr. Buzard (and others) persuaded the House of Delegates to defeat the motion to postpone indefinitely by a narrow 209 to 190 vote. But the House of Delegates did not adopt the resolution (10A) in the form originally championed by the New York State Bar Association. Instead, the delegates adopted a much shorter resolution, which did not refer to U.S. News by name and called only generally for a study of lawyer and law firm rankings. The final resolution said simply:

“RESOLVED, That the American Bar Association examine any efforts to publish national, state, territorial and local rankings of law firms ….”

The Ethics 20/20 Commission’s Informational Report

Shortly after the February 2010 meeting, ABA President Carolyn Lamm asked the Ethics 20/20 Commission to create a Working Group to study law firm rankings and ratings. For more than a year, the Working Group conducted research and engaged in extensive outreach and analysis. It reached out to dozens of bar associations, to disciplinary counsel, to consumer groups, to entities that rank and rate lawyers, and to professors who had engaged in social science research pertaining to the legal profession. On April 27, 2011, after a year of study, the 20/20 Commission released a “Draft informational Report” and filed it with the ABA House of Delegates. The draft was slightly revised in August of 2011. I will now discuss that Report in detail.

Regarding its outreach efforts, the section of the informational Report headed “Lawyer Disciplinary Counsel” was especially interesting. The 20/20 Commission asked all chief disciplinary counsel for information regarding “any complaints filed in the last five years regarding lawyers’ participation in ratings or rankings services or lawyers’ use of the results of those services.” The responses were as follows:

• Disciplinary counsel from 13 jurisdictions, including three of New York’s Judicial Departments, advised that no complaints had been filed and no disciplinary actions had been instituted against lawyers for their participation in rating or ranking services or for their use of the results of those services.

• Disciplinary counsel from one jurisdiction indicated that its office had received “less than five” complaints, that all those complaints had been from other lawyers and none warranted discipline of any kind.

• Disciplinary counsel from another jurisdiction stated there had been one grievance filed during the past five years, but that no cases had resulted in public discipline over that time.

Regarding methodology and validity — which were the key concerns of those who were disturbed about rankings and ratings — the professors consulted by the Working Group advised that they were not aware of any standardized methodology for evaluating law firm and lawyer ratings and rankings. Accordingly, to evaluate a rating or ranking service, a methodology would have to be created. Professor Deborah Hensler, Ph.D., the Judge John W. Ford Professor of Dispute Resolution and Associate Dean of Graduate Studies at Stanford Law School, indicated that the process of developing a methodology would require a team that includes (i) psychometricians and other measurement experts, (ii) survey research methodologists, sampling statisticians, (iii) experts on legal education, (iv) legal sociologists who have focused on the organization of law firms and other legal service providers, and (vi) industrial organization and labor economists who have focused on the legal services industry. Professor Hensler pointed out that different methodologies could come to different conclusions about any particular rating or ranking service. In addition, a comprehensive determination of the statistical validity of a rating or ranking service was likely to be very expensive — perhaps exceeding a million dollars.

The Informational Report then addressed some of the widely varying methodologies used by the various rankings and ratings providers. For example:

• A few providers rank law firms in numerical order according to specific criteria.

• Others rank lawyers or firms by placing them in tiers, bands or categories.

• Some list lawyers or firms in directories according to certain criteria.

• Consumer feedback websites were emerging as a “popular vehicle” for distinguishing lawyers from one another.

Some rating and ranking services were national, but many were regional or local, and were distributed by business publications of varying types. The Informational Report identified about 35 local or metropolitan areas throughout the country with such ratings or rankings, as well as 25 states with state and region-wide publications expressing views on top lawyers or firms in various categories. Overall, the Informational Report identified “several hundred” different lists purporting to identify top lawyers or firms in communities around the country, many of which had done so for years. In fact, ratings and rankings have become “so prevalent” that some law firm marketing consultants now specialize in assisting law firms to maximize their rating and ranking.

Legal Authorities on Regulating Law Firm Rankings and Ratings

The Informational Report next discussed some problems of regulating rankings and ratings services and with regulating the use to which lawyers and law firms put their rankings and ratings. Whether in print or online or both, the ratings and rankings services enjoyed First Amendment protections, and to that extent the legal profession and the organized bar “have no authority to directly govern their conduct.”

The Report noted that many, if not the majority, of ratings and rankings services are not dependent on the lawyer’s or law firm’s cooperation. The services obtain information from third parties (including clients and other lawyers) or from other publicly available sources. Moreover, since Bates v. State Bar of Arizona, the Report said, it has been clear that First Amendment protection extends specifically to lawyer advertising as a form of commercial speech. The Court in Bates held that a state may not constitutionally prohibit a lawyer’s advertisement of fees for routine legal services but also held that a state may prohibit commercial expression that is false, deceptive, or misleading and may impose reasonable restrictions as to time, place, and manner of an advertisement. The Supreme Court expanded on this guidance in Central Hudson Gas & Electric Corporation v. Public Service Commission of New York, 447 U.S. 557 (1980), which explained that while a state may constitutionally prohibit inherently misleading speech or speech that has been proven to be misleading, a state may impose other restrictions only if they serve a substantial state interest, directly advance that interest, and are no more restrictive than reasonably necessary to serve that interest.

The Ethics 20/20 Commission found little case law on the subject of ratings. However, in Mason v. Florida Bar, 208 F.3d 952 (11th Cir. 2000), the court held that a lawyer who truthfully claims that he is “AV Rated, the Highest Rating in the Martindale-Hubbell National Law Directory” is not making a misleading or potentially misleading statement. The Florida Bar had contended that it had “an interest in encouraging attorney rating services to use objective criteria.” The court found no “value in the distinction between objective and subjective criteria” in the specific context before it, and therefore rejected the Florida Bar’s argument. The Eleventh Circuit also rejected the argument that the public’s unfamiliarity with the Martindale-Hubbell Law Directory made the reference potentially misleading. The court said that a state “cannot satisfy its burden to demonstrate that the harms it recites are real and that its restrictions will alleviate the identified harm by rote invocation of the words ‘potentially misleading.’”

The Informational Report then surveyed state ethics rules specifically directed to rankings and ratings. For example, New Jersey Rule 7.1(a)(3) states:

A lawyer shall not make false or misleading communications about the lawyer, the lawyer’s services, or any matter in which the lawyer has or seeks a professional involvement. A communication is false or misleading if it: …

(3) compares the lawyer’s services with other lawyers’ services, unless

(i) the name of the comparing organization is stated,

(ii) the basis for the comparison can be substantiated, and

(iii) the communication includes the following disclaimer in a readily discernable manner: “no aspect of this advertisement has been approved by the Supreme Court of New Jersey…”

New Jersey’s Comment to this language states:

A truthful communication that the lawyer has received an honor or accolade is not misleading or impermissibly comparative for purposes of this Rule if: (1) the conferrer has made inquiry into the attorney’s fitness; (2) the conferrer does not issue such an honor or accolade for a price; and (3) a truthful, plain language description of the standard or methodology upon which the honor or accolade is based is available for inspection either as part of the communication itself or by reference to a convenient, publicly available source.

North Dakota Rule 7.1 was amended in 2010 to state:

… A communication is false or misleading if it: …

(d) compares the lawyer’s services with other lawyers’ services based on the lawyer having received an honor or accolade, unless:

(1) the name of the comparing organization is stated, and

(2) the basis for the comparison can be substantiated.

North Dakota also adopted a Comment that is nearly identical to New Jersey’s Comment.

The Informational Report also summarized all existing bar association ethics opinions regarding rankings and ratings, many of which refer by name to Super Lawyers or Best Lawyers. These ethics opinions may prove instructive to New York lawyers in deciphering the meaning of “bona fide professional ratings” expressly permitted by Rule 7.1(b)(1). As summarized by the Ethics 20/20 Commission, the opinions provide as follows:

Alaska Opinion 2009-2 concludes that it is permissible for a lawyer to refer to a listing in Super Lawyers, Best Lawyers or “another commercial professional ranking so long as the reference includes the publication name, date and the practice area, if one was specified, in which the lawyer was ranked or selected.” …

Arizona Opinion 05-03 concludes that a lawyer may refer to a listing in an advertisement if the lawyer indicates the year of the publication and the specialty for which the lawyer is listed.

Connecticut Advisory Opinion 07-00188 concludes that the designation “Connecticut Super Lawyer” is potentially misleading and the use of the designation in an advertisement must include an explanation and disclaimer in order to avoid creating an unjustified expectation.

Delaware Opinion 2008-2 concludes that it is permissible for a lawyer to advertise that he or she has been designated a “Super Lawyer” or “Best Lawyer” as long as the “lawyer states the year and particular specialty or area of practice of the designation” and the ad is otherwise compliant.

Iowa Opinion 07-09 indicates that a lawyer may advertise the fact that the lawyer is listed in Best Lawyers or Super Lawyers because they are open to all lawyers regardless of whether they subscribe to the services.

Michigan Opinion RI-341 concludes that a lawyer may state that he or she is listed in Super Lawyers, but cannot state that he or she is the best as a result of the listing.

North Carolina Opinion 2007-14 states that a lawyer’s advertisement that mentions inclusion in a listing in north Carolina Super Lawyers is not misleading or deceptive provided that the listing uses objective, verifiable standards; the ad contains the standards or information on how to obtain them; and no compensation is paid for inclusion in the listing.

Virginia Advertising Opinion A-0114 states that a lawyer may advertise that he or she is listed in a publication such as Best Lawyers, but may not communicate credentials that are not based upon objective criteria or legitimate peer review, but instead on willingness to pay a fee.

The Informational Report’s Conclusions

Finally, the Informational Report stated a series of conclusions, which I quote in full:

1. Based upon the information set forth above, the [Ethics 20/20] Commission concludes that Rule 7.1 of the Model Rules of Professional Conduct, as currently written, is sufficient to govern a lawyer’s participation with entities that rate or rank lawyers or law firms. State versions of Model Rule 7.1 have been the basis for ethics opinions that define the boundaries of a lawyer’s participation in a rating or ranking service. These opinions make it clear that the credential may not be based on payment; the lawyer may not overstate the credential, such as stating he or she is the best lawyer because he or she was listed in the Best Lawyers Directory; and the lawyer must include details about the selection, such as the name of the publication, date of selection, and field of practice when publicizing the credential.

States facing specific and identifiable concerns about consumers being misled by ratings or rankings or situations where there is a demonstrated need for the jurisdiction’s lawyers to have additional guidance should make relevant resources available. Those resources could take the form of an online bibliography or the development and adoption of practice suggestions addressing the range of issues pertaining to a lawyer’s participation with entities that rate or rank lawyers or law firms and/or communicate the results of those ratings or rankings.

The ABA Standing Committee on Ethics and Professional Responsibility may also wish to consider whether further explication in a Formal Ethics opinion is appropriate or whether the evidence warrants amendments to the Comment to model Rule 7.1 consistent with the language of the comments to the New Jersey and North Dakota Rules of Professional Conduct cited above.

2. The Commission further requested that the Standing Committee on Ethics and Professional Responsibility consider whether a Formal Ethics opinion on the application of Model Rule 1.6 to a lawyer’s participation with entities that rate and rank lawyers and law firms is needed. Current Model Rule 1.6, governing confidential client information makes clear that a lawyer does not have the authority to provide to a third party information about the representation unless the lawyer obtains the client’s informed consent to the disclosure.

3. Based upon its examination of efforts to publish national, state, territorial, and local ratings and rankings of law firms as described above, the Commission on Ethics 20/20 concludes that the ABA need not, at this time, undertake, support or contribute further resources to the study of this subject for the following reasons:

A. There is a paucity of evidence that there exists a pervasive problem that warrants such an undertaking by the ABA. Further, lawyers and law firms are rated or ranked by hundreds of entities and undertaking a scientific evaluation of the methodologies for all of them is not feasible. Conducting selective evaluations for any particular individual or group of providers could expose the Association to accusations of, or possible litigation for, preferential treatment or inappropriate targeting.

B. Hiring an expert or team of experts to conduct such an analysis would be prohibitively expensive. Because the rating and ranking providers are constantly modifying their methodologies, the ABA would have to retain these experts on an ongoing basis to ensure currency of results.

C. The potential finding that a methodology is “statistically invalid” pursuant to the standards of a psychometrician or team of experts would not necessarily mean that a lawyer’s reference to the rating or ranking is false or misleading pursuant to the Model Rules, particularly when the methodology is available to the public.

D. Were the ABA to engage an expert or team of experts to conduct and publish the results of this analysis, it would open the door for providers to claim that their methodologies had received the “ABA seal of approval” or endorsement. The Commission believes that, even with disclaimers, the risk of such behavior in an internet-driven world is far too high.

Conclusion: The Future of Law Firm Rankings and Ratings

During the debate over law firm rankings and ratings at the ABA’s February 2010 meeting, one of the speakers said that the debate reminded him of the story of King Canute, who ordered the tide not to come in. He said that’s what the proponents of regulating law firm rankings and ratings were trying to do. I agree. The tide is coming in. Just as the legal profession could not stop the tide of lawyer advertising from coming in more than three decades ago in Bates v. State Bar of Arizona, so the legal profession cannot stop the tide of law firm rankings and ratings from coming in today. For example, the team of U.S. News and Best Lawyers, which modestly describes its publication as “the leading survey of lawyers worldwide,” now rank nearly 9,000 firms in 81 practice areas in 171 metropolitan areas and 7 states.

Of course, lawyers must abide by the ethical duty of confidentiality when supplying information to any rankings and ratings entity, and must advertise rankings and ratings truthfully and in a manner that is not false, deceptive or misleading. But when lawyers want to advertise their rankings and ratings, New Jersey’s simple approach should be sufficient. A truthful communication that a lawyer or law firm has been ranked or rated is not “misleading” under Rule 7.1 if: (1) the ranking or rating entity has inquired into the attorney’s fitness; (2) the entity does not issue rankings or ratings for a price; and (3) a truthful, plain-language description of the standard or methodology for the ranking or rating is available either as part of the advertisement itself or by reference to a convenient, publicly available source.

Professor Roy Simon is the author of Simon’s New York Rules of Professional Conduct Annotated. The brand new 2015 edition analyzes more than 100 new cases, ethics opinions, and other developments critical to New York practice. It’s the legal ethics bible for all New York-area lawyers. To purchase, click here.

In addition, Professor Simon advises lawyers and law firms on questions of professional conduct and serves as an expert witness in cases raising issues of lawyer conduct. You may reach Professor Simon at 516-463-5289 or Roy.Simon@hofstra.edu.

DISCLAIMER: This article provides general coverage of its subject area and is presented to the reader for informational purposes only with the understanding that the laws governing legal ethics and professional responsibility are always changing. The information in this article is not a substitute for legal advice and may not be suitable in a particular situation. Consult your attorney for legal advice. New York Legal Ethics Reporter provides this article with the understanding that neither New York Legal Ethics Reporter LLC, nor Frankfurt Kurnit Klein & Selz, nor Hofstra University, nor their representatives, nor any of the authors are engaged herein in rendering legal advice. New York Legal Ethics Reporter LLC, Frankfurt Kurnit Klein & Selz, Hofstra University, their representatives, and the authors shall not be liable for any damages resulting from any error, inaccuracy, or omission.

Lawyer Rankings/Ratings Roy Simon