USNWR law school voters sank Yale Law and Harvard Law for the first time in rankings history.

The USNWR “peer score” is the single most heavily-weighted component of the law school rankings. USNWR surveys about 800 law faculty (the law dean, the associate dean for academics, the chair of faculty appointments and the most recently-tenured faculty member at each law school). Respondents are asked to evaluate schools on a scale from marginal (1) to outstanding (5). There’s usually a pretty high response rate—this year, it was 69%.

Until this year, Yale & Harvard had always been either a 4.8 or 4.9 on a 5-point scale in every survey since 1998.

But this year, Harvard's peer score was a 4.7. And Yale's was a 4.6.

What precipitated the drop (e.g., Harvard could be close to a rounding error, it may have been a 4.76 in the past and a 4.74 in the past) is anyone’s guess. But respondents do tend to react to certain influences, it would seem, and one could only speculate what might have prompted such responses in the fall 2021 or early 2022 when this cohort was surveyed.

Some dramatic swings as USWNR introduces new bar exam metric

The latest USNWR law school ranking has some significant swings in the bar exam component. It made three significant changes: increasing the weight from 2.25% to 3%, and measuring “all graduates who took the bar for the first time,” and including graduates who were admitted via diploma privilege in both a school’s passers and the overall passers. From the methodology:

Specifically, the bar passage rate indicator scored schools on their 2020 first-time test takers' weighted bar passage rates among all jurisdictions (states), then added or subtracted the percentage point difference between those rates and the weighted state average among ABA accredited schools' first-time test takers in the corresponding jurisdictions in 2020. This meant schools that performed best on this ranking factor graduated students whose bar passage rates were both higher than most schools overall, and higher compared with what was typical among graduates who took the bar in corresponding jurisdictions.

For example, if a law school graduated 100 students who first took the bar exam – and 88 took the Florida exam, 10 the Georgia exam and two the South Carolina exam – the school's weighted average rate would use pass rate results that were weighted 88% Florida, 10% Georgia and 2% South Carolina. This computation would then be compared with an index of these jurisdictions' average pass rates – also weighted 88-10-2. (For privacy, school profiles on usnews.com only display bar passage data for jurisdictions with at least five test-takers.) Both weighted averages included any graduates who passed the bar with diploma privilege. Diploma privilege is a method for J.D. graduates to be admitted to a state bar and allowed to practice law in that state without taking that state's actual bar examination. Diploma privilege is generally based on attending and graduating from a law school in that state with the diploma privilege.

In previous editions, U.S. News divided each school's first-time bar passage rate in its single jurisdiction with the most test-takers by the average for that lone jurisdiction. This approach effectively excluded many law schools' graduates who took the bar. Dividing by the state average also meant the location of a law school impacted its quotient as much as its graduates' bar passage rate itself. The new arithmetic accounts for average passage rates across all applicable jurisdictions as proxy for each exam's difficulty and reflects that passing the bar is a critical outcome measure in itself.

The new methodology really changes the results for two kinds of schools. (The increase in the weight from 2.25% to 3% obviously also benefits schools that do well and harms schools more that do poorly.)

First, it benefits good schools in jurisdictions with tougher bars and strong out-of-state placement.

Second, it harms Wisconsin’s two law schools.

Let’s start with the first. Which schools benefited most from 2022 (measuring the 2019 bar) to 2023 (measuring the 2020 bar)? (These charts exclude a handful of schools that did not include their bar passage statistics this time around.)

	Pass rate 2019		Jurisdiction rate	Cumulative pass rate 2020	Cumulative jurisdiction rate	USNWR score delta
San Francisco	38.7%	CA	59%	78.4%	78%	0.0497
William & Mary	86.7%	VA	78%	96.9%	81%	0.0323
Washington & Lee	80.0%	VA	78%	92.6%	81%	0.0317
Emory	84.5%	GA	77%	91.7%	78%	0.0295
Minnesota	94.0%	MN	81%	98.9%	82%	0.0279
Georgia	94.5%	GA	77%	94.4%	76%	0.0272
Kentucky	78.4%	KY	75%	90.6%	80%	0.0266
Montana	88.9%	MT	85%	92.5%	82%	0.0255
Penn State-Dickinson	88.5%	PA	80%	91.7%	79%	0.0249
Drexel	77.1%	PA	80%	84.0%	78%	0.0249

On the left are the school’s pass rate in 2019 with its modal jurisdiction, and that jurisdiction’s pass rate. Next is cumulative pass rate in 2020 along with the cumulative jurisdiction rate. Finally is the delta of the USNWR score—how much better the school did this year compared to last year in the weighted Z-score.

(I noted last year that we saw major swings at some schools in 2020. We see how those are playing out here.)

The University of San Francisco saw a tremendous improvement in California of almost 40 points (aided in part by a lower cut score in California in 2020). But the next three schools are telling. William & Mary and Washington & Lee are strong schools in a very tough bar exam market (Virginia is one of the toughest bars in the country), and Emory in Georgia in an above-average difficulty bar. Each did reasonably well in 2019. But when adding in performances in other jurisdictions, their scores climbed. ABA data shows W&M went 15-for-15 in DC, 15-for-15 in Maryland, and 10-for-10 in New York. All were excluded in the old metrics; all are easier bars than Virginia. W&L grads went 13-for-14 in DC, 13-for-13 in North Carolina, and 9-for-10 in New York. Emory went 21-for-21 in New York and 11-for-11 in Florida.

In other words, a diffuse and successful bar exam test-taking base redounds to the benefit of these schools.

Let me add one more detail. The new methodology puts law schools closer to parity with one another when comparing bar passage rates, especially those outside the “outliers.” The more graduates you have taking the bar, across jurisdictions, the less likely the difficulty of the bar matters in the end; and the inclusion of “diploma privilege” (or adjacent) admissions lifts the results. The 2019 “denominator” of the bar exam ranged from 55% at the low end of law schools (i.e., Maine) to 87% at the top end (i.e., Kansas), a gap of 32 points. That shrunk a bit in 2020 with the new methodology, from 70% to 99% (29 points). But the difference between the 10th and 90th percentiles shrunk significantly, from 2019 (61% and 81%, 20 points) to 2020 (75% to 86%, 11 points). In other words, there differences between the 19th and 168th law schools in terms of their “jurisdiction pass rate” was about half as much in the “overall pass rate” this year compared to last year.

Let’s look at the worst-performing schools.

	Pass rate 2019		Jurisdiction rate	Cumulative pass rate 2020	Cumulative jurisdiction rate	USNWR score delta
Western State	56.7%	CA	59%	51.7%	78%	-0.0688
Ohio Northern	95.7%	OH	79%	66.7%	81%	-0.0658
Golden Gate	43.9%	CA	59%	44.1%	78%	-0.0620
Faulkner	81.8%	AL	77%	60.7%	79%	-0.0584
Marquette	100.0%	WI	71%	98.2%	99%	-0.0538
Southern Illinois	59.4%	IL	79%	50.6%	82%	-0.0513
Wisconsin	100.0%	WI	71%	100.0%	99%	-0.0497
CUNY	74.5%	NY	74%	66.7%	86%	-0.0493
Pepperdine	81.0%	CA	59%	78.6%	78%	-0.0455
Pace	76.0%	NY	74%	69.6%	85%	-0.0422

You can see that several schools performed worse, or relatively worse, compared to their 2019 figures (again, consistent with what I noted earlier, major swings at some schools in 2020). But note outliers. Marquette (98.2%) and Wisconsin (100%) both have extraordinarily high bar passage rates, due principally to in-state diploma privilege.

In the past, this redounded to their benefit, as ordinary test-takers who took the bar exam performed substantially lower than 100% (see 71% in 2019), giving them a huge advantage. The new USNWR methodology, however, includes all of those diploma privilege admittees as “passers” in cumulative jurisdiction’s pass rate, too. Wisconsin and Marquette used to perform 30 points above the average; they’re now basically at the average.

In one sense, there’s a greater honesty to the metric in comparing similarly-situated graduates to one another. But it comes at the cost of punishing two schools whose graduates are all (or nearly all) immediately able to practice law. That’s a tremendously valuable outcome for law students.

It might be beneficial for USNWR to instead include two factors, absolute passers and relative passers (like this one). Some (especially California deans!) critique an “absolute” passer rate that lacks accounting for the difficulty of the bar. But if we care about law students’ ability to practice law, it seems to me that it’s important to capture whether your graduates are successfully getting students there, regardless of how hard or easy the bar exam is. (Of course, relative performance also should matter, I think, at least to some degree, as it suggests that some schools are improving opportunities for their graduates.) I confess, others would disagree.

How did other schools, like those in Utah, Washington, or Oregon, not perform much better or worse despite emergency “diploma privilege” being introduced? Recall it’s a mixed bag, depending on the school and the state, history and out-of-state test-takers. February 2020 did not have such exemptions, which are partially included in the figures above. Utah and Oregon still had a decent set of in-state test-takers, as diploma privilege did not extend to everyone—but schools in those states didn’t see as dramatic changes in overall passing rates, as in both states they were keyed to pre-set levels of test-taker success (86%, with an exception in Oregon for in-state schools), and that meant most people taking the test would have passed, anyway. Washington, in contrast, opened up diploma privilege to essentially all test-takers, and the corresponding increase in passers put the University of Washington near the bottom of changes from 2019 to 2020 (suffering something that Wisconsin and Marquette experienced this year).

It’s a seemingly small change in methodology, and it’s hard to know what a number like “0.0497” means to an overall score. But it’s worth identifying that the changes are not value-neutral and can affect similarly-situated schools quite differently.

Federal judges have already begun to drift away from hiring Yale Law clerks

On the heels of the latest controversy at Yale Law School, which David Lat ably describes over at Original Jurisdiction, a federal judge penned an email to fellow judges: “The latest events at Yale Law School, in which students attempted to shout down speakers participating in a panel discussion on free speech, prompt me to suggest that students who are identified as those willing to disrupt any such panel discussion should be noted. All federal judges—and all federal judges are presumably committed to free speech—should carefully consider whether any student so identified should be disqualified from potential clerkships.”

The truth is, Yale Law has already seen falling clerkship placement numbers in recent years. Incidents like this may harden some judges’ opposition. (There are caveats, of course, about what factors affect a judges hiring practices, the political salience of the issues here, and so on.)

I closely track federal judicial clerkship placement, and I have in recent years included a three-year average of clerkship placement in a report I release every two years. The latest version of that report is here. But we can look at some trends among a handful of schools. I select eight of the (historically) highest-performing: Yale, Stanford, Chicago, Harvard, Duke, Virginia, Michigan, and UC-Irvine. I’ll look at the last eight years’ placement. (Any choice of schools and window of time is a bit arbitrary, and I could go back for more data or more schools if I wanted. I didn’t look at 2012 or earlier data, so I don’t know what I’m missing with this cutoff.)

View fullsize

Let me start by pointing out that the total placement among recent graduates has been fairly steady (see the chart). Schools report between 1150 and 1250 placements per year.

Some declines may well be attributable to vacancies in the federal judiciary that were unfilled. It does not appear that there is a “trend” of hiring materially fewer recent law school graduates in favor of clerks with work experience.

But this means that there’s roughly a fixed set of possible clerkship positions each year. If some schools are declining in placement we would expect to see other schooling improvement in placement. We can’t necessarily make those as one-to-one tradeoffs (e.g., a judge “stops” hiring from Yale and “starts” hiring from Chicago), but we can watch some aggregate trends.

I’ll start with percentage of graduates placed into a full-time, long-term federal clerkship. Admittedly, this doesn’t capture those who work then clerk. But there is some consistency in the reporting of data over the years. It makes no distinction among competitiveness of clerkships or types of judges (e.g., appellate or district court). Percentages can also fluctuate with the class size or be deceptive based on class size; I’ll dig into the raw figures in a moment.

View fullsize

A few items stand out. Yale would typically place between 25% and 35% of its class into federal clerkships. Its number is low in 2020, but not the lowest in this time period. A couple of times, Stanford has placed a higher percentage of clerks than Yale.

But noteworthy is Chicago’s climb, from 10% of the class in 2013 to a whopping 27.6% in 2020, for the first time in recent memory besting Yale.

A few other trends are noteworthy. Apart from Irvine’s decline (which may coincide with the departure of founding Dean Erwin Chemerinsky), we see that the University of Virginia placing fourth with 17.5% placement. It’s done well in recent years, including occasionally edging out Harvard, but (apart from a 2017 dip) shows a trendline of consistent and perhaps improving placement.

Let’s now look at the raw totals of placement. Recall that these figures are going to help assess placement into the market of roughly 1150 to 1250 total new clerks a year.

View fullsize

Harvard tops the list, as its 15-20% placement into clerkships still means a whopping 80 to 120 clerks a year, given its tremendous class size. But, it is notable to see it at an eight-year low in placement. Yale, which had consistently been second in raw placement for the previous seven years, has slipped to fourth in 2020, as both Chicago (56) and Virginia (55) placed more federal clerks than Yale (52).

Now, it’s perhaps no coincidence that Yale graduates just 197 students in the Class of 2020, its smallest class in this eight-year period, and perhaps correspondingly saw a decline in overall placement in different ways. Still, federal judges needed clerks in 2020. They simply looked elsewhere at slightly higher rates.

But at a larger level, it’s worth noting that federal judges do change their hiring preferences, and we may be witnessing some of that right now, regardless of whether some judges are “investigating” whether some graduates of some law schools have acted in a disruptive manner at a public event. There are, of course, any number of reasons why federal judges looked elsewhere, returning to a point at the top of this post. It could be that law students at some law schools, more than others, are self-selecting out of applying to federal judges (option for lucrative large law firm placement, competitive government positions, or the booming public interest sector).

And finally, it could also be that this blip is hardly a “trend,” and we’ll wait for a month to see what the Class of 2021 figures show.