Does the bar exam adequately test prospective lawyers' minimum competence?

The critiques of the bar exam have grown louder over the last few years on the heels of declining bar pass rates. But the most popular critiques have changed somewhat. It used to be that external factors--such as the ExamSoft debacle--were a target. Then came charges that the bar exam was harder than usual. But the most recent charges are actually quite a longstanding critique of the bar exam--it simply isn't a good measure of prospective lawyers' "minimum competence."

The bar has attempted to adjust in the last fifty years. Many states now have a "performance test," a component designed to simulate what lawyers do--test-takers are given some law and some facts and asked to address the problem with a legal task. That said, performance tests moderately correlate with other elements of the bar exam and perhaps are not performing the function some hoped they would serve.

Regardless, critiques of the bar exam are longstanding, and some of the most popular critiques look something like this: why did a state, like California, pick this score as a passing score for "minimum competence"? And why is the bar exam any good at testing the kinds of things that lawyers actually do? The bar exam is a three-day (in California, beginning this July, two-day), closed book test with multiple choice and timed essay questions that in no way resembles the real world of law practice. Why should we trust this test?

It's a fair point, and it's one best met with a question: what ought the bar test? And, perhaps a more subtle question: what if it turns out that the answer to what the bar ought to test actually aligns quite closely with the results from the existing bar exam?

A study in 1980 in California is one of the most impressive I've seen on this subject. And while it's a little old, it's the kind of thing that ought to be replicated before state bars go about making dramatic changes to their exams or scoring methods. I'll narrate what happened there. (For details, consider two reports on the study and the testimony presented to California lawmakers asking the exact same questions in 1984, after the particularly poor performance of applicants to the state bar on the July 1983 bar exam--a historically low score essentially matched in the July 2016 administration.)

After the July 1980 bar exam in California, the National Conference of Bar Examiners teamed up with the California Committee of Bar Examiners to run a study. They selected 485 applicants to the bar who had taken the July 1980 exam. Each of these applicants took an additional two-day test in August 1980.

The two-day test required participants to "function as counsel for the plaintiff in a simulated case" on one day, and "counsel for the defendant in a different simulated case" the other day. Actors played clients and witnesses. The participants were given oral and written tasks--client interviews, discovery plans, briefs, memoranda, opening statements, cross-examination, and the like. They were then evaluated among a number of dimensions and scored.

In the end, the scores were correlated to the applicants' bar exam scores. The relationship between the scores and the general bar exam scores were fairly strong--"about as strong as the underlying relationship between the Essay and MBE section of the [General Bar Exam]." "In short," the study concluded, the study and the bar exam "appear to be measuring similar but not identical abilities."

Additionally, a panel of 25 lawyers spent more than two days with extended in-depth evaluation of 18 of these participants. The panelists were clinical professors, law professors, attorneys, judges, and others with a variety of experience. The panelists were asked to evaluate these 18 participants' performance among the various dimensions along a scale of "very unsatisfactory" (i.e., fail) to "borderline" to "very satisfactory" (i.e., pass). The panel's judgments about the pass/fail line was consistent with the line where it was drawn on the California bar exam (with the caveat that this was a sample of just 18 applicants).

It might be that there are different things we ought to be testing, or that this experiment has its own limitations (again, I encourage you to read it if you're interested in the details). But before anything is done about the bar exam, it might be worth spending some time thinking about how we can evaluate what we think ought to be evaluated--and recognize that there are decades of studies addressing very similar things that we may ignore to our peril.

Whittier's challenges may have been unique to California

On the heels of my analysis of the challenges facing Whittier, I starting thinking about how Whittier compared with a great many other law schools in the country that are facing the same challenges--a shrinking law school applicant pool, declining quality of applicants, continued challenges in bar exam pass rates and graduate employment statistics. Whittier's incoming class profile isn't unique. What makes its situation different from other schools?

The answer, I think, lies in California, along three dimensions--state bar cut scores, transfers, and employment.

I recently read a law professor suggest that Whittier was making a significant mistake closing because it was located in Orange County, California, a place that would experience great demand for legal services in the near future. I tend to find just the opposite--if Whittier were dropped into just about any of the other 49 states in the country, it likely would not be facing the same pressures it faces in its current location. (This is, of course, not to say that it wouldn't be facing the same kinds of pressures in legal education generally, but that its problems are exacerbated in California.)

I looked at the incoming class profiles from 2013 and picked 11 other schools that closely matched Whittier's overall incoming LSAT profile.

School Name	Matriculants	75th LSAT	50th LSAT	25th LSAT
Atlanta's John Marshall Law School	235	152	149	146
John Marshall Law School	404	152	149	146
Mississippi College	159	153	149	145
New England Law | Boston	238	153	149	145
Nova Southeastern University	305	152	149	146
Oklahoma City University	162	153	149	145
Suffolk University	450	153	149	145
University of Massachusetts Dartmouth	78	151	148	145
University of North Dakota	83	153	148	145
Western New England University	120	152	149	145
Whittier Law School	221	152	149	145
Widener-Commonwealth	74	151	148	145

First, I looked at the first-time bar pass rate for the July 2016 bar, with each state's cut score and the state's overall first-time pass rate among graduates of ABA-accredited schools. (As of this writing, neither the Mississippi state bar nor Mississippi College have disclosed school-specific bar pass rates yet.)

School Name	State	Cut score	July 2016 Pass Rate	Statewide Pass Rate
Mississippi College	MS	132	*	75%
Widener-Commonwealth	PA	136	79%	75%
Western New England University	MA	135	74%	81%
New England Law | Boston	MA	135	73%	81%
University of North Dakota	ND	130	73%	73%
Suffolk University	MA	135	70%	81%
University of Massachusetts Dartmouth	MA	135	69%	81%
Oklahoma City University	OK	132	67%	75%
John Marshall Law School	IL	133	65%	77%
Nova Southeastern University	FL	136	63%	68%
Atlanta's John Marshall Law School	GA	135	43%	73%
Whittier Law School	CA	144	22%	62%

A Pepperdine colleague blogged last year that if Whittier were in New York, it would likely have had a 51% first-time pass rate instead of a 22% pass rate. New York's cut score is relatively low--a 133.  (Whittier's average combined California bar score for first-time test-takers in July 2016 was a 135.5, above 133 and well below California's 144.) If Whittier were in Massachusetts or Georgia, it might have had something near 51%. If it were in Mississippi or North Dakota, its pass rate may have approached 60%. A first-time pass rate of 3 in 5 is still not something to be happy about, but it's a far cry from a first-time rate around just 1 in 5.

It isn't that some of these schools figured out how to help their students to pass the bar and that Whittier lagged; it's that California's high cut score makes it more difficult to pass the bar than if Whittier grads had taken the bar in almost any other state. (This isn't to say that a higher or a lower cut score is better or worse; it's simply to describe the situation that California schools face compared to others.)

Second, a factor in bar pass rate includes the loss of high-performing students as transfers elsewhere. I looked at transfer rates among these schools in 2014, a loss of students who matriculated to the school in 2013.

School Name	Transfers Out Pct	Transfers Out
Atlanta's John Marshall Law School	19%	45
Nova Southeastern University	13%	41
Whittier Law School	13%	28
John Marshall Law School	12%	50
New England Law | Boston	11%	26
University of Massachusetts Dartmouth	10%	8
Western New England University	8%	10
Suffolk University	8%	37
University of North Dakota	5%	4
Oklahoma City University	3%	5
Widener-Commonwealth	3%	2
Mississippi College	3%	4

It may come as little surprise that larger states with many competitive schools that shrunk the incoming class sizes to preserve their LSAT and UGPA medians tended to rely on transfers to help backfill their classes. Atlanta's John Marshall lost 20 students to Emory, 10 to Georgia State, and 5 to Mercer; Nova lost 16 to Miami and 5 to Florida State; and Whittier lost 15 to Loyola-Los Angeles. These schools lost a number of their best students, and, unsurprisingly, had some of the worst bar outcomes among this cohort. (Four of these schools are in Massachusetts, and perhaps no single school attracts the bulk of transfer attention; and schools in less competitive states like Mississippi, North Dakota, and Oklahoma experienced insignificant attrition.)

Third, Whittier had low job placement in full-time, long-term, bar passage-required and J.D.-advantage positions, but it's a reflection of the fact that the placement rate of California schools lags most of the rest of the country. (It's also exacerbated by the low bar pass rate.) Consider each school's placement in FTLT BPR & JDA positions, and the statewide placement rate into such (unfunded) jobs.

School Name	FTLT BPR+JDA	Statewide emp	Delta
Mississippi College	76%	72%	4
Oklahoma City University	73%	77%	-4
Widener-Commonwealth	68%	83%	-15
Suffolk University	66%	79%	-13
John Marshall Law School	66%	77%	-11
New England Law | Boston	60%	79%	-19
Atlanta's John Marshall Law School	58%	77%	-19
Nova Southeastern University	58%	65%	-7
Western New England University	57%	79%	-22
University of North Dakota	57%	57%	0
University of Massachusetts Dartmouth	55%	79%	-24
Whittier Law School	39%	64%	-25

Whittier lags in placement here, too, but in part because California has unusually low placement. (North Dakota, a state with just one flagship law school, serves as an outlier.) This is not a total defense of a particular school's outcomes, either--Florida also appears to have a relatively high number of law school graduates, and its employment rate shows similar challenges. But coupled with Whittier's low bar passage rate, one can see why securing students in positions, particularly "bar passage required" positions, would be even more difficult. Several other schools show employment rates that are 19 to 24 points behind the state average. (School like Oklahoma City, one of three; Mississippi College, one of two; and North Dakota, the only school in the state, may distort these comparisons somewhat.)

I then read another piece from a graduate from the 1970s lamenting that Whittier had "lost its way" in training graduates ready to take the bar. I pointed out Whittier's challenges were hardly recent, as the ABA had placed Whittier on probation in 2005, which lead to efforts that bolstered Whittier's first-time bar pass rate in California past 84%.

But it's worth looking back to the 1970s, when Whittier first sought accreditation, to consider its situation and aspirations. Here's an excerpt from the Los Angeles Times in 1978 when Whittier received ABA accreditation:

View fullsize

In 1978, a 550 was around the 51st percentile of LSAT scores--something like a 151 today. Its tuition in 1974 was $1200 per year, or around $6000 per year in 2017 dollars. It was on pace to increase to $2900 per year in just four years, or about $11,000 in 2017 dollars. There are obviously significant benefits that arise from becoming an ABA-accredited law school. But there are also costs with accreditation--and I'm not sure that a law school with tuition levels at $11,000 a year would be facing the same kinds of pressures among selecting prospective students.

I don't pretend to understand the dynamics of legal education in California in the last 40 years, with more than 20 ABA-accredited law schools and a number of California accredited and unaccredited schools. But I do think some context about the California market suggests that some of the problems Whittier faced were exacerbated by the California market in particular.

Visualizing law school federal judicial clerkship placement, 2014-2016

The release of the latest ABA employment data offers an opportunity to update the three-year federal judicial clerkship placement rates. Here is the clerkship placement rate for the Classes of 2014, 2015, and 2016. Methodology and observations below the interactive visualization. The "placement" is the three-year total placement; the "percentage" is the three-year placement divided by the three-year graduating class total.

The placement is based on graduates reported as having a full-time, long-term federal clerkship. (A one-year term clerkship counts for this category.) I thought a three-year average for clerkships (over 3600 clerks from the graduating classes of 2014, 2015, and 2016) would be a useful metric to smooth out any one-year outliers. It does not include clerkships obtained by students after graduation; it only includes clerkships obtained by each year's graduating class.

I included some schools that had only one or two year's worth of data, like the separate Penn State schools. Additionally, I merged the entries for William Mitchell and Hamline into Mitchell|Hamline. The three schools in Puerto Rico are excluded.

I should add that we've actually seen a slight decline in graduates placed into federal clerkships, just under 1200 for the second year in a row. Given last year's figures, some might think this is a trend toward judges hiring more clerks with work experience. I'm not sure that's the case. Instead, I would venture to guess that because the Senate last confirmed a federal judge in November 2015, we may be experiencing an unusual number of vacancies--and, therefore, lack of slots for clerkship hires. In the event the President nominates, and Congress confirms, these judges, we could see a few hundred more clerkship openings in the near future. And if Congress chooses to create more judgeships consistent with the recommendations of the Federal Judicial Center, we'd see even more.

I'll highlight two smaller charts first. The first is New York law school placement.

School	Pct	Total Clerks
Cornell University	6.5%	36
New York University	5.8%	84
Columbia University	5.0%	64
Brooklyn Law School	2.4%	26
Fordham University	2.0%	25
Syracuse University	1.8%	10
University of Buffalo-SUNY	1.2%	7
St. John's University	1.2%	9
Cardozo School of Law	1.2%	13
Albany Law School	1.1%	6
City University of New York	1.1%	4
Pace University	0.7%	4
New York Law School	0.7%	8
Hofstra University	0.7%	6
Touro College	0.0%	0

The second is California law school placement.

School	Pct	Total Clerks
Stanford University	27.1%	153
University of California-Irvine	12.5%	40
University of California-Berkeley	12.3%	110
University of California-Los Angeles	4.0%	39
Pepperdine University	3.5%	20
University of Southern California	2.9%	18
University of California-Davis	2.8%	14
Loyola Law School-Los Angeles	2.3%	26
University of San Diego	2.0%	15
University of California-Hastings	1.7%	17
Thomas Jefferson School of Law	0.7%	5
California Western School of Law	0.6%	4
McGeorge School of Law	0.4%	2
Chapman University	0.2%	1
University of San Francisco	0.2%	1
Southwestern Law School	0.1%	1
University of La Verne	0.0%	0
Western State College of Law	0.0%	0
Golden Gate University	0.0%	0
Whittier Law School	0.0%	0
Santa Clara University	0.0%	0

An overall raw chart is below.

St	School	Pct	Total Clerks
CT	Yale University	31.0%	200
CA	Stanford University	27.1%	153
MA	Harvard University	17.6%	312
IL	University of Chicago	15.8%	98
VA	University of Virginia	15.2%	159
NC	Duke University	12.7%	82
CA	University of California-Irvine	12.5%	40
CA	University of California-Berkeley	12.3%	110
MI	University of Michigan	11.1%	119
TN	Vanderbilt University	10.3%	58
PA	University of Pennsylvania	9.8%	77
TX	University of Texas at Austin	9.4%	100
IL	Northwestern University	8.0%	66
AL	University of Alabama	7.6%	35
MT	University of Montana	7.5%	18
IN	University of Notre Dame	7.0%	37
LA	Tulane University	6.6%	45
KY	University of Kentucky	6.5%	26
NY	Cornell University	6.5%	36
VA	Washington and Lee University	6.1%	24
IA	University of Iowa	5.9%	25
VA	William and Mary Law School	5.8%	36
NY	New York University	5.8%	84
GA	University of Georgia	5.8%	36
NC	University of North Carolina	5.7%	40
VA	University of Richmond	5.5%	25
NY	Columbia University	5.0%	64
TX	Baylor University	5.0%	20
MN	University of Minnesota	4.9%	37
PA	Temple University	4.8%	34
MO	Washington University	4.5%	32
MS	University of Mississippi	4.3%	19
DC	Georgetown University	4.1%	81
AR	University of Arkansas, Fayetteville	4.1%	15
UT	Brigham Young University	4.1%	17
WA	University of Washington	4.0%	22
CA	University of California-Los Angeles	4.0%	39
WV	West Virginia University	3.8%	14
UT	University of Utah	3.8%	14
GA	Mercer University	3.8%	16
DC	George Washington University	3.7%	59
DC	American University	3.7%	49
GA	Emory University	3.6%	31
KS	University of Kansas	3.6%	13
IL	University of Illinois	3.6%	19
CA	Pepperdine University	3.5%	20
MO	University of Missouri	3.4%	13
MA	Boston College	3.3%	25
WY	University of Wyoming	3.3%	7
VA	Regent University	3.0%	10
SD	University of South Dakota	3.0%	6
TX	Texas Tech University	3.0%	18
TN	University of Memphis	2.9%	10
NC	Wake Forest University	2.9%	15
CA	University of Southern California	2.9%	18
CA	University of California-Davis	2.8%	14
PA	Pennsylvania State University	2.8%	5
GA	Atlanta John Marshall Savannah	2.8%	1
MS	Mississippi College	2.7%	12
MD	University of Maryland	2.7%	21
GA	Georgia State University	2.7%	16
IN	Indiana University - Bloomington	2.6%	16
TX	Southern Methodist University	2.6%	19
NV	University of Nevada - Las Vegas	2.6%	10
VA	George Mason University	2.6%	12
LA	Louisiana State University	2.6%	15
SC	University of South Carolina	2.5%	15
KY	University of Louisville	2.5%	9
OH	Ohio State University	2.5%	14
AZ	University of Arizona	2.4%	10
FL	Florida State University	2.4%	17
NY	Brooklyn Law School	2.4%	26
LA	Loyola University-New Orleans	2.4%	15
NE	Creighton University	2.4%	9
ME	University of Maine	2.4%	6
CA	Loyola Law School-Los Angeles	2.3%	26
TN	University of Tennessee	2.3%	10
CT	University of Connecticut	2.2%	11
OH	University of Toledo	2.2%	7
DC	Howard University	2.2%	8
CO	University of Colorado	2.2%	11
FL	University of Florida	2.1%	20
CA	University of San Diego	2.0%	15
PA	Widener-Commonwealth	2.0%	5
NY	Fordham University	2.0%	25
WI	University of Wisconsin	1.9%	12
AZ	Arizona State University	1.8%	11
NY	Syracuse University	1.8%	10
NJ	Rutgers Law School	1.8%	22
OH	Case Western Reserve University	1.7%	7
CA	University of California-Hastings	1.7%	17
NE	University of Nebraska	1.7%	6
OR	Lewis and Clark College	1.6%	10
WI	Marquette University	1.6%	10
NM	University of New Mexico	1.5%	5
NC	Elon University	1.5%	4
OH	University of Cincinnati	1.5%	5
TX	University of Houston	1.4%	10
MO	University of Missouri-Kansas City	1.3%	6
AR	University of Arkansas, Little Rock	1.3%	5
OH	Ohio Northern University	1.3%	3
ND	University of North Dakota	1.3%	3
NJ	Seton Hall University	1.3%	8
AL	Samford University	1.2%	5
IL	Southern Illinois University-Carbondale	1.2%	4
NC	Campbell University	1.2%	5
NY	University of Buffalo-SUNY	1.2%	7
NY	St. John's University	1.2%	9
KY	Northern Kentucky University	1.2%	5
NY	Cardozo School of Law	1.2%	13
MA	Boston University	1.2%	8
PA	University of Pittsburgh	1.2%	7
PA	Villanova University	1.2%	7
TX	Texas Southern University	1.1%	5
OK	University of Oklahoma	1.1%	5
NY	Albany Law School	1.1%	6
PA	Penn State - Dickinson Law	1.1%	1
NY	City University of New York	1.1%	4
OK	University of Tulsa	1.1%	3
MA	Northeastern University	1.1%	6
SC	Charleston School of Law	1.1%	5
PA	Penn State Law	1.0%	2
FL	Stetson University	1.0%	9
VA	Liberty University	1.0%	2
MI	Michigan State University	1.0%	9
WA	Gonzaga University	1.0%	4
PA	Drexel University	1.0%	4
MI	Wayne State University	0.9%	4
OR	University of Oregon	0.9%	4
ID	University of Idaho	0.9%	3
FL	University of Miami	0.9%	10
NY	Pace University	0.7%	4
NY	New York Law School	0.7%	8
NH	University of New Hampshire	0.7%	2
NY	Hofstra University	0.7%	6
VT	Vermont Law School	0.7%	3
PA	Duquesne University	0.7%	3
IL	Loyola University-Chicago	0.7%	5
FL	Florida A&M University	0.7%	3
CA	Thomas Jefferson School of Law	0.7%	5
MO	Saint Louis University	0.7%	4
IN	Valparaiso University	0.6%	3
CA	California Western School of Law	0.6%	4
IL	John Marshall Law School	0.6%	7
OH	University of Dayton	0.6%	2
TN	Belmont University	0.6%	1
WA	Seattle University	0.6%	5
CO	University of Denver	0.6%	5
TX	St. Mary's University	0.6%	4
IA	Drake University	0.6%	2
OH	Cleveland State University	0.5%	2
DE	Widener University-Delaware	0.5%	3
MN	University of St. Thomas (Minnesota)	0.5%	2
OH	University of Akron	0.5%	2
IL	Chicago-Kent College of Law-IIT	0.5%	4
TX	South Texas College of Law	0.5%	5
AZ	Arizona Summit Law School	0.5%	4
DC	Catholic University of America	0.4%	2
AL	Faulkner University	0.4%	1
IL	Depaul University	0.4%	3
FL	Ave Maria School of Law	0.4%	1
LA	Southern University	0.4%	2
CA	McGeorge School of Law	0.4%	2
MD	University of Baltimore	0.3%	3
IL	Northern Illinois University	0.3%	1
FL	St. Thomas University (Florida)	0.3%	2
TX	Texas A&M University	0.3%	2
KS	Washburn University	0.3%	1
IN	Indiana University - Indianapolis	0.3%	2
CA	Chapman University	0.2%	1
OK	Oklahoma City University	0.2%	1
MA	Suffolk University	0.2%	3
MI	University of Detroit Mercy	0.2%	1
NC	North Carolina Central University	0.2%	1
CA	University of San Francisco	0.2%	1
MN	Mitchell|Hamline	0.2%	2
FL	Barry University	0.1%	1
FL	Nova Southeastern University	0.1%	1
MA	New England Law | Boston	0.1%	1
CA	Southwestern Law School	0.1%	1
NC	Charlotte School of Law	0.1%	1
TN	Lincoln Memorial	0.0%	0
ID	Concordia Law School	0.0%	0
VA	Appalachian School of Law	0.0%	0
CA	University of La Verne	0.0%	0
MA	University of Massachusetts Dartmouth	0.0%	0
CT	Quinnipiac University	0.0%	0
HI	University of Hawaii	0.0%	0
RI	Roger Williams University	0.0%	0
CA	Western State College of Law	0.0%	0
DC	District of Columbia	0.0%	0
MA	Western New England University	0.0%	0
CA	Golden Gate University	0.0%	0
OR	Willamette University	0.0%	0
OH	Capital University	0.0%	0
CA	Whittier Law School	0.0%	0
NY	Touro College	0.0%	0
GA	Atlanta's John Marshall Law School	0.0%	0
FL	Florida International University	0.0%	0
CA	Santa Clara University	0.0%	0
FL	Florida Coastal School of Law	0.0%	0
MI	Thomas M. Colley Law School	0.0%	0

More details on the legal job market: small law firm, business jobs disappearing

Last week, I posted a perspective on the changing legal market and the outcomes for the Class of 2016. Troublingly, job placement in full-time, long-term, bar passage-required positions declined from 25,787 for the Class of 2013 to 22,874 for the Class of 2016; that said, placement improved from 55.9% to 62.4% because of the shrinking graduating class size.

I thought I'd dig into job-specific data to see what may be leading the decline. The problem with the industry-specific data is that we don't know whether the jobs are bar passage-required, J.D.-advantage, professional, or non-professional. That said, we can make some guesses; most entry-level hiring in law firms with 501 or more attorneys are probably bar passage-required, for instance. Regardless, I looked at the full-time, long-term job categories for each position and catalogued notable areas.

I had two instincts. First, perhaps big law hiring has declined and law firms are relying on greater productivity, greater outsourcing, increased reliance on technology, higher retention of junior associates, and delayed retirements. Second, perhaps government hiring has declined in eras of partisanship and budget stalemates.

Both instincts were wrong.

FTLT	Class of 2013	Class of 2016	Net	Delta
Solo	926	444	-482	-52.1%
2-10	6,947	5,490	-1,457	-21.0%
11-25	1,842	1,640	-202	-11.0%
26-50	1,045	906	-139	-13.3%
51-100	846	768	-78	-9.2%
101-205	1,027	940	-87	-8.5%
251-500	1,041	993	-48	-4.6%
501+	3,978	4,204	226	5.7%
Business/Industry	5,494	3,796	-1,698	-30.9%
Government	4,360	4,034	-326	-7.5%
Public Interest	1,665	1,398	-267	-16.0%
Federal Clerk	1,259	1,184	-75	-6.0%
State Clerk	2,043	2,021	-22	-1.1%
Academia	490	352	-138	-28.2%

Law firms with 501 or more attorneys were the only category that saw an increase in the last three years, a modest 226-person increase but a pleasant increase at that.

Yes, Government saw a decline, but not as significant as other categories. Other relatively small areas had some big declines. Public Interest jobs declined, perhaps because of combinations of student debt levels or public interest organization funding, and because of law school-funded positions drying up (although it remains somewhat counterintuitive at a time when law schools have been increasing their hiring and development of clinical education). Solo practitioners fell by half--it's become something less attractive for new graduates due to its uncertainty, perhaps.

Instead, two areas saw thousand-student-plus declines in three years.

First, hiring in small law firms, those with two to 10 people, declined 21% in three years.  This has been the single biggest employer of new graduates in recent years, and it has seen a significant decline. These are not firms that typically arrive at on campus interviews. But they are also firms, I think, that probably need new graduates to pass the bar on the first attempt. Failure to do so is a real challenge--they likely can't absorb someone who's unable to practice until the next bar go-around. And these are probably places that are most willing to dip into lower-ranked schools and students with lower grades. The decline in bar passage rates may be impacting this area the most--but that's just speculation on my part.

Second, hiring in business & industry positions has declined nearly 31% in three years. As business & industry jobs are a major source of J.D.-advantage positions, it would explain the decline in J.D.-advantage positions, too. But while bar passage-required jobs in business might also suffer a decline in placement as bar pass rates decline, why J.D.-advantage positions, too? Perhaps--again, some speculation--it's that, in a (relatively) robust economy with many strong college graduates, businesses may no longer be valuing the J.D., or they may be finding greater value in M.B.A. students.

For schools looking to solve their employment challenges, addressing the reasons why there's been a steep drop in demand in these two important employers is crucial. A few speculated reasons here are hardly a beginning to explore the reasons for the changes in climate.

The incredible shrinking law school

One of the things that jumped out at me in my analysis on legal employment outcomes was the shrinking graduating class size. That started me to examine what's happening on a school-by-school basis.

I started with a quick qualitative look and saw wildly divergent stability. Thomas M. Cooley dropped from 1143 graduates in its Class of 2013 to 462 for the Class of 2016, a 60% decline. Florida Coastal saw its graduating class drop from 562 to 299. Two large, elite schools saw little change--Georgetown rose from 645 to 652; Harvard from 578 to 598.

View fullsize

But the decline has been largely across the board. I calculated the median graduating class size over the last four years. In 2013, it was 206--meaning half of law schools had graduating classes larger and half had smaller. Last year, however, it was down to 161. The decline is not isolated but quite widespread--probably not too much of a surprise given the market, but worth considering from the data.

Additionally, I thought I'd divide up schools into categories based on size--100 or fewer graduates, 101 to 200 graduates, and more than 200 graduates. The Class of 2013 had mostly large graduating classes--no surprise given the median of 206. 107 law schools saw graduating classes larger than 200, and another 80 were between 101 and 200. Just 11 schools were 100 or fewer--four of them flagship schools in rural states (Montana, North Dakota, South Dakota, and Wyoming).

View fullsize

The Class of 2016, however, saw the number of schools with 100 or fewer graduates rise to 36. (The 36th-smallest graduating class in 2013 had 135 students.) And the number of schools with more than 200 grads dropped to just 67.

Law school downsizing comes with it all of the turbulence we've seen in media reports--faculty buyouts, closures, mergers, and the like--but this offers a display about why such pressure exists at the school-specific level.

We should expect at least some stability over the next few years: incoming class sizes have been very stable for the last three years, and early reports are that we'll see some such stability this cycle, too. (The quality of the class, and the resulting changes to bar pas rate as a result, are another matter.)

Visualizing legal employment outcomes in DC-Maryland-Virginia in 2016

This is the eighth and last in a series of visualizations on legal employment outcomes for the Class of 2016. Following posts on outcomes in Texas, New York, Illinois, California, Florida, Ohio, and Pennsylvania, here is a visualization for legal employment outcomes of graduates of Pennsylvania law schools for the Class of 2016. (More about the methodology is available at the Texas post.)

Total jobs in unfunded bar passage-required and J.D.-advantage positions increased slightly, counter to much of the rest of the country: there were 2765 positions, up from 2735 last year. 455 J.D.-advantage jobs is more than most markets have, and, as you can see from the chart, can vary fairly significant from school to school. Graduates declined from 3740 to 3602, yielding a fairly strong 79% employment rate. 90 school-funded positions came mostly from two schools, Georgetown (44) and Virginia (19). That leads to some variance in the visualization (sorted by unfunded job placement), and the table below (displaying overall placement, including school-funded positions).

As always, please notify me of any corrections or errata.

View fullsize

Peer score	School	2016	YoY%	BPR	JDA	LSF	2015	BPR	JDA	LSF
4.4	University of Virginia	96.1%	0.7	293	5	19	95.4%	311	9	30
4.1	Georgetown University	87.1%	6.9	486	38	44	80.2%	456	50	38
2.7	George Mason University	86.5%	-0.3	86	25	4	86.8%	94	34	4
3.1	Washington & Lee University	84.2%	2.0	73	7	0	82.2%	131	11	1
3.0	University of Maryland	83.6%	8.3	126	51	1	75.3%	156	47	1
3.2	William & Mary Law School	81.3%	5.5	162	21	0	75.8%	118	17	0
3.4	George Washington University	79.8%	1.5	373	61	9	78.3%	301	55	8
2.5	University of Richmond	77.0%	2.8	95	19	0	74.2%	97	21	0
2.1	University of Baltimore	76.2%	2.0	142	69	0	74.2%	142	56	0
1.2	Regent University	70.5%	5.2	57	5	0	65.3%	67	10	0
2.8	American University	66.3%	4.9	219	56	0	61.4%	205	76	4
1.5	District of Columbia	66.0%	14.5	33	30	1	51.5%	15	19	1
2.2	Catholic University of America	65.2%	-5.2	53	37	0	70.4%	63	37	0
1.2	Liberty University	63.8%	9.7	32	4	1	54.1%	31	1	1
2.4	Howard University	63.2%	3.7	65	20	1	59.5%	57	12	0
1.2	Appalachian School of Law	52.4%	-10.9	15	7	0	63.3%	32	4	2

Perspective on legal employment outcomes for the Class of 2016

The American Bar Association has finally released the employment statistics for the Class of 2016.

I say finally deliberately. Last year, the ABA released them around May 2. The ABA here disclosed them May 11.

In 2013, the ABA approved a change to reporting employment statistics. It used to be "employment outcomes 9 months after graduation," or an employment date of February 15. The ABA moved it back to 10 months for the Class of 2015, or an employment date of March 15. Deans of law schools in New York and California in particular objected that due to late bar exam results (often in November), followed by a holiday season, it was difficult for some students to secure employment by February 15. That artificially depressed employment figures in these jurisdictions compared to the rest of the country.

That pushed back the deadline for schools to report employment outcomes from March 15 to April 15. April is a significant season for prospective law school matriculants. Schools typically have "deposit deadlines" in April, requiring a few hundred dollars' nonrefundable deposit to secure a seat in the class. (I Googled a few: Washington & Lee requires $250 by April 15; Cornell requires $300 by May 1; Georgia requires $150 by April 15; New Mexico requires $200 by April 15.)

It's worth noting that if these employment disclosures are supposed to advance a "consumer protection" rationale (although I'm not sure much has been studied about students' reliance on these disclosures), the decision to move from 9 months to 10 months has undermined some of the "consumer protection" rationale. While most schools have put these results on their websites, several (I'll leave them nameless) have not; the ABA disclosures would be the first release of this information.

Additionally, the putative reason for moving the reporting deadline back a month does not appear to have been borne out. Consider New York & California employment outcomes in the last few years: it's not obvious that the employment situation materially improved after the date change. Granted, some of this is likely attributable to other causes, the peril of isolating one factor among many possible factors. But it would be worth the ABA considering evidence from these law schools that the change did materially improve outcomes in those jurisdictions--evidence we just don't have.

It's also a time to discuss recent predictions about this class. Professor Ted Seto in 2014 offered an optimistic prediction for the Class of 2016: "If the new BLS projections are accurate, we should see demand and supply in relative equilibrium in 2015 and a significant excess of demand over supply beginning in 2016."

Perhaps the BLS data wasn't quite accurate, or perhaps it was an overly-rosy view of what employment might look like. In what's surely a disappointment, nationwide employment in unfunded full-time, long-term, bar passage-required positions has been steadily declining. Placement in such positions was just 22,874, down from 25,787 three years ago. (UPDATE: please note that I excluded Puerto Rico's three law schools from this analysis.)

	Graduates	FTLT BPR	Placement	FTLT JDA
Class of 2012	45,751	25,503	55.7%	4,218
Class of 2013	46,112	25,787	55.9%	4,550
Class of 2014	43,195	25,348	58.7%	4,774
Class of 2015	40,205	23,895	59.4%	4,416
Class of 2016	36,654	22,874	62.4%	3,948

The good outcome is that the percentage of graduates placed has improved as the number of law school graduates has declined sharply. So why a decline in raw jobs? There are many possible reasons, and I only offer a few to speculate here for future investigation.

Some of this is likely attributable to declines in bar passage scores. But perhaps more troubling has been the decline in J.D. advantage positions, too. For years, the versatility and flexibility of J.D. has been a common point of defense among law schools, not without some controversy. But those positions--which not only highlight the versatility of the J.D., but aren't contingent on passing the bar--have been declining, too.

It may be that law schools have too readily assumed the fungibility of a law degree. That is, even if the total number of law school graduates have declined, it may be that employers still prefer individuals who graduate in the top X% of the class, from a top-Y law school. It may be that they have been reluctant to dip lower into classes or to lower-ranked schools. That is, they're not just looking for any law school graduates; they're still looking for a certain quality of graduate, and they may not be convinced that they should move off their previous expectations. Or it may be that BLS or other labor data has simply been too optimistic about the new lawyer market.

Last year, I noted the sharp decline in law school-funded positions. The same is true for this year. Through a combination of changes to reporting requirements (schools must pay at least $40,000 a year), the requirement under the Affordable Care Act that they provide health insurance if they are working more than 30 hours a week, and the USNWR decision to cut the weight of such positions in its methodology, we likely won't see a resurgence in this category anytime soon.

In short, the good news of the employment picture is almost exclusively a result of law school graduating classes shrinking significantly. Raw job placement has grown slightly and steadily worse over the last few years, while the placement rate has ticked up somewhat--7 points of improvement in the last few years.

Visualizing legal employment outcomes in Pennsylvania in 2016

This is the seventh in a series of visualizations on legal employment outcomes for the Class of 2016. Following posts on outcomes in Texas, New York, Illinois, California, Florida, and Ohio, here is a visualization for legal employment outcomes of graduates of Pennsylvania law schools for the Class of 2016. (More about the methodology is available at the Texas post.)

Pennsylvania is new to my series of visualizations. Total jobs in these unfunded bar passage-required and J.D.-advantage positions declined somewhat, from 1077 to 1036. There were about 250 fewer graduates, from 1414 to 1260. As a result of the significant decline in enrollment, overall prospects for graduates improved: the overall employment rate was 82.7% (including all funded positions), up from 76.2% last year.

It's worth noting that some of these schools are quite small: Penn State Dickinson had just 34 graduates; Penn State Law (in State College) had 95; Widener had just 53. The visualization below can look somewhat deceiving. Ideally, the width of the bars would be variable based on the total number of graduates, (larger classes would have thicker bars to display the total grads they placed), but that apparently is not readily available in Excel. I'll look into improving the display for next year. Blogging is constantly a work in progress!

As always, please notify me of any corrections or errata.

View fullsize

Peer Score	School	2016	YoY%	BPR	JDA	LSF	2015	BPR	JDA	LSF
4.3	University of Pennsylvania	97.3%	0.9	230	17	4	96.3%	221	10	6
2.0	Pennsylvania State University - Dickinson Law	88.2%	18.1	28	2	0	70.2%	34	6	0
2.1	Drexel University	83.7%	4.8	111	12	0	78.9%	83	18	0
2.7	Temple University	83.2%	7.7	135	32	1	75.5%	149	42	0
2.4	Villanova University	82.0%	5.9	119	21	1	76.1%	132	29	1
2.3	Penn State Law	74.7%	6.4	63	8	0	68.4%	52	15	0
2.6	University of Pittsburgh	74.3%	6.4	108	22	0	67.9%	105	28	0
1.8	Duquesne University	74.2%	8.7	76	16	0	65.5%	77	18	0
1.6	Widener Commonwealth	67.9%	2.5	26	10	0	65.4%	40	11	0

Congress, the executive, and the FBI: what makes a "constitutional crisis"?

A few longer, meandering thoughts from a few Twitter threads overnight....

President Donald Trump fired Federal Bureau of Investigation ("FBI") Director James Comey last night. As I tweeted last October, the Office of Legal Counsel has long held the view that the FBI director may be removed "at the will of the president." The removal is certainly constitutional. There is a design of independence in the FBI director--he is given a 10-year position, which is designed to insulate him from political pressure, such as the pressure of renewal by the same president who hired him. But that does not mean that he is legally independent.

Much of the commentary that erupted has elided some of these legal and political distinctions. But it's worth noting that the firing itself has elided these distinctions. And it's worth emphasizing why our constitutional order is functioning quite well--hardly a "constitutional crisis." That said, the next political steps will be significant in the extent to which they protect the institutions, and the checks, the Constitution has created.

President Trump could have fired Mr. Comey for no reason whatsoever. But he didn't. He provided reasons linked to Mr. Comey's handling of Hillary Clinton's email server. As commentators have already noted, elements of this justification seem oddly post hoc or a solution in search of a justification. Some have speculated that the true reason was on account of the investigation into the possible relationship between members of the Trump campaign and Russia. (Indeed, President Trump mentions in the cover letter that he is gratified that Mr. Comey has informed him "on three separate occasions" that he is not under investigation.)

Here we see an important legal/political distinction. Legally, President Trump needed no such justification; politically, he felt compelled to come up with a reason. (Apparently, that reason has not been sufficiently persuasive to many.) Because of a political tradition of rarely firing the director of the FBI (President Bill Clinton is the only other to do so, after a lengthy investigation and fact-finding)

Some have opined about the problems of this regime--how can the President be able to fire at will the very person investigating him? Consider Justice Antonin Scalia's words in dissent in Morrison v. Olson (1988) (some citations excluded):

Is it unthinkable that the President should have such exclusive power, even when alleged crimes by him or his close associates are at issue? No more so than that Congress should have the exclusive power of legislation, even when what is at issue is its own exemption from the burdens of certain laws. No more so than that this Court should have the exclusive power to pronounce the final decision on justiciable cases and controversies, even those pertaining to the constitutionality of a statute reducing the salaries of the Justices. A system of separate and coordinate powers necessarily involves an acceptance of exclusive power that can theoretically be abused. As we reiterate this very day, "[i]t is a truism that constitutional protections have costs." While the separation of powers may prevent us from righting every wrong, it does so in order to ensure that we do not lose liberty.  The checks against any branch's abuse of its exclusive powers are twofold: First, retaliation by one of the other branch's use of its exclusive powers: Congress, for example, can impeach the executive who willfully fails to enforce the laws; the executive can decline to prosecute under unconstitutional statutes; and the courts can dismiss malicious prosecutions. Second, and ultimately, there is the political check that the people will replace those in the political branches (the branches more "dangerous to the political rights of the Constitution," Federalist No. 78, p. 465) who are guilty of abuse. Political pressures produced special prosecutors - for Teapot Dome and for Watergate, for example - long before this statute created the independent counsel. See Act of Feb. 8, 1924, ch. 16, 43 Stat. 5-6; 38 Fed. Reg. 30738 (1973).

As Professor Adrian Vermeule has carefully pointed out, the remedies here are political. And they are considerably more powerful, I think, than many otherwise anticipate.

First, the Senate has the power to consent to appointment of the next FBI director.

Second, the House can initiate impeachment proceedings.

Third, Congress can authorize the creation of a special prosecutor to investigate (who might still be removable at the will of the Attorney General).

Fourth, Congress can create an independent commission to investigate the matter.

The likelihood is perhaps another matter. It is worth noting that these political solutions work under limited circumstances: if party that controls the Senate (or Congress) is not the President's party, or if there is bipartisan support for these political solutions, or if one waits for an intervening election, or if Congress can override the President's veto on new legislation. There is some suggestion that this may be a bipartisan moment, at least among some influential moderate and independent Senators. Time will tell. But these are the costs of a political system like we have, as Justice Scalia pointed out in Morrison.

Two notable solutions are likely unavailable.

The first is the independent counsel--the very thing that was approved in Morrison v. Olson. Much time has passed since 1988, and many view Justice Scalia's dissent as unusually prophetic. It may be that the Supreme Court would overrule Morrison--indeed, Justice Elena Kagan offered remarkable praise for Justice Scalia's dissent, and it may well be the case that there are five votes to overturn Morrison. Some in Congress have already mentioned such a possibility, but I believe that would be dead on arrival in Congress, much less in the courts.

The second is a judicial remedy. The battle here will play out between Congress and the executive--and into the political realm in 2018, given our frequency of elections in the United States. The federal courts--absent, perhaps, weighing in on the constitutionality of some such possible new legislation in the future--will remain on the sidelines.

Finally, I've seen people refer to this as a "constitutional crisis," but, as Professor Orin Kerr notes, this phrase has become something too broad. This isn't the Civil War; this is a significant political controversy, to be sure, that will be carried out in Congress, in the executive, and in the election cycle. But it's something our Constitution is actually fairly equipped to handle. What the results will be, and whether one finds those results acceptable, is, I think, quite a different matter.