When computers generate legal text, that matters for everyone—not just lawyers
Some thoughts on "generated legal texts"
As soon as it started seeming possible that generative AI would reach levels of quality where it would become viable to use commercially, people started wondering about the legal industry. One particularly significant early prediction was a report by Goldman Sachs in March 2023 estimating that 44% of tasks in the legal profession were “exposed to automation.” And you can understand why: a very big part of what lawyers do happens through text. Laws and regulations are text; the briefs that argue for particular interpretations of those laws are text; the opinions resolving legal disputes are often text; government agencies’ decisions and pronouncements are typically text; the contracts that memorialize legal agreements between private parties are usually text; and the advice that lawyers give to their clients is often given in the form of text, too.
But as that list suggests, significant changes to how we produce and manage text are not just relevant to those who worry about lawyers’ employment. The use of automated tools in legal contexts has the potential to affect everyone. Regardless of how AI tools should be used, it is clear that they are being used by legal institutions in areas that affect everyone—including at least some forays into judicial opinion writing, issuing or revising regulations, and drafting laws.
My latest academic piece, written with Kevin Tobia, tries to take a first step toward grappling with what’s going on here. We look at what we call “generated legal texts”—texts that are partly or entirely generated by computer software, such as generative AI, for use by legal institutions. In the first part of the article, we report on the results of a wide survey of examples of people and institutions using generated legal texts. We find that the results are broad and deep: people are using AI to generate texts across a wide range of legal contexts, and they are at times relying on it intensively or for highly important activities. AI is being used to draft pleadings, to help write briefs, to translate evidence in court, to draft contracts, to propose revisions to regulations, and, as I mentioned above, even in some places to draft judicial opinions and legislation. Some of these uses are idiosyncratic, and may fizzle out or be regulated out of existence. Others are being adopted wholesale by institutions. And this is still just in the first couple of years since the release of ChatGPT, by a set of actors (courts, agencies, law firms) that often have the reputation of being slow adopters of new technology. It seems likely that we will see much more of this in the near future.
One thing we try to do in the piece is to look at text as the relevant unit of analysis. We think it can be useful to do so because there are some concerns and responses that accompany generated legal texts across different contexts—whether the issue is a public agency’s notice-and-comment rulemaking or litigation between private parties, for instance. We think some of these concerns are familiar from the use of AI in other contexts, but others arise in more distinctive ways when you focus on text.
As for the familiar issues, we point to concerns about bias and discrimination; about overreliance on limited tools; or about inaccuracy—the widespread problem of hallucinations, for instance. Although I’ve written elsewhere about the ways in which AI can be useful for lawyers despite its limitations, it also seems clear that there are some serious potential downsides—especially at this early stage of adoption when many of its limitations do not seem to be well understood by its users.
In addition to these familiar AI concerns, there are also issues that arise with text that look more distinct. We identify a few, but have no illusion that we’ve exhausted this category. A couple of the ones we discuss:
Floodgates: when it’s cheap and easy to generate texts nearly instantaneously, that raises the possibility of overwhelming institutions built implicitly around the costs and speeds associated with text written by humans. We’ve already seen, for instance, that federal agencies have been flooded with machine-generated comments during notice-and-comment periods. And even if it becomes possible to ban astroturfing and you just consider genuine disputes with real individuals, the huge background set of legal issues that currently go unresolved because of cost could threaten to overwhelm institutions if those cost curves change significantly (see, e.g., Yonathan Arbel’s writing about this potential problem).
Insincerity: when an AI-generated text is supposed to represent the reasoning, beliefs, or feelings of a person, that creates a potential “sincerity gap”—a difference between what the text represents to the world about the person and what that person actually thinks or feels. Obviously, this is true even of non-AI-generated text, too (hopefully it doesn’t come as a surprise to you that the reasons given by legal institutions can sometimes be pretextual). But we have norms of honesty and integrity in many areas of the law, even if those norms aren’t always lived up to. We view it as a problem if a judge writes their opinion one way while they secretly have other reasons for deciding the case the way they did. Having a machine that can automatically generate plausible reasons is likely to increase the ease with which that situation can arise—and could even make it feasible for judges to generate opinions without first doing the work to come up with sincere reasons and beliefs to begin with. Or consider a defendant at a criminal sentencing making a statement about their acceptance of responsibility, or an explanation of their beliefs or emotions at the time of the criminal act in question. The task of assessing sincerity in such a context is already difficult; adding in the possibility that a machine could have generated the text of the statement makes the problem even more difficult.
The paper also discusses what has, so far, been the primary policy response to concerns about generated legal text: ratification. By “ratification,” we mean a person’s acceptance of responsibility for a text. So, for instance, when a lawyer signs a brief that contains generated text, that lawyer is accepting responsibility for that text. That’s part of why lawyers are getting in trouble for filing briefs with hallucinations in them—they have ratified those briefs, so they are on the hook for their errors. Many courts have leaned into this approach, issuing standing orders or other policies explicitly saying that by signing briefs you are taking responsibility for AI-generated text, and/or by requiring disclosure alongside a signature.
Ratification has some nice features, in that it draws on many longstanding legal norms: clerks draft text for judges to use in their opinions, associates draft text for partners to put in briefs or contracts, and we have more- and less- formal ways of assigning ultimate responsibility for end-product texts in these scenarios with multiple authors. But ratification also has limitations, and can risk being a fig leaf in situations where the person ratifying the text at issue doesn’t have the right incentives to, e.g., actually do a good job ensuring the quality or accuracy of the text.
The current landscape of ratification is a kind of microcosm of the entire universe of generated legal texts. You have a set of novel issues, and some old institutions that are trying to come to grips with it largely by leaning on preexisting norms with some light updating. That fix works to some extent, but leaves a variety of concerns unaddressed.
The article tries to make some headway, but it is largely an exercise in diagnosis and description rather than prescription. Our hope is that the text-focused lens will be a useful one, as there may be solutions (or at least mitigations) that are shown to work in one domain that can be ported over to another. We are ultimately still in the relatively early days of legal institutions’ adaptation to AI. But these developments also aren’t hypothetical anymore, and generated legal texts are starting to become a real and regular part of the law—even if legal institutions aren’t quite ready.