AI will change the world. There is little doubt that in many ways it already has. The pace of change, however, is uneven and often oversold. Particularly over-hyped is the degree to which AI makes “everything different.” AI is not ready to replace humans and my expectation is that it never will. When you think AI, think “Iron Man” Tony Stark, not Samantha the voice-only cyber-girlfriend in “Her.” Buyers of legal services with AI components should not leave their skepticism at the door. AI tools are rapidly improving but they are not perfect, and in the wrong hands, can make a mess of your project. Be sure that you understand what the AI platform does and make sure that the work plan your vendors propose has room for learning and course-correction as it unfolds. Most importantly, make sure you understand how the work-plan uses humans. That human element remains crucial, regardless of impressions that whiz-bang marketing pitches of service providers and staffing companies might give.
My Story
My task was straightforward: oversee all legal aspects of the sale of a business for which I had served as general counsel for 18 years. At heart, this job centered around creating and populating a virtual data room (VDR) and gaining sufficient working knowledge of the important risks that the non-financial documents loaded to the VDR would reveal to an alert buyer. I had a strong feeling I knew where to look and what to look for when searching our contracts and other documents for things that could slow down the transaction or get in the way of buyers paying the highest price. Still, only a fool acts as their own lawyer, and for similar reasons no corporate manager should feel that they are the best choice for doing the due diligence review without outside, expert assistance. Moreover, this being the latter half of the 2010s, there were alternatives to paying large law firms to mobilize their high-paid associates to review all the documents the old fashioned way: document by document, front to back. In short, there were tools! Artificial Intelligence tools!
When I asked our outside M&A law firm for advice, I was told my hunch was correct, they were not going to be able to perform the essential document review functions at a competitive cost. A new generation of legal services firms (FTI Consulting, Axiom and ThomsonReuters among others) were ready to provide document review and staffing services. We contacted several firms and chose one well-known staffing provider and legal service firm who promised that they could reduce costs without compromising accuracy by the intelligent use of (yes, you guessed it) “AI-empowered tools.” In answer to follow-up questions, the service firm revealed that the AI engine for their service would be Relativity One. A search revealed it was a leading platform for e-discovery and legal document review. I was sold.
The service firm asked that we gather up the documents (it was determined that we would start with a review of contracts for “consent to assignment” issues) and provide a list of search terms that the service provider would use to “train” the AI engine to look for relevant text. We collaborated with the provider on a set of instructions designed to isolate what I thought were all the variants of “assignment” and “successor” language so that we would have the ability to create work plans for follow-up depending on whether the ultimate transaction was structured as an asset sale or a sale of shares (these structures can have very different implications for disposition of contracts).
During the 10 days or so that the project was in the hands of the service provider, no requests came back to me from the vendor to help them refine or improve the accuracy of the search terms. We only got reports saying that things were going well but that we should be patient as there was a chance that they would need a bit more time. When we finally received the results, they surprised us.
The number of “positives” (i.e. contracts that required additional review to determine how they were to be managed through the transition to the new buyer) was much higher than I had anticipated. With the help of a contract lawyer and a bright and experienced new hire, we came to the conclusion that the service provider had not understood the task. When we pressed, it was revealed that the heart of the service was a group of trained human document reviewers who made the final decision on how to “code” the clauses that were tee-ed up by the Relativity software. The problem was that these human “coders” did not understand what they were doing.
The good news was that after testing a large number of contracts, that the pattern was one of systematic over-inclusion, not under-inclusion. Thus the work was not wasted; however, it was nowhere near as efficient as advertised. My team had to put great deal of time on the back end was to individually vet each contract and reassign it to the proper list for action (obtain consent or give notice) or no-action. These tasks were unwelcome and unexpected, as we had a multitude of other things we needed to do to finish the deal.
Key Learnings
In the end, to effectively manage transactional document review, it is more important to build a process that uses human talent effectively than to choose a vendor offering the right software. The software can be important, but only in the context of the overall plan.
- AI tools are aids to human document reviewers. They are not substitutes for human judgment. Therefore it is crucial that the flesh-and-blood reviewers are well-trained and well-supervised.
- If you decide not to use your outside law firm as the manager of the due diligence process, then you are better off using a service firm that can assign a lawyer with real experience in transactions to oversee the team of reviewers that will be using the search platform. Otherwise, be prepared for you or a trusted in-house counsel to be trained on the review software yourself so that you can supervise the process.
- The work-plan needs to build in enough time to allow for up-front training and for mid-project check-ins where issues can be flagged and the course corrected if need be. The worst thing is to think you have set a deadline by which everything would be completed only to find out that the quality of the work requires additional steps to ensure its accuracy and to avoid needless do-loops with third parties that add cost and risk to a transaction.
Looking towards the Future
The particular Relativity software used by our service provider is no longer the state-of-the-art in the industry. More importantly, software companies are becoming much more transparent about their offerings as their customer base has become more sophisticated. Cloud-based solutions providers such as Exterro and DISCO offer materials that explain how make the most of the tools and stress the “garbage-in/garbage-out” trap that misuse of tools can create. Machines can only learn what they are taught. Many providers stress the importance of real humans beginning the document review and teaching the AI-system to complete the task on a much larger pool of documents. The marketing materials are increasingly making clear that key to the success of a review project is assessing and making corrections that increase the probability that the right documents (based the correct textual inferences) will be found. Human review should be used to train the machine. Reliance on up-front rules (so-called “seed sets”) should be minimized.
What is clear is that the analogy of “Iron Man” Tony Stark (a real human with a set of force-multiplying tools surrounding him) is the most apt way of thinking about AI and how to harness AI to make you a better manager of your transaction.
Richard Paul Pasquier is a partner at the law firm, Practus, LLP. Mr. Pasquier helps clients navigate mergers and acquisitions, business transactions, commercial contracts, litigation, compliance, regulatory affairs, human resources, and labor matters. From his experience leading compliance and human resources functions in the chemical industry, he combines skills of a business lawyer with expertise in antitrust, trade associations, public policy, community relations, product safety (chemicals, pesticides, and food safety), and environmental health and safety.