Using AI to Land Better Cases? Why Not?

Darrow co-founders, Evya and Gila, were recently interviewed by Christopher Anderson, host of the Un-Billable Hour podcast, which is part of the Legal Talk Network. It was a great conversation about how attorneys can leverage AI as a business development tool, effectively cutting down on unbillable hours or time spent between cases.

Listen to the episode on Spotify, Apple or Google, or read an abbreviated transcript below.

–

Chris: In today’s episode, we are going to discuss AI and what it means for law firms in production and marketing. I’m honoured to have my guests today: Darrow co-founders, Evyatar Ben Artzi and Gila Hayat. I know you guys are probably familiar with the lawyer who just had AI write his brief and it made up cases and submitted them. You know, there are these stories that people are hearing. So let’s talk about the serious uses that we can put this stuff to that could really help lawyers and law firms achieve more success for their business. But before we do that, what brought you to found Darrow. What was the driving vision? What are you trying to accomplish?

Evya: Well, it was about four years ago. I was clerking at the Supreme Court with a friend from law school, and we realized the serious level of friction that the system has. You basically see a lot of cases that never really make it to court, just by doom scrolling on your phone at the end of a workday, right? You go on Instagram or Twitter or whatever, and you’re like “oh, there are so many legal violations in the world. Most of them don’t get to my desk every morning. And as a clerk, that was frustrating. Once we caught that and we learned that most legal violations never even get detected to begin with. People don’t even know that they’ve been harmed, most of the time. They just don’t know. And we shared the problem with Gila and she immediately recognized that what we were talking about was this fluffy macroeconomic problem in the world about inefficient enforcement of the law or something like that. She said immediately, this is about an information gap. It’s an intelligence problem. 

Gila: The fact that there is knowledge somewhere or intelligence that is scattered around is not something that’s unique to the legal system. In general there are a lot of opportunities and a lot of exposure in knowing what the other side or any side of that story holds. So, working in intel and understanding at the macro level: what is the story behind it? What are the driving forces? It requires a lot of data and requires not only data, but also a methodology to understand that to tell the story at scale and gather that evidence. So when we met and we started looking around this problem, we understood that it’s not about the legal profession. It’s the ability to acquire knowledge, work around it and work around data in order to support those stories or those narratives. As a data professional, for me, what is required is the story behind it –  the legal rationale, that is the DNA of the story that we’re trying to convey. Because in the end, it’s very difficult to understand. What can you do with raw data without the story behind it? Because raw data doesn’t tell a story, it’s barely information. 

Chris: We’re talking about unstructured data as well ? Like just data from all sorts of sources that isn’t neatly in a database in a nicely categorized way, Yeah? 

Gila: Yeah. Most data is dirty. Data is not built to serve us for a purpose, it just exists. It takes a lot of knowledge and work to understand what you can derive from data in order to make it into a coherent, cohesive story.

Chris: Sure. Okay, so let’s connect the dots a little bit. So Evya described a problem – that lawyers are spending a lot of time on less impactful cases, and yet we’re aware of more impactful things out in the world –  but they’re not making their way into the legal system. And now, Gila, you’ve talked about somehow harnessing the unstructured data that’s out there – “dirty” as  you called it, to discover these problems and tell  a story about it. So now let’s connect those two dots. How does that help lawyers? How does it help legal practices? How does that help the world?

Evya: Well, lawyers care about that deeply because they spend about 20% of their working hours on developing the business. At least in litigation teams that means hunting for those cases, and that’s a lot of work that goes into something that isn’t legal work. 

Gila: It’s unbillable work that is basically creative. So it’s about developing your business in creative ways, especially when it’s understanding what kind of social phenomena are out there in the world. And that really requires harnessing creative skill that lawyers are expected to have in order to bring business in. 

Evya: This podcast is called the unbillable hour, right? We’re trying to cut down on the unbillable hours that lawyers have because when you’re developing a new case, it doesn’t mean that you’re going to be able to bill that time. And that’s the thing that we’re looking at as the business development work for law firms, at least for litigation teams.

Chris:  A lot of lawyers wait for new business to come knocking on their door. They  wait for their clients to realize they have a problem, or an opportunity, and then bring it to them. But what you guys are talking about is saying, hey, you’ve got a problem, and you might not even know about it. Which in marketing we would refer to as being higher up the funnel. And what it means in practice is there’s a much larger market to address, with a lot of opportunity, that might otherwise never even be recognized. Am I catching it right? 

Evya: Perfectly. 

Chris: So who are you helping right now? What kind of law firms, and  lawyers, and what kind of practices are amenable to this with you right now? 

Evya: So we mainly work with  plaintiff side litigators – partners and associates that do litigation. In general, we work with the rainmakers of the firms – those top litigators who bring in the business. 

Chris:  Let’s take a break here for a second and talk about how you’re able to bring them business. I want to talk about a couple of practice areas, and understand what kind of discovery you’re doing for them? What kind of business are you being able to bring that they might not otherwise have? So who’s using this? What practice areas have been successful using these methodologies?

Evya:  We are able to bring plaintiff’s attorneys new business in a number of areas including consumer protection, product liability,  environmental issues, securities, fraud, and general financial misconduct. As well as antitrust, pharmaceuticals, employment, and privacy. 

Gila: The technology we built allows us to understand cases by looking at patterns. So  we’re applying law on raw data. That’s what we’re doing. This allows us to expand very quickly to new types of legal domains and new types of cases.

Chris: Sure, as long as there’s data out there. You said based on the available data –  do these data sources tend to be publicly available data sources? Or do companies bring you their own data that you can then sift through?  What do you feed into the algorithms to give you an idea of  what the story is starting to look like? 

Gila: We look at everything that is publicly available – a lot of openly available databases and data sources, as well as a lot of texts. Although what we look at is publicly available, they tend to be big data dumps that although publicly available,  no regular person will ever be able to put their  hands on it  just using a browser. 

Evya: Just sometimes it may not be publicly noticeable, but it’s available. 

Gila: There’s a funny anecdote that the best place to hide a dead body is on page three of Google search. Well, we’re looking there, and we’re going to the deep web and beyond in order to find those data sources. What’s also interesting about what we do is we make combinations. So for example,  one publication or data source is not enough to tell a full story. But if the story is told in one place, then its out there in other places too. We have the ability to craft and understand the narrative of a story,  and where it’s spread,  in order to find the full picture of the story. So we’re basically connecting both the legal dots and the evidential dots in order to create a full cohesive story. 

Chris: Let’s take a minute to step back, because there are a lot of conversations around AI. People are using the word AI, people are familiar with the big stories in AI, and of course, chatGPT is out there and people are familiar with that. But at the same time people  have skipped over actually understanding what it is. So what is AI? Chat GPT is a language system, it’s one way of using AI – but it doesn’t really encompass what AI is and the extent of what AI can be. So how would you describe what artificial intelligence is in the context of your company? 

Gila: Artificial Intelligence, in a nutshell, is a system that has ingested huge amounts of human intelligence that has been curated for a long period of time. It might look like magic, or even dark magic for some people. But at the core of things, AI is a machine that has been exposed to a huge amount of data, that due to the sheer volume, couldn’t be comprehended by a single human being. Thanks to the exposure, the machine then has the ability to derive or predict the answer to any kind of question. So the more data you put into it, the better results you will get. It’s actually similar to the way you would teach a child to talk. The more they’re exposed to a language, the more they will be able to speak it. And when they’re not sure, just like when chatGPT is not sure of something, they’re very keen to give good results, so they just work with the best they could, based on what they’ve been taught. So language models can look like dark magic when they say things that sound coherent, but they are only as good as the data they were trained on.  And from what we’ve seen with general AI models, like ChatGPT for example, they talk about legal matters in a way that a civilian would, but they don’t really get to any depths. For example the intuition  of what a story is really about, or what kind of narrative can  make it a more understandable story. What we’ve been working on for the past three years is giving AI in the legal sphere that missing component, that intuition, that ability to make a real legal decision.

Chris: What you’re talking about sounds amazing, and everybody should jump on it. However there are going to be skeptical questions which deserve to be asked and answered. One thing you just mentioned is AI’s need to answer a question, even if it doesn’t have sufficient data to answer it,  which leads to BS and the problem of GIGO (garbage in garbage out.) You said you go deep in the web, and I don’t know what the statistic would be but I would venture to say that there is substantially more misinformation or quasi misinformation than solid information out there. So how can law firms and attorneys be sure that they’re not chasing a phantom problem that is just stoked by misinformation? 

Evya:  First of all,  the idea that there’s a lot of misinformation on the web is something that we all intuitively feel is right. Not because people have bad intentions just because sometimes there is misinformation. The capability of finding and cross referencing a source is not unique to humans, humans do it. But machines can also do it. And anyone using generative AI should employ techniques for cross referencing their sources, making sure that the information they’re providing is the best available. Anything that anyone does today, there is a trace online, it’s there. And there is true information online.  And so deciding not to do anything about it because of the risk of doing something wrong,  that’s not a good thing for the further technological advancement of law. We have to do something. So cross referencing sources is important. And if for example you get a case out of ChatGPT that seems odd, then you might want to cross reference it and check whether it’s accurate.  And that doesn’t mean asking the model again,  we’ve seen that doesn’t work. The techniques of cross referencing sources have to be advanced thoroughly. That’s what we’re working on and  there are a bunch of other companies that are doing this as well. Hats off to everyone working on this problem, trying to figure out how to prevent hallucinations!  We see a lot of e-discovery companies doing this, and that’s  a space where you really can’t get it wrong. The stakes are too high. Justice is at stake, and sometimes people’s lives are at stake. You can’t in any way get this wrong. So using AI without the proper breaks, that doesn’t work. You have to have checks and balances in your models and in the architecture.

Gila:  The initial fear from commodity language models was that they are a single model that is a complete black box, that we know nothing about how they work.  It’s insane to call them commodities now because a year ago we were not even able to imagine it. But they do not exist by themselves. They are not self-driving machines.

Chris: Not yet

Gila: Not yet. There’s a tremendous amount of investment going into the infrastructure. The ability to call for humans, or to cross reference. It’s not necessarily just one statistical model, or even a family of models or a group of models that are working together. But it’s also the ability to apply criticism and  see if  it really checks out – and clarify whether a source is real and truthful. So when we’re talking about AI,  It’s not just a coin where you ask a question, and you get the full legal case. If that was so easy, then we wouldn’t be here talking about this. There are a lot of different steps to finding legal cases from A to Z, and AI is just one part of it. But it also requires a lot of human intelligence, as well as many different checks that need to be applied to the AI. So it’s not just this one model working. There’s a lot of software and steps involved into understanding if we are getting close to a story, whether it’s truthful and whether it’s feasible 

Evya:  And there are a lot of humans in this loop. We’re 90 People at Darrow – in Israel and New York. The AI companies, the ones developing models, machine learning architectures, and building AI,  are trying to be the beating human heart of this machine. They’re trying to educate the machine in order to make it sufficient for our needs. And that can’t be just this idea that we’re building a machine and that’s it. It’ll do everything for us. There is a whole operation around this. That’s what AI companies are about. The software is part of it. It’s not everything.

Chris: In the beginning, you talked about the using AI to eliminate the unbillable hour – which I take personally –  let’s call it optimizing the unbillable hour, and really allowing attorneys to do that in a reliable way. So how does a law firm come to you? What’s the question they should be asking? 

Evya: It’s the question of do I want to have discipline in my business development and marketing practice? I’m developing new cases. I want new cases for my litigation. What is going on? Am I doing this in a disciplined way? Am I systematic? Am I methodological?  It could be the managing partner of a firm saying, I feel that we’re not being disciplined enough, or I see some of my litigation teams having trouble with a lot of downtime and I want this to be fixed. Or it could just be a litigation team that wants to insert some discipline into their practice 

Gila: And bring their A game.

Evya:  Most legal tech companies today are focused on making firms more efficient. At Darrow we are focused on helping firms grow. So any law firm that wants to grow and has a litigation team, they will usually come to us and say “Hey, how can you help us?” And we start out by saying, we’re not going to help reduce your time spent on certain tasks. We’re not going to be helping you perform discovery tasks in a simple manner. There are companies that do this but we don’t. We will help you generate more cases and more new business for the firm. This is powered by AI and has the ability to mitigate the risk of spending a lot of time on  business development tasks. And that’s the major value that firms that use Darrow are looking for. They’re looking to reduce the downtime, and get more business into the firm to grow. 

Gila: There’s a lot of risk in growing by expanding into new domains or expanding the practice shown in court. Because you don’t necessarily know if you can find more cases, or find the plaintiffs. All of that work without both the discipline and the data is very dangerous. We mitigate this risk for law firms.  We take a data driven approach so we’re able to remove the guesswork around validating new fields and understanding the best strategy to move forward in growing the firm. 

Chris: So let’s take that last step now. Gila mentioned that this stuff didn’t even exist a year ago, and some of it is already commoditized today. It’s improving at an exponential rate. The models are becoming smarter, and faster. So what’s the future? How does this look? I don’t even know how far I could ask you to look down the road. Is it fair to say a year from now? What do you see coming? 

Gila: A year ago I was overwhelmed just by the event,  by the adoption, because the ability that has been introduced to the world is mind blowing. So it’s very hard to guess where we’re going to be in a year. During the past three years we’ve learnt that we as humans, or as professionals, are going to distill the human cognitive things that cannot be replaced. We’ve also learned that asking good questions is better than thousands of good answers when you don’t know what question you’re trying to answer. A lot of the task work is going to be redundant because it’s going to be replaced. What is not going to be replaced is the ability to ask questions, and to find problems. So I think that what language models bring us, or in general just the commoditizing of AI, is that it will turn everyone into product managers. Lack of technical skill is something that you can quickly overcome. However, the human component around problem-ing rather than solution-ing is going to be key. We’ve seen that working with lawyers is to carefully draft that question, carefully identify what sources do I trust or not? What kind of methodologies? What kind of train of thought is aligned with my mission? That is something that is very hard to reproduce just by prompting simply. It’s really a process. The ability to learn that process, ask the good questions, and formulate what the problems are that we’re trying to solve, is going to be key. It will completely redefine the way we interact with technology. I can’t promise anything about what the next user interface will look like, but in terms of cognitive skills, we as human species are going through hyper growth of learning how to ask questions. When it’s quick enough to get that answer then the loops are going to close much faster.

Chris:  Is that part of what you do for law firms is teach them how to ask the questions?

Gila:  Absolutely. So we’re asking law firms a lot of questions to understand their practice and then see how to apply those machine skills. There’s a lot of machine learning but also a lot of machine teaching. And interacting with our clients is the core component of teaching that machine. So by asking good questions, to both the machines and the lawyers that we’re working with, is what will make it happen. And we take pride on the ability to translate that, and introduce more context, more content, and more data into that. But in the end, it’s about the question that matters. 

Evya: And I think you don’t have to be a technologist per se, to ride this wave. That’s the cool thing about it. If you’re a lawyer, and looking to understand how you can implement these capabilities into your practice, you don’t have to go and study computer science for 10 years. 

Chris:  And in a sense, lawyers and law firms are an excellent market for you. Because the one thing we’ve been trained to do for many, many years is to ask questions. That’s what we do.