The New “Chaos” in Enterprise Data: How Experts Are Reacting to GenAI Prompt Data

The GenAI Prompt Data Boom

Based on estimates within the industry, there were 250 million generative AI (GenAI) prompts in January 2024. This number tripled to 750 million by July 2024 and is on pace to hit 1 billion monthly prompts by the end of the year.

Those are only the prompts. A 15- or 20-word prompt can generate 10 or 100 times more content. Simply put, the scale is massive. The legal technology industry hasn’t seen anything like this, and it has implications for eDiscovery, litigation support, and regulatory compliance professionals.

Casepoint hosted a webinar about GenAI and enterprise data challenges for more than a dozen industry experts. Oliver Silva, Vice President of Product Operations at Casepoint, and David Cowen, President of The Cowen Group, a career development and advancement group for legal innovators, co-led the conversation.

Collecting Prompt Data: Current or Future Priority?

Attendees gave a wide variety of responses regarding whether their organizations are collecting prompt data or whether they’re seeing it happen around them.

“Nobody in our realm is collecting or considering prompt eDiscovery content at this point,” an eDiscovery and litigation technology manager at a government law office said. “However, I am beating the drum that this is going to happen, especially somewhere where you can have incredibly asymmetrical litigation.

“You're going to have pro se clients that are essentially turning themselves into miniature law firms, and rightly so, in an access to justice way. Therefore, lots of things are going to start popping out of the world now, and we're going to start having, for instance, an HR system that terminates an employee based upon some analysis by a generative AI module. I can see that happening immediately. Getting people to start to think about that here is fun, but it's also kind of a challenge because it's not where they're living right now.”

Other experts chimed in on the question:

• A managing director for a top-tier law firm said they are “absolutely” seeing companies preserving and collecting prompt data.

• A compliance enablement manager for a multinational oil and gas company said they are “avoiding saving the prompt,” but the stance will evolve moving forward.

• A chief legal and compliance officer for a global fleet services company was called on to give a general counsel's point of view. The officer expressed how the plan is to launch GenAI, but “I haven’t even thought about some of the ideas that are coming out of this call, so I’m a little bit nervous.”

The professional wasn’t alone in their nervousness. Why? Due to the size and scope of the problems GenAI prompt data poses.

Describing the “Chaos” Organizations Face

The webinar leaders brought the discussion back to the sheer scale of what organizations have to contend with for GenAI data.

“It’s a very big problem that we’re solving for our customer,” Silva said. The volume of data generated by prompts and responses can get into the gigabytes or terabytes. It’s a massive data and value problem, and the industry is at the beginning of it all.

“Once upon a time, eDiscovery was chaos,” Cowen said. “Then it became organized chaos. Then it became, ‘OK, we’ve got some ad hoc solutions we’re cobbling together.’ Then we operationalized. Now we're back into a state where it's just complete chaos.”

So, if GenAI prompt data warrants a new era of chaos, so to speak, what does that mean for organizations? What issues do they need to work out?

“We're talking about IT policy considerations, information governance, security — like how does this data work, and how does it get transferred over to a company or a partner like Casepoint?” Silva said. “From a legal operations perspective, what does this mean for us keeping this data? Right now, we can try to ignore it, but we know that going forward, this will become an issue.

“We need to start planning for it. This will hit all of our plates in one way or another in the very, very near future where we'll have to deal with it. The thing is, we know how to deal with this. We've solved problems like this before. This is a little different. We just have to apply our previous learnings about what we did to this new problem.”

How to Respond: Advice From a Veteran Legal Ops Professional

A lot of industry experts expressed how they were early in the process of approaching GenAI prompt data and/or were concerned with the situation.

As a result, Cowen called on an eDiscovery veteran for advice and tips. Her first call-out was to ensure leaders understood that the credibility and responsibilities of legal operations are necessary to support the interactions and infrastructure of any in-house team.

“In terms of where we're headed and where we're coming along, it really is making sure that you're keeping a lens on the needs and pain points of your business leaders that you're partnered with,” a director of legal operations at an automated data integration platform said. “So it's not just looking within your own legal team, but legal as a support function. What about those teams that you're supporting? What's on their roadmap? What do they care about? What's the risk tolerance across the organization as a whole and at your executive team level? And then you're able to hone in on those critical areas of concern and focus, so as you're building out either the chatbots or the AI infrastructure, to know how your team is supporting the rest of the teams — helps guide that lens of what do you really need to spend your time, resources, and budget on, in terms of these new opportunities and tools that are coming forward.

“And I encourage you to continue in these kinds of conversations, to open your eyes to what those challenges may be and to take up the ‘Casepoints’ of the world on their offers to have an educational session, a demo and don't just do it yourself. Bring others along with you, because the more of you across your team that know what's road-mapped and available, the better position you're going to be in to help build the buy-in because there's less buy-in for convincing them they need it. It's showing them what's on the horizon and what can be done for them.”

Contact us to discuss how your team can meet the new data challenges presented by generative AI efficiently and securely.