Of all the places to try out machine learning, the legal market would probably not be the logical choice, given its reputation as a laggard in technology adoption. But about five years ago, machine learning was given a go – in the form of something called predictive coding in a certain area of legal software called e-discovery.
It’s fair to say that did not go down well. But the reason it didn’t take off was not because it didn’t work – quite the opposite. It did not prove popular in e-discovery precisely because it did work. Therein lies a lesson.
E-discovery is the pre-trial process of gathering and analyzing evidence in litigation. It is also used in investigations at large corporations. Predictive coding is the name given to technology and workflows, whereby samples of the entire corpus of documents – which could be a few hundred to millions in size – are run through a text-analytics engine. And through the use of machine learning, the software ‘learns’ what is relevant, and not relevant, in the case.
The process is repeated – all the while getting input from lawyers – until the software reaches a threshold where it is satisfied that no more relevant documents exits. This has the potential to save huge amounts of money, because lawyers don’t have to look at every single document in a corpus to determine relevancy (or ‘responsiveness’ in legal jargon), which is the way it had been done in the past – and still is in many instances.
However mundane document review of this sort can be, lawyers who get paid by the hour are not particularly interested in using technology that could dramatically cut the number of hours they bill clients for. But predictive coding using machine learning and text analytics can do precisely that, and can cut down the amount of documents lawyers have to read as part of the discovery process. The savings can be dramatic. This is technology that uses both supervised and unsupervised machine learning to solve the ‘show me more like this’ problem. And it’s good at it.
The next big thing
Now, according to IBM, Google, Microsoft and many other vendors, machine learning is the next big thing. And that idea goes beyond the large technology vendor, although it’s fair to say they are driving it. 451 Research covers vendors regardless of size and maturity, and there are too many to list here, but what follows is a sample of applied machine learning that we have published research on in the past three months. (Note that this does not include the various machine learning toolkits; this report is intended to show the various applications in the market).
We are seeing machine learning used in all sorts of advanced analytics tools in software QA testing during application development. Systems management companies are using it to diagnose problems in applications, and learn how to fix them automatically. We also see it in the form of robotic-process automation the outsourcing industry.
These are just a few examples, but there are many more, because there are still huge inefficiencies in the way most of us work – doing repetitive and mundane tasks that software can – and will – do better.
Why now?
So why is machine learning popular now? There are a number of reasons. Compute power is available as never before. And data is available in volumes never dreamt of, which means the algorithms have more to get their teeth into (machine learning lives or dies depending on the quantity and quality of the data available).
The increasing use of containers of software development means small machine-learning applications can be placed into software without breaking the rest of the application. And the digitization of all sorts of industries means there are more places, more data, and more compute power than ever before.
So although some participants in more conservative markets such as the legal sector may prove resistant to machine learning’s charms, we believe the technology has reached a point where any predictions about the future software that don’t include extensive use of machine learning will be wide of the mark.
This is an extract from my recent 451 Research report, which 451 clients can access here. For trial access to 451 Research, click here.
