We live in a world in which business decisions are increasingly driven by data. And that data isn’t just the structured variety stored in a relational database and accessed via BI tools. The majority of data is unstructured and can be hard to find. This is where enterprise search comes in.
The market for enterprise search software has everything going for it in terms of macro trends, but little going for it in terms of visibility in the enterprise, market growth or a vibrant ecosystem of startups. So what’s wrong with what’s on offer, and how could it change for the better?
Principal enterprise search vendors
|
*in the process of being sold to Micro Focus
Points in favor
First, every survey on how people look for and find information in the enterprise shows that people regularly complain about loss of productivity due to the inability to find information. This unsolved problem will only get more challenging – and, thus, is an ever bigger opportunity as data continues to explode in size, type and scope.
Second, the volume and the rate of data growth continues to increase, which means the problem is only getting larger. The locations where corporate data is stored have become more diverse as increasing amounts of corporate information has moved to the cloud, but significant amounts will always remain on premises, behind the firewall. And the Internet of Things has introduced whole new classes of information sources throwing off new types of data at unprecedented frequencies.
Third, data types are ever more diverse. Consider the way teens communicate with images via Snapchat, let alone all the texts, emails and other forms of messaging flying around. So, technologies – such as text analytics, which underpins many search engines – is important as are ways of analyzing other forms of media such as image recognition.
Fourth, machine learning is the hottest thing around right now. It is something that most enterprise search vendors are well-versed in the benefits of – having used it for years to handle tasks such as categorization, summarization and ranking. Machine-learning-driven text analytics, along with driving some sort of effective search or discovery tool, is the only feasible way to handle the explosion in data because it is beyond human curation at this point.
And fifth, open source software has become widely accepted in the enterprise as a viable option. Large ecosystems have built up around the two main technologies – Apache Lucene/Solr and Elasticsearch.
Points against
Now let’s look at the points against enterprise search (in no particular order of importance).
First, the return-on-investment story for enterprise search is a vague one – at least when it comes to general-purpose, horizontally-applicable enterprise search.
Second – and related to the first point – search isn’t something that requires much of a workflow or defined business process. Applications, such as salesforce automation, are a no-brainer for commercial organizations because they can understand the process – starting with a lead coming into the company to its conversion into a sale. But many search problems are quite different from one another and don’t lend themselves to being easily encapsulated into a business process.
Third – and although it might sound trivial – names like ‘search’ and ‘retrieval’ don’t sound to C-level executives like spending priorities. That’s why more recent search vendors have favored terms like ‘discovery,’ which hints at telling the user something they didn’t know before. The business intelligence market has a great name, even if it is only looking at a small portion of the total available data. And roles such as ‘data scientist’ sound a lot more interesting than ‘search manager’ or even ‘knowledge manager.’
Fourth, if you take venture capital and other forms of investment as a proxy for a broad, potential customer interest in the space, you’ll find a noticeable lack of startups there – certainly compared to a decade or so ago. Of course, this is more of an effect of the market’s nature than a cause of its problems. And the lack of interest from the largest software players is also pronounced. Sure IBM, Microsoft and Oracle have products in the market, but they’re hardly front and center for those vendors. And Google, which launched its enterprise search appliance in 2002, told its partners and customers in early 2016 that it would be discontinuing the product by 2018. Google didn’t give a reason for it, but we assume it was due to the lack of revenues in the context of its overall business. It had stopped selling a smaller version called the Mini in 2008.
Fifth, after the old on-premises document management market largely ignored the search problem in the early 2000s (which in turn created opportunities for search vendors back then – the modern counterparts in the enterprise file sync-and-share market, such as AeroFS, Box, Citrix, Dropbox, Egnyte, Google and others) offer what we would call ‘good-enough’ search. It’s considerably better than what is offered by the old document management vendors, but it is usually confined to searching within each vendor’s tool and not meant as a general-purpose enterprise search engine.
Sixth is search’s old nemesis, information security. Security is tough for search because in most enterprises users can’t open documents they don’t have permission to see, and the documents should not appear in their search results. This requires highly granular security, often at the sub-document level, to be successful. However, many implementations struggle with that, so a more coarse-grained security approach is taken. This often results in users not finding what they are looking for because they’re excluded from seeing things they do have permission to see.
Seventh and last is Microsoft SharePoint. Although Microsoft no longer sells stand-alone enterprise search (it rebuilt SharePoint’s search in part using the technology it got from its 2008 acquisition of FAST Search & Transfer), as with the document management vendors mentioned earlier, for the millions of users of SharePoint Server 2016 and earlier version and now SharePoint Online, its search is good enough. As such, it is a massive impediment to independent search vendors that have to convince IT departments of the need to replace SharePoint search with their own tools.
What works in search?
Domain or role-specific search has been – and will continue to be – successful, in our view. Examples here include Attivio in data source discovery, Coveo in the customer service market, Dassault Systems’ Exalead in manufacturing, Splunk in log file analysis, or the myriad of eDiscovery vendors in that field whose products are based on search and text analytics.
Open source search seems to have hit a nerve as well. LucidWorks is basing its business on the Apache Lucene and Solr platform, while Elastic’s Elasticsearch products and ecosystem are also experiencing a rapid rise.
One of the main areas of interest our customers have asked about is cloud-based search or search across both on-premises and cloud repositories. The main issue here appears to be connectors and latency rather than any other major challenge, but we think there is an opportunity here. To that end Google has just launched its Cloud Search as it seeks to maintain those Search Appliance customers while winning new ones.
This is an edited version of a report that first appeared on 451 Research’s website here. For trial access to 451 Research, click here.
