Are You Finding Quick Enough? Discovering Enterprise Search

Searching is human and finding is divine. In all likelihood, you have experienced this many times and it does hurt for not possessing the celestial power to quickly reach that essential piece of information or document or a data entry, which can serve as a silver bullet in your critical transaction. Especially, in an enterprise scenario, fate of a critical business decision depends on timely access to relevant and accurate information at minimum effort and expenditure for the enterprise representative.

With the exponential growth in the volume of information, it has become imperative to possess appropriate tools to enable search in the maze of information present inside the logical enterprise walls. Without an effective tool, you and your colleagues will be found starving for the information that matters. A tool offering such capability is called Enterprise Search Engine and has the potential to unleash the incredible power hidden inside the digital information owned by the enterprise.

Enterprises and Enterprise Users – Current Sentiment

In one of the recent surveys by Association for Information and Image Management (AIIM) - http://mis-asia.com/technology_centre/enterprise_applications/study-companies-need-better-enterprise-search - in which they interviewed enterprise users, it was found that employees are complaining about intranet search and are seeking answers to why can’t we have Google in-house?

While Google ( http://www.google.com/enterprise/gsa/ ) and many other vendors provide enterprise search offerings, the scale and speed of adoption has been fairly sluggish so far. The above survey seems to indicate that less than half of the businesses do not have any formal plans of enterprise search, while almost 69 percent have more than 50 percent of their business data undiscovered – making it virtually unreachable even if the most potent search implementation is deployed.

Still worse, only 10 percent of the survey respondents indicated success in searching greater than ¾th of enterprise content.

Another fact uncovered from the survey was that most enterprises are holding onto diverse search tools over their different information silos leading to the problem of non-standardisation and non-unification.

Conceptual Introduction

In simple terms, Enterprise Search Engine can be considered as an evolved cousin of Database Search. However, given below are the constraints or challenges faced by Enterprise Search Engines:

1. Sources and data types of information to be searched can be numerous, enormous, diverse and distributed and they keep evolving with changing times
2. Security (access control) over enterprise information is typically distributed (remote) and not necessarily centralised
3. User query is typically neither structured nor controlled. It is difficult to translate the user’s exact search motivation and intent into a suitable query
4. Updates to the information happen outside the purview of the search engine
5. Structured and unstructured information has to be dealt in accordance with their format, intent and context

It takes additional support for the Enterprise Search Engine to address the above challenges and taste success.

How Does An Enterprise Search Engine Function?

To begin with let us discuss the basics about the functioning of a typical Enterprise Search Engine and then highlight several enrichment possibilities.

Enterprise Search Ecosystem

An extendible ecosystem is necessary to enable the enterprise search engine to fulfill its responsibilities and go beyond. The ecosystem comprises connectors that can crawl and retrieve content from variable sources, top-notch indexing and query processing algorithms that form the core of the engine and a flexible result display and navigation framework that can render matching information and help result navigation and refinement in a user-friendly manner. All this coupled with tight security embedded with the enterprise’s access control policies.

Enterprise Search Engines will usually support certain content sources like file systems, databases and web-enabled content out-of-the-box. For other content repositories, content connectors are required. These connectors, usually proprietary in nature, will leverage standard or proprietary APIs and other access mechanisms provided by the 3rd party content repositories to retrieve content hierarchy and related structures, package them into normalised format understood by the search engine and submit for indexing.

Connectors are also required to bridge and normalise any diverse authentication and authorisation models between the search engine and 3rd party content repository.

Tokenisation and Indexing of the retrieved content is core to the search engine. This is heart of the search engine behaviour and the richness of search-n-find capabilities largely depend on it. Tokenisation helps the search engine to breakdown the document and identify what words inside the content are valid candidates for indexing. Indexing is the process of recording such tokens with their corresponding access points for fast retrieval of matching documents.

Query Processing is a run-time activity that will accept user query inputs, tokenise it and run the keywords through the indexed catalogue to retrieve matching documents/records. This part of the search ecosystem positions most relevant results by operating on the user’s behavioural pattern, choices of data and other parameters.

Security Is No Compromise

Enterprise security is honoured by ensuring that only authorised users are presented with matching records. Some standard ways of ensuring security are mentioned below:

1. Early binding – Here, the enterprise search engine will also seek authorisation lists from the content sources while rest of the content is being retrieved. The authorisation information is then stamped on the content being indexed. These authorisations are then used when processing the query to filter any unauthorised records for a particular enterprise user. This is the most optimal mode but has the risk of leveraging stale authorisation data.

2. Late binding – In this mode, the enterprise search engine will seek, at query runtime, to resolve authorisation on-the-fly. For the records that match user query, further filtering is exercised by checking authorisation against the content source. This imposes some run-time overhead, but ensures no security leakage.

3. Hybrid binding – In this mode, combination of early and late bindings is exercised providing a fairly up-to-date and optimal solution.

Most search engines will provide in-built support for one or more security integration modes.

Performance and Scalability

As businesses grow, the volume of information sources and quanta, user base and information requirement rises. There are several outward and inward techniques to enhance performance of search engine response and scale to higher volume of content. Some are design time and others deployment time. Obviously, the goal is to stay at a 10 times higher scale than the enterprise’s current need.

Conclusion

While enterprise search realm has come a long way and evolved a lot over the last couple of decades, it is still in a nascent stage with respect to widespread adoption, demonstrating complete RoI and overpowering the problem of information discovery. While the Enterprise Search Engine vendors are building and will continue to build more functionality and infrastructure around their platforms, it is equally or even more important that those enterprises and their enterprise search stakeholders inject catalyst elements in terms of:

• Allocation of right budget and resources
• Analysing and defining precise success factors – use cases
• Maintaining momentum and executive-level insight

The above coupled with the right choice of enterprise search will go a long way in ensuring successful implementation.

Paranjpe is senior project manager in the Enterprise Search and Connectors Competency at Persistent Systems.