Over these weeks we saw how analysis of unstructured and semi-structured data locked inside supply and procurement channel documentation can lead to better spend and revenue forecasting (Supply and Procurement Intelligence: Analytics from Unstructured and Semi-structured Information ).

We also saw how an early adoption of information classification standards could help businesses streamline their supply and procurement chains (Information Classification Standards: An Overview).

Most recently, K-Praxis offered a perspective on how unstructured information analysis can help manage and leverage product information management and on its integration across various e-business channels (Intelligent Product Information and Catalog Management)

Having gained some understanding of the various practical applications and uses of unstructured information management across various e-business channels and chains it is now important to link these understandings and insights to some of the broader themes that emerge out of such analysis.

Here are these themes:

Enterprise Information Integration

• Introduction

When we talk about integration of unstructured and semi-structured information management into more structured information and numerical data we are talking of something like a complete Enterprise Information Integration or EII ( a term supposedly coined by Aberdeen Group). Adopting this approach companies will be able to gain access to a 360-degree-view of complete organizational information - whether that information lies in databases in numerical format or lies in textual format somewhere hidden in a text field inside an RFP. EII - or something similar to EII - will be able to offer an overarching information-umbrella to integrate: Corporate Information Portals and Enterprise Content Management Systems; Enterprise Applications:CRM, ERP and SCM; Business Intelligence and Decision Support Applications; Web services; and, Unstructured and Semi-structured Information Management Platforms.

• Achieving Enterprise Information Integration: Shift to a New Class of EII Applications or Integrate Individual Applications or Modules

An ideal EII application or methodology should allow you to capture/extract, organize/structure, standardize/normalize information and data from a variety of disparate and heterogeneous sources (structured, semi-structured, and unstructured) and formats (text fields in commercial documents, database tables, XML, word and PowerPoint documents, emails, PDF documents, information on the Internet and public sources, and other media - data captured through monitoring television and newspapers).

If a potential buyer decides to opt for an EII solutions he/she will be faced with a natural question: whether to migrate to one of the new class of platforms offered by various EII vendors (list of vendors given below) or to look for an individual module and just plug it into your existing enterprise software infrastructure, and the latter (plug-and-play-approach) seems to be much more cost effective, but if you are a big company with huge amount of turn-key information-processing requirements then it makes sense to have a look at these platform as at some point in future EII is going to be an inevitability for your organization

• Vendors Offering EII Platforms

Here are a few key vendors who are offering solutions for information integration (ordered on the basis of technological offerings as regards to unstructured information management) :

MetaMatrix; Metatomix; Semagix ;Attunity ; Enosys ; Nimble ; Sypherlink; Sagent ; iPedo ; Callixa ; Dicision Support Inc . Out of these, Metamatrix, Metatomix and Semagix seem to be particular worth investigating in terms of technological capabilities.

• Database vendors and EII

Another trend towards management of unstructured information and information integration is that the database vendors themselves are offering capabilities to handle structured, semi-structured and unstructured data through the databases themselves. Oracle and IBM are leading towards this direction.

Regulatory Pressures

As noted earlier, management of unstructured data is important to provide a 360-degree-view of organization information. And more so, since in the recent years there have been growing regulatory and compliance-related pressures making it binding on companies to opt for more transparent information management and discovery standards (SEC or new legal discovery compliance regulations).

Requirement for Interpolarity and Standards

There is also a push from various interpolarity and standards organizations, such as Semantic web, Dublin Core, UNSPSC, XBRL, ebXML, etc. (a comprehensive list of various standards is listed in Information Classification Standards: An Overview) to adopt various information standards to achieve better intra/inter-organizational information interpolarity.

Coming of Age of Solutions based on Machine Learning, AI and Language Processing Technologies

More and more vendors are now offering solutions that use AI-based machine learning and natural language processing (statistical, semantic and syntactic) technologies. Remarkable thing about "the new-AI" in these solutions is, that instead of trying to fulfill the classic AI premise of offering "the complete automation", these solutions are trying to tackle small-scale, practical and yet more immediate profit-oriented problems. Besides, it is important to note that use of natural language processing technology has gone much ahead of the usual suspects: search and retrieval! There are companies like Softface that provide solutions to syntactically and semantically tag commercial information (going down to sentence level) that provide unparallel decision support. Judging from these trends, we are bound to see more of a "convergence" between Information Management, AI and adoptive machine learning techniques and language processing technologies.

Milestones for Promoting Unstructured Information Management and Greater Enterprise-wide Information Integration

• Evangelizing the Unstructured Information Management and Adoption of Standards

Companies and vendors providing these solutions will have to play a greater evangelizing and proselytizing role in educating and empowering the customers about the immediate importance of EII and the role of unstructured data management in EII. This evangelizing will include reaching out to those who matter or decision-enablers through a completely different combination of marketing and consultancy. This Integration of consulting and marketing will help vendors achieve better reach by using lead-generation and influencing the decision makers through the Internet based promotions.

• Offering Better Professional Services and Consultancy to Gain Maximum Benefits out of Evangelizing

Evangelization and viral promotion campaigns will also lead to imparting a new role to professional services and consultancy-practices that could be built around the product centroid.

• Feeding Results of Customer Interactions and Consultancy back into Product Development

The vendors will also be required to tighten the evangelizing, consultancy and professional service feedback loop to create more customer-oriented products.

• Giving Equal Importance to Information Management as well the Underlying Technology

Lastly one the most important aspects of these solutions will be that vendors are willing to give equal importance to the "information management" and "information analysis" part of the problem - both domain expert based as well generic information analysis expertise during the product and business development, marketing and pre and post sale consultancy phases. Usually, technology organization are reluctant to understand the importance information management aspect of this loop and tend to overlook this aspect. Technology here is just an enabler but the key driver for offering an intelligent solution has to be information management and analysis.