K-Praxis - Topic: Content Analysis - Public Information: A Need for Automated Content Analysis-Print

Public Information: A Need for Automated Content Analysis
August 29, 2003

If you are following the Lord Hutton Inquiry into the death of the British weapons scientist Dr David Kelly or even have heard about it and you are somebody who does not really care about British politics, but somebody who understands information management technologies you might want to stop and take a note of the amount of information that has been submitted by the Hutton Inquiry on the Internet. You also may want to take a note of the various forms of information being made available in digital format: emails, private notes, reports and "aide memoires" - something one never thought would come out in publicly accessible format. In this article K-Praxis points out that both governments as well information management vendors could help in making massive quantities of public information more accessible using automated and intelligent content analysis technologies that aid information discovery.

Recent Examples of Publicly Available Information

The Hutton Inquiry (referred as Inquiry hereafter) is unique because it constitutes a significant mile-stone - at least in the developed world - in terms of public information availability. These examples given below show the range of information that has been made available to public in the past month of so:

The report submitted by the Joint Inquiry into Intelligence Community Activities Before and After the Terrorist Attacks of September 11, 2001 (9/11 Report)(1000+ pages);
Lord Hutton Inquiry (10000+ pages);
And yesterday 2000+ pages of transcripts released by the Port Authorities of New York and New Jersey depicting the last conversations in the twin towers ( Excerpts from CNN ).

Content Analysis: Need for Automation?

How do we make sense out of more and more information that is available? And remember public information is crucial to people - it affects peoples' lives, or for some others, their jobs and duties revolve around interpreting this public information.

In the light of this analysis, it seems appropriate that we should have to have more technological tools for analyzing publicly available information. Use technologies that can create relationship maps and information visualization (an example of a similar public interest relationship map was described in this pervious K-Praxis article:Collaborative Information Visualization?! ), discover concepts and themes, analyze the language of these documents. Basically it is possible to use technologies that can pin-point useful bits of information and summarize this vast amount of information either for public consumption or for people who need to derive actionable intelligence from this information - and for that matter even government agencies will require these tools to reach out to public.

Think of a scenario how effectively a Hutton Inquiry website would be if it were to use relationship maps and information visualization graphs of the information submitted to give us an objective overview and analysis?

Conclusion

Commercially there are hundreds of vendors providing intelligent and automated information management technologies to analyze information - and notably, these technologies form the bed-rock of technology analysis sites like K-Praxis.

But there is a need to make these technologies available for publicly available information. The mute point will be who will bear the cost - here the efforts are required from both the sides: the vendors could provide their technologies as free demo for public information purposes and the governments on the other hand, need to invest in such technologies to make information accessible to public.

So in the future the responsibility of public/civic bodies will not only stop at making the information available, but it will be equally imperative that they make it accessible in easily digestible format.

Addendum:

In the article we were talking about who should come forward to help the governments? And here is an example, Gary Price of ResourceShelf suggested a link to an article from UPI (Software can detect text inconsistencies) that talks about the Glasgow Accident Analysis Group (Glasgow University).This group of researchers - engaged in accident analysis and in incident reporting techniques in order "to improve our understanding of system `failures' and human `error'" - has tried to analyze the Hutton Inquiry documents by using argument mapping techniques.

	Making sense of online textual information and information management technologies

	Public Information: A Need for Automated Content Analysis August 29, 2003 If you are following the Lord Hutton Inquiry into the death of the British weapons scientist Dr David Kelly or even have heard about it and you are somebody who does not really care about British politics, but somebody who understands information management technologies you might want to stop and take a note of the amount of information that has been submitted by the Hutton Inquiry on the Internet. You also may want to take a note of the various forms of information being made available in digital format: emails, private notes, reports and "aide memoires" - something one never thought would come out in publicly accessible format. In this article K-Praxis points out that both governments as well information management vendors could help in making massive quantities of public information more accessible using automated and intelligent content analysis technologies that aid information discovery. Recent Examples of Publicly Available Information The Hutton Inquiry (referred as Inquiry hereafter) is unique because it constitutes a significant mile-stone - at least in the developed world - in terms of public information availability. These examples given below show the range of information that has been made available to public in the past month of so: The report submitted by the Joint Inquiry into Intelligence Community Activities Before and After the Terrorist Attacks of September 11, 2001 (9/11 Report)(1000+ pages); Lord Hutton Inquiry (10000+ pages); And yesterday 2000+ pages of transcripts released by the Port Authorities of New York and New Jersey depicting the last conversations in the twin towers ( Excerpts from CNN ). Content Analysis: Need for Automation? How do we make sense out of more and more information that is available? And remember public information is crucial to people - it affects peoples' lives, or for some others, their jobs and duties revolve around interpreting this public information. In the light of this analysis, it seems appropriate that we should have to have more technological tools for analyzing publicly available information. Use technologies that can create relationship maps and information visualization (an example of a similar public interest relationship map was described in this pervious K-Praxis article:Collaborative Information Visualization?! ), discover concepts and themes, analyze the language of these documents. Basically it is possible to use technologies that can pin-point useful bits of information and summarize this vast amount of information either for public consumption or for people who need to derive actionable intelligence from this information - and for that matter even government agencies will require these tools to reach out to public. Think of a scenario how effectively a Hutton Inquiry website would be if it were to use relationship maps and information visualization graphs of the information submitted to give us an objective overview and analysis? Conclusion Commercially there are hundreds of vendors providing intelligent and automated information management technologies to analyze information - and notably, these technologies form the bed-rock of technology analysis sites like K-Praxis. But there is a need to make these technologies available for publicly available information. The mute point will be who will bear the cost - here the efforts are required from both the sides: the vendors could provide their technologies as free demo for public information purposes and the governments on the other hand, need to invest in such technologies to make information accessible to public. So in the future the responsibility of public/civic bodies will not only stop at making the information available, but it will be equally imperative that they make it accessible in easily digestible format. Addendum: In the article we were talking about who should come forward to help the governments? And here is an example, Gary Price of ResourceShelf suggested a link to an article from UPI (Software can detect text inconsistencies) that talks about the Glasgow Accident Analysis Group (Glasgow University).This group of researchers - engaged in accident analysis and in incident reporting techniques in order "to improve our understanding of system `failures' and human `error'" - has tried to analyze the Hutton Inquiry documents by using argument mapping techniques.