Are there any nice examples of Libraries/Archives using Natural Language Processing to generate descriptive metadata from full text content?

Question

I would be curious to see examples of NLP techniques (in particular, Named entity recognition, Relationship extraction and Topic modeling) applied to generate descriptive metadata and any thoughts on what techniques and tools are producing useful results. I'm up for hearing about experimental work, but I would be particularly interested to hear about any projects making good use of NLR to generate descriptive metadata from full text content at scale.

Answer 1

I am really just starting to get familiar with this field, but could this paper be of use?

LIGHTWEIGHT PARSING OF NATURAL LANGUAGE METADATA Aliaksandr Autayeu, Fausto Giunchiglia, Pierre Andrews, Ju Qi

http://eprints.biblio.unitn.it/1619/1/028.pdf

Also: in Natural Language Processing for Digital Libraries (NLP4DL) Workshop, Viareggio, Italy, June 15th 2009.

From the Conclusions:

In this paper we have first shown that a standard part of speech (POS) tagger could be accurately trained on the specific language of the metadata. A large scale analysis of the use of POS tags showed that the metadata language is structured in a very limited set of patterns that can be used to develop accurate lightweight Backus-Naur form grammars. We intend to use parsers based on these grammars to allow deeper understanding of metadata semantics, important for such tasks as translating classifications into lightweight ontologies for use in semantic matching. In the future, we plan to simplify the grammars and try to unify them. We plan to analyse other metadata datasets and check the scalability of our approach and to investigate the possibility to automate the creation of grammar production rules.

They used OpenNLP, of which I suppose you are aware

http://en.wikipedia.org/wiki/OpenNLP (The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text. It supports the most common NLP tasks, such as tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, chunking, parsing, and coreference resolution).

Hope this helps - I used to work there so probably I am biased, but it seemed interesting.