Large Scale Concept Ontology for Multimedia

The Large-Scale Concept Ontology for Multimedia (LSCOM) project was a series of workshops held from April 2004 to September 2006 [Naphade, "et al", "Large Scale Concept Ontology for Multimedia: VACE Workshop Report,"] for the purpose of defining a standard formal vocabulary for the annotation and retrieval of video.

LSCOM Mandate

Sponsored by the Disruptive Technology Office(DTO), LSCOM brought together representatives from a variety of research communities, such as multimedia learning, informationretrieval, computational linguistics, library science, and knowledge representation, as well as "user" communities such as intelligence agencies and broadcasters, to work collaboratively towards defining a set of 1,000 concepts. [" [http://nrrc.mitre.org/NRRC/LSCOM%20Overview.ppt| Naphade, "et al", "A Large Scale Concept Ontology for Multimedia Understanding," ppt presentation published by MITRE] ] . Individually, each concept was to meet the following criteria: [ " [http://www.ee.columbia.edu/~lyndon/pubs/mm2006-lscom.pdf| Naphade, "et al.", "Large-Scale Concept Ontology for Multimedia," IEEE MultiMedia, vol. 13, no. 3, pp. 86-91, July-September 2006.] ]

*Utility: the concepts must support realistic video retrieval problems
*Feasibility: the concepts are capable or will be capable of detection given the near-term (5 year projected) state of technology
*Observibility: the concepts occur with relatively high frequency in actual video data sets

Jointly, these concepts were to meet the additional criterion of providing broad (domain independent) coverage. [ " [http://www.ee.columbia.edu/~lyndon/pubs/mm2006-lscom.pdf| Naphade, "et al.", "Large-Scale Concept Ontology for Multimedia," IEEE MultiMedia, vol. 13, no. 3, pp. 86-91, July-September 2006.] ] High-level target areas for coverage included physical objects, including animate objects (such as people, mobs, and animals), and inanimate objects, ranging from large-scale (such as buildings and highways) to small-scale (such as telephones and applicances); actions and events; locations and settings; and graphics. The effort was led by researchers from Carnegie Mellon University, Columbia University, and IBM. [Naphade, "et al", "Large Scale Concept Ontology for Multimedia: VACE Workshop Report,"]

LSCOM Development Tracks

The project had two main "tracks" -- the development and deployment of keyframe annotation tools (performed by CMU and Columbia), and the development of the LSCOM concept hierarchy itself. The second track was executed in two phases: The first consisted in the manual construction of an 884 concept hierarchy, was performed collaboratively among the research and user community representatives.

The second track, performed by knowledge representation experts at Cycorp, Inc., involved the mapping of the concepts into the Cyc knowledge base and the use of the Cyc inference engine to semi-automatically refine, correct, and expand the concept hierarchy.The mapping/expansion phase of the project was motivated by a desire to increase breadth--the mapping had the effect of moving from 884 concepts to well past the initial goal of 1000--and to move LSCOM from a one-dimensional hierarchy of concepts, to a full-blown ontology of rich semantic connections. [ " [http://www.ee.columbia.edu/~lyndon/pubs/mm2006-lscom.pdf| Naphade, "et al.", "Large-Scale Concept Ontology for Multimedia," IEEE MultiMedia, vol. 13, no. 3, pp. 86-91, July-September 2006.] ]

Project Results

The outputs of the effort include: [Naphade, "et al", "Large Scale Concept Ontology for Multimedia: VACE Workshop Report,"]

#A "lite" version of the LSCOM concept hierarchy consisting of a subset of 449 concepts.
#A corpus of 61,901 video keyframes, taken from the 2006 TRECVID data set, annotated using LSCOM "lite."
#The full LSCOM taxonomy of 2,638 concepts, built semi-automatically by mapping 884 concepts, manually identified by collaborators, into the Cyc knowledge base, and querying the Cyc inference engine for useful additions.
#The full LSCOM ontology, in the form of a 2006 ResearchCyc release that contains the LSCOM mappings into the Cyc ontology.

Use of LSCOM in Larger Research Community

Since its release, LSCOM has begun to be used successfully in visual recognition research: Apart from research done by LSCOM projectparticipants, it has been used by independent research in concept extraction from images [" [http://www.google.com/url?sa=t&ct=res&cd=1&url=http%3A%2F%2Fwww.few.vu.nl%2F~guus%2Fpapers%2FSnoek07a.pdf&ei=A373RfziMaOKjAG8nKzBBg&usg=__cdCfgrpN4UHWSnybSrfKt_rI-Bk=&sig2=UNjhQ4xlxxTju9pIhuGnxA| Snoek, "et al.", "Adding Semantics to Detectors for Video Retrieval," "forthcoming" in "IEEE Transactions on Multimedia, 2007"] ] [ Worring, "et al", "The MediaMill Large-lexicon Concept Suggestion Engine," "forthcoming, in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing", Honolulu, Hawaii, USA, April 2007.] , and has served as the basis for a video annotation tool [" [http://www.eurecom.fr/util/publidownload.en.htm?file=/homesdocs/publications/htdocs/mm/arnaem-061206.pdf| Emilie Garanaud, Smeaton, A., and Koskela, M., "Evaluation of a Video Annotation Tool Based on the LSCOM Ontology," in "Proceedings of The First International Conference on Semantics And Digital Media Technology", Athens, Greece, 6-8 December 2006.] ] .

See also

* Multimedia Web Ontology Language (MOWL)

References