Lexical Markup Framework

Lexical Markup Framework (LMF) is a work in progress within International Organization for Standardization ISO/TC37 in order to define a common standardized framework for the construction of natural language processing (NLP) and machine-readable dictionary (MRD) lexicons. Project scope is standardization of principles and methods relating to language resources in the contexts of multilingual communication and cultural diversity.

Objectives of LMF

The goals of LMF are to provide a common model for the creation and use of lexical resources, to manage the exchange of data between and among these resources, and to enable the merging of large number of individual electronic resources to form extensive global electronic resources.

Types of individual instantiations of LMF can include monolingual, bilingual or multilingual lexical resources. The same specifications are to be used for both small and large lexicons, for both simple and complex lexicons, for both written and spoken lexical representations. The descriptions range from morphology, syntax, computational semantics to computer-assisted translation. The covered languages are not restricted to European languages but cover all natural languages. The range of targeted NLP applications is not restricted. LMF is able to represent most lexicons, including WordNet, EDR and PAROLE lexicons.

History of LMF

In the past, lexicon standardization has been studied and developed by a series of projects like GENELEX, EDR, EAGLES, MULTEXT, PAROLE, SIMPLE and ISLE. Then, the ISO/TC37 National delegations decided to address standards dedicated to NLP and lexicon representation.The work on LMF started in Summer 2003 by a new work item proposal issued by the US delegation. In Fall 2003, the French delegation issued a technical proposition for a data model dedicated to NLP lexicons. Early 2004, the ISO/TC37 committee decided to form a common ISO project with Nicoletta Calzolari (Italy) as convenior and Gil Francopoulo (France) and Monte George (US) as editors.Since this date, 13 versions has been written, dispatched (to the National nominated experts), commented and discussed during various ISO technical meetings.

Current stage

The ISO number is 24613. The LMF specification is currently (March 2008) in FDIS stage (i.e. Final Draft for International Standard). The schedule is to reach a final publication in September 2008.

LMF as one of the members of the ISO/TC37 family of standards

The ISO/TC37 standards are currently elaborated as high level specifications and deal with word segmentation (ISO 24614), annotations (ISO 24611 aka MAF, ISO 24612 aka LAF, ISO 24615 aka SynAF, and ISO 24617-1 aka SemAF/Time), feature structures (ISO 24610), multimedia containers (ISO 24616 aka MLIF), and lexicons (ISO 24613).These standards are based on low level specifications dedicated to constants, namely data categories (revision of ISO 12620), language codes (ISO 639), scripts codes (ISO 15924), country codes (ISO 3166) and Unicode (ISO 10646).

The two level organization forms a coherent family of standards with the following common and simple rules:
* the high level specification provides structural elements that are adorned by the standardized constants;
* the low level specifications provide standardized constants as metadata.

Key standards used by LMF

The linguistics constants like /feminine/ or /transitive/ are not defined within LMF but are recorded in the Data Category Registry (DCR) that is maintained as a global resource by ISO/TC37 in compliance with ISO/IEC 11179-3:2003 [http://hal.inria.fr/docs/00/12/14/74/PDF/LREC2006WS-RI-20AprilBis.pdf] . And these constants are used to adorn the high level structural elements.

The LMF specification complies with the modeling principles of Unified Modeling Language (UML) as defined by Object Management Group (OMG). The structure is specified by means of UML class diagrams. The examples are presented by means of UML instance (or object) diagrams.

An XML DTD is given in an annex of the LMF document.

Model structure

LMF is composed of the following components:
* The core package which is the structural skeleton which describes the basic hierarchy of information in a lexical entry.
* Extensions of the core package which are expressed in a framework that describes the reuse of the core components in conjunction with the additional components required for a specific lexical resource.

The extensions are specifically dedicated to morphology, MRD, NLP syntax, NLP semantics, NLP multilingual notations, NLP morphological patterns, multiword expression patterns, and constraint expression patterns.

A tiny example

In the following example, the lexical entry is associated with a lemma "clergyman" and two inflected forms "clergyman" and "clergymen". The language coding is set for the whole lexical resource. The language value is set for the whole lexicon as shown in the following UML instance diagram.

The elements "Lexical Resource", "Global Information", "Lexicon", "Lexical Entry", "Lemma", and "Word Form" define the structure of the lexicon. They are specified within the LMF document.On the contrary, "languageCoding", "language", "partOfSpeech", "commonNoun", "writtenForm", "grammaticalNumber", "singular", "plural" are data categories that are taken from the Data Category Registry. These marks adorn the structure. The values "ISO 639-3", "clergyman", "clergymen" are plain character strings. The value "eng" is taken from the list of languages as defined by ISO 639-3.

With some additional information like "dtdVersion" and "feat", the same data can be expressed by the following XML fragment:

This example is rather simple, while LMF can represent much more complex linguistic descriptions the XML tagging is correspondingly complex.

External links

Related web sites

* [http://www.lexicalmarkupframework.org/ LMF web site]
* [http://lirics.loria.fr/ LIRICS web site]

ome recent scientific communications about LMF

* Gesellschaft für linguistische Datenverarbeitung GLDV-2007/Tubingen: Lexical Markup Framework ISO standard for semantic information in NLP lexicons [http://hal.inria.fr/docs/00/12/14/83/PDF/Coling2006LMFForMultilingualResources19Mai.pdf]
* Language Resources and Evaluation LREC-2006/Genoa: Lexical Markup Framework (LMF) [http://hal.inria.fr/docs/00/12/14/68/PDF/LMFPaperForLREC2006FinalSubmission31March06.pdf]

ome related scientific communications

* Language Resources and Evaluation LREC-2006/Genoa: The relevance of standards for research infrastructures [http://hal.inria.fr/docs/00/12/14/74/PDF/LREC2006WS-RI-20AprilBis.pdf]

ee also

* Computational lexicology
* Lexical semantics
* Morphology (linguistics) for explanations concerning paradigms and morphosyntax
* Machine translation for a presentation of the different types of multilingual notations (see section "Approches")
* Morphological pattern for the difference between a paradigm and a paradigm pattern
* WordNet for a presentation of the most famous semantic lexicon for the English language
* UTX for a user-oriented, alternative format for machine-readable dictionary