Analogical modeling

__NOTOC__Analogical modeling (hereafter AM) is a formal theory of exemplar-based analogical reasoning, proposed by Royal Skousen, professor of Linguistics and English language at Brigham Young University in Provo, Utah. It is applicable to language modeling and other categorization tasks. Analogical modeling is related to connectionism and nearest neighbor approaches, in that it is data-based rather than abstraction-based; but it is distinguished by its ability to cope with imperfect datasets (such as caused by simulated short term memory limits) and to base predictions on all relevant segments of the dataset, whether near or far. In language modeling, AM has successfully predicted empirically valid forms for which no theoretical explanation was known (see the discussion of Finnish morphology in Skousen et al. 2002).

Implementation of the model

Overview

An exemplar-based model consists of a general-purpose modeling engine and a problem-specific dataset. Within the dataset, each exemplar (a case to be reasoned from, or an informative past experience) appears as a feature vector: a row of values for parameters that describe the problem. For example, in a spelling-to-sound task, the feature vector might consist of the letters of a word. Each exemplar in the dataset is stored with an outcome, such as a phoneme or phone to be generated. When the model is presented with a novel situation (in the form of an outcome-less feature vector), the engine algorithmically sorts the dataset to find exemplars that helpfully resemble it, and selects one, whose outcome is the model's prediction. The particulars of this algorithm distinguish one exemplar-based modeling system from another.

In AM, we think of the feature values as characterizing a context, and the outcome as a behavior that occurs within that context. Accordingly, the novel situation is known as the "given context." Given the known features of the context, the AM engine systematically generates all contexts that include it (all of its "supracontexts"), and extracts from the dataset the exemplars that belong to each. The engine then discards those supracontexts whose outcomes are inconsistent (this measure of consistency will be discussed further below), leaving an "analogical set" of supracontexts, and probabilistically selects an exemplar from the analogical set with a bias toward those in large supracontexts. This multilevel search exponentially magnifies the likelihood of a behavior's being predicted as it occurs reliably in settings that specifically resemble the given context.

Analogical modeling in detail

AM performs the same process for each case it is asked to evaluate. The given context, consisting of n variables, is used as a template to generate two-to-the-n supracontexts. Each supracontext is a set of exemplars in which one or more variables have the same values that they do in the given context, and the other variables are ignored. In effect, each is a view of the data, created by filtering for some criteria of similarity to the given context, and the total set of supracontexts exhausts all such views. Alternatively, each supracontext is a theory of the task or a proposed rule whose predictive power needs to be evaluated.

It is important to note that the supracontexts are not equal peers one with another; they are arranged by their distance from the given context, forming a hierarchy. If a supracontext specifies all of the variables that another one does and more, it is a subcontext of that other one, and it lies closer to the given context. (The hierarchy is not strictly branching; each supracontext can itself be a subcontext of several others, and can have several subcontexts.) This hierarchy becomes significant in the next step of the algorithm.

The engine now chooses the analogical set from among the supracontexts. A supracontext may contain exemplars that only exhibit one behavior; it is deterministically homogeneous and is included. It is a view of the data that displays regularity, or a relevant theory that has never yet been disproven. A supracontext may exhibit several behaviors, but contain no exemplars that occur in any more specific supracontext (that is, in any of its subcontexts); in this case it is non-deterministically homogeneous and is included. Here there is no great evidence that a systematic behavior occurs, but also no counterargument. Finally, a supracontext may be heterogeneous, meaning that it exhibits behaviors that are found in a subcontext (closer to the given context), and also behaviors that are not. Where the ambiguous behavior of the nondeterministically homogeneous supracontext was accepted, this is rejected because the intervening subcontext demonstrates that there is a better theory to be found. The heterogeneous supracontext is therefore excluded. This guarantees that we see an increase in meaningfully consistent behavior in the analogical set as we approach the given context.

With the analogical set chosen, each appearance of an exemplar (for a given exemplar may appear in several of the analogical supracontexts) is given a pointer to every other appearance of an exemplar within its supracontexts. One of these pointers is then selected at random and followed, and the exemplar to which it points provides the outcome. This gives each supracontext an importance proportional to the square of its size, and makes each exemplar likely to be selected in direct proportion to the sum of the sizes of all analogically consistent supracontexts in which it appears. Then, of course, the probability of predicting a particular outcome is proportional to the summed probabilities of all the exemplars that support it.

(Skousen 2002, in Skousen et al. 2002, pp. 11-25, and Skousen 2003, both passim)

Formulas

Given a context with $n$ elements::total number of pairings: $n^2$:number of agreements for outcome "i": $n\_i^2$:number of disagreements for outcome "i": $n\_i(n\; -\; n\_i)$:total number of agreements: $sum\{n\_i^2\}$:total number of disagreements: $sum\{n\_i(n\; -\; n\_i)\}\; =\; n^2\; -\; sum\{n\_i^2\}$

Example

This terminology is best understood through an example. In the example used in the second chapter of Skousen (1989), each context consists of three variables with potential values 0-3

:Variable 1: 0,1,2,3:Variable 2: 0,1,2,3:Variable 3: 0,1,2,3

Historical Context

Analogy has been considered useful in describing language at least since the time of Saussure. Noam Chomsky and others have more recently criticized analogy as too vague to really be useful (Bańko 1991), an appeal to a "deus ex machina." Skousen's proposal appears to address that criticism by proposing an explicit mechanism for analogy, which can be tested for psychological validity.

Applications

Analogical modeling has been employed in experiments ranging from phonology and morphology (linguistics) to orthography and syntax.

Problems

Though analogical modeling aims to create a model free from rules seen as contrived by linguists, in its current form it still requires researchers to select which variables to take into consideration. This is necessary because of the so-called "exponential explosion" of processing power requirements of the computer software used to implement analogical modeling. Recent research suggests that quantum computing could provide the solution to such performance bottlenecks (Skousen et al. 2002, see pp 45-47).

References

*cite book|title=Analogical Modeling of Language|author=Royal Skousen|year=1989|location=Dordrecht|publisher=Kluwer Academic Publishers|pages=xii+212pp|id=ISBN 0-7923-0517-5|format=hardcover
*cite journal | author=Miroslaw Bańko | title=Review: Analogical Modeling of Language | journal=Computational Linguistics | month=June | year=1991 | volume=17 | issue=2 | pages=246–248 | url=http://acl.ldc.upenn.edu/J/J91/J91-2010.pdf
*cite book|title=Analogy and Structure |author=Royal Skousen |year=1992 |location=Dordrect| publisher=Kluwer Academic Publishers
*cite book|title=Analogical Modeling: An exemplar-based approach to language|editor=Royal Skousen, Deryle Lonsdale, Dilworth B. Parkinson|location=Amsterdam/Philadelphia|publisher=John Benjamins Publishing Company|year=2002|format=Human Cognitive Processing vol. 10|id=ISBN 1-58811-302-7|pages=x+417pp
* Skousen, Royal. (2003). [http://humanities.byu.edu/am/bls_paper3.pdf "Analogical Modeling: Exemplars, Rules, and Quantum Mechanics."] . Presented at the Berkeley Linguistics Society conference.

ee also

*Computational Linguistics
*Connectionism

External links

* [http://humanities.byu.edu/am Analogical Modeling Research Group Homepage]
* [http://linguistlist.org/issues/14/14-110.html LINGUIST List Announcement] of "Analogical Modeling", Skousen et al. (2002)