Vector space model

Vector space model (or "term vector model") is an algebraic model for representing text documents (and any objects, in general) as vectors of identifiers, such as, for example, index terms. It is used in information filtering, information retrieval, indexing and relevancy rankings. Its first use was in the SMART Information Retrieval System.

Definitions

A document is represented as a vector. Each dimension corresponds to a separate term. If a term occurs in the document, its value in the vector is non-zero. Several different ways of computing these values, also known as (term) weights, have been developed. One of the best known schemes is tf-idf weighting (see the example below).

The definition of "term" depends on the application. Typically terms are single words, keywords, or longer phrases. If the words are chosen to be the terms, the dimensionality of the vector is the number of words in the vocabulary (the number of distinct words occurring in the corpus).

Applications

Relevancy rankings of documents in a keyword search can be calculated, using the assumptions of document similarities theory, by comparing the deviation of angles between each document vector and the original query vector where the query is represented as same kind of vector as the documents.

In practice, it is easier to calculate the cosine of the angle between the vectors instead of the angle:

:$cos\{\; heta\}\; =\; frac\{mathbf\{v\_1\}\; cdot\; mathbf\{v\_2\{left|\; mathbf\{v\_1\}\; ight|\; left\; |\; mathbf\{v\_2\}\; ight$

A cosine value of zero means that the query and document vector were orthogonal and had no match (i.e. the query term did not exist in the document being considered).

Example: tf-idf weights

In the classic vector space model proposed by Salton, Wong and Yang the term specific weights in the document vectors are products of local and global parameters. The model is known as term frequency-inverse document frequency model. The weight vector for document "d" is $mathbf\{v\}\_d\; =\; [w\_\{1,d\},\; w\_\{2,d\},\; ldots,\; w\_\{N,d\}]\; ^T$, where

:$w\_\{t,d\}\; =\; mathrm\{tf\}\_t\; cdot\; log\{frac\{|\{t\; in\; d\}|$ is inverse document frequency (a global parameter). $|D|$ is the total number of documents in the document set; $|\{t\; in\; d\}|$ is the number of documents containing the term "t".

In a simpler Term Count Model the term specific weights do not include the global parameter. Instead the weights are just the counts of term occurrences: $w\_\{t,d\}\; =\; mathrm\{tf\}\_t$.

Limitations

The vector space model has the following limitations:

#Long documents are poorly represented because they have poor similarity values (a small scalar product and a large dimensionality)
#Search keywords must precisely match document terms; word substrings might result in a "false positive match"
#Semantic sensitivity; documents with similar context but different term vocabulary won't be associated, resulting in a "false negative match".
#The order in which the terms appear in the document is lost in the vector space representation.

Models based on and extending the vector space model

Models based on and extending the vector space model include:
* Generalized vector space model
* (enhanced) Topic-based Vector Space Model [http://kuropka.net/files/HPI_Evaluation_of_eTVSM.pdf] (eTVSM) — Extends the vector space model by removing the constraint that the term-vectors be orthogonal. In contrast to the generalized vector space model the (enhanced) Topic-based Vector Space Model does not depend on concurrence-based similarities between terms. The enhancement of the enhanced Topic-based Vector Space Model (compared to the not enhanced one) is a proposal on how to derive term-vectors from an Ontology.
* Latent semantic analysis
* DSIR model
* Term Discrimination

Further reading

* G. Salton, A. Wong, and C. S. Yang (1975), " [http://www.cs.uiuc.edu/class/fa05/cs511/Spring05/other_papers/p613-salton.pdf A Vector Space Model for Automatic Indexing] ," "Communications of the ACM", vol. 18, nr. 11, pages 613–620. "(The article in which the vector space model was first presented)"
* [http://isp.imm.dtu.dk/thor/projects/multimedia/textmining/node5.html Description of the vector space model]
* [http://kuropka.net/files/HPI_Evaluation_of_eTVSM.pdf Description and Evaluation of the enhanced Topic-based Vector Space Model]
* [http://www.miislita.com/term-vector/term-vector-3.html Description of the classic vector space model by Dr E Garcia]

ee also

*Inverted index
*Compound term processing