Overfitting (machine learning)
- Overfitting (machine learning)
:"For the statistical concept see Overfitting
The concept of overfitting is important in machine learning. Usually a learning algorithm is trained using some set of training examples, i.e. exemplary situations for which the desired output is known. The learner is assumed to reach a state where it will also be able to predict the correct output for other examples, thus generalizing to situations not presented during training (based on its inductive bias). However, especially in cases where learning was performed too long or where training examples are rare, the learner may adjust to very specific random features of the training data, that have no causal relation to the target function. In this process of overfitting, the performance on the training examples still increases while the performance on unseen data becomes worse.
In order to avoid overfitting, it is necessary to use additional techniques (e.g. cross-validation, regularization, early stopping, that can indicate when further training is not resulting in better generalization. The process of overfitting of neural network during the training is also known as overtraining. In treatment learning, overfitting is avoided by using a minimum best support value.
Literature
* Tetko, I.V.; Livingstone, D.J.; Luik, A.I. Neural network studies. 1. Comparison of Overfitting and Overtraining, [http://www.vcclab.org/articles/tetko.html#overtraining J. Chem. Inf. Comput. Sci., 1995, 35, 826-833]
External links
* http://www.cs.sunysb.edu/~skiena/jaialai/excerpts/node16.html
* [http://www.vcclab.org/articles/tetko.html#overtraining Overtraining]
Wikimedia Foundation.
2010.
Look at other dictionaries:
Overfitting — Noisy (roughly linear) data is fitted to both linear and polynomial functions. Although the polynomial function passes through each data point, and the linear function through few, the linear version is a better fit. If the regression curves were … Wikipedia
Overfitting — blau: Fehler bzgl. Trainingsdatensätzen rot: Fehler bzgl. Testdatensätzen Wenn der Fehler bzgl. der Testdatensätze steigt, während der Fehler bzgl. der Trainingsdatensätze stetig fällt, dann befindet man sich möglicherweise in einer… … Deutsch Wikipedia
Supervised learning — is a machine learning technique for learning a function from training data. The training data consist of pairs of input objects (typically vectors), and desired outputs. The output of the functioncan be a continuous value (called regression), or… … Wikipedia
Decision tree learning — This article is about decision trees in machine learning. For the use of the term in decision analysis, see Decision tree. Decision tree learning, used in statistics, data mining and machine learning, uses a decision tree as a predictive model… … Wikipedia
Neural Lab — Original author(s) Sergio Ledesma … Wikipedia
Granular computing — is an emerging computing paradigm of information processing. It concerns the processing of complex information entities called information granules, which arise in the process of data abstraction and derivation of knowledge from information.… … Wikipedia
Regularization (mathematics) — For other uses in related fields, see Regularization (disambiguation). In mathematics and statistics, particularly in the fields of machine learning and inverse problems, regularization involves introducing additional information in order to… … Wikipedia
Cluster analysis — The result of a cluster analysis shown as the coloring of the squares into three clusters. Cluster analysis or clustering is the task of assigning a set of objects into groups (called clusters) so that the objects in the same cluster are more… … Wikipedia
Natural language processing — (NLP) is a field of computer science and linguistics concerned with the interactions between computers and human (natural) languages; it began as a branch of artificial intelligence.[1] In theory, natural language processing is a very attractive… … Wikipedia
Data mining — Not to be confused with analytics, information extraction, or data analysis. Data mining (the analysis step of the knowledge discovery in databases process,[1] or KDD), a relatively young and interdisciplinary field of computer science[2][3] is… … Wikipedia
