Letter frequencies

Letter frequencies

The frequency of letters in text has often been studied for use in cryptography, and frequency analysis in particular. No exact letter frequency distribution underlies a given language, since all writers write slightly differently. Linotype machines sorted the letters' frequencies as etaoin shrdlu cmfwyp vbgkqj xz based on the experience and custom of manual compositors. Likewise, Modern International Morse code encodes the most frequent letters with the shortest symbols; arranging the Morse alphabet into groups of letters that require equal amounts of time to transmit, and then sorting these groups in increasing order, yields e it san hurdm wgvlfbk opjxcz yq. Similar ideas are used in modern data-compression techniques such as Huffman coding.

More recent analyses show that letter frequencies, like word frequencies, tend to vary, both by writer and by subject. One cannot write an essay about x-rays without using frequent Xs, and different authors have habits which can be reflected in their use of letters.

Everyone writes differently – for example, Hemingway's writing style is visibly different from Faulkner's. Letter, bigram, trigram, word frequencies, word length, and sentence length can be calculated for specific authors, and used to prove or disprove authorship of texts, even for authors whose styles aren't so divergent.

Accurate average letter frequencies can only be gleaned by analyzing a large amount of representative text. With the availability of modern computing and collections of large text corpora, such calculations are easily made. The Deafandblind link details examples from a variety of sources, (press reporting, religious text, scientific text and general fiction] and there are differences especially for general fiction with the position of 'h' and 'i'. The example differs from the linotype 'etaoin shrdlu' to come out as 'etaoHn Isrdlu'. There is an unproven statement that conversation is similar in frequency to general fiction.

Cryptographic sources suggest for code-breaking purposes a sequence of 'etaonrish'.

The 'top twelve' letters comprise about 80% of the total usage.

Relative frequencies of letters in the English language

Letter frequencies: [ [http://pages.central.edu/emp/LintonT/classes/spring01/cryptography/letterfreq.html English letter frequencies ] ]

Relative frequencies of letters in other languages

-*See Turkish dotted and dotless I

Based on these tables, the 'etaoin'-equivalent results for each language is as follows :-

French - esait nrulo'; (Indo-European: Romanic)

Spanish 'eaosr niltu'; (Indo-European: Romanic)

Italian 'aeion lrtsu'; (Indo-European: Romanic)

Esperanto 'aieon lsrtu' (Synthetic language - influenced by Indo-European Languages, romanic, germanic mostly)

German 'enisr atulo'; (Indo-European: Germanic)

Swedish 'eantr sildo'; (Indo-European: Germanic)

and Turkish 'aeinr ldkmu'. (Turik -- is a non Indo-European Language)

All these languages using a basically similar 25+ character alphabet.

ee also

* Corpus linguistics
* ETAOIN SHRDLU
* RSTLNE
* Frequency analysis (cryptanalysis)
* Linotype machine
* Most common words in English
* Scrabble

References

External links

* [http://pages.central.edu/emp/LintonT/ A site with content of "Cryptographical Mathematics"] by Robert Edward Lewand
* [http://www.bckelk.ukfsn.org/words/etaoin.html Some examples of letter frequency rankings in some common languages]
* [http://www.imn.htwk-leipzig.de/~dborkman/letter.html Java-Application for building letter frequencies out of a text file]
* [http://deafandblind.com/word_frequency.htm A detailed listing of letter frequencies]


Wikimedia Foundation. 2010.

Игры ⚽ Поможем написать реферат

Look at other dictionaries:

  • Letter frequency — The frequency of letters in text has often been studied for use in cryptography, and frequency analysis in particular. No exact letter frequency distribution underlies a given language, since all writers write slightly differently. Linotype… …   Wikipedia

  • Letter beacon — Signal of letter beacon D on 5137.5 kHz Letter beacons are radio transmissions of uncertain origin, which consist of only a single repeating Morse Code letter. They are also often referred to as: SLB, or Single Letter Beacons SLHFB, or… …   Wikipedia

  • Television channel frequencies — The following tables show the frequencies assigned to broadcast television channels in various regions of the world, along with the ITU letter designator for the system used. The frequencies shown are for the video and audio carriers. The channel …   Wikipedia

  • List of comic book letter column titles — The typical comic book letter column until its general disappearance in the early 2000s had its own title, usually a reference to the book’s hero or heroes. Cape and Cowl Comments ( World s Finest Comics ), JLA Mailroom ( Justice League of… …   Wikipedia

  • Index of coincidence — In cryptography, coincidence counting is the technique (invented by William F. Friedman[1]) of putting two texts side by side and counting the number of times that identical letters appear in the same position in both texts. This count, either as …   Wikipedia

  • Vigenère cipher — The Vigenère cipher is a method of encrypting alphabetic text by using a series of different Caesar ciphers based on the letters of a keyword. It is a simple form of polyalphabetic substitution.The Vigenère (pronEng|ˌviːdʒɪˈnɛəɹ, veedj ih nair )… …   Wikipedia

  • cryptology — cryptologist, n. cryptologic /krip tl oj ik/, cryptological, adj. /krip tol euh jee/, n. 1. cryptography. 2. the science and study of cryptanalysis and cryptography. [1635 45; < NL cryptologia. See CRYPTO , LOGY] * * * Introduction …   Universalium

  • information theory — the mathematical theory concerned with the content, transmission, storage, and retrieval of information, usually in the form of messages or data, and esp. by means of computers. [1945 50] * * * ▪ mathematics Introduction       a mathematical… …   Universalium

  • Frequency analysis — In cryptanalysis, frequency analysis is the study of the frequency of letters or groups of letters in a ciphertext. The method is used as an aid to breaking classical ciphers.this is wrong is based on the fact that, in any given stretch of… …   Wikipedia

  • Dvorak Simplified Keyboard — The modern Dvorak Simplified Keyboard (US layout) Typing a text excerpt at 115 WPM with the Canadian French Dvorak keyboard layout. The Dvorak Simplified Keyboard is a k …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”