Words of Estimative Probability

Words of Estimative Probability (WEP or WEPs) are terms used by intelligence analysts in the production of analytic reports to convey the likelihood of a future event occurring. They express the extent of their confidence in the finding. A well-chosen WEP provides a decision maker with an unambiguous estimate upon which to base a decision. WEP usage is not standard across the US Intelligence Community (IC). Some intelligence and policy failures appear to be related to an imprecise use of estimative words.

A well-chosen WEP gives a decision maker a clear and unambiguous estimate upon which to base a decision. Ineffective WEPs are vague or misleading about the likelihood of an event. An ineffective WEP places the decision maker in the role of the analyst. The decision maker has to infer the prediction alone, thus increasing the likelihood of poor decision making or snap decision making.

=History=

Intelligence

In 1964 Sherman Kent, one of the first contributors to a formal discipline of intelligence analysis, addressed the problem of misleading expressions of odds in National Intelligence Estimates (NIE). In "Words of Estimative Probability" Kent distinguished between ‘poets’ (those preferring wordy probabilistic statements) from ‘mathematicians’ (those preferring quantitative odds). To bridge the gap between them and decision makers, Kent developed a paradigm relating estimative terms to odds. His goal was to "… set forth the community's findings in such a way as to make clear to the reader what is certain knowledge and what is reasoned judgment, and within this large realm of judgment what varying degrees of certitude lie behind each key judgment."citation
url =https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/docs/v01n5p.htm
title = Sherman Kent and the Profession of Intelligence Analysis
publisher = Center for the Study of Intelligence, Central Intelligence Agency
date = November 2002
page =50 ] Kent’s initiative was not adopted although the idea was well received and remains compelling today.

However, the NIC’s own discussion of this paradigm seems to undercut its chances of being effective:

Intelligence judgments pertaining to likelihood are intended to reflect the Community’s sense of the probability of a development or event. […] We do not intend the term “unlikely” to imply an event will not happen. We use “probably” and “likely” to indicate there is a greater than even chance. We use words such as “we cannot dismiss,” “we cannot rule out,” and “we cannot discount” to reflect an unlikely—or even remote—event whose consequences are such it warrants mentioning. Words such as “may be” and “suggest” are used to reflect situations in which we are unable to assess the likelihood generally because relevant information is nonexistent, sketchy, or fragmented. [ National Intelligence Council, 2007, p5 ]

This explanation is ‘a half-step forward, half-step backward.’ An agency-sponsored WEP paradigm "is" progress. However--an estimative statement that uses “maybe” , “suggest” , or other weasel words is vague and symptomatic of the problem at hand--not its solution. In 1964 Kent railed against the “restort to expressions of avoidance…which convey a definite meaning but at the same time either absolves us completely of the responsibility or makes the estimate enough removed … as not to implicate ourselves.”citation
url =https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/docs/v01n5p.htm
title = Sherman Kent and the Profession of Intelligence Analysis
publisher = Center for the Study of Intelligence, Central Intelligence Agency
date = November 2002
page =63 ]

Mercyhurst experience with standardized WEPs

The Mercyhurst College Institute for Intelligence Studies conducted several experiments investigating [http://sourcesandmethods.blogspot.com/2008/03/what-do-words-of-estimative-probability.html the IC’s interpretation of WEPs] (results variedCitation
last = Wheaton
first = K.
title = What Do Words Of Estimative Probability Mean? An Exercise In Analyst Education
date = 24 March 2008
year = 2008
url = http://sourcesandmethods.blogspot.com/2008/03/what-do-words-of-estimative-probability.html
accessdate = 2008-04-24] ) and citation
last = Kesselman | first = Rachel
contribution = Estimative Words of Probability Trends in National Intelligence Estimates
title = International Studies Association 49th Annual Convention: Bridging Multiple Divides
date= 26 March 2008
url = http://www.allacademic.com/meta/p251711_index.html] their use of WEPs in NIEs over the past three decades] to determine the significant changes in the ways the NIC has been articulating its intelligence judgments over time. See Analysis of Competing Hypotheses

Mercyhurst’s WEP paradigm reduces Kent’s schema to its least ambiguous words:
Analytic Confidence and Source Reliability are expressed on a 1 to 10 scale, with 10 high.

Weasel Words

Table 4 contains a non-exhaustive list of common terms that are especially vague, known pejoratively as Weasel Words. Their use in estimative statements is almost certain to cause confusion; they should be avoided at all costs.

References

Further reading

cite web
last = Blair
first = Bruce
title = The Logic of Intelligence Hype and Blindness
publisher = [http://www.cdi.org Center for Defense Intelligence]
date = 2003
url = http://www.cdi.org/blair/logic-updated.pdf
format =PDF
accessdate = 2008-04-21|format=PDF

Citation
last = Chido
first = Diane E.
last2 = Wheaton
first2 = Kristan J.
last3=Seward
first3=Richard M.
last4=Altman
first4= Kristina M.
last5=Kelly
first5=James J.
author5=James J. Kelly
title = Structured Analysis of Competing Hypotheses: Theory And Application
place = Erie, PA
publisher = Mercyhurst College Institute of Intelligence Studies Press
year = 2006
url = http://www.mciis.org/
doi =
id =
isbn =0-9773881-0-7 .

cite web
authorlink = Department Of Defense
title = Joint Intelligence 2.0
publisher = Department of Defense
date = 2007
url = http://www.fas.org/irp/doddir/dod/jp2_0.pdf
format =PDF
accessdate = 2008-04-21|format=PDF

Citation
first = Richards J. | last = Heuer
title = The Psychology of Intelligence Analysis
place = Langley, VA
publisher = [https://www.cia.gov/library/center-for-the-study-of-intelligence/index.html CIA Center for the Study of Intelligence]
year = 1999
isbn = 1 929667-00-0

cite web
last = Ward
first = David L.
authorlink = David L. Ward
title = Definition of Some Estimative Expressions
publisher = https://www.cia.gov/library/center-for-the-study-of-intelligence/index.html CIA Center For The Study Of Intelligence
date = 1964
url =https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/sherman-kent-and-the-board-of-national-estimates-collected-essays/6words.html
accessdate = 2008-04-23