Test and learn

Test and Learn is a set of practices followed by retailers and consumer-focused companies to test ideas in a small number of locations or customers to predict impact. The process is often designed to answer three questions about any tested program before rollout:

# What impact will the program have on key performance indicators if executed across the network or customer base?
# Will the program have a larger impact on some stores/customers than others?
# Which components of the idea are actually working?

Historical context

One of the pioneers of Test and Learn is Capital One. Founded in 1988, Capital One tested highly customized financial products, tailoring offers by customer. Capital One founding members have spoken of Test and Learn as a competitive advantage, crediting the company’s rapid success to frequent, rigorous, and systematic application of test and learn across their customer base.

Criticism

Companies wanting to conduct Test and Learn must mobilize, track and analyze site and customer data at very granular levels. This can often be a massive and expensive undertaking. Increasingly however, exponential increase in computing power, and decrease in its cost, have made testing more accessible.

Examples

* Retail Banks running different commercials in different markets.
* Fast Food restaurants launching a new menu item or product for a limited time in select locations
* Big-box retailers evaluating “store of the future” programs
* Companies evaluating pricing strategies
* Retailers testing merchandising layouts across their door base

See also

* Clinical trials
* Scientific method
* Design of experiments
* Evidence-based management
* Business process reengineering

External links

* [http://www.predictivetechnologies.com/docs/solutions_case.cfm Case Studies]
* [http://norvig.com/experiment-design.html Experimental Design]