Collegiate Learning Assessment

The Collegiate Learning Assessment (CLA) is a standardized testing initiative in United States higher educational evaluation and assessment. It uses a "value-added" outcome model to examine a college or university's contribution to student learning which relies on the institution, rather than the individual student, as the primary unit of analysis. The CLA measures are designed to test for critical thinking, analytic reasoning, problem solving, and written communication skills. The assessment consists of open-ended questions, is administered to students online, and controls for incoming academic ability. An institution's average score on the CLA measures correlates highly with the institution's average SAT score (r = 0.90)^[1]. Institutional results are not published.

History and Test Format

The CLA was first launched in 2000 by the Council for Aid to Education (CAE), a national nonprofit organization based in New York City. Rather than testing for specific content knowledge gained in particular courses or majors, the intent was to assess “the collective and cumulative result of what takes place or does not take place over the four to six years of undergraduate education in and out of the classroom.^[2] Of the entire test, the most well-developed and sophisticated part is its performance task component^[3], in which students are given ninety minutes to respond to a writing prompt that is associated with a set of background documents. The testing materials, including the documents, are accessed through a computer. The CAE has published several examples of representative performance tasks, one of which is described below (from Academically Adrift: Limited Learning on College Campuses):

The “DynaTech” performance task asks students to generate a memo advising an employer about the desirability of purchasing a type of airplane that has recently crashed. Students are informed: “You are the assistant to Pat Williams, the president of DynaTech, a company that makes precision electronic instruments and navigational equipment. Sally Evans, a member of DynaTech’s sales force, recommended that DynaTech buy a small private plane (a SwiftAir 235) that she and other members of the sales force could use to visit customers. Pat was about to approve the purchase when there was an accident involving a SwiftAir 235.” Students are provided with the following set of documents for this activity: newspaper articles about the accident, a federal accident report on in-flight breakups in single engine planes, Pat Williams’s e-mail to her assistant and Sally Evans’s e-mail to Pat Williams, charts on SwiftAir’s performance characteristics, an article from Amateur Pilot magazine comparing SwiftAir 235 to similar planes, and pictures and descriptions of SwiftAir models 180 and 235. Students are then instructed to “prepare a memo that addresses several questions, including what data support or refute the claim that the type of wing on the SwiftAir 235 leads to more in-flight breakups, what other factors might have contributed to the accident and should be taken into account, and your overall recommendation about whether or not DynaTech should purchase the plane."^[4]

The CAE also publishes its scoring rubric.^[5] The design of both the prompts and the criteria for evaluation demonstrates the CLA's focus on complex, holistic, real-world problem-solving as a measurement of high level learning. As a result, the argument goes, institutions that attempt to “teach to the test” will be schools that teach students “to think critically, reason analytically, solve problems, and communicate clearly."^[6]

Criticisms

Again according to Academically Adrift, there are four primary criticisms of the CLA:

1. The first is an aversion to standardized testing in general and a belief that there is accountability in higher education due to market forces. That is, if students were not gaining something through education then they would not be paying for it. In this case, the CLA is a waste of time and money.
2. By focusing on general skills rather than concrete knowledge and specialization, some critics argue that there is real learning taking place that the CLA does not pick up.
3. Others have questioned the instrumental validity of the test, although some of these concerns have now been addressed by a 2009 test validity study organized by the Fund for the Improvement of Postsecondary Education (FIPSE). The resulsts showed that while these tests should not be used as a basis to make institutional decisions about students as individuals (e.g., promotion or course placement), when aggregated in larger samples they can provide reliable estimates of institutional or group-level differences in performance on these tasks.^[7][8]
4. The fourth criticism is not targeted against the test itself but rather against the idea that you can measure the value-added from higher education, when so much of what students gain from college is outside of academics.

See also

Voluntary System of Assessment--an initiative developed by American Association of State Colleges and Universities (AASCU) and the National Association of State Universities and Land-Grant Colleges (NASULGC) for 4-year public colleges and universities. The VSA endorses the use of the CLA for reporting out student learning outcomes through the College Portrait.

References

1. ^ Benjamin, Robert; Chun, Marc (2003). "A New Field of Dreams: The Collegiate Learning Assessment Project". Peer Review 5 4: 26-29. http://www.aacu.org/peerreview/pr-su03/pr-su03feature2.cfm. Retrieved 11/19/11. 
2. ^ Hersch, Richard (2007). "Going Naked". Peer Review 9. 
3. ^ Arum, Richard; Roksa, Josipa (2011). Academically Adrift: Limited Learning on College Campuses. 536: University of Chicago Press. 
4. ^ Arum, Richard; Roksa, Josipa (2011). Academically Adrift: Limited Learning on College Campuses. University of Chicago Press. 
5. ^ "Collegiate Learning Assessment Common Scoring Rubric". Council for Aid to Education. New York: Council for Aid to Education, 2008. http://www.cae.org/content/pdf/CLA_Scoring_Criteria_(Jan%202008).pdf. Retrieved 11/19/11. 
6. ^ Shavelson, Richard (2008). "The Collegiate Learning Assessment". Ford Policy Forum 2008: Forum for the Future of Higher Education. 
7. ^ Arum, Richard; Roksa, Josipa (2011). Academically Adrift: Limited Learning on College Campuses. University of Chicago Press. 
8. ^ Klein, Stephen; Ou Lydia Liu, and James Sconing (September 29). Test Validity Study (TVS) Report. http://www.voluntarysystem.org/docs/reports/TVSReport_Final.pdf. 

External links

• Collegiate Learning Assessment