PRINT AS PDF
On Tuesday, June 7, 2017, The Wall Street Journal, drawing upon Collegiate Learning Assessment+ (CLA+) exam results, published a report suggesting that “many colleges” did not improve their students’ critical thinking skills. “At more than half of schools, at least a third of seniors were unable to make a cohesive argument, assess the quality of evidence in a document or interpret data in a table,” the newspaper declared. The article then went further, revealing, “At some of the most prestigious flagship universities, test results indicate the average graduate shows little or no improvement in critical thinking over four years.” In explaining the credibility of the CLA+ outcomes concerning stunted critical thinking development, The Journal cited a PayScale Inc. survey that found 50% of employers believed that the graduates they hired were not workplace-ready. Inadequate critical reasoning skills were the top explanation for this lack of preparedness.
The CLA+ exam, which is administered to freshmen and seniors, is comprised of two sections. The first section involves a performance task. Students are given a real-world scenario, assigned a role e.g., an employee within a company, and given a task to be completed. This section requires students to answer an open question based on information provided in a document library comprised of 4-9 documents that may include technical reports, memos, e-mails, articles, tables, or graphs. The second section involves response questions. In that section, students are expected to analyze data, tables, graphs, among other information, and then answer multiple choice questions using that information. A total of 25 multiple choice questions examine students’ scientific and quantitative reasoning, critical reading and evaluation, and their ability to critique arguments.
The article provided a link to results from 68 institutions, including seven CUNY senior colleges. All institutions listed in the table had at least 75 freshmen and seniors who took the exam. Lehman College was not listed, as 60 seniors took the CLA+ exam. For academic and other policy reasons, CUNY discontinued the use of the CLA+ after the 2013-14 academic year.
The average scores for the seven CUNY institutions were 1,107 for seniors, a +28-point change in the average score between freshmen and seniors, and a mean value added score of -0.71 (“Near Expected”). The respective numbers for Lehman College were 1,054, +25-point change, and -0.51 (“Near Expected”).
Those above numbers, and other CLA+ outcomes, should be taken with a degree of caution. First, the exam is a single assessment tool. Multiple measures should be utilized to corroborate findings. Second, student participation is voluntary. That nature of the exam could create sampling issues. Third, freshmen-senior comparisons are not drawn from cohorts that include the same students. Use of different students could lead to measurement issues. Fourth, variability in the outcomes on a year-to-year basis, possibly in part due to the second and third points, raises some questions about reliability.
Those issues notwithstanding, the CLA+ can provide a degree of insight. At least some of the issues identified in the CLA+ should, in theory, show up in additional assessments of student learning. To look beyond the CLA+, I examined how my own BBA 407 Strategic Management students performed on a critical reasoning task. I reviewed data for the three most recent semesters (93 students).
In my class, the rubric used to grade the semester research project involves three critical thinking elements. The rubric’s scale is as follows: 1-not evident; 2-emerging; 3-proficient; and, 4-excelling. During the three most recent semesters, 45% of students fell below borderline proficient (less than 2.5) and 55% were borderline proficient or better (2.5 or higher). 21% had proficient (3.0) or higher skills in this area. Overall, the anecdotal evidence from my own class was broadly consistent with the findings documented by The Wall Street Journal and those of Lehman College’s seniors when the CLA+ was last administered in 2014.
Critical thinking skills remain in high demand in the workforce. According to a recent report published by American Student Assistance (ASA), problem-solving was the second most in-demand skill based on a survey of employers, with only the ability to work in a team ranked slightly higher. Therefore, even if institutions choose not to administer the CLA+, the use of multiple assessment tools aimed at measuring student progress in critical thinking remains crucial to evaluating an institution’s overall effectiveness, especially as it relates to preparing students for their lives beyond college.