What Makes a Metric
February 25, 2018
Folks in the Whitman community may have glossed right over the word “metric” in President Murray’s most recent email following the Board of Trustees February meeting. “Metrics” only warranted two mentions, though hashing out specifics over their implementation is sure to prove contentious.
The conversation about metrics began in earnest following the finalization of Whitman’s Strategic Priorities. The Board hopes to create metrics through which they can assess their progress towards those five initiatives. During the most most recent meetings, committee chairs brought the draft metrics to the full Board for consideration. Though those drafts are not publicly available, they will be after the meeting in May, when the metrics are expected to be complete. It also implicates Whitman’s on-going accreditation process.
Associate Dean for Academic Affairs and Associate Professor of Biology Kendra Golden elaborated on the connection between accreditation and metrics in an email to The Wire.
“Our hope is that using more formalized ways of assessing student learning will tell us something about how well our programs are actually serving our students, or not, independent of accreditation. To be able to determine whether our programs are serving students or not will be based on actual evidence if we employ metrics, which can hopefully then guide future planning and allocation of resources.”
Golden continued, “Metrics provide evidence, and th[at] evidence can be used to justify maintaining the status quo, or it can be used as justification to change things up … Accreditation is part of the reason that we are being more concrete about assessing student success, but it is not the only reason.”
Given this context and the importance of metrics to Whitman’s long term future, it is strange that metrics have not been the subject of more conversation on campus.
Metrics are hard to talk about because they are difficult to pin down. Neal Christopherson, Director of Institutional Research, called them “a measurement of something.” The breadth of this definition is indicative of how ethereal metrics can be.
Part of the difficulty in creating these metrics lies in their subjectivity.
“When you go outside you can feel on your skin what the temperature is, the difficulty is assigning a number to that,” Christopherson said. “For example, the same day in the United States and the United Kingdom would have totally different numbers attached to it [Fahrenheit and Celsius] … It’s important to remember that these metrics aren’t totally neutral purveyors of information.”
Baker Ferguson Chair of Politics and Leadership Timothy Kaufman-Osborn is similarly cautious. He defines metrics slightly differently than Christopherson: “By definition, the use of metrics involves the collection of data that can be expressed in quantifiable terms. As such, metrics are ill-equipped to assess any phenomena that cannot be captured in those terms.”
For Kaufman-Osborn, it is crucial to trace the phrase back to its corporate roots. “The use of metrics has now migrated to many other institutional realms, including higher education. As such, some have argued, the adoption of metrics within higher education is yet another manifestation of what is sometimes called the ‘corporatization’ of the academy,” Kaufman-Osborn said.
Kaufman-Osborn is not alone in his caution regarding the use of metrics to evaluate institutions of higher education. Indeed, debates about the use and abuse of metrics in colleges and universities have been raging for quite some time.
Whitman College President Kathy Murray sees things differently. “The Board’s attitude is that not everything which is important can be measured, but that we should measure those things which we can,” she said.
For Murray, it is impossible to dissociate the use of metrics from the Strategic Priorities. “What the Board is trying to measure is their own performance, and their progress towards success with our Strategic Priorities. They want to figure out how they will know if they are moving towards those goals, or even if they have been met.”
An example is the “Life After Whitman” initiative.
“We might look to see how many of our students are employed three years after they [graduate]. How many are employed nine months after they leave. Let’s take into account the ones who go to graduate school, but let’s figure out exactly what that number is,” Murray said.
Murray continued, “[The Board of Trustees has] acknowledged throughout the whole process that there are some things which cannot be measured, but they are also vehement that there needs to be a way to know if progress is being made.”
Kaufman-Osborn recognizes this utility, but is wary about going too far. “We must recognize that metrical analyses may have some limited utility, but they cannot substitute for careful judgement about the centrality of specific disciplines and areas of study to the form of education we claim to value at Whitman.”
Christopherson echoed this point: “It’s so crucial to make sure that the metric doesn’t become the ruler of everything, and that we are careful to really look hard in order to figure out what the metric is really expressing.”
Christopherson, Murray and Kaufman-Osborn all agree that metrics can be a valuable tool to measure institutional progress. What remains unclear is exactly how metrical analysis will be employed.
An example of the application of a metric is the College’s efforts to return to a 10-to-1 student-to-faculty ratio, a goal stated by Chair of the Board of Trustees Brad McMurchie and reiterated by Chair Elect Nancy Serrurier. The student-to-faculty ratio can be viewed as a metric in action, says Kaufman-Osborn.
“[The student-to-faculty ratio] can be useful in certain contexts. But … this ratio in and of itself does not answer the question of whether our current student-faculty ratio does or does not represent an institutional problem. Moreover, if this ratio is, in fact, deemed a problem, the ratio alone tells us nothing about how best to respond to it,” Kaufman-Osborn said.
The application of this specific metric has been met with widespread chagrin, as Whitman prepares to enter the 2018-2019 school year without a Professor specializing in Twentieth Century American History or European Renaissance Art History.
The application of metrics in the context of financial constraints such as under enrollment is what worries Kaufman-Osborn. “I am not suggesting that Whitman is currently employing purely metrical analyses … I am, however, expressing a concern that, especially in times of austerity, metrical analyses become more important.”
Kaufman-Osborn continued, “In arguments about such matters, the use of metrical analysis has a rhetorical advantage insofar as the results of these analyses can be expressed in quantitative terms and so can lay claim to objectivity. By way of contrast, arguments on behalf of the ‘intrinsic’ importance of certain disciplines to a liberal arts education are likely to appear comparatively mushy and hence subjective.”
To this, President Murray demurs. “The life of the mind, [is] still the essence of what we do, but I think we have to do this too,” she said, referencing the use of metrics. “I think it’s essential, I don’t think we have a choice anymore.”
Justin Lincoln • Mar 4, 2018 at 11:48 pm
One can reach further back than the corporate world to see the roots of this focus on metrics. It was used by the military before it melded with industry. Positivism might be the most relevant philosophical instantiation but it goes back many centuries. Some people believe that measuring things allows greater control, and in some cases that is true. However, in a complex system with many inputs and outputs it’s very difficult to find accurate linear cause and effect patterns ….though it’s easy to imagine them, which defeats the purpose of such measures in the first place. The Scientific Method and it’s basis in Empiricism has been valuable in many ways, but it should not be the only model used in a Liberal Arts environment. One could look to readings in Encounters ranging from Heisenberg to the Bhagavad Gita that might call the metric focus into question or complicate it. I distinctly remember the notion that how we measure a thing fundamentally can change the thing being measured, and the call to focus on right action, not the fruits of action. Thank you for your thoughtful article.