The Complicated Relationship of Numbers, Data and Values in Policy Analysis: Guest Blogger Marigold Setsuko Holmes

For some time now, I have pondered the question “what does ‘fact’ really mean?” and “are data really neutral?” I have been somewhat skeptical of the notion that data are impartial and therefore the grounds by which to make policy decisions. As a doctoral student and aspiring scholar, I tend to make sense of the world through an interpretivist lens and the lived experiences of those associated with the issues that I study. The nature of these studies lend themselves to considering values – the values of the people who share their stories with me, as well as the values that I hold as the researcher who interprets these stories and presents them as data. Yet, as a professional engaged in international education, I have navigated the world through a functionalist lens, and am well aware of the power of data, which are often portrayed as an inpartial indicator of performance, effectiveness, and worth. I have sat through countless trainings about data-driven inquiry and decision-making. I cannot count the number of times I have been told ‘we need facts!’ or ‘The anecdotal evidence is nice, but what do the numbers say?’ ‘The story of the student overcoming challenges is interesting, but who hired her, what is her current title, and how much is she making?’ Thus, our reports become a colorful montage of charts, graphs, numbers and ‘facts’ that somehow carry the air of neutrality, fairness, and truth. The rich stories somehow fail to achieve validity in the eyes of decision-making.

Facts are what win government appropriations for exchange programs. Numbers are what increase departmental allocations at universities. As many universities in the U.S. turn to performance-based funding models, academic programs scramble to increase graduation rates, vying for limited resources. Never mind that a department provides a more holistic educational experience through smaller teacher to student ratios and strong experiential learning opportunities. If they do not show the numbers, they risk losing the resources to continue providing these experiences. However, this decision to use numbers as an indicator of success, is in and of itself based on values. Performance, as measured by graduation and retention rates, are valued over experience. I have spent countless hours poring over data, graphs and charts, to set forth evidence of effectiveness. These facts, carefully crafted to provide the narrative we wish to tell, are also value laden. We decide which figures to include and how to analyze them so that data is presented in our favor. The choice of what counts as evidence or how we display that evidence is based on the value judgements that we make – for example, when we decide which data to disclose as most compelling or important.

To this end, the Policy Analysis course with Karen Seashore has enabled me to understand that data is, in fact, tied to values and so too are policy decisions.  Moreover, I appreciate the affirmation that it is acceptable, in fact critical, that we consider values in policy analysis. This might be a best-kept secret, especially for those who ascribe to the notion that policy is developed on the premise of neutrality and therefore must be universally applied to maintain its integrity and promote equity.

As we put together a policy analysis (or embark on our dissertation research), I believe that it is extremely important to understand the undeniable role that values play in our endeavors. To borrow the words of Heck (2004) “…the collection of data is not neutral. Data collection represents political decisions about what types of information are useful” (as cited in Alexander, 2013, p. 720). It is also important to understand that numbers do not ensure equity. As Alexander points out “numbers by themselves do not necessarily tell you about power relationships and why a problem exists; they do not necessarily reveal hidden biases or differential interpretation and implementation of policy (p. 67). I do not think that this means we must stop making data-driven decisions. I still believe in the practical application of data. However, what I now know from taking this course is that values matter. In other words, understanding the complex relationship between fact, data and values will, to borrow the words of Alexander “allow [us] to be transparent in the decision process and in the evaluative criteria that [we] establish” (p. 80). Moreover, by acknowledging our values, we will be able to bring integrity to our policy analyses and by extension, I hope that we can help decision makers establish policies that are inclusive and mindful of the basic tenant of education ‘to do no harm’.

References:

Alexander, N. A. (2013). Policy analysis for educational leaders: A step-by-step approach. Boston, MA: Pearson Education, Inc.

Categories Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this:
search previous next tag category expand menu location phone mail time cart zoom edit close