I’ve fallen behind on my usual schedule of posting and replying to comments. The reason is, of course, my adventures in the realm of assessment. This began in 2004 when I was assigned to eternal membership on the General Education Assessment Committee. I am now a co-chair of the committee. I was also assigned to do the unit assessment for philosophy and religion. The basic idea of assessment is to assess using various direct and indirect measures. As might be imagined, this has little to nothing to do with philosophy, although I do try to sneak in the occasional philosophical bits. These are, as you might guess, typically edited out when the documents are reviewed.
My university is in the process of re-accreditation, something all schools do on a regular basis. My task is to complete a major document for a specific standard–the document is currently at 11, 184 words.
I have some upcoming essays that I hope to complete tomorrow and perhaps the assessment grind will permit me to get back to my usual writing a reply cycle. And, you know, teaching and stuff.
But, here is a look at what sort of stuff I write for assessment with some philosophy. Philosophy that will be excised in the final version, of course.
Overview of Target Levels and Measure of Success
The establishment of target levels and measuring competence requires addressing two basic concerns. One is determining what counts as competence in each assessed area. The second is setting a percentage goal for student competence.
The second is easy to address. In the United States educational system (broadly construed), 70% has been established as the minimal level of adequacy. As such, adopting the broad standard that 70% of the students assessed will perform at a level of adequate competency or better is justified by this established measure. Justification for this measure, in general, can be sought in whatever theoretical, practical and philosophical foundations were used to make this the national standard. The first is rather more challenging to address.
Justifying a standard of competence is difficult because of an epistemic problem raised by the ancient Greek Skeptics. If a standard is not self-justifying, it must be justified. If the justification is not self-justifying, it must be justified. Philosophically, this must lead to either a regress (infinite or circular) or a self-justifying foundation. As there seem to be no self-justifying foundations for standards, the regress problem wins the day and all standards are ultimately arbitrary. Fortunately, there is a pragmatic solution to this problem: presenting a plausible narrative for the standards that convinces the relevant authorities to accept them. This is what follows.
To measure the competence of an individual student in an assessment area, there must be an established standard of what counts as competent. To use the obvious analogy, to measure the height of a person, there must be an established and consistent means of measuring. One way to define competence in education is in terms of how the average student performs in that area. This is analogous to sorting out what is “normal” height—it is based on what is average in the relevant population. As such, assessing the competence of Florida A&M University students required knowing the national average for comparable students in the relevant competency areas. To this end, the ETS Proficiency Profile (EPP) was utilized to set the standard—specifically the national mean. This standard is used in the areas the EPP tests: Communication, Critical Thinking, and Quantitative Reasoning. Since this method is accepted by the relevant authorities in assessment, it is justified.
While the use of standardized tests solves some of the assessment problems, it does not solve all of them. Specifically, it does not solve the problem of assessing areas that are not well-covered by standardized tests (such as Social/Ethical Responsibility) and it does not solve the problem of assessing individual artifacts, such as philosophy papers. Fortunately, there is an established solution to this problem, namely the use of rubrics. The main challenge with a rubric is developing it so that it properly and consistently sorts students into the specified levels of competence. While all rubrics are flawed in some manner, Florida A&M University began in 2004 with established rubrics from other universities and refined them over the years in accord with both national and local findings to ensure that best practices were being used. Since these rubrics are accepted by the experts in the field of assessment, they are justified as means of assessment.
Other methods of assessment, such as focus groups and surveys, are also established as accepted methods by the relevant experts in the field of assessment. These methods are, of course, crafted and deployed in accord with the best-practices as established by the relevant experts in the field. Thus, these methods are also justified.