Archives for May 2023

What is the difference between undertaking evaluation of finished thing (say, a sculpture, or an essay, or the yield of a crop) as distinct from making a forecast based on some kind of data (e.g. a weather prediction)?

Segmentalism is certainly having a moment, and while I do agree that breaking things into little parts to better understand some larger artefact (say, a body) makes general scientific sense, I’m not convinced that this logic is directly transferable to the evaluation of semiotically complex phenomena (e.g. the quality of an argument).

A couple of days ago a posted a few recommendations for assessment, with a view to being practical. This list, retrieved from a now defunct blog, is on the other end of the spectrum—it is perhaps less pragmatic, but I think the ideas are quite important.

This is a list of propositions (not facts) on the topic of what makes for a robust educational assessment practice—it was originally produced to help structure a discussion about assessment in university music degrees. It is to some extent a very direct summary of principles boiled down from my exposure to Sadler’s work. To enact these principles properly is very hard in the current higher education environment.

A key point is that the traditional requirement for the use of grades (beyond an end-of-degree pass/fail) has a very strong flow-on effect on what comprises a basis for legitimate assessment.

1. If grades cannot be done away with, they must have integrity to satisfy ethical and legislative requirements

2. For grades to have integrity, standards must be robust within the context

3. To make that work there must be group consensus about those standards

4. For those standards to be useful over time, the group consensus must be maintained

5. For that to be possible, there must be an available bank of exemplars, consulted by the relevant team on a regular basis

6. Assessment of this kind does not apply to learning, it is an appraisal of quality of work completed, performed, and/or submitted

7. Grades are only valid for the work assessed, in that time and place, in which they were awarded—aggregating grades creates a spurious representation of cumulative learning and achievement

I spend a lot of time critiquing the way educational assessment works. In the interest of being pragmatic, I feel like I should try to suggest some things that educators can do without having to really change the system.

First, a principle worth clarifying: Formal assessment at school or university usually involves assigning students’ work a value—it could be a simple pass-fail, a broad score (e.g. 1 – 5), or quite large, as in the case of awarding points out of a total of 100. Here are some things I think should always feature in an assessment process, from the assessor’s side:

1. Always (always!) explain the reasons for which the grade was not higher than it is—in an educational context, if this cannot be explained, the grade is not justified (the same is not necessarily true in other contexts).
2. Spot-moderation is insufficient; for consistency of standards between markers through time, sharing of perspectives on the quality of students’ work should be as constant as possible within an assessment team.
3. The stakes are highest for students on the borderline between passing and failing; so additional emphasis is often warranted on point 1 above.
4. Let students know in advance how assessment processes actually work from the assessors’ side—this helps them to build empathy with assessors (handy if you have 100+ students and a team of assessors), and also lets them know what they can reasonably expect to receive in terms of feedback. Even better, involve them in a mock assessment process.
5. If use of a rubric is mandated, start with a global appraisal of the work, produce an explanation of the grade (as in point 1), and only then proceed backwards to consider the criteria.

I’m used to understanding the concept of atrophy in a biological context, where it has to do with something wasting away—usually a muscle, in its most popular usage. The Oxford English Dictionary calls it the “wasting away of the body, or any part of it, through imperfect nourishment”.

This week I heard someone use the term creative atrophy to describe a decline in creativity over time.

I appreciate this idea, and am tempted to offer two provocations in response.

First, this view seems to approach creativity from a primarily psychological standpoint. There are other ways to understand creativity that play a little further outside the psychological realm, and which have less to do with it being a trait and more to do with it being a practice (and therefore, somewhat mundane to everyday life).

Second, in the biological context the counterpart to atrophy is hypertrophy, which has to do with growth. Surely this is worth emphasising as well? If we are designing creative interventions, it might serve well as a less scarcity-oriented metaphor.

Interestingly, the OED offers a few figurative interpretations of atrophy from centuries gone by. I like this one, from Thomas Carlyle’s On Heroes (1841): “For the Scepticism..is..a chronic atrophy and disease of the whole soul” (v. 282).

Fairytales are an excellent example of transferable principles. Here’s three, very brief reimaginings of The Little Mermaid:

1. There was once a naughty young mermaid who ignored the conventional advice regarding witches. She made a deal with the local witch and got what she deserved.

2. There was a prince who lived by the sea. Sometimes, on the calm nights while voyaging at sea, he spied the glinting of scales in the ebb of the placid ocean. The sailors were superstitious, and believed the glinting to be merfolk spying on the from within the waves. One night, having had a little too much to drink, he dove into the ocean in search of the mermaids, and was never heard from again.

3. There was once a witch who lived deep under the sea. She had once been married to the king of the merfolk, and they had a daughter. The merfolk were afraid of the witch, and so she was banished into exile. One day, her daughter came in disguise, seeking the help of the witch in escaping the ocean to live on land. The witch did not recognise her daughter, and in a desire to spite the mermaid, placed a curse upon her.

What if the little mermaid was a tattoo? A constellation? The name of a seedy bar?