Jack Walton

What’s the difference between making things and finding things?

In some contexts, we are presented with situations that involve selecting the best possible things from existing options—it could be picking a course to study, players for a team, or a restaurant for dinner. In these situations, we are informed by our impressions of the quality or skill of the selectees.

In other situations, the emphasis is on making something from closer-to-scratch—sitting down to write one of these posts is a good example.

Decision-making seems to be different in both contexts (or perhaps, relative to both contextual emphases). When the issue is to do with selection, part of the challenge is in being able to perceive the right pre-existing characteristics (an exercise in evaluative judgment). In design scenarios, success seems more about being able to suspend judgment to enable something useful to emerge, almost as a function of probability in combination with latent situational characteristics (e.g. person, environment, state of mind, amount of coffee—things that include but are not limited to psychological characteristics).

The above is a little too dichotomous for my taste. If my experiences trying to write papers or produce music are any indication, a seesaw is a better metaphor for the interplay between design and selection. As soon as we produce something, the evaluative gears start turning. Being able to exercise the right state at the right time seems a practice worth figuring out.

What is the difference between undertaking evaluation of finished thing (say, a sculpture, or an essay, or the yield of a crop) as distinct from making a forecast based on some kind of data (e.g. a weather prediction)?

Segmentalism is certainly having a moment, and while I do agree that breaking things into little parts to better understand some larger artefact (say, a body) makes general scientific sense, I’m not convinced that this logic is directly transferable to the evaluation of semiotically complex phenomena (e.g. the quality of an argument).

A couple of days ago a posted a few recommendations for assessment, with a view to being practical. This list, retrieved from a now defunct blog, is on the other end of the spectrum—it is perhaps less pragmatic, but I think the ideas are quite important.

This is a list of propositions (not facts) on the topic of what makes for a robust educational assessment practice—it was originally produced to help structure a discussion about assessment in university music degrees. It is to some extent a very direct summary of principles boiled down from my exposure to Sadler’s work. To enact these principles properly is very hard in the current higher education environment.

A key point is that the traditional requirement for the use of grades (beyond an end-of-degree pass/fail) has a very strong flow-on effect on what comprises a basis for legitimate assessment.

1. If grades cannot be done away with, they must have integrity to satisfy ethical and legislative requirements

2. For grades to have integrity, standards must be robust within the context

3. To make that work there must be group consensus about those standards

4. For those standards to be useful over time, the group consensus must be maintained

5. For that to be possible, there must be an available bank of exemplars, consulted by the relevant team on a regular basis

6. Assessment of this kind does not apply to learning, it is an appraisal of quality of work completed, performed, and/or submitted

7. Grades are only valid for the work assessed, in that time and place, in which they were awarded—aggregating grades creates a spurious representation of cumulative learning and achievement

I spend a lot of time critiquing the way educational assessment works. In the interest of being pragmatic, I feel like I should try to suggest some things that educators can do without having to really change the system.

First, a principle worth clarifying: Formal assessment at school or university usually involves assigning students’ work a value—it could be a simple pass-fail, a broad score (e.g. 1 – 5), or quite large, as in the case of awarding points out of a total of 100. Here are some things I think should always feature in an assessment process, from the assessor’s side:

1. Always (always!) explain the reasons for which the grade was not higher than it is—in an educational context, if this cannot be explained, the grade is not justified (the same is not necessarily true in other contexts).
2. Spot-moderation is insufficient; for consistency of standards between markers through time, sharing of perspectives on the quality of students’ work should be as constant as possible within an assessment team.
3. The stakes are highest for students on the borderline between passing and failing; so additional emphasis is often warranted on point 1 above.
4. Let students know in advance how assessment processes actually work from the assessors’ side—this helps them to build empathy with assessors (handy if you have 100+ students and a team of assessors), and also lets them know what they can reasonably expect to receive in terms of feedback. Even better, involve them in a mock assessment process.
5. If use of a rubric is mandated, start with a global appraisal of the work, produce an explanation of the grade (as in point 1), and only then proceed backwards to consider the criteria.

I’m used to understanding the concept of atrophy in a biological context, where it has to do with something wasting away—usually a muscle, in its most popular usage. The Oxford English Dictionary calls it the “wasting away of the body, or any part of it, through imperfect nourishment”.

This week I heard someone use the term creative atrophy to describe a decline in creativity over time.

I appreciate this idea, and am tempted to offer two provocations in response.

First, this view seems to approach creativity from a primarily psychological standpoint. There are other ways to understand creativity that play a little further outside the psychological realm, and which have less to do with it being a trait and more to do with it being a practice (and therefore, somewhat mundane to everyday life).

Second, in the biological context the counterpart to atrophy is hypertrophy, which has to do with growth. Surely this is worth emphasising as well? If we are designing creative interventions, it might serve well as a less scarcity-oriented metaphor.

Interestingly, the OED offers a few figurative interpretations of atrophy from centuries gone by. I like this one, from Thomas Carlyle’s On Heroes (1841): “For the Scepticism..is..a chronic atrophy and disease of the whole soul” (v. 282).