Reverse Engineering Assessment Criteria; or, That’s what you meant by critically analyse!

Last month, I attempted to better understand my students by reverse engineering their essays. It was a surprisingly popular piece. During a feedback session, one of my students, quite out of the blue, informed me that I was ‘absolutely right about conclusions’ and that they had always had difficulty understanding their true purpose.

Although a number of students have, in the past, commented upon my blog, either online or in person, this was something new. The student had not mentioned they had been reading my blog. They simply turned to me, in the midst of our conversation, and referred back to it, as if it were a continuation of a previous encounter. The power of new media.

But onto the matter at hand. Having reverse engineered their essays, fair play demanded that I reverse engineer my assessment criteria as well.

First, let us look at the criteria in context. The following are descriptions of first or A quality work from institutions with which I have worked.

  • Work of a distinguished quality, which is based on very extensive reading and which demonstrates an authoritative grasp of the concepts, methodology and content appropriate to the subject and the assessment task. There is clear evidence of originality and insight and an ability to sustain a sophisticated argument, to think analytically and to critically and perceptively synthesise material effectively.
  • Persuasive and direct answer to the question, establishing the wider significance of the issues concerned. Comprehensive coverage of the relevant material; accuracy in the details. A direct and coherent argument, well supported by relevant evidence. Critical analysis of relevant concepts, theoretical or historiographical perspectives or methodological issues. Fluent and engaging writing style; persuasive presentation and structuring of arguments.

The main themes that appear in both are

  • Breadth of Reading
  • Understanding of Historical Concepts
  • Understanding of Historical Methodologies
  • Understanding of Relevant Historical Content
  • Clear and Structured Argument
  • Evidence of Analysis and Synthesis

Only the later mentions clarity of expression, but this should be considered requisite for any piece of formal writing, not merely those of exceptionally high quality. With these six points in mind, how do we go about explaining them?

Breadth of Reading

There is no magical formula for the number of historiographical or primary sources you should consult for an essay. Extensive reading is demonstrated not only by the number of items listed in the footnotes or bibliography, but by the persuasiveness of your argument in conjunction with those references. If a point can be sufficiently made with three references, then there is no need to offer six. However, if a point is not conclusive after three authors (because there are significant gaps in chronology or geography), then a fourth or fifth may be needed.

I offer my students the following formulas as a good starting point:

  • For every 1000-words in an essay, you should have 2-3 substantive sub-arguments that support your overall thesis.
  • For every sub-argument in your essay, you should have at least 2 sources that provide historiographical arguments (with which you agree or disagree, or that you partially accept) and 3 additional sources that provide direct evidence to help you evaluate these arguments.

Thus a 2000-word essay should contain roughly 15-25 sources, allowing for some duplication between sub-arguments. I have only ever had a handful of first-year essays actually achieve this, but it is a good goalpost to aim for.

Clear and Structured Argument

One of the most difficult points of the assessment criteria to explain, and indeed mark, is structure. To many, its a matter of socialisation–they know a good structure when they see it. But this is simply not true. Rhetoric is an art and a science, something that has been lovingly crafted for centuries. Under the advice of my colleague Matthew McCormack (@historymatt) I returned to the 18th century for inspiration and texts on rhetoric and composition. I, too, had been the victim of the 5-paragraph mantra, and learning the meaning and evolution of essay structure was a liberating experience for me.

Yet, while I encourage all my readers to explore their rhetorical roots, for the time being, it shall suffice to address the three main parts of a short undergraduate essay: the introduction, the conclusion, and the body.

Introduction

The introduction is a much maligned component of the essay. Repository of all extraneous material (often labeled as background or context), it rarely serves its true purpose, to introduce the essay. At its core, an introduction should provide three things:

  1. An introduction to the historiography you are engaging with. For example, you can explain a particular historiographical debate you are weighing-in on, or give the vital statistics (date, location and individuals) of a lesser known event that you will be centering your essay upon.
  2. Any contextual information your reader NEEDS to understand the specific arguments in your body. This does not mean ‘all your evidence’, but rather those points that are not common knowledge, but which are required to understand all or most of the sub-arguments you intend to make. For example, if you are speaking about a conflict between two parties, you may wish to clarify exactly who, when and where you will be discussing.
  3. A clear, concise and definite thesis statement (your argument).

That’s it. Longer essays can play with this formula, adding and omitting to suit style and taste, but for a short 1500-2500 essay, these three things will more than suffice.

As a matter of preference, I would also prohibit beginning an introduction with the phrase ‘The Oxford English Dictionary defines”–but to each his own.

Conclusion

The conclusion, too often a repository for the thesis and a hasty summary of the evidence presented, is meant to provide your reader with a sense of clarity, completion, and perhaps curiosity.

Clarity is achieved by settling the questions that were raised by your introduction. Although your body should flow logically from point to point, it is helpful to your reader to draw together the loose ends and remind them what their overall judgement should be.

Rather than restate the thesis, however, your conclusion should provide a more nuanced and qualified version of it, one that is only possible to understand, or believe, because of the proof and sub-arguments presented in your body. For example, whereas your thesis might have been

  • The Salem Witch Trials were primarily caused by pre-existing social tensions among the local inhabitants, and later exacerbated by mass hysteria.

Your concluding statement might be

  • Longstanding rivalries between the Putnams and other Salemite families fostered resentment and a sense of persecution, which was amplified by personal tragedies outwith their control.

In the latter, you are moving beyond a simple repetition of either your thesis or your evidence. Instead, you are adding nuance and specificity to your original argument and clarifying how your evidence proves your thesis to be correct.

Completion is achieved in much the same way. While your first section may have discussed pre-existing conditions in Salem, and a later section may have detailed the events immediately preceding the accusations, your conclusion ties these together with your thesis to form a complete picture.

Finally, curiosity can be created by exploring what might come next. If your essay has satisfactorily explained the Salem Hysteria, might your theories also be applied to other cases? Which would be most appropriate to examine next?

Body Structure

The most common form of body structure remains the three- or five-paragraph model, wherein the author develops three to five individual points that help support the overall thesis. There is nothing wrong with this model; indeed many of my colleagues swear by it to this day. But I offer a slightly revised version.

Your body paragraphs should each provide a separate point or argument, but they should also build upon each other, as much as your argument allows. What you say in your first paragraph should ideally provide evidence not only for its own main point, but provide evidence and background for the second paragraph as well. For example, your first paragraph might explore the most practical and basic causes for discontent in colonial New England, while your second might explain both the ideological origins of the revolution and how these were connected to the practical reasons listed previously. In this way, your paragraphs are well linked and unnecessary repetition is avoided.

Understanding of Historical Concepts

A historical concept is a seemingly straightforward addition to any undergraduate paper. However, when I typed the phrase into relevant search engines, I found a number of students querying Yahoo!Answers and other similar services. More worryingly, the answers to these queries were eclectic to say the least. In fairness, when I teach essay construction, I never use the term, preferring ‘historiographical argument’, which I feel gives more agency to the individual authors and hopefully prevents phrases such as ‘everyone agrees that…’

Before beginning your essay, you should have a firm understanding of the most prevalent ‘understandings’ or ‘conceptions’ of the historical period, event or individual you are discussing. You should also be able to distinguish between them based on chronology (when they were first argued, or when they were most popular), on geography (British v. American v. African perspectives), and on the background of the authors or evidence (sociological, economic, literary, cultural, political). If you can sit down to your keyboard understanding how Reconstruction, or the Scottish Enlightenment, has been conceived at different times and by different scholars, you are ready to begin your paper.

When it comes to conveying that information, however, a simple list will never suffice. Sometimes you have spent so long trying to understand something that you feel a great desire to show that knowledge off. But before you include something in a short essay, you must ask yourself, does this help my reader understand my argument? If not, think twice.

In truth, most short essay questions will lend themselves to an extensive discussion of historiography. For example, you cannot answer ‘What caused the American Civil War’ without acknowledging the fundamental debate between Slavery and States’ Rights–each of which has its own motley crew of supporters that you can parade and examine in turn. The point is to include this information when it helps support or clarify your argument, not as a machine-gun attack on your reader’s recall memory. We are looking for understanding, not recollection.

Understanding of Historical Methodologies

Unlike historical concepts, historical methodologies is deceptively straightforward. Over your university career you will be introduced to a variety of methodologies–statistics, interviews, literary analysis–and schools–Marxism, The Annales School, Ethnohistory, Gender Studies, Regional Studies, Intellectual History. What is difficult is using these in a meaningful way.

Too often, students carelessly, often erroneously, use the phrase ‘Marxist historian’ or ‘Revisionist historian’ and thus achieve no greater effect than simply using the author’s first name. Marxist historian Feeley argues this. Revisionist historian Reid disagrees. Others acknowledge that both numbers and words exist, but do not explain how these each affect the conclusions of the researcher.

As you sift through your evidence and arguments, look for themes and contradictions. Does all your qualitative data lead to one understanding, while your statistical information leads to another? What does this tell you? Or, do two authors use roughly equivalent evidence sets, but arrive at wildly different interpretations? How can you explain this?

Again, understanding, not recollection.

Understanding of Relevant Historical Content

It may seem strange, but this is ingredient most student historians neglect to include. So worried are they about including methodologies and concepts that the basic details of who, what, when and where are completely excluded. One student this year helpfully suggested that ‘Well, you know all that already, so I didn’t think I needed to write it out.’

You do.

Your tutor probably does know most of the historical details you will utilize in your short essay (though we always appreciate being pleasantly surprised), but novelty is not what are looking for. What we are assessing is your ability to look at 100 facts and chose the 10 most relevant and persuasive.

By choosing to focus and develop particular events, individuals or processes, you are demonstrating that you have understood their relative weight or importance to the period your are writing about. If you neglect them, your tutor is likely to question the range or depth of reading you undertook, or your understanding of who and what were most important.

On the other hand, by omitting details that are irrelevant (or tangential), you are demonstrating that you understand the question fully and know which pieces of information will best answer it.

Your choice of evidence and detail also demonstrates that you understand agency, causation, and correlation as well as applicability to specific groups and regions. One of the most common comments given on ‘historical content’ is ‘generalisation’ or applying an argument indiscriminately across a long period or equally to all individuals.

So, be specific, be discriminating, and be careful.

Evidence of Analysis and Synthesis

The final nail in the coffin of a poorly written essay is a lack of critical analysis. This is usually indicated with words such as ‘narrative’ or ‘descriptive’, but might also be characterized as an ‘unsupported’ or ‘uncritical assertion’, the latter being VERY common in first-year efforts.

So what does it mean to analyse critically? There is no single answer, but a key characteristic is that the author understands the difference between an impression, a personal opinion, an assertion, and an argument.

  • An impression is the first idea that comes to mind when coming across the evidence or material. It is often fragmentary, relying on a general perception of word choice and content. It is often described as ‘shallow’ or ‘surface’ and usually only manifests in essays that are researched or written the night before submission.
  • A personal opinion is an argument that is based solely on impressions and personal experience. It is how it seems to me. These are usually quite tentative, or defensive, and prefaced by phrases such as ‘I believe, ‘I think’ or ‘I feel’. While having a personal opinion is nothing to be ashamed of, putting it forth in an essay is. It suggests that you are not sure your conclusion is correct, and therefore do not want to be criticized if you are wrong.
  • An assertion is, in some ways, the opposite of a personal opinion. It is a statement that something is true, absolutely and without qualification, but without any evidence to support or prove it. This is usually not the result of arrogance, but rather a believe that there is a correct answer, and you have found it. The problem is not that you are incorrect, because you are not being assessed solely or even largely on your correctness. You are being assessed on your ability to make and support a reasoned argument. An assertion, however correct, is never put forth critically and can therefore never fulfill this requirement.
  • An argument is a statement that something is true (or at least more likely to be true than any other argument put forth thus far) that is supported by clear, relevant, and specific evidence. Any flaws are admitted and either accounted for (this only applies to this group or place) or excused for the moment (the data has not yet been found to state this conclusively, but should be explored further). It is logical (all the steps are clearly put forth) and persuasive (not open to easy rebuttal).

Thus, to provide a critical analysis is to first provide a fully thought-out and supported argument. It is as simple as that. You can critically analyse the work of others, stating where their argument is lacking, but it is not enough to say they are probably wrong. You must demonstrate your own critical argument, one that is so well constructed that it can defend itself against the critical appraisal of others. When it can do that, you have demonstrated your ability to critical analyse the evidence and the secondary literature. If your reader can say ‘What about…’ after every statement, your paper isn’t ready to be submitted.

Will my students be able to reconstruct a perfect essay from these reversed engineered plans? Perhaps not, but perhaps my marking will finally rely less on my gut and more on my rubric.

**Image courtesy of Nick in exsilio

 

2 thoughts on “Reverse Engineering Assessment Criteria; or, That’s what you meant by critically analyse!

  1. Dave Andress

    Enjoyed that very much. Disagree horribly with you about intro’s though – should absolutely be a place for stating the outline of your argument, NOT reciting a contextual background: that’s something to integrate with your explanations in the main body.

    And that’s my unsupported assertion for the day!

    I’d also say that a critical analysis and an argument are importantly different things – one can ‘make an argument’ simply by using commonly-agreed empirical data to present the case for one side of a dispute; but to ‘critically analyse’ the nature of the dispute implies understanding it from the inside out, in terms of the methodologies and positions that bring others to interpret evidence in certain ways, or to argue for the validity of certain interpretive methods and conclusions…

  2. Thanks for this, Dave! I used to agree with you 100% about introduction. I like them short and sweet and structured. But, sadly, I seemed to be in the minority within my respective departments and wanted to allow for (minimal) context. As I said though, only what your reader absolutely must know at the start–everything else should be brought in where relevant.

    As for critical analysis, I did not mean to imply (but upon re-reading may have done so) that arguments were the same as critical analysis. What I hoped to convey is that you cannot begin to critically analyse something before you have an argument about a dispute, which you ‘understand from the inside out’, and you can therefore support against the naysayers and doomsayers and sayers of all kind.

Leave a Reply