Reply To: Empowering Students to Inquire in a Digital Environment

[Jenny Toerien Keymaster]

A really interesting discussion here about the CRAAP test – and Elizabeth’s link to Getting beyond the CRAAP test is well worth following however I would say that the issue that both Mike Cauldfield and John Royce have with the CRAAP test is not the idea itself, more the way it is taught and presented. Yes, it can be presented as a mindless list of a million things to check off a list before you can use a website, and no, that isn’t helpful. Particularly if, as Mike says, in The problem with the checklist approach you ask students to look for outdated things like how professional a site looks, whether there are spelling errors and how many ads there are, or if you dwell too much on the importance of the domain name. His SIFT approach does look really interesting, and it is fantastic that he has provided all the materials under Creative Commons (we’re big fans of that here!) – but it is largely asking students to do the same thing as the CRAAP test, just presented in a different way. What are they looking for under “Investigate the source” if it isn’t some sense of:

• Is this Current?
• Is it Relevant to my inquiry?
• Does it seem to be reasonably Accurate? (How do I know? Have I checked it against other sources? Does anything about the claims feel odd?)
• Does the writer/site have any Authority to make this claim? (How do I know? Have I checked their general reputation, not just the qualifications they claim to have?)
• What is the Purpose of this story? (What are they trying to convince me of and why? Everyone is selling something!)

Having said that, I will be having a good look at the Check please site, because I think there is a lot there that could be really helpful (even if I choose to combine it with CRAAP). Thanks Elizabeth!

I think Mike is absolutely right that it is easy for students to get overloaded, and as such I tend to go for a ‘light touch CRAAP test’. Rather than mechanically go through the motions of the whole list, I would encourage students to focus on the area they think is most important for that source. As Barbara says, Authority and Purpose are pretty much non-negotiable. Currency and Relevance are very quick (if it isn’t relevant, there is no point continuing with the rest!), and Accuracy is often tied up with Authority.

There are many ways to teach source evaluation, and all of them have advantages and disadvantages, but what we are actually teaching students is a sense of healthy suspicion. Does it seem too good to be true? Does something seem a bit odd? Does it tie up with what you knew already? Then giving them the tools to follow up their suspicions. It doesn’t have to be very time-consuming if you start with reasonably reputable sources – but they need guidance on where to find reasonably reputable sources and what they ‘feel ‘ like. And we need to talk honestly and non-judgementally about Wikipedia. What is it good for? What isn’t it good for? Why do you need to be careful with it? (This is part of a presentation we use on this topic occasionally).

In the article Mike says (emphasis added):

“The CRAAP approach assumes that students need to learn, for example, that authority on a subject matters, that funding matters, that conflicts of interest matter, and asserts if we train students in a twenty minute catechism that reviews these issues each time they’ll make better decisions. The reality is checklist or no checklist we know most of these issues when we see them. Most people get intuitively that Russian state media is not be the best source on whether Russia was involved with the downing of MH17. The difference between the professional fact-checker and the student is a set of digital habits that quickly reveal that RT is a Russian propaganda arm, or that a particular naturopathic cancer center has a reputation for quackery. The gap isn’t the understanding, it’s the missing context.” 

I do actually disagree with him here (and this may be because I am mostly working with younger students). Many of our students don’t have a good understanding of authority (which is why Wikipedia can be a problem – because they don’t really understand, or sometimes care, what they are dealing with). They also aren’t great at realising that all of the sources that they read have an agenda of some kind and that they need to be alert to that. I’m not convinced that many untrained high school students would automatically have a problem with a Russian state media source giving information about the downing of MH17. Particularly if they were new to the story and didn’t really understand the agendas involved. Yes, the gap is the missing context, but that context needs to be taught and the CRAAP test is one way to do that (as is SIFT). And it doesn’t have to be taught as a 20 minute checklist.

In that sense (and I don’t find myself saying this very often!) I am with Professor E. D. Hirsch (a well-known opponent of Inquiry Learning) – we cannot treat children as if they have the same level of social and cultural literacy that adults have. Where Professor Hirsch and I deviate, as far as I understand it, is that he feels that this means that inquiry is an unsuitable way for young people to acquire new understanding, where as I believe that it means that we need to give them tools and support to do what an expert would do automatically. We need to give them a scaffolding that can drop away once they fully grasp the underlying skills and concepts.

I have used this presentation and accompanying resources to teach the CRAAP test to both Year 9 and Year 12. An important point about it is that the Breitbart article on which it is based is a very well-written, well-presented article – and they have largely not heard of Breitbart (a highly questionable, very right-wing US media outlet). The key here is getting them to spend a few minutes looking for information about the organisation and the article outside the original site (a vital point that I picked up very strongly from John Royce’s article) before reading too deeply. The other key point is to give them some useful tools to help them to check the leanings of unknown sites (such as https://mediabiasfactcheck.com/ ). Those of us from the UK, for example, know that The Daily Mail is a right wing tabloid, prone to hyperbole, often inaccurate and downright misleading, whereas The Times is a broadsheet that is right-centre in tone but largely accurate in its reporting. Do our students know that? Or do they just think that we would prefer them to read the Times because the Mail is a bit gossipy and more ‘fun’? Do many adults realise quite how misleading the Mail can be, and how dangerous a news diet based a single media source is, particularly one like this? It is a national paper, after all. We need to be giving students practical tools like Media bias/Fact check, and keeping pace with change. This is a good tool now – just as domain names were a good tool 15 years ago. In 10 years time, I’m sure we will need different tools to support something like the CRAAP test. But that doesn’t mean the test itself is a bad one.

As Stephanie says – let’s teach them not to judge a book by its cover. But then we do need to give them some other criteria to judge it by, given the whole point is that their ‘gut feel’ is not yet well-developed.

P.S 1: after looking at the Breitbart article with Year 12, one student was quite aggrieved that it got such a low score. He said “But they’ve worked so hard to make it look good. Don’t they deserve some credit for that?”. Perhaps that is an argument for doing away with the scoring aspect of CRAAP. We aren’t giving the sites a mark for effort – we’re looking at whether they are suitable sources to use in our work!

P.S. 2: the Times/Mail discussion is another powerful argument for promoting reading and literacy (digital or otherwise) in our schools generally. One of the startling facts that Alice Visser-Furay highlighted in her excellent presentation for CILIP SLG in July 2020 (also available from her website, Reading for Pleasure and Progress) was that the average reading age of an article in the Sun (UK tabloid) was 8 years old, while for the Guardian (UK broadsheet) it was 14 years old. There is a social imperative to make sure that the next generation of the voting public have the literacy skills (and inclination) to get their news from reputable sources.