Hi Sara. Thank you for the really important question, and sorry for the delay in getting back to you. I had been thinking that I needed to add something on questionnaire design to our (very new and still developing) Extended Essay LibGuide, but it would be even better to be able to offer a lesson on it. Next year I would love to add an optional session during our February EE Investigation week for all those planning to use questionnaires, so am very keen to hear feedback from your session when you deliver it. [Note: the LibGuides are new this year so are still in the process of being written as we work through the IB Extended Essay process with the students. Most of the guidance beyond the initial subject selection is in the individual subject-specific guides e.g. IB English A ]
How long is the lesson likely to be? Is it for EPQ and HPQ students together, or two separate lessons? What stage in the process will they be at (i.e. do they already have a good idea of their area of interest? Could they come out of the lesson having made some practical decisions about their own questionnaires?). It also depends on the group size and level of interaction you are able to have. Are you doing this online during the lockdown, or planning something for afterwards?
Frustratingly we have a couple of really good books on questionnaire design, which are currently under coronavirus lockdown at school. However, it seems that there are two key angles to address:
1. The background to writing the questionnaire:
- What information are you trying to find?
- Who do you need answers from? (what is your target population?)
- Ethical considerations
- How are you going to administer the questionnaire? (paper, interview, online – what are the options and tools available? The online tools are important to explore before students get started. I had a student come to me during our EE investigation week in February who had designed a SurveyMonkey survey and only discovered once she had finished that she was only allowed 10 questions on a free account, but she needed more. Like many schools, however, we use Office 365 and Forms would allow her up to 100 questions, so she had to remake her questionnaire in Forms. This is my first year in this particular role in the school and this was a lesson to me that we need to address this earlier.)
- The vital role of a small pilot study to check it works the way you think it does!
2. The questions themselves:
- Open vs closed. Important to think about what you want to do with the data – open questions can produce some very interesting answers that you wouldn’t otherwise have considered, but are very difficult to analyse, particularly with larger sample sizes. Closed questions are good for quantitative analysis, but do (sometimes artificially) restrict the answers you get.
- Bad questions: ‘Leading’, confusing and double-barrelled questions
- Scales of agreement/disagreement (lots of interesting discussion to be had about Likert scales if you have the time)
- Forced choice questions
- Keeping it short, relevant and interesting for the respondent and ordering and grouping questions appropriately.
- Explaining what it is for and levels of confidentiality
[I found these two videos from helpful in getting my brain going in the right direction: Designing a questionnaire, Writing good survey questions , but there are many others out there.]
I’m sure you know all this better than I do though – I’m just starting to think about a session for next year, and I have to think through the content I want to deliver before worrying about the format. You were actually asking for ideas for the format of the session. I think your idea of a ‘bad’ questionnaire is a good one – although perhaps it could be more nuanced and have some good and some bad questions.
How about getting students to take a short survey at the start, as a ‘lesson starter’ as they come in perhaps, or even electronically before the session using something like Forms? If you design that with some deliberately bad and good questions, you could ask them to critique the survey. Which were ‘bad’ questions? Why? How could they improve them? Which were ‘good’ ones? Why?
If they submit their answers before the session, you could look at some of the results and show which answers are difficult to analyse, and which questions produce what they might consider to be surprising results. Likert scales are quite interesting to look at for acquiescence bias and central tendency bias but you probably won’t have time for this!
It would be good for them to emerge from the session with something concrete for their inquiry if they are at a suitable stage. Perhaps you could design a form for them to fill in looking at those five background stages above in relation to their own inquiry: What? Who? How? Ethics? And Pilot (how are they going to do one?) or ask them to design one good question for their survey and test in on a partner.
One final thought, which is perhaps more challenging given the current situation – is there any opportunity to connect with subject-teaching colleagues in subjects that use questionnaires (off the top of my head Geography and DT are obvious examples) to ask what they teach about this and when, and what they would like students to know but don’t actively teach? What are common problems they see that they would like you to address? One of the advantages we have in libraries is that we can be a point of connection between subject departments and provide students with a joined up experience.
I would love to continue this conversation and to hear your thoughts and any feedback on how you get on – this is definitely something I need to develop for next year.
.
