2 Qualitative questions

The purpose of this chapter is to get you thinking about how to formulate good questions for different qualitative research methods, and what things to avoid. Do the activities below on your own and then discuss your answers with your group.

2.1 Focus Groups

When you are conducting qualitative research, thinking about the questions you are going to ask your participants is really important to ensure your data will be rich and of good quality. You want to write questions that are clear, open-ended and prompt the participants to discuss/share their experiences. Here are ten tips for writing good focus group questions:

Ensure that the questions on your Focus Group Schedule are relevant and allow you to explore and possibly answer your research questions.
Remember you are not doing quantitative research! The beauty of qualitative research is that your participants can respond in a variety of ways and tell you all about their experiences. In order to allow them to do this, don’t ask them questions, such as: Do people like A more than B? Does one thing cause another?
Make sure that your questions are open-ended, such as: What are your experiences of… , Can you think of a situation when…? This will keep conversation going. Don't ask questions that prompt a simple yes/no answer.
Don’t write questions that are leading e.g. Do you agree that.... You are investigating something you don’t know the answer to. Even if you think you might know the answer your questions should not reflect this.
Think about ethics! Are the questions you are asking people ethically problematic? If they are, it doesn’t always mean you should not ask them, you just have to be more cautious when you do - it's important that we respect our participants and give them the opportunity to decide how much they are willing to share!
Think about time. The focus group is normally around 30-40 mins long. It takes quite a bit of time to get through a focus group schedule, especially if you have to ask additional questions. We would recommend you have 5 or maybe 6 questions/discussion points on your schedule.
It is useful to put a star next to/highlight the really important questions you want to ask. This means that if you do start to run out of time then you can miss some of the other questions and go straight to these questions.
Think about the order of the questions on the schedule. You want to create a discussion that is as natural as possible, and this means one question should follow from the next. If you structure the questions well at this point you may find that in the focus group you have to say very little in order to cover everything.
Have an opening question that makes people feel relaxed and gets them talking. Something that people can discuss easily and something they don’t need to think about too much.
Additional prompts underneath the questions can be helpful for the person running the focus group, especially when you are new to qualitative research. See below for some examples from a study Ashley was involved with a few years ago. So, think of some probing questions that might allow people to discuss the question/topic in more detail. You might not have to use these, but they are there if you do!

2.2 Interviews

Many of the rules for good focus group questions also apply to interviews (e.g. writing open-ended questions that are not leading, having prompts ready etc.). However, in an interview, the researcher is typically one-on-one with a participant, which means that the discussion may in some cases be a bit less structured than a focus group schedule and may need to include more probes.

This is one section from a semi-structured interview schedule Ashley conducted, and is provided purely to give an example of how you might follow up your questions with prompts or probes. Please note that these were sensitive interviews that required NHS ethical approval, and therefore will likely be very different to any research you will be doing during the ODL course in terms of content/tone.

Questions about receiving anxiety interventions	PROBES
What help have you received for your anxiety?	What help did you receive in the past? Are you currently receiving help? Who helps/helped you?
Do you feel as if the help you received (or are receiving) has made any difference to your anxiety? It may have made a positive difference, a negative difference or no difference at all.	Anything else?
What was the best thing about the help you received for your anxiety?	How did it help? Anything else?
What was the worst thing about the help you received for your anxiety?	Can you expand on this? Anything else?
Some interventions will be adapted for the specific needs of the client. Did the help you received seem to be modified in any way? If so, what changes were made, in your opinion.	If there were changes, were they helpful? What helped the most? If any changes at all could have been made, what would your suggestions be?
If someone was to develop an intervention to help adults with ASD with their anxiety, what would help the most, in your opinion?	What would be the best type of intervention? What changes should be made? What information could you give that would help the person giving the intervention?

2.3 Qualitative surveys

Many of the principles discussed above also apply to qualitative survey questions - you want your questions to be clear, qualitative, and open-ended. However, due to the different response format and the fact that you are not there to clarify or provide prompts, there are some additional things to take into account when designing qualitative surveys. Here are some tips for writing good qualitative survey questions:

You don't get instant feedback from your participants the same way you do in focus groups or interviews - hence, it's really important to ensure your questions are clear before starting data collection. Pre-testing the questions with people who are not involved in the project is a good way to do this.
Clarity of the questions is particularly important - make sure there is no ambiguity and keep your questions short.
Avoid questions which have multiple parts (as participants may just pick up on one) - if your question has multiple parts, it might be better to split it into two
Think of the order of your questions - if you start with a broad, over-arching question, participants might only answer this question and write "see above" in others, which reduced the richness of your data
You can provide examples in brackets after the question that work as "prompts" and help the participant answer the question. However, be careful not to steer them too much or assume what their answer will be. In a recent qualitative survey that Wil conducted, this was one of the questions: "What do you think language barriers in teaching are, and what are your experiences (and/or those of your colleagues) of them in teaching international students? (Here you may want to reflect on your experience of teaching, tutoring, supervising and interacting with international students.)"
Include a final open question that allows the participant to share anything they would like to add. For example: "Is there anything else you would like to add about … ?"
Don't include too many topic questions - this will reduce the quality of your data as participants will get tired and more likely to drop out (you may have some demographic questions which are shorter and that's fine).

2.4 Activities

2.4.1 Activity 1: Which of these questions would you change and why?

Have a look at these fictional focus group questions and write down which ones you would change and why.

Research question: How does students' sense of belonging relate to participation in team sports?

How many hours of group sport do you participate in each week?
What does being a part of a team mean to you?
What does belonging mean to you?
How has your sense of belonging improved since joining your sports team?
I am going to go around the group one-by-one and I want everybody to tell me about a time when they felt that they haven't belonged to a team, and how that impacted them personally?
Tell me on a scale from 1-5, how much do you feel like you belong to a sports team
Do you like being part of a sports team?

Question 1. This question will result in a numeric answer that does not produce meaningful qualitative data. Be careful not to accidentally ask quantitative questions!
Question 2. and 3. These two questions are open-ended and qualitative, and will likely provide rich answers. One thing to think about is what question to start with and where to include prompts - number 3 is quite big and vague, and participants may struggle to get started with this, especially if it's early on in the FG.
Question 4. This is a leading question - it implies that their sense of belonging has indeed improved when this may not be the case. A better way to phrase this would be: "Has your sense of belonging changed since joining a team? If so, how? If not, why not?"
Question 5. This question puts participants on the spot, which is something you never want to do, especially with a sensitive question like this. Instead, the question should be given to the group, so participants can answer if they want to, and disclose as much or as little they wish to.
Question 6. This is a quantitative question and does not provide meaningful qualitative data.
Question 7. This is a yes/no-question. You should avoid phrasing questions like this, as they don't prompt rich answers. Instead, you may want to ask something like: "What has been the best part about joining a team and why? or What aspects of being part of a team do you like? Why? If you ask a yes/no-question, make sure to open it with some prompts (see example above for question 4).

2.5 Activity 2: Re-wording questions

Have a look at these poorly written fictional focus group questions. Try to re-write them to improve them for clarity and the quality of data they would give you.

Research question: Exploring international students' experiences of their sense of belonging and homesickness

Have you ever felt homesick while studying abroad?
How has this negatively affected you?
How often do you feel like you don't belong?
Does homesickness ever make you feel like you don't belong?
What do you do to feel better?

When you are happy with your own answers, go through the suggested answers below.

Question 1: This question prompts a yes/no answer - a better way to phrase this would be: "What have your experiences of homesickness been while studying abroad?"
Question 2: This question can be a bit sensitive - a better way to phrase this would be to say "How do you think this has affected you?" so the participant doesn't feel like they need to share a negative experience if they don't want to. Alternatively you could ask: "What about this has been challenging to you and how have you managed that?"
Question 3: This question doesn't produce meaningful qualitative data - if they answer "Mostly on weekends", you don't have much to analyse. A better way of phrasing the same idea it would be: "How have your experiences of belongingness been?"
Question 4: This question also prompts a yes/no answer. This is not necessarily a very good question in the first place because you're not directly looking at a causal relationship the same way you would in a quantitative study, and it becomes easily a leading question assuming there is a relationship between the two. A better question would be: "How do you think your experiences of belongingness and homesickness compare or connect to each other?"
Question 5: This question is a bit ambiguous and it's not really clear what you are asking - feel better about what? This also assumes that homesickness is always a negative state, which it might not be to the participants. A better way to phrase this would be: "How do you like to manage your feelings of homesickness?" or "What do you do when you are feeling homesick?"

2.6 OPTIONAL: Research skills session - Developing questions

Ashley has recorded optional research skills sessions on different aspects of qualitative research. These sessions take a little bit deeper dive into some of the topics and may be particularly useful if you are planning on doing a qualitative dissertation next year.

Watch the video here

2.6.1 Overall structure

It's important to think about the overall structure of an interview or focus group schedule. A good structure to use is: A warm up question, the main questions you want to ask, and an exit question. It is likely particularly important to use a warm up/ice breaker question for a focus group, especially if participants don't know each other particularly well.

Warm up Question

The aim is to allow participants to introduce themselves, ensure everyone understands the procedure, and to encourage people to feel comfortable talking. Previous projects have used photos, cartoons, and media, or asked participants to generate meanings of the terms used in the discussion (e.g., “poor sleep”,”well-being”) to get people talking. What is your idea for an ice breaker that might lead naturally into the main discussion?

What makes a good/poor icebreaker?

Thinking about the points above: What do you think of the following ice-breakers?

A FG on lived experience during COVID-19 of international students: What is your favourite meal from your home country?
A FG on coping strategies in students: What do you think is the best way to reduce stress for students?
A FG on feelings of belonging in a work context: What do you think about the graphic below? Does it resonate in any way?

Figure 2.1: Belonging icebreaker

Main Questions

Bear in mind that the discussion may wander off track or focus on one aspect. Consider which part of the research question you think is most important to get people to answer. Start with this and be prepared to come back to it at the end if you need to. If you have more sensitive questions in your research schedule, they should be included later on, so people answer them when they are more comfortable.

What should the questions be like? Avoid asking questions that ask two or three things in one sentence (e.g., about behaviour and impact on mood). Break it down into simpler questions. Limit the number of direct questions that elicit yes/no answers. These can be useful to gather information but are less helpful in generating a discussion. Add in probes/prompts to elicit further information if required.

Try to:

Frame questions in an open way that will facilitate discussion (e.g. 'do you use facebook?' is closed (can be answered as yes/no) whereas 'can you tell me about your experiences with social media?' is a bit more open - you could then follow up with prompts to specifically ask about types if you want to get that information
Frame questions in a way that is not leading. For example, 'how did lockdown negatively impact on your ability to study?' assumes that it did, when this may not be the case. A better question would be ''Do you think that lockdown restrictions impacted on students' experiences of studying? If so, how?"
If you have sensitive questions, try to frame questions in a way that is not personal (i.e. you give people a chance to answer related to themselves, but they can also choose not to). This apporach also invites people who may not have lived experience to respond, e.g. "How do you take care of yourself when you are anxious" would be better phrased as "How might students support themselves when they are feeling anxious?" This is more important for focus groups, as people might feel put on thespot and go beyond their comfort levels.
Not ask lots of different things within the one question, e.g. "Can you tell us about your experiences of studying in Semester 1, how it made you feel and how that affects your studying in Semester 2?" You could ask about experiences of studying generally, and then could follow up with specific sub-questions asking them about semester 1 and 2, if that is something that was important to you.

Closing question

The aim here is to give people an opportunity to add anything that might have been missed in the discussion/to wrap things up.

Examples:

Thank you very much for all the great points you had today!Before we wrap things up, you have any final points that you would like to make, that perhaps wasn't covered in the interview?
Today we covered X, Y, and Z in the focus group. Is there anyone who has any final thoughts about what we covered here today that they'd like to share?
That's us finished with the main part of the focus group. We'd like to end by giving you all an opportunity to add anything that you think is important but has not been covered in the focus group so far. If you'd like to add something, please press raise hand on teams and we can make sure we go round those people before we finish up!

2.6.2 Braun & Clarke (2013) activity

Here, we have an adapted version of an interview schedule that is included in the Braun & Clarke (2013) book, on the topic of Partner Relationships. This is purposefully poor. What problems can you spot?

If you are not already married, are you planning to get married within the next few years?
I love going to the cinema with my partner, do you?
Do you have a good time with your partner?
Would you go to Relate if you had problems in your relationship?
When and how did you meet your partner?
Do you earn more money that your partner?

In general Overall, the interview schedule lacks coherence – it covers a wide range of different topics, and it’s not clear what the focus of the interview is. The ordering of the questions is also very poor (very sensitive questions early in the interview; moving straight from questions about positive aspects of the relationship to relationship difficulties). The researchers should decide on the focus of their research, and their research question, and design the interview schedule with these in mind. The schedule should also be reordered so it starts with less sensitive questions (these should be moved to later in the interview) and the questions flow logically. The interview should also end with a general ‘clean-up’ question (e.g., ‘Is there anything else you would like to add?’). (NB. Our answers to this exercise assume that the research is being conducted in Britain)

Question 1

Problems First, this is a closed question which does not encourage participants to provide rich and detailed answers. Second, unless the researcher already has quite a bit of information about the participant, it is not an ideal opening interview question because it makes a number of problematic and potentially alienating assumptions, which may damage rapport. For example, unless the study is specifically focused on heterosexual couples (‘partner relationships’ suggests a inclusive focus), or the researcher already knows how the participants identify their sexuality, assuming that people are married, or asking whether they are planning to marry, when marriage is not legally available to same-sex couples, reinforces heteronormativity. Furthermore, unless the research is specifically focused on marriage, asking whether people are ‘planning to get married’ assumes a particular model of relationships whereby marriage (and civil partnership) is treated as the ‘gold standard’ of relationships, the ideal end-point of a couple’s relational journey, and not everyone views marriage in this way (some couples are critical of marriage and choose actively not to get married).

Solutions The question needs to be reworded so that it is inclusive of same-sex and heterosexual relationships and does not ‘buy into’ assumptions about marriage as gold standard of relationships. The question should also be reworded as an open question and probably moved to later in the interview.

Question 2

Problems Again, a closed question and extremely leading – it invites the participant to agree with the researcher’s preferences! This is not the best way of accessing participants’ perspectives.

Solutions Reword the question so that it is open and not leading (do not use the researcher’s preferences in this way).

Question 3

Problems Again closed, and rather vague (what constitutes a ‘good time’?).

Solutions Reword the question so that it is open and more concrete.

Question 4

Problems Again closed, and assumes that the participants know what Relate is (a UK charity that that offers relationship counselling and sex therapy to heterosexual and same-sex couples experiencing relationship difficulties). Making assumptions about what the participants know (and don’t know) can alienate participants (making them feel stupid for not knowing) and damage rapport. Moreover, Relate is not the only provider of relationship counselling in the UK.

Solutions Replace this with an open question and one that does not make assumptions about the participants’ knowledge (e.g., ‘relationship counselling’ is a good substitute for Relate).

Question 5

Problems Again, a double-barrelled question and also one that would work better earlier in the interview.

Solutions Reword the question to remove the double-barrelled element (decide what is more important to know when [how long ago] they met their partner or how they met them?). This would work better as the opening interview question (it makes little sense to ask questions 1-6 before asking this question) because it is a relatively ‘gentle’ question and possibly one that participants would be expecting to answer in an interview on partner relationships. It provides participants with an opportunity to ‘tell the story’ of their current relationship at the start of the interview.

Question 6

Problems Again a closed question and also a very sensitive one!

Solutions Again, this needs to be replaced with a more open question that invites detailed responses. The researcher should reflect on what they want to know and how this question relates to their research question.

2.6.3 Further things to consider with qualitative surveys

No opportunity for clarity, so wording has to be clear
Similar points to above in terms of not asking too many different things, having a sensible structure
No rapport, unlikely to be an 'ice breaker' (although might have an intro question)
People might be more comfortable answering sensitive questions as they can be anonymous
Try not to have lots of questions as your participants will likely lose patience - what are the key things you want to know?
Can use branching e.g. in Qualtrics

1 Qualitative designs

3 Data collection