This Fall, OpenNorth’s Applied Research Lab worked with the Laboratoire d’innovation urbaine de Montréal (LIUM) to draft the City of Montreal’s first Open Data Action Plan. Foremost, the draft of the Plan addresses the need to make open data more accessible and re-usable for a wide audience. The Plan also aims to reflect the realities and needs of City employees, their teams, and departments, in their roles as data publishers and also as key users of open data. In order to understand the day-to-day experiences that employees face in using open data, as well as the processes relating to publishing data, OpenNorth and the LIUM organized and facilitated focus group discussions with city employees.
Focus group discussions are often associated with marketing companies, as a tactic they use understand the behaviours and values of their target clientèle. However, focus groups are a research method that can be used in number of settings, to capture views, perceptions and interactions amongst a group of individuals on a certain subject matter. In this blog post we discuss focus groups : what they are and in what cases are they useful as a research method. Then, based on our experience leading focus groups at the Ville de Montreal, we present some considerations for focus group design and analysis.
What are focus groups?
Focus groups are a qualitative research method that can provide valuable insight and perspectives to researchers, as they facilitate open group conversations around a certain subject matter. Typically, a focus group consists of a one-time meeting among a small group of individuals who share a common experience, to discuss an issue (Carey 2015). While the recommended number of participants vary, some researchers have suggested that the ideal number of participants is between six to ten (Rabiee 2004; Krueger & Casey 2009).
While focus groups can provide a unique depth of understanding around a research question, they should not be used as a substitute for statistical data collection. This is due to sampling issues that arise in focus group composition : the opinions of a small, hand-picked group of people should not be generalized to represent the beliefs of an entire population (Morgan 1998).
Therefore, focus groups are best used by researchers in cases where they wish to examine complex human behaviours and motivations in depth; not for gathering a statistically representative sample. To open the conversation, researchers can ask for participants to share their general thoughts on a topic, which will reveal an initial set of beliefs and values from each participant. Then, as the conversation unrolls within the group, participants may begin to reflect on how their own experiences relate to others, which can bring to the surface contrasts, curiosities, and feelings which perhaps participants were not previously aware of. Drawing information from this web of social interactions is one of the distinct features of focus group methodology, which provides the researchers a depth of understanding which could not have been captured using a survey or through a one-on-one interview (Morgan 1998; Rabiee 2004).
When considering using focus groups for research, it is also important to remember in which situations they are less useful. The goal of focus groups should be to generate free flowing conversation, where each individual who participates feels that their contributions are respected by the rest of the group (Rabiee 2004). Accordingly, focus groups should not be used when participants are unfamiliar, or uncomfortable with the topic at hand.
Next, we present some best practices for focus group design, to help researchers ensure the validity of the results at all stages of design, planning and analysis.
Focus group composition
Even before the focus group begins, the researchers need to decide on the membership of the group. To promote an environment where participants feel welcomed, it is often recommended to create focus groups that are homogenous. Seeking participants with the same socio-economic background, or participants with the same level of experience in the topic at hand, is one way of doing this (Hennick 2014). Another approach is to segment focus groups by gender and age (Rabiee 2004; Hennick 2014).
Whatever the chosen methodology, it is important to remember that the group selection will impact the content of the conversation, the group dynamics and ultimately the analysis. For instance, if the researcher chooses to group together participants that already know each other, this may make participants more comfortable sharing their feelings. It could also have the opposite effect: if participants already know each other, they may provide fewer details in their response (Hennick 2014).
We took these factors into consideration when segmenting the focus groups for the Ville de Montréal discussions; for instance for each group we tried to ensure that the participants came from different teams and departments. This provided for interesting group dynamics, as most participants had different backgrounds, but many shared similar experiences in trying to publish or use open data.
Moderating techniques
To encourage participants to share their experiences and feelings, it is important to support an open environment, and to generate free-flowing discussion which contain valuable insight for the researcher. However at the same time, the conversation needs to stay on topic and on time. Achieving this balance, while ensuring equal participation among participants, requires a skillful moderator. Ideally, the moderator is also neutral, and actively ensures that each participant feels like their contribution is valued equally (Hennick 2014).
Whether or not you are a seasoned moderator, there are techniques that can be used to better structure and lead focus group discussions. First, presenting a set of guidelines to participants before starting the discussions can help to set the expectations and name the boundaries of the conversation. For example, before we began the focus group conversations for employees at the City, we asked participants to not speak over each other and to direct their response to the moderator. This helped to set a respectful tone for the conversation, and made it easier for the moderator to interject if someone is speaking out of turn. Another easy but effective practice, is to task an individual to keep track of the time for the moderator, so that the time-tracker can indicate to the moderator when he or she should move on to the next question. Lastly, to ensure we had a balance participation for each question, we assigned each participant a number (e.g. participant 1, participant 2) which made it easier for the moderator to ensure that each participant contributed, or had the chance to contribute to the given question.
Collecting and structuring the data
Recording the content of focus group conversations is critical, to enable the research to collect and fully analyze the information that comes from the discussion. This process of data collection should rely on rigorous, systematic methods in order to convert the speech recording into a full, annotated transcript which can then be coded for themes and analyzed. While speech to text technology is available for generating transcripts, it should be used cautiously. For instance, the ability of these softwares to pick up on colloquial language and nuances is not guaranteed.
This is why we chose to transcribe the recordings we collected manually – but even then, it can take considerable time to record all of what what said accurately, especially taking into account the considerable differences between spoken and written language. Among other things, we found it tricky to deal with spelling, fragmented sentence structures and in cases where two respondents were speaking over each other. While there may be different ways of dealing with these issues, it is important to apply the same principles across the transcript, and document the process. This way, any potential question that occurs in later stages can be traced back and properly acknowledged.
Contribute your thoughts to the Open Data Action Plan
OpenNorth had the opportunity to lead open discussions and gather thoughts that directly contributed to the City of Montréal’s Open Data Action Plan. This gave us important insight into applying focus group methodology to capture individual and group reactions to complex topics, such as open data publication, a topic which requires action from individual city employees and from across government departments. We’re very grateful for this opportunity to work closely with the Laboratoire d’innovation urbain de Montréal.
In the foregoing, we emphasized that focus groups need to be planned and designed with care, to generate open discussions, and gather a variety of viewpoints. Considerable attention must be put into segmenting the groups, moderating the conversation, and transcribing the recordings with systematic methodology in order to generate reliable and valid results.
Lastly, the City is still looking for input on the Open Data Action Plan. The draft of the Open Data Action Plan is available for the public to view and comment, until May 15th 2019. We highly encourage you to provide your contribution! Provide your comments here: http://bit.ly/MTL_DO_plan
Note: OpenNorth recently received a grant from the J.W. McConnell Family Foundation Social Innovation Fund to develop its Applied Research Lab. Part of this work includes knowledge mobilization and reflection on methodological issues that may be faced by other organisations in the non-profit sector. This post represents some of our ongoing thoughts towards that objective.
Works Cited
Carey (2015). Focus Groups. International Encyclopedia of the Social & Behavioural Sciences, 2nd edition, 9, 274-279. http://dx.doi.org/10.1016/B978-0-08-097086-8.10543-4
Rabiee, F. (2004). Focus-group interview and data analysis. The Proceedings of the Nutrition Society, 63(4), 655-60.
Hennink, M. (2014). Focus group discussions (Understanding qualitative research). New York, NY: Oxford University Press. (2014).
Krueger, R., & Casey, M. (2009). Focus groups : A practical guide for applied research (4th ed.). Los Angeles: SAGE.
Morgan, D. (1997). The focus group guidebook (Focus group kit). Thousand Oaks: SAGE Publications.