Surveys & Questionnaires
Surveys involve asking a series of questions to participants. They can be administered online, in person, or remotely (e.g. by post/mail). The data collected can be analysed quantitatively or qualitatively (or both). Researchers might carry out statistical surveys to make statistical inferences about the population being studied. Such inferences depend strongly on the survey questions used (Solomon, 2001) meaning that getting the wording right is crucial. For this reason, many test out surveys in pilot studies with smaller populations and use the results to refine their survey instrument.
Sampling for surveys can range between self-selection (e.g. where a link is shared with members of a target population in the hope they and others contribute data and share the survey) through to the use of specialised statistical techniques (“probability sampling”) that analyse results from a carefully selected sample to draw statistical conclusions about the wider population. Survey methodologies therefore cover a range of considerations including sampling, research instrument design, improving response rates, ensuring quality in data, and methods of analysis (Groves et al., 2011).
One common question format is to collect quantitative data alongside qualitative questions. This allows a more detailed description or justification for the answer given to be provided. Collecting ordinal data (e.g. ranking of preferences through a Likert scale) can be a way to make qualitative data more amenable to quantitative analysis. But there is no one superior approach: the crucial thing is that the survey questions and their phrasing aligns with the research question(s) correctly.
Surveys are widely used in education science and in the social sciences more generally. Surveys are highly efficient (both in terms of time and money) compared with other methods, and can be administered remotely. They can provide a series of data points on a subject which can be compared across the sample group(s). This provides a considerable degree of flexibility when it comes to analysing data as several variables may be tested at once. Surveys also work well when used alongside other methods, perhaps to provide a baseline of data (such as demographics) for the first step in a research study. They are also commonly used in evaluations of teaching & learning (i.e. after an intervention to assess the impact). However, there are some noteworthy disadvantages to using surveys. Respondents may not feel encouraged to provide accurate answers, or may not feel comfortable providing answers that present themselves in a unfavourable manner (particularly if the survey is not anonymous). “Closed” questions may have a lower validity rate than other question types as they might be interpreted differently. Data errors due to question non-responses may exist creating bias. Survey answer options should be selected carefully because they may be interpreted differently by respondents (Vehovar & Katja Lozar, 2008).
Surveys & Questionnaires: GO-GN Insights
Marjon Baas collected quantitative data through a questionnaire among teachers within an OER Community of Practice to explore the effect of the activities undertaken to encourage the use of the community on teachers’ behaviour in relation to OER.
“I used several theoretical models (Clements and Pawlowski, 2012; Cox and Trotter, 2017; Armellini and Nie, 2013) to conceptualise different aspects (that relate to) OER adoption. This enabled me as a researcher to design my specific research instruments.”
Judith Pete had a deliberate selection of twelve Sub-Saharan African universities across Kenya, Ghana and South Africa with randomly sampled students and lecturers to develop a representative view of OER. Separate questionnaires were used for students (n=2249) and lecturers (n=106).
“We used surveys to collect data across three continents. Online survey tools were very helpful in online data collection and, where that was not possible, local coordinators used physical copies of the survey and later entered the information into the database. This approach was cost-effective, versatile and quick and easy to implement. We were able to reach a wide range of respondents in a short time. Sometimes we wondered, though, whether all those who responded had enough time to fully process and understand the questions that they were being asked. We had to allocate a significant amount of time to curating the data afterwards.”
Samia Almousa adopted Unified Theory of Acceptance and Use of Technology (UTAUT) survey questionnaire, along with additional constructs (relating to information quality and culture) as a lens through which her research data is analysed.
“In my research, I have employed a Sequential Explanatory Mixed Methods Design (online questionnaires and semi-structured interviews) to examine the academics’ perceptions of OERs integration into their teaching practices, as well as to explore the motivations that encourage them to use and reuse OERs, and share their teaching materials in the public domain. The online questionnaire was an efficient and fast way to reach a large number of academics. I used the online survey platform, which does not require entering data or coding as data is input by the participants and answers are saved automatically (Sills & Song, 2002). Using questionnaires as a data collection tool has some drawbacks. In my study, the questionnaire I developed was long, which made some participants choose their answers randomly. In addition, I have received many responses from academics in other universities although the questionnaire was sent to the sample university. Since I expected this to happen, I required the participants to write the name of their university in the personal information section of the questionnaire, then excluded the responses from outside the research sample. My advice for any researcher attempting to use questionnaires as a data collection tool is to ensure that their questionnaire is as short and clear as possible to help the researcher in analysing the findings and the participants in answering all questions accurately. Additionally, personal questions should be as few as possible to protect the identity and privacy of the participants, and to obtain the ethical approval quickly.”
Olawale Kazeeem Iyikolakan adopted a descriptive survey of the correlational type. The author research design examines the relationship among the key research variables (technological self-efficacy, perception, and use of open educational resources) and to identify the most significant factors that influence academic performance of LIS undergraduates without a causal connection.
“The descriptive research design is used as a gathering of information about prevailing conditions or situations for the purpose of description and interpretation (Aggarwal, 2008). My research design examines the relationship among the key research variables (technological self-efficacy, perception, and use of open educational resources) to identify the most significant factors that influence academic performance of Library & Information Science undergraduates without a causal connection. Ponto (2015) describes that descriptive survey research is a useful and legitimate approach to research that has clear benefits in helping to describe and explore variables and constructs of interest by using quantitative research strategies (e.g., using a survey with numerically rated items.
“The reason for the choice of descriptive survey research instead of ex-post-facto quasi-experimental design is that this type of research design is used to capture people’s perceptions, views, use, about a current issue, current state of play or movements such as perception and use of OER. This research design comes with several merits as it enables the researcher to obtain the needed primary data directly from the respondents. Other advantages include: (1) Using this method, the researcher has no control over the variable; (2) the researcher can only report what has happened or what is happening. One of the demerits of this type of research design is that research results may reflect a certain level of bias due to the absence of
statistical tests.”
Useful references for Surveys & Questionnaires: Aggarwal (2008); Fowler (2014); Groves et al., 2011); Lefever, Dal & Matthíasdóttir (2007); Ponto (2015); Sills & Song (2002); Solomon (2001); Vehovar & Manfreda (2008); Vehovar, Manfreda, & Berzelak (2018)
