Surveys are an effective measure to collect information, test hypotheses and have an insight into populations. But when the survey is poorly constructed it gives a false impression, wastes time, money and effort put into research. The precision of the data provided to you is directly related to the coherency of your questionnaire.
In Mars Publications, we observe excellent research work being damaged by a poor survey design. By preventing highly prevalent design errors, yet perilous, we enable researchers to gather clean and reliable information. Follow this guide to audit the survey tool that you are using or are planning to use and guarantee the integrity of your results.
Pitfall 1: Biased and Leading Questions
A leading question is used to indirectly influence the respondent to a particular answer and creates response bias and invalidates your results. This occurs in instances where the question has non-neutral language or assumes an answer.
The Mistake
- Sample Leading Answer: “How content were you with the excellent customer service you have had today? (Suppose that the service was good).
- The Issue: The words excellent customer service make the respondent be able to affirm the positive rating which biases the gauge of satisfaction towards the right.
The Fix: Embrace Neutrality
- Redo it: What was your rating of the customer service today?
- Be sure to provide Belly and Bones: Provide a complete spectrum of answers, i.e. Very Dissatisfied to Very Satisfied, with a middle ground between them.
Pitfall 2: Use of unclear and vague words
The most important thing in the design of surveys is that of clarity. Loose language, ambiguous ideas, or sector lingo perplex the respondents, which results in a lack of consistency in meaning and a lack of dependable information.
The Mistake
- Vague Question: “Do you use social media frequently?
- The Issue: Frequently has more than one definition in the minds of people. To one, it may consist of once a week; to another ten times a day. The information gathered cannot be compared amongst the respondents.
The Fix: Operationalize and Define
- Define Key Terms: In case of using such terms as healthy diet, regular exercise, or frequency of use, you should define them within the question or give an operating range.
- Give Specific Choices: You do not want to ask how often, but you can ask: How often do you use social media? with fixed options:
(a) Multiple times per day
(b) Once per day
(c) A few times per week
(d) Less than once per week
Pitfall 3: Double-Barreled Questions
A two-tailed question is one in which two different matters are being enquired about. The respondents are not in a position to give an answer logically, on the basis that they may support one statement and disagree with another one.
The Mistake
- Samples of Double-Barrel Question: “Do you think that the university should reduce tuition and more money to sports programs?
- The Issue: A student may hold firmly the case that the tuition should be reduced, yet argue against sports more funding. A simple yes or no answer does not reflect their real view on either of the issues.
The Fix: Split the Query
- Separate the Ideas: It is always important to separate the elements of the question into two. Do you suppose that the university should reduce tuition? Do you feel that the university should allocate more money to sport programs?
- Check Conjunctions: Your survey should be actively scanned with the conjunctions such as and, or, and but as they are usually used to indicate a double-barreled trap.
Pitfall 4: Response Scales Mismanagement
The scale used (e.g. Likert, semantic differential) and labels you place on it significantly influence quality of data. Such pitfalls as uneven scales or the inability to take into consideration missing data are common.
The Mistake
- Example of Uneven Scale: The satisfaction scale will be provided as a 4-point scale (Excellent, Very Good, Good, Fair).
- The Issue: This scale does not have negative answers so the respondent has to choose between positive answers only and this creates an artificially high mean score of positive.
The Fix: Create a Balance and Completeness
- Apply Balanced Scales (e.g., 5- or 7-point Likert): It is important to have equal number of positive and negative items on a scale, typically with a neutral center.
- Sample: Strongly Disagree, Disagree, neutral, agree, strongly agree.
- Insert N/A. or Don’t Know: All respondents will not have an opinion or experience to comment on every question. The presence of a Not Applicable or Don’t Know option helps avoid the situation when the respondent has to choose a response that is not relevant to them, and the answer is forced, which significantly enhances data validity.
- Active Voice Proposit: Be coherent. In the case of a 5-point scale, in one section, then attempt to apply the one on the rest of the survey to reduce fatigue among the respondents.
The Next Steps to the Perfect Data Collection
Effective survey design is a procedural approach of reducing bias and improving clarity. It is very demanding in terms of attentiveness to detail, but the pay off is quality, reliable data that can be used to make sound conclusions.
- Pilot Test: It is always advisable to conduct the pilot test using a small and representative sample. Request them to single out any confusing questions or words
- Review Logic: Skip logic and screening questions provide an opportunity to make sure that respondents answer only relevant items.
- Refine Continuously: See survey design as a process.
Mars Publications is dedicated to making it possible to publish research that is founded on strong grounds. You are also instrumental in making a way to a successful influential and credible publication through mastering these design principles. Are you willing to construct a questionnaire that is going to reflect reality?