By Published On: November 1, 2022

In Part 1 of this two-part blog series, we looked at the basics of survey construction. In Part 2, we’ll take a more in-depth look at the psychological factors that can help you develop better questions.

Psychological factors to consider

With any kind of survey, it’s important to keep a few tenets of human behavior in mind. People may have an emotional investment in your success, but this sentiment only goes so far.

If you make the survey too cumbersome the desire to help will fade quickly. Similarly, asking too much of a person (such as going back for follow-up questions or asking for repeated participation) may create greater resistance. If your audience is captive, like a group of employees at your company, you may have more latitude, but even then, psychological factors can skew results, so questions should be constructed carefully.

Drafting your questions

To avoid the risks of survey abandonment and unreliable conclusions, provide a limited number of choices for each question. For instance, if you ask “What is your favorite fruit?” as a closed-ended question and provide only “Apple, Orange, Banana” as possible answers, you are going to get no answer or an unreliable answer from all the strawberry and kiwi lovers in your audience. If you’re basing your next flavor concoction on this customer survey, you may be disappointed in its success.

When you’re using rating scales, keep them consistent in structure between questions. If 1 is the low score for questions 5 and 6, it should also be the low score for questions 12 and 17. Once you’ve trained your audience how the scale works, you don’t want to accidentally trick them into giving you an incorrect response.

Drafting your questions

To avoid the risks of survey abandonment and unreliable conclusions, provide a limited number of choices for each question. For instance, if you ask “What is your favorite fruit?” as a closed-ended question and provide only “Apple, Orange, Banana” as possible answers, you are going to get no answer or an unreliable answer from all the strawberry and kiwi lovers in your audience. If you’re basing your next flavor concoction on this customer survey, you may be disappointed in its success.

When you’re using rating scales, keep them consistent in structure between questions. If 1 is the low score for questions 5 and 6, it should also be the low score for questions 12 and 17. Once you’ve trained your audience how the scale works, you don’t want to accidentally trick them into giving you an incorrect response.

Refining and editing

Review your questions for wording that may trigger psychological resistance from your audience. You want to allow for both negative and positive feedback, but you do not want to lead their answers as it could alienate survey takers who feel differently. If you signal the answers you are hoping for, you’re likely to make the response data less valid.

Beta testing

Even if your beta testers work through all the questions with no issues, you should still evaluate their answers upon completion. If their answers are typical of the ones you might receive from all survey responses, ask yourself if you would have achieved your goals. If not, consider refining your vocabulary or further focusing your questions to get the kind of information you need.

Final analysis

Analyzing qualitative information from open-ended questions can be much more challenging than aggregating percentages from your closed-ended questions. Look for patterns of responses and consider creating your own ranking scale based on the trends you identify. For example, if you asked for three words that describe your customer service, you can create a word cloud from the responses to see which ones stand out or assign a value to each response and create a %positive/%negative graph, making perception easier to visualize.

Finally, keep your emotions in check as you analyze the information you gather. Negative feedback can be difficult to hear, but is often necessary for planning your way forward. Similarly, too much positive feedback can stifle your opportunity to think creatively about how to make use of the information. You can’t solve a problem if don’t get to the root of it. And you probably wouldn’t be issuing a survey if you weren’t looking to improve in some way.

Best of luck with your survey! We’re here to help if you need us.

Building an effective survey Part 1: Survey construction
Read more

About the Author: Beth Seitzberg

Beth Seitzberg
During her career crafting creative Beth has conceptualized, designed, developed, strategized and overseen the building of brands, campaigns, and creative platforms for large corporations as well as for dozens of regional and local companies in every sector including financial services, manufacturing, retail, medical, and non-profit. This range of experience with clients of all sizes has honed a specialization in brand management and application of master brand strategy across channels and tactics. With a background in psychology and sociology she brings both a researcher’s behavioral approach and an artist’s instinct to her work. Beth specializes in designing outstanding, strategic creative that ties into business goals and communicates the client’s message clearly and distinctly in their unique voice.
Building an effective survey Part 1: Survey construction
All posts