Survey Design with Smartphones In Mind
- March 16, 2015
In a meeting today, someone said, “There’s no point in making this segmentation survey smartphone-friendly or shorter than 30 minutes, since no one would take this on their smartphone anyway.” That’s simply not true!
People try to take segmentation surveys all the time on their smartphones (especially fresh river or client sample respondents, who have not yet been burned by the attempt.) They will even complete it on these devices, but only if the survey is of tolerable length and complexity. The issue of optimized survey design does not only apply to segmentation research, but rather to any long survey, which some in the industry have somewhat arbitrarily defined as 20-plus minutes.
Let’s examine your choices.
You can go with a long survey and prevent or discourage completion via smartphone, realizing this makes it really hard to have a representative sample. Yes, you may fill the objective demographic quotas, but the softer psychographic and lifestyle traits of who you are getting will be off from the general population. As an analogy, if you keep dialing landlines long enough, you will eventually fill your quota for young adults… but the people you get are surely not going to be representative of typical young people!
If you go long and don’t discourage entry on smartphones, be aware that smartphone respondents will break off at double the rate of other devices due, in part, to a miserable survey experience that decreases the chances that these respondents will even complete another survey… undermining the long-term integrity of survey samples.
The bottom line? We need to adapt our research approaches to this emerging, smartphone-based reality if we hope to have valid and projectable results. Mobile devices make potential respondents more accessible than ever, but the number and type of questions you can ask in mobile format are limited. Trying to design your survey to play well on a small screen in an on-the-go environment shines a bright light on just how long and complicated some question types are… for ANY device.
Avoid the following: long “select-all” lists which generate under-reporting due to satisficing, large grids which trigger break-offs, and questions that impose a heavier cognitive load which cause people to think the survey was even longer than it really was, lowering satisfaction with the survey-taking experience and reducing future cooperation rates.
Alternatively, you can simplify question formats, shorten brand lists, reduce attribute battery lengths, and focus each survey on only the content that needs to be included to address the immediate business issue at hand. Moreover, you might consider breaking large strategic pieces of research into survey sets, rather than trying to cram everything in into a single survey instrument.
And if you are wondering whether this blog is EVER going to end…. just imagine that you were taking it as a survey!