The 7 Step Software Evaluation Approach
My company is both of a user of several software …Read more
If you make a Google search for online survey software, you can go through several pages finding different products. However, few of these offer mixed modes or multimode surveys. Ignoring the paid for advertising, the first page of the search at the time of writing this article threw up three review sites (Capterra, PCMag, a 2014 blog post) and seven products (Smart Survey, SoGoSurvey, SurveyMonkey, Qualtrics, Snap Surveys, SurveyGizmo and QuestionPro).
The three review sites all lean towards the free products. Indeed, the 2014 blog post on Wordstream advocates Google forms as one of the best 7 platforms. Of the survey products, they also tend to lean towards the same free products. Of course, this makes perfect sense, there are plenty of people who want to do free surveys for their studies, clubs and 101 other reasons. And, what’s more, these products generally do the job well, so why would you want to pay if you can get it free. If you offered me a decent car that did everything I wanted for free, I might well drive it rather than the car I drive!
Often, research agencies need something more powerful than the free/budget tools available. There are plenty of reasons for this. These includes – more flexibility/better looking questionnaires, complex question routing, handling of sample, linking data from a database, analysis tools, reporting and more. This was discussed in a previous blog article called “The advantages and disadvantages of using free online survey software”
Most survey data collection software systems are focused on collecting data online. This is fine if you only want to collect data online. Most of the free/budget systems will only collect data online; you will need to decide whether you want a system that can collect data using mixed modes or has a multimode capability.
Some samples are hard to reach online. Others work better with paper questionnaires – an event organiser told me that response rates are very good if questionnaires are completed as people leave a conference whereas online reminders tend not to work so well. Sometimes, giving respondents the choice of how they can take part in a survey will be important – this is particularly true where you have hard to reach samples. Paper questionnaires may be diminishing but it may be some time before they are redundant.
When MRDC Software started in 1992, most data came from paper questionnaires. The list has now grown to:
There’s a difference between mixed mode and multimode and it’s one to consider carefully if you want to use a mode other than CAWI (online surveys). True mixed mode, in my opinion, means that you can collect data by more than one mode (e.g. paper, CAPI and CAWI) AND have all the results available together in one merged file. If any data recoding is needed to pull together, say, the CAWI data and the paper questionnaires, I would call this multimode availability only. It can be an important difference.
Whether it is best to buy mixed mode or multimode software for your data collection will be dependent on your needs. I am not sure that there is any product that supports all the above modes. Depending on your work, you may decide that text surveys and mall surveys are not important, for example.
Besides the importance of the functionality of a software tool, the scope of each mode may need to be considered. Here’s a brief review:
Some software suppliers will claim that CAPI is available but you may need an internet connection to carry out the CAPI interviews. This may not be a problem if surveys are always being conducted in a venue with wifi or a stable internet connection, but otherwise it is likely to be a problem. The best CAPI software tools work offline or online. Usually, they will send the data to a central server if there is an internet connection, but store the data on the device if there is no internet connection or it is a poor connection.
CAWI surveys need to be easy to use for respondents. Some products will work equally well on any device, allowing for questionnaires to appear differently on each type of device (PC, tablet, smartphone, for example) if necessary. Others will be less flexible and not adapt to some devices well. For example, if a respondent needs to scroll to answer questions on a smartphone, it is likely that the respondent will give up more quickly.
Most CAWI surveys can be used for CATI surveys. However, full CATI systems will have tools to deal with call backs, appointments, supervisor control etc. This might not be important if you want to add a small number of telephone interviews, but most online survey platforms will not have full CATI functionality meaning that full mixed mode capability is limited.
Mall surveys can be carried out successfully at airports, exhibitions and other public places. These are self-administered using touch screens usually. If a respondent leaves midway through an interview, the system needs to be able to reset itself to the beginning of the interview for the next respondent.
Scanning still offers a good solution for processing large volumes of self-completion forms or questionnaires. Very few online survey platforms have a scanning capability.
Some surveys are carried out by the exchange of texts using SMS. These have limited use as the surveys need to be very short and concise and can incur phone tariffs.
And finally, we come to the oldest methodology – paper questionnaires. Being able to produce a paper questionnaire can be important.
Snap is the product we offer for true mixed mode capability. Data collected from any method can be stored together without any recoding or manipulation. Here’s a summary of what Snap offers:
Mixed mode - Yes
Multimode - Yes
CAPI: Yes – offline or online
CAWI – Yes
CATI – Yes, but using CAWI tools only
Mall surveys – Yes
Scanning – Yes
Text surveys – No
Paper surveys – Yes
If you need help with choosing and understanding the range of mixed mode options available within the research industry, you can contact me at email@example.com