How to handle market research data analysis and tables

Like many computer-based tasks, data analysis and tabulations from market research surveys can often be easy but can become very time-consuming when things get even slightly more complex. Indeed, on several occasions, I have seen companies take what I believe to be 20 times more hours than necessary because they do not have the right expertise or the right tools to do the job. This guide discusses FIVE levels of software on the market for crosstabs and looks at TEN typical tabulation analysis tasks.

Before we start

By definition, these five levels overlap. A product in Level 2 may have a feature usually found in Level 4 and above; conversely, a Level 4 product may have a feature usually found in Level 3 missing, though this is less common. Less associated with the level of software are tools that sit around the basic cross-tabbing engine – for example, charting, importing and exporting data. Each product should be assessed separately for these peripheral tasks, with suitability subject to specific needs.

Software product levels

Level 1 – Basic only – you can crosstab one question by another, possibly with a neat drag-and-drop interface, but there are few options to control the output – cell counts and percentages only, usually.

Level 2 – Basic with some features -this broad category will allow you to control many table output requirements, such as decimal places, statistical options, simple weighting, grouping answers, and analysing questions by several banner points/categories. The user interface will usually be easy to use.

Level 3 – Comprehensive software – this category will have tools to produce most types of tables, including summary tables, most types of weighting, significance testing and more. If there are limitations, they may be handling complex data, all types of data, including hierarchical data, or exporting to other formats. The user interface may be more complex or have many options to navigate, making regular use desirable for best productivity.

Level 4 – Comprehensive software with power – this is a separate category as some software systems may be comprehensive in terms of features but lack the power to efficiently handle large volumes, such as large data sets, large volumes of tables or complex data structures, or data loops found in diaries or questionnaires with other repetitive sections.

Level 5 – Fully-featured and powerful – only a few products are in this category. They aim to allow you to produce more or less any output you need and have the power to process large projects. They generally use a scripting or programming language, which means they need regular usage to make the most significant gains. They will have features to deal with complex or repetitive tasks in an efficient way.

Beware the workaround

Before I continue, let’s talk about workarounds. I have seen software demonstrations where overclaiming has been used. There may be a way to achieve what you want, but it needs a workaround. I saw a recent demonstration where someone dropped data from their product into Excel, performed several tasks in Excel, and, hey presto, there was the required output. It was extremely cumbersome and painful to watch to produce one table. Imagine if there had been ten tables like this! Workarounds are great if they are simple and quick, but they often mean something is lacking in a software product.

10 tabulation analysis tasks explored

Let’s look at ten typical analysis problems and explore how each type of product measures up.

     1. The basics

In my view, the items listed below are the basics, although most Level 1 products will not have all these features, and most Level 2 products will have most. I would expect Level 3 (and above) products to have all these features.

• Filters (selection of subsets of data) may need to be applied
• Figure and percentage options (decimal places, vertical/horizontal percents)
• Ranking
• Table statistics (mean score, standard deviation, error variance, median etc.)
• Significance testing
• Able to cross tables by more than one question or variable
• Outputs as text, PDF, CSV or Excel
  

  2. Summary tables – summary of rating scales / mean scores

Summary tables are good ways to compare rating statements or summarise scores and volumes. They are a necessary part of crosstab analysis for most surveys. Level 3 and above products will have these tools usually, but only Level 5 may offer ultimate flexibility to present output exactly how you want and to make calculations on incremented tables. Summary tables of mean scores, net preferences (NPS) summaries, rating scales summaries, etc., are more likely to be possible from the higher-rated products

    3. Tables showing volumes/volumetrics

Sometimes, you want to produce tables based on volumes rather than people giving answers in a survey. Tables of this type are often called volumetric tables, showing figures in terms of money spent, fuel consumed, etc., rather than respondents. Some Level 2 products may offer basic tabulation outputs of this type, but Level 3 and above products will often be needed.

    4. Hierarchical data

Not many products handle hierarchical data. By hierarchical data, I am referring to data where there is more than one level of data – for example, where you interview a doctor and then ask the doctor about his last five patients, or maybe a different number of patients for each doctor. Here, you have two levels of data – doctors and patients. You could even have a third level as you could record each drug that each patient received, for example. For such tasks, you need advanced software. Surveys covering eating occasions, diaries or purchase occasions are usually similar. While some Level 4 products may handle this type of analysis, it is typically the domain of Level 5 products.

    5. Tracking study data

Tracking studies where the data is consistent are relatively easy to handle. I would expect all Level 2 and above products to handle such surveys with varying levels of efficiency. However, you may need to manage multiple data files from period to period and probably show trends. This might be a case where lesser software can handle such tasks, but a more powerful product will leave you better placed in the long term. Some Level 3 and Level 4 may have tools for tracking surveys, but they may be cumbersome to use or highly prone to human error.

    6. Tracking study data with questionnaire variations

Analysis of these types of projects can be challenging to manage, even though conceptually, it might seem easy. Products at the top end of the range tend to handle these types of tasks using powerful data handling tools, whereas products at the lower end tend to manage by editing and reformatting data to make the data uniform in some way. It’s an important question to ask potential suppliers if you have such projects. Inefficient solutions to these problems can require many times more staff time and have a higher risk of error.

    7. Data or tables requiring complex calculations or algorithms

Again, power wins, but with power comes greater learning. If you need one complex variable per project, it doesn’t matter too much if it takes 30 minutes to prepare rather than 5 minutes. However, if you have many complex calculations, algorithms and variables, more power becomes a necessity. Some Level 5 programs allow you to build templates or functions, making repeated use of complex requirements easier.

    8. Projects with target, factor or rim weighting

More and more products now offer target and rim weighting. However, some don’t, so check if it is a must-have. Most Level 2 products will have a basic target weighting function such as scaling to age group within region. However, if you need more complex variables as part of the weighting process, need tools to find empty cells, test or display effective sample size (often called weighting efficiency), you may need to move up the scale. Some Level 2 products will handle rim weighting, but having tools to deal with complications that can arise from rim weighting are far more likely to be available in Level 5 products.

    9. Projects where data from a database needs to be merged with survey data

Few products can merge data from an external source, or if they can, it is achieved by complex or awkward data manipulation. If this is a one-off task, it is not a problem. If you need to plug in an updated database every week, it really needs to be readable in situ. Again, many mid-range products would not have this facility.

    10. Data that needs to be available in other software

This topic is more difficult and focuses on one of my pet hates. In my view, modern software should make it easy to read data from other sources AND make data available readily for other sources. In other words, software should be as open as possible, with import and export flexible and straightforward. It’s becoming more critical as the number of types of software tools increases and the scope of market research widens. Some software products are good at reading data from other sources but then trap you in their system with no export tools, links or APIs; this is not good, in my view. Generally, the higher-level products are better, but there is no hard and fast rule, so it is best to check. As a minimum requirement, I expect market research software to have good links to and from SPSS, Triple-S, Excel, CSV, and ASCII data.

Using the software

The further up the scale you go, the more learning is required to use a product. Of course, the trade-off is efficiency. An inadequate product may either mean that you cannot deliver what you want or that it will take you many times more than some using a more advanced product. Further, complex projects, such as large tracking studies, can become considerably more cumbersome to manage as time goes on if they are not handled correctly using the right software. Good video tutorials can reduce the impact of using a higher-level product and are essential, in my view.

Where do MRDC Software’s products fit

    1. MRDCL

Our flagship product, MRDCL, a powerful scripting language, sits in the Level 5 category. No products match MRDCL’s power and scope for building variables and generating crosstabs. The competitors are small in number. Quantum is the oldest and a good competitor, but it has not been developed for many years and is no longer capable of many modern data processing needs. Meanwhile, although full of features, Merlin has ceased to progress over the last five years or more. Uncle is a third competitor, but it has made little progress over the previous ten years and has little information on its website. Ruby, a fourth competitor, has a good range of tools and comes with a hefty user interface. However, you must use an external programming language rather than tools within the product to get the greatest power. MRDCL does everything within its own system, making long-term maintenance of complex projects far more manageable. Read article link

    2. QPSMR

QPSMR is a friendly user interface that gives access to a subset of MRDCL’s capabilities. However, it uses the same engine as MRDCL, meaning that it is a Level 4 product, although some data management tasks require more effort than MRDCL would require. QPSMR is fully compatible with MRDCL.

    3. Snap

Snap is a data collection, analysis and reporting system all in one. It is a Level 3 to 4 product for analysis and can produce most user needs. It does not have access to a scripting language, but it is easy to transfer a project from Snap to MRDCL if you need to handle more complex requirements.

    4. Resolve

Resolve is our free secondary analysis tool, which means you can distribute products to colleagues or clients free of charge. You will need the cheapest MRDCL or QPSMR licence to generate Resolve files, but it means others can produce basic tables easily for Level 1 to 3 requirements. There is a Premium version of Resolve, which allows more power at a small fee; this is a Level 4 product.

What next?

Buying decisions for analysis software are difficult. You do not want a product that is unnecessarily complex to use or overly expensive, but at the same time, lower-level products can mean more staff time or an inability to meet client needs. We are probably not the right supplier if you are only looking for Level 1 software unless you want ways to distribute projects or a big project to many people. However, we are experienced in questioning and understanding your needs so that we can offer the right tools for you from our range of products. Feel free to contact us for a free consultancy by contacting nikki.sunga@mrdcsoftware.com