How to handle data from multi-country surveys

From a data processing perspective, multi-country surveys can range from being easy to handle to being highly complex. Two things, however, can make a big difference: planning and using the right software. Some of the main problems are:

• Managing and dealing with different country/regional stakeholders
• Handling multiple languages
• Dealing with differing questionnaires
• Dealing with different data structures
• Producing combined analysis and reporting
• Automating analysis and reporting

So, let’s unpack these problems and look at how to mitigate them and minimise their potential impact. Let’s start with the ideal solution.

The ideal solution

The most straightforward multi-country surveys are those where the questionnaire is the same in every country, the questionnaire is in the same single language, and the data is stored in one format, using the same data fields or data locations. If all those things are possible, your multi-country is probably no more complex than any other survey of a similar size. In reality, most multi-country surveys will be more difficult for one or more of these factors, potentially making the project significantly more time-consuming. Let’s dig deeper.

The software you need

Without the right software, the problems above can be, at best, challenging to handle and prone to error or, at worst, almost impossible to manage given the deadlines we all work to. While other products can address some of these complications well, we believe that our suite of products is cost-effective and capable of dealing with all these potential complications well.

1. A brief introduction to our products and the solutions you need

The three products I will discuss are Snap and QPSMR for managing data collection and analysis and MRDCL, our tabulation and reporting scripting software, for analysis and automation. Let’s look at the main issues more fully.

       a. The questionnaire – different languages

Whether face-to-face, CAPI, CATI or online, questionnaires often need to be available in multiple languages. Snap makes it very easy to input translations and apply a chosen language by having an initial question identifying the required language. QPSMR makes it easy to produce translations by dropping texts into a spreadsheet so that you can make translations. Applying the translations for paper questionnaires is easy but requires some work for CATI surveys.

       b. The questionnaire – different questions

Adding questions only applicable to one or selected countries is easy enough in most survey platforms. It is as simple as adding a question and applying a filter for the country or countries in which the question needs to be present. Problems can start to arise when you analyse the data. For example, data maps/locations may vary from country to country, particularly if different fieldwork agencies have been responsible for generating a local questionnaire. Further, tabulations and analysis can be cumbersome, as you may need different tables for each country’s report. All our products handle these problems well, but MRDCL has some powerful tools which allow you to produce individual outputs for each country whilst maintaining one script to control everything.

       c. The questionnaire – different response codes/brand lists

Now, things start to get more complex. It’s common in a multi-country survey to want different response lists for some or many questions in each country. There are obvious things, such as demographics, that are subtly different for each country and brand lists, which may be similar, overlap partially or be entirely different. Though conceptually straightforward, the logistics of managing these issues can become complex, particularly for tracking studies in areas where brand lists, for example, change regularly or there are often new sub-brands. Snap and QPSMR are both good at holding master response lists, but MRDCL can manage this through an Excel template that can contain all the information MRDCL needs to automate your country and combined country or global reports. This feature alone can save you hours, if not days, of your time.

       d. The data – different fields/data locations

When data is collected from more than one country, particularly if more than one fieldwork agency or team is involved, the likelihood of different data layouts and differently named data fields becomes almost a certainty. You need tools to deal with these differences and easily viewable systems that anyone can handle, regardless of staff illness or someone leaving. MRDCL handles this aspect of multi-country data processing exceptionally well, not relying on ‘black boxes’ to sort out country differences. In extreme cases, I have seen such tasks cut by 90% once you use the right software.

       e. The analysis – aligning data

I am always fascinated by how staff underestimate the time they spend manipulating data that is similar but different from several countries. Recoding data to a single format may work if the differences are minimal, but the task can grow exponentially with each difference. Recoding data manually or reformatting it to a single format is highly error-prone, especially if it is controlled by complex programming code. MRDCL handles this by using a viewable template that allows you to merge data how you need to, however complex. The advantage of this approach is that it takes it away from programmers and scripters and puts it in the hands of administratively efficient people. Further, it is self-documenting, so spotting errors and other glitches is easier.

       f. The analysis – automating analysis

Once you have prepared your analysis, you want it to be as easy as pressing a button or two to produce all the reports you need. Of course, this may not be an issue if you only need a combined report for all countries. However, you may want a report for each country, each group of countries, and a global report. Further, if you are handling a tracking study, you may need outputs for different periods within these geographical reports – for example, a yearly, quarterly and monthly report. You need software that automates this for you for this type of work. MRDCL has these tools, with additional tools to automatically output your data to other formats in an easily accessible format for further analysis or to provide data for internal databases. Your customers will not want to face any of the problems cited above, so you need MRDCL to streamline this process and provide customers with data that is ready to use.

       2. Managing a multi-country survey from a DP perspective

There are two points in running a multi-country survey where you can simplify processes and improve efficiency. These are pre-data collection and post-data collection. Let’s look at each.

       a. Pre-data collection management

In almost all cases, the more you can align your project before data collection, the easier it will be to process. One way to simplify analysis and reporting is to ensure that data is stored in a uniform format by storing questions that only appear in some countries in reserved data locations or fields. Further, unifying brand/response lists wherever possible will make life easier. QPSMR and Snap both manage country differences well by giving you complete control over how your data is stored. This is not true of all software products.

       b. Post-data collection management

Sometimes, you have little or no control over how data is available for different countries. Aligning the data is vital for two reasons – firstly, it will make analysis more straightforward and secondly, providing data for a later process will be more viable. Arguably, this second reason is gathering greater importance; there is an ever-increasing need to put data into dashboards, provide to clients in a database or output to another product such as SPSS. In short, data mobility is becoming a prerequisite. This prerequisite means that the analysis software shouldn’t ‘patch round’ these complexities but allow you to manage variables so that they can be redeployed with minimal effort and delay. MRDCL is, in our view, the best tool on the market for this type of task.

       c. Maintaining your systems

If you repeat a multi-country survey or it forms part of a tracking study, there will be a need for some good housekeeping. Such projects can start well-organised even if you use an ideal tool like MRDCL. I have seen multi-country tracking studies that look well-organised after two or three waves but suddenly become a tangled mess after four or five waves. Such projects can change beyond expectations after two or three waves. I believe the only solution is to allocate time (and a budget) for maintaining the project. It is easy to plough on with projects that become messy because staff can find workarounds. It doesn’t take long for these workarounds to become a tangled mess. Of course, this is less likely to happen if you start the project using MRDCL, but maintenance is an integral part of any long-term project work.

       d. Start with the right tools

If there is a ‘call-to-action’ here, it is to encourage you to use the right software from the start if you have a multi-country survey to handle. Some software products may have some good ways of getting around knotty problems. However, their shortcomings start to unravel as the intricacies, often conceptually simple, make project management unwieldy or, worse still, impossible to handle. If you want help with your multi-country projects, contact nikki.sunga@mrdcsoftware.com.