Efficient vs Inefficient Market Research Data Processing
I am sometimes shocked to see how inefficiently s…Read more
We like to be open our pricing. This article is aimed at helping you understand what things will affect the price of handling a cross tabulation project for you. Here’s the big five things that have the biggest impact on 90% or more of the projects we handle for clients. As we have handled crosstab projects ranging from £50 to £25,000, here are the key indicators.
Most research and commercial can be supplied to us in ASCII format using codes to represent each answer. ASCII data can be edited or viewed in most editors, including free tools such as Notepad. This is easy for us to work with and, provided that a map of the data is available, it will be easy to navigate. ASCII data that contains all or much of the data as text fields will be harder to work with.
Triple-S or SPSS are two other standard formats that make it easy for us to import the data into our software so that we can start to prepare your cross tabulations. As long as they are annotated properly, they are easy to work with.
Data supplied in Excel format or as a CSV file are usually easy to work with. The same conditions for ASCII data apply to Excel data – data as codes is easier to work with than free texts.
For all the above, it is easy for us to work with your data. However, if data is provided in some proprietary format or is difficult to understand without a lot of questions, costs will start to increase. You can minimise these costs by providing concise, clear documentation.
Some projects will have more complex data. Most data sets in market research have one record per respondent, but more complex data structures are sometimes used. Examples of these would be:
All of these types of data issues will add cost to the project. On a small project, they can represent a significant percentage of the total project cost. On a larger project, they will impact less.
Some projects that we handle have all or most of the texts with the data provided. For example, Triple-S files contain the data and, usually, texts relating to a project. This minimises the amount of work we have to do. SPSS files can also help, although multiple response questions are not processed efficiently in SPSS, which means that data from SPSS can be less easy to work with if there are a large number multiple response questions.
If the texts are available in Excel format, we can usually read the text directly from the worksheets without the need to re-enter. Sometimes, texts supplied in text files or in Word can also be used.
If we just receive a printed questionnaire, it is likely that we will need to re-enter all of the texts for you. This can be a labour-intensive task, particularly on long questionnaires where different texts are used for most questions. On a complex project that needs senior tabulation script writers, costs can increase if you are effectively paying a highly skilled person to enter texts.
We are specialists in handling tracking studies and multi-country projects. These types of projects typically have similar questionnaires and data files – from wave to wave in the case of a tracking study – from country to country in the case of multi-country studies. How this is managed within the data can have a huge effect on project costs. There is a separate blog article on this, but we would recommend that you speak to us at the earliest opportunity if you want us to help with these types of project. The cost of taking over and re-writing a badly managed project of this type can be high.
This is a more obvious metric – the more variables or tables that there are, the more that they will cost.
However, there are two important things that can increase or decrease costs. If there are a lot of repetitive variables or tables, they will be much cheaper to produce than lots of differently structured variables or tables. A questionnaire that has a large number of rating scales that all need to be analysed the same way can be very cheap to prepare and process.
For some projects, a lot of derived variables are needed. These may take time to prepare and check. If you require tabulations which show data as the questions are on the questionnaire, this will reduce costs.
Tabulating a series of questions by one standard banner (or crossbreaks) is usually the cheapest way to produce a volume of crosstabs. Costs increase where there are summary tables, complex calculations, precise table layouts and where weighting is used. Generally, keeping your requirements to a pattern or style will reduce costs. If every table needs to be crafted differently, this will increase costs.
This might sound like a rude thing to say – and I apologise in advance. But, clients who provide clear, concise instructions and know what they want can enjoy much lower prices. If you see a run of crosstabs as an experiment before you produce the final set of crosstabs, costs will be much higher.
By clear, concise instructions, I mean that you should specify anything that is not obvious, but don’t specify anything that is obvious or can be explained easily by email, phone or as a standard instruction.
Whether this is the first project you have worked on or you have worked with different agencies, it is always best to talk about your project before it happens. We may be able suggest ways that you provide data, analyse your data or instruct us which will make a big difference to the cost of the work. Just ask!