Phil Hearn: Blogger, Writer & Founder of MRDC Software Ltd.

Ways to organise market research data analysis and data processing

It’s very easy to set up a system for data analysis and processing and never change it. As a company, one of the things we like to do once a year is to question how we do everything. This covers everything we can think of – how new sales leads are handled, how we manage purchase invoices and expenses, what each customer might think of us, how impactful our software developments are, etc. You name it; we try to question it. It’s a good exercise and can involve everyone too.

Getting the right people doing the right work

 A client recently underwent a similar process regarding their market research agency’s data analysis and processing system. The manager observed that whilst he had an excellent team of data analysts, the team was over-skilled for many projects they handled. To a great extent, this had arisen because the company had taken on an increasing number of smaller, fast online projects. When I suggested that the overqualified staff probably did excellent or good work on complex projects but only average quality work on simple projects, my client hesitated for a second and then nodded in agreement. Most people do better work when they are challenged.

The risk of using lower-level software

The problem was that the data processing team had been set up to use Quantum, a leading scripting language of its era for survey analysis. Since then, they have switched to MRDCL, a similar product on the surface but with some significant differences. Indeed, both are powerful software products needing skilled staff to use them. However, a more straightforward product would suffice if the requirements are easy. The question was whether using a lesser product for some projects would improve productivity or complicate the process. The risk of using a lower-level product with less qualified staff is that client requests may escalate and go beyond the scope of the lower-level product.

Getting the best value from staff

One of the problems with survey analysis is that a particular project often becomes one person’s responsibility. This has three adverse effects. Firstly, it means that staff cannot share projects by helping out or covering during illness and holidays easily. Secondly, skilled staff often spend too much of their week doing simple work. Thirdly, costs for data processing are higher than they need to be.

Solution #1 – Use software where projects can be transferred to skilled staff when necessary

The main benefits of using a scripting language are that complex or repetitive requirements can be generated in a fraction of the time it takes with less advanced systems. You don’t want to lose that benefit, particularly if you have a heavy workload or tight time constraints. Our solution is that complex projects are handled in MRDCL, whilst more straightforward projects are handled using QPSMR. As QPSMR and MRDCL are compatible and use the same engine, an expert can take over a QPSMR project if the going unexpectedly gets tougher. Everything you do in QPSMR can be taken and used by an MRDCL user, as QPSMR effectively generates an MRDCL script and runs it for you. In other words, the same engine is running your analysis.

Solution #2 – Separate the difficult parts of the work from the easy parts

There are two main phases to producing survey analysis – defining the variables needed for analysis and specifying the tables you want from those variables. Often, though not always, the variable definition task is the more complex part of the work. Therefore, it can make sense for many projects to script the variables in MRDCL and then move the project into Resolve, the free tabulation analysis tool, so that less skilled data processing staff or researchers produce their own tables. If there are a couple of complex tables, the MRDCL team can write some script that can be included in the Resolve table run or run them separately.

Solution #3 – Use Excel Productivity Scripting (EPS)

The third solution is to use the powerful EPS tools within MRDCL. EPS is a technique within MRDCL where you can build templates in Excel to store instructions that MRDCL processes. The technique needs some upfront work to build the templates and handle everything you need, but once in place, you have a customised system that controls requirements through an easy Excel interface. And, what’s more, the methodology takes advantage of one of Excel’s great strengths: it keeps stuff organised and easy to understand. If some special MRDCL script is needed to specify any variables or tables, the MRDCL analyst can easily include these in the program. It means that less skilled staff can do most of the work, even complex or repetitive requirements, while more qualified team members spend a greater proportion of their time doing what they are good at.

Solution #4 – Working in partnership with MRDC to make use of EPS

One of the criticisms of EPS is that it takes a lot of learning to implement tools for less skilled staff to utilise. There is an element of truth to this, although many MRDCL users have been surprised that the steep learning curve is just a short learning curve. There are only five or six techniques to learn before a whole range of new possibilities present themselves. However, suppose you only need to use EPS occasionally or have fairly standard data processing needs. In that case, we are always happy to partner with clients to put a system in place that meets exact needs and improves productivity.

Reappraising how work is handled

Let’s step back now and summarise what I am advocating. I think it is essential to be in a position where, on a project-by-project basis, you can choose the most effective approach. A one-size-fits-all approach can sometimes work, but data analysis and data processing needs are becoming more disparate rather than being standardised for most agencies. Sometimes, having one system for all projects might look attractive and easier to manage, but whether you are getting the best from your team, giving your clients the best service and fulfilling your staff’s aims is another matter.

Why MRDCL, QPSMR and Resolve

We believe that our combination of products – MRDCL, QPSMR, Resolve Base and Resolve Premium offer more choices than any other products on the market. Most products will have strengths for one type of survey or analysis, but our products give you the flexibility to ensure high productivity across all surveys, from the most challenging tracking studies to small online surveys and from large-scale surveys with large samples to small, complex studies.