8 Reasons Why Online Communities Fail – and, how we can help
At conferences, I’ve heard speakers say that you …Read more
Target weighting is a technique used in market research to scale a sample of respondents to a known or desired target or population. For example, you may have 40% males and 60% females but want 50% of both males and females. MRDC Software can provide you with a free working model in Excel if you email firstname.lastname@example.org.
The purpose of this article is to explain how target weighting works and to understand the effects of the weighting on your data. After you have applied weighting factors to your data, the effective sample size will usually be reduced.
The article only explains how target weighting works and some basic checks that you should apply.
If you want a working model in Excel, please email email@example.com and you can receive our free target weighting calculator which also give you the effective sample size. There is a video that shows you how to use the target weighting and effective sample size calculator.
If you want to know how to carry out target weighting using the free target weighting and effective sample size calculator, please see this video.
Target weighting allows you to scale survey respondents to a targeted number of respondents. To do this, you will usually be weighting respondents to one or more questions or variables. For example, you may wish to adjust your sample so that it represents 50% males and 50% females. If you survey sample had 40% males and 60% females, you would apply a weighting factor to the male sample of 0.83333 (50% divided by 60%) and to the female sample of 1.2 (50% divided by 40%). If these weighting factors are applied to the data, you will get a weighted sample which is 50% males and 50% females.
Target weighting is often carried out using a two-dimensional matrix. For example, gender within age group. For example, you may have the following targets:
Males 18-34 – 20%
Males 35-54 – 15%
Males 55+ - 15%
Females 18-34 – 15%
Females 35-54 – 25%
Females 55+ - 10%
You should note that weighting targets can be figures or percentages. The examples here use percentages but the working model allows either to be used.
In this example, if Males 18-34 were 15% of the sample, they would get a weight of 1.33333 (20% divided by 15%). The weights for the other five cells would be calculated in the same way.
The targets do not need to be a matrix – you could have four age groups for females and three for males, for example. However, it is important that the percentages add to 100% and that every respondent falls into one (and only one) of the categories.
Three (or more) dimensional weighting matrices are also possible, but care should be taken not to spread the sample too thinly as your effective sample size may reduce too much (see next section).
You will note that a factor is calculated for each cell of your weighting matrix by dividing target by the actual number of respondents.
The effective sample size is the sample which could have been achieved by an effective unweighted random sample. In other words, if your sample size of 200 has an effective sample size of 100, you could have achieved results that were as statistically accurate with a random sample of 100 that met your target criteria.
The effective sample size is an important figure that should always be checked when weighting survey data. An extreme example shows this. Let’s say your targets are 50% males and 50% females in a sample of 200 people, but you have 99% males and 1% females in your sample. When you apply weighting as described in section 2, your effective sample would be 7.92. In other words, your sample of 200 people has reduced to the validity of a random sample of 7.92 people.
What does this mean? It is easy to think that target weighting can be used to solve sampling defects. If it can, it comes with a penalty. It is good practice to observe the weighting factors being applied to each cell of your weighting target matrix, but, as an absolute minimum, you should always check (and preferably show) the effective sample size in any analysis you do.
If you have any further questions, please do not hesitate to contact me.
Just email firstname.lastname@example.org and I will send you a working example in Excel that you can use to calculate target weights and effective sample size yourself.