- The right sample size for a research project depends on the desired accuracy
- It is important to know how the results will be analyzed when setting a sample size
Note: In this blog “sample size” refers to the number of completed interviews.
How big does your sample size need to be to get results that accurately reflect your target population?
The answer depends on two things:
1. How accurate do you want your answer to be? (This is the confidence interval.)
2. How confident / certain do you want to be of the accuracy? (This is the confidence level.)
The confidence interval is the “+/- “figure reported in survey results. A commonly-used confidence interval is 3%. This means that if the sample of people has given an average answer to a question of 54%, you can be reasonably certain that the answer you would have obtained had you asked the entire population that question (the unknown “true answer”) is between 51% and 57%. (54% +/- 3%)
How sure can you be that this would be the case? That is determined by the Confidence Level. A commonly-used confidence interval is 95%. This means that in 95% of cases, if you re-surveyed using a fresh sample of the same size, that sample should give an answer within the confidence interval.
To summarize, you may want your answer to be accurate to within 3% points either way, and may want to be 95% certain that the answer you get is within that +/- 3% range. If you want to be 99% certain, or if you want to be accurate to within less than 2% points either way, your sample size needs to be larger.
The confidence interval is almost entirely dependent on the sample size for the project. It is marginally affected by how positive or negative the answer is (the confidence interval is narrower as the answer approaches 0 or 100). It is not affected at all by the population size from which the sample is drawn, unless everyone in the target population is being interviewed. Whether you are interviewing in a small country or a large country the desired sample size is the same.
As the sample size gets bigger, the accuracy improves. However, the marginal gains are small. As a rule of thumb, it is necessary to quadruple the sample size to cut the error in half.
When considering the right sample size for a research project, it’s important to think about how you will analyze the data after you have the results. Think about the groups you are likely to analyze and report on separately, and make sure you have adequate sample in those groups to provide the accuracy and confidence you need.
Finally, remember that of course this is all about probability. Just because you say you are “95% sure that the true answer lies between 42% and 50%” does not mean it is so. The answer could in theory be anything.
That is why statistics do not prove anything. They can only tell us the probability that something is so.
For more information on calculating sample size, contact us.