Understanding Crowdsourcing and Sample Sizing

If you look at the above calculation, which was published recently by the Survey Monkey organization, you may be easily scared away from from any discussion of crowdsourcing statistics, such as calculating a "sample size". We can't blame you, when we, too, think back about the terror of attending Stats class in college. But, please hang in there and read further about this topic! For one thing, Survey Monkey has come to the rescue with their handy sample size calulator which you can access it here

But, let's dig a little deeper and cover some of their other interesting information which is very relevent to us in the world of industry-driven crowdsourcing and benchmarking. First of all, let's place some context in place on our use of the phrase, "industry-driven". This phrase, which we use all the time, means that the individuals who are a part of a sample survey population are segmented by either NAICS or SIC code. This places the survey results into a specific industry vertical. When that industry vertical is combined with similar industry verticals, we call that merged survey population an "industry cluster". An industry cluster survey population might also be relevant for our Executive Think Tank surveys if we're also using the NAICS and / or SIC codes as segmenting vehicles. 

OK, now that this context has been set, let's look at the basics of survey statistics (note: surveys are also known as "crowdsourcing" projects) to refresh your memories of "stats". "How many people do you need to take your survey? Even if you’re a statistician, determining sample size can be tough", says Survey Monkey. They go on to help us with the following information...

What is a sample size?

The number of completed responses your survey receives is your sample size. It’s called a sample because it only represents part of the group of people (or population) whose opinions or behavior you care about. As an example, one way of sampling is to use a so-called “Random Sample,” where respondents are chosen entirely by chance from the population at large.

Understanding sample sizes

Here are a few key terms you’ll need to understand to calculate your sample size and give it context:

Population size: The total number of people in the group you are trying to reach with your survey is called your "population size". If you were taking a random sample of people across the United States, then your population size would be about 317 million. Similarly, if you are surveying your company, the size of the population is the total number of employees.

Margin of error: A percentage that describes how closely the answer your sample gave is to the “true value” is in your population. The smaller the margin of error is, the closer you are to having the exact answer at a given confidence level.

If you want to calculate your margin of error, check out Survey Monkey's margin of error calculator.

Confidence level: A "confidence level" is a measure of how certain you are that your sample accurately reflects the population, within its margin of error. Common standards used by researchers are 90%, 95%, and 99%.

As an example, say you need to decide between two different names for your new product. By your estimates there are 400,000 potential customers in your target market. If you decide that the industry standard of 3% margin of error at a 95% confidence level is appropriate, then you will need to get 1065 completed surveys.

Calculating your sample size

If you’d like to do this sample size calculation by hand, use the above calculation and the following formula variables:

  • Population Size = N  

  • Margin of error = e

  • z-score = z
  • e is percentage, put into decimal form (for example, 3% = 0.03).

  • The z-score is the number of standard deviations a given proportion is away from the mean. To find the right z-score to use, refer to the table below:

  • sample size calculation - z-scores

Things to watch for when calculating sample size

A smaller margin of error means that you must have a larger sample size given the same population.

The higher your confidence level, the larger your sample size will need to be.

Tips for using Survey Monkey's sample size calculator

If you are making comparisons between groups within your sample, you will need to take that into account when calculating sample size. If, for example, you break your sample out into two groups of equal size, your sample size for each group is cut in half and your margin of error will increase. This can make it difficult to make meaningful comparisons between groups in your survey.

Returning to the scenario from earlier, your have a population of 400,000 potential customers, and you need 1065 respondents to get to a 95% confidence level with a 3% margin or error. If you wanted to see how the opinions of women and men differ (presuming they each make up 50% of the sample), you would wind up with a sample size 533 of for a population of 200,000. With those numbers, your margin of error would go up—or you need to increase your sample size.

Let us know what we might have missed in this discussion and we will post it here.


Post new comment

The content of this field is kept private and will not be shown publicly.