Course 11: The Importance of Quality Sample Size

When conducting research, quality sampling may be characterized by the number and selection of subjects or observations. Obtaining a sample size that is appropriate in both regards is critical for many reasons. Most importantly, a large sample size is more representative of the population, limiting the influence of outliers or extreme observations. A sufficiently large sample size is also necessary to produce results among variables that are significantly different.(1) For qualitative studies, where the goal is to “reduce the chances of discovery failure,” a large sample size broadens the range of possible data and forms a better picture for analysis.(2)

Sample size is also important for economic and ethical reasons. As Russell Lenth from the University of Iowa explains, “An under-sized study can be a waste of resources for not having the capability to produce useful results, while an over-sized one uses more resources than are necessary. In an experiment involving human or animal subjects, sample size is a pivotal issue for ethical reasons. An under-sized experiment exposes the subjects to potentially harmful treatments without advancing knowledge. In an over-sized experiment, an unnecessary number of subjects are exposed to a potentially harmful treatment, or are denied a potentially beneficial one.”(3)

Theoretical Case Study: Dangers of Small Sample Size

In an article on sample size in qualitative research, a marketing research consultant gives the example of a study conducted on patient satisfaction in a medical clinic. The medical clinic has one staff member known to aggravate 1 out of every 10 patients visiting. A research budget permits only one focus group with 10 clinic patients, and all respondents report feeling satisfied with their visit. However, when performing data analysis, it is critical to consider the population represented by a study of only ten patients. The probability that the sample failed to include an unsatisfied patient is calculated to be 35%. In other words, approximately 1 in 3 random samples of ten patients would overlook the actual statistic of aggravation (1 out of every 10 patients). To see how this calculation was performed, visit: http://www.icology.co.uk/qualitativesamplesize.html.(4)

Determining Sample Size:

There are many different ways to determine an appropriate sample size. For in-depth qualitative studies, Abbie Griffin and John Hauser found that “20-30 in-depth interviews are necessary to uncover 90-95% of all customer needs for the product categories studied.”(5) Thus, the authors determined that a sample size of 30 respondents would provide a reasonable starting point. This number is corroborated by Dr. Saiful, a clinical researcher, who states that a “sample size larger than 30 and less than 500 are appropriate for most research,” adding that sub-samples also require at least 30 observations when applicable.(6)

Determining the exact sample size necessary for a study usually requires extensive statistical calculations. However, a reasonable sample size acceptable in most studies utilizes the calculated margin of error. An estimation of margin of error at 95% confidence level (where there is only a 5% chance that the sample results differ from the true population) is given by 1/√N, where N is the number of participants or sample size. This means that a sample size of 10 would have a 31.6% margin of error (1/√10=0.316).

To demonstrate this calculation through example, we can walk-through a study on fear of heights. If researchers survey 10 people and find that 6 respondents are afraid of heights, this means that there is a 95% chance that between 2.8 (6 – 3.16) and 9.2 (6 + 3.16) of the population is actually afraid of heights. With such a large range, the data is not very conclusive. However, if the researchers survey 100 people, the margin of error falls to 10%. Now, if 60 participants report a fear of heights, there is a 95% chance that between 50 (60 – 10) and 70 (60 + 10) of the population actually has a fear of heights. The greater N is, the smaller the margin of error and more useful the measurable results.(7)

In addition to the yield of statistical significance and confidence in results, quality sample size must consider the rate of response. Incomplete or illegible responses are not useful observations. Thus, the total sample size must account for these potential issues.(8)

Methods of Sampling:

Purposive Sampling

A common strategy for sampling in qualitative research studies, purposive sampling places participants in groups relevant to criteria that fits the research question. Factors that affect sample size include available resources, study time, and objectives. However, sample sizes are also determined by the concept of “theoretical saturation,” or “the point in data collection when new data no longer bring additional insights to the research questions.”(9) Generally, studies that use purposive sampling have a target number of participants, rather than a set requirement.

Quota Sampling

Quota sampling predetermines the number of participants desired. While designing the study, researchers may determine sample size, along with appropriate proportions of subsamples, when identifying participants of certain characteristics. With this criteria, researchers can then recruit participants appropriate to the “location, culture, and study population…until [meeting] the prescribed quotas.”(10)

Snowball Sampling

This third type of sampling uses existing participants or contacts to reach their social networks and refer the researcher to other potential participants. Snowball sampling helps to recruit “hidden populations” that may not be found from other methods of sampling.(11)

Strategies to Obtain a Quality Sample:

• Focus your study. Ensure that the research question is feasible and the study answers a question with clear variables.

• Find a representative sample. Determine the essential inclusion and exclusion criteria for the study population such that findings can accurately generalize or specify results to the target group.(12)

• Determine a recruitment strategy. Specify a plan to identify and enroll study participants. This may involve screening or establishing criteria for number, location, and sampling method.

• Consult with the community to identify and recruit potential participants. Examine existing infrastructure to find venues of contacting appropriate samples. Researchers must also be respectful and responsive to the advice of local experts and leaders.

• To avoid selection bias, it is also necessary to recruit an appropriate comparison group.

• Do not give up after the first attempt to recruit a potential participant. Follow-up using various communication strategies, including personal, written, or electronic messaging. “The significance of personal contact should not be underestimated.”(13)

• Allow for flexibility in the process. If current recruitment strategies are resulting in “inadequate enrollment” or modifications are made in criteria of participants, the solution may be to alter the sampling plan.

Case Study: Participant Recruitment in Reproductive Health Research in India

A study conducted in India on reproductive health found that when female recruiters approached patients in the waiting room of an outpatient OB/GYN clinic, only 23% of those screened were eligible for the study. Overall, only 7% of those screened enrolled.

When recruiters adopted an alternative method to utilize community resources and networks to find participants, they found greater success. By inquiring within women’s microeconomic self-help groups, 87.9% of those screened were eligible for the study. Of the women screened, 85.2% enrolled in the study. Moreover, those recruited in community clinics had higher retention rates and were more likely to attend their first follow-up visit. In particular, 97% of recruitments from community groups attended their first-follow up, while only 72% of participants recruited from the clinic attended. The drastic differences in enrollment and retention between the two methods suggests that a “community-supported recruitment process may facilitate access to young women in the community, increase general knowledge and health seeking on reproductive health issues, and produced better overall study retention.”(14)

The study suggests reasons for low recruitment through clinics as patient fear of healthcare settings, barriers to transportation, social stigma from attending the clinic, and restricted female autonomy. While sociobehavioral research may use findings to explore such issues, this case study demonstrates the value of sampling strategies, including the employment of community infrastructure and the need for flexibility throughout the sampling process.(15)

Footnotes

(1) Patel, M., Doku, V., and Tennakoon, L. “Challenges in recruitment of research participants.”Advances in Psychiatric Treatment. 9. (2003)

(2) DePaulo, P. (2000). Sample size for qualitative research. Quirks Marketing Research Review, 1202.

(3) Lenth, R. V. (2001). Some practical guidelines for effective sample size determination. The American Statistician, 55(3), 187-193.

(4) DePaulo, P. (2000). Sample size for qualitative research. Quirks Marketing Research Review, 1202.

(5) Ibid.

(6) Ibid.

(7)“Sample Size: How Many Survey Participants Do I Need?”https://www.sciencebuddies.org/science-fair-projects/references/sample-size-surveys.

(8) Patel, M., Doku, V., and Tennakoon, L. “Challenges in recruitment of research participants.”Advances in Psychiatric Treatment. 9. (2003).

(9) Mack, N., Woodsong, C., MacQueen, K., Guest, G., and Namey, E. “Qualitative Research Methods: A Data Collector’s Field Guide.” Family Health International. https://www.fhi360.org/sites/default/files/media/documents/Qualitative%20Research%20Methods%20-%20A%20Data%20Collector's%20Field%20Guide.pdf

(10) Ibid.

(11) Ibid.

(12) Ibid.

(13) Patel, M., Doku, V., and Tennakoon, L. “Challenges in recruitment of research participants.”Advances in Psychiatric Treatment. 9. (2003).

(14) Krupp, K., et. al. “Novel recruitment strategies to increase participation of women in reproductive health research in India.”Global Public Health. 2.4 (2007).

(15) Ibid.