What’s the difference between case studies & surveys? Because they sample from a wide population, surveys can generalize findings to a large(r) population. Case studies, since they investigate relationships among discrete variables, aren’t as generalizable.
N = target population (the group for whom you hope your findings are generalizable)
n = sample population. n needs to be representative of the target population for external validity.
Confidence limits – the closer n is to N, the more precise your findings will be…the smaller the confidence limit…If your n is very close to N (around 70%), you don’t need to use the confidence limit equation. The percentage in the confidence interval refers to the number of times out of 100 that your finding is not due to chance. Also, you can only really use confidence intervals if you have a random sample.
**Random sample does not necessarily mean a representative sample.**
- True random sample – Number everyone in the population, then use a random number generator to select subjects.
- Systematic random sample – Number everyone in the sample, then pick, for example, every 3rd person do be in the study.
- Quota sampling – Representative, but not random.
- Stratified sample – Pick a segment (strata) of the population, and sample exclusively from it.
- Cluster sample – Used to study individual units within a larger population (e.g. pick 1 class to study all English 103 students). This is the least strong technique for making a compelling, generalizable claim for your findings.
To have strong findings from a survey, you should shoot for an 80-90% response rate. Anything less, and the sample is no longer representative and generalizable. There is self-selection bias–you’ll get polarized results at the 30% and less response rate. People will argue with your findings because they’ll say the only ones who filled out your survey had a grudge or were bored, etc.
n: k ratio – k are variables. for every k, you should have 10 n’s to get statistically reliable data. 1: 10. This is a common mistake in the published research which deals with surveys. Most studies will have many more variables than respondents.
What kind of data can you collect with surveys?
- So, you’ll be looking at measures of central tendency: mean, standard deviation.
Standard deviation – measure of agreement among respondents. A low SD will mean a high rate of agreement (i.e. people answered the question in exactly the same way). A high SD means a lot of disagreement among the respondents: a polarized situation.
Reporting the mean without the SD means that there’s no way of interpreting the specific responses. Therefore, when you report mean, you must also note the SD, for full study transparency.
One way to influence SD is the wording you use in the survey–the way you phrase the questions. Strive for neutral, unambiguous phrasing.