Sampling & Study Design

Population vs sample

• Population: the entire group you want to learn about.
• Sample: the subset you actually measure.
• Parameter: a number describing the population (like the true mean $\mu$) — usually unknown.
• Statistic: a number computed from the sample (like $\bar{x}$) — used to estimate the parameter.

We sample because measuring everyone is usually impossible. The whole game is getting a sample that represents the population.

Why randomness matters

Random selection protects against bias and lets us actually quantify uncertainty. Without it, hidden patterns in how you chose can quietly distort everything.

Sampling methods

• Simple random sample (SRS): every possible group of size $n$ is equally likely — like drawing names from a hat.
• Stratified: split the population into similar groups (strata), then randomly sample within each. Good when the groups differ from each other.
• Cluster: split into clusters (often by location), randomly pick whole clusters, and measure everyone in them. Cheaper to run.
• Systematic: order the list and take every $k$-th item after a random start.
• Convenience (not random): whoever is easiest to reach. Prone to bias — avoid it for real inference.

Bias — the real enemy

Bias systematically pushes estimates away from the truth.

• Undercoverage: some groups can't be selected (a landline survey misses people without landlines).
• Nonresponse: selected people don't answer, and non-responders differ from responders.
• Response bias: question wording or social pressure skews the answers.
• Voluntary response: only people with strong opinions reply (most online polls).

The crucial point: a bigger sample does not fix bias. It just gives you a more precise version of the wrong answer.

Observational study vs experiment

• Observational study: you measure without intervening. It can show association, but not causation, because of lurking variables.
• Experiment: you actively impose a treatment and compare groups. A well-designed experiment can establish causation.

Principles of a good experiment

• Control: compare against a control or placebo group.
• Randomization: randomly assign subjects to groups to balance out lurking variables.
• Replication: use enough subjects to detect a real effect.
• Blinding: keep subjects (and ideally the evaluators) unaware of group assignment. Double-blind removes expectation effects on both sides.

Example. To test a drug, randomly assign patients to drug vs placebo (randomization + control), with neither patients nor doctors knowing which (double-blind), using enough patients to see an effect (replication).

Key takeaways

• Statistics estimate parameters; randomness is what makes the estimate trustworthy.
• Know SRS, stratified, cluster, and systematic; avoid convenience and voluntary-response samples.
• Bias is systematic error — a larger biased sample is still biased.
• Only a randomized, controlled experiment can establish causation.