Research Methods in Political Science

Units

1. 1

   The scientific study of politics

   4 lessons

   1. Politics as a scienceDistinguish empirical from normative questions, state Popper's criterion of falsifiability, and describe the cycle of the scientific method from theory to hypothesis to observation and back.
   2. Theories and hypothesesDefine a theory as a general causal explanation and a hypothesis as a testable statement relating an independent variable to a dependent variable, and write directional hypotheses about turnout as arrow diagrams.
   3. Four hurdles to causalityApply Kellstedt and Whitten's four causal hurdles, a credible mechanism, ruling out reverse causation, covariation between X and Y, and controlling for confounders Z, to the claim that education causes voting.
   4. Good research questionsJudge a research question by three tests, a puzzle, real-world importance and a contribution to the literature; distinguish deductive from inductive reasoning; and use a literature review to frame a project.
2. 2

   Concepts and measurement

   4 lessons

   1. Conceptualization and operationalizationMove from a concept to a definition to indicators to a measure, operationalize democracy, political knowledge and ideology, and explain why every measure is a choice that can change a finding.
   2. Levels of measurementClassify political variables as nominal, ordinal, interval or ratio, with party identification, Likert scales, feeling thermometers and vote share as examples, and say which statistics each level allows.
   3. Validity of measuresDistinguish face, content, construct and criterion validity, and explain systematic measurement error and bias with examples from measuring corruption, democracy and turnout.
   4. Reliability of measuresExplain test-retest and inter-coder reliability and random measurement error, and use the target picture to separate measures that are reliable but not valid, valid but not reliable, both, or neither.
3. 3

   Research design

   3 lessons

   1. Experimental and observational designsTell experimental from observational studies, describe cross-sectional, time-series and panel designs, and choose a unit of analysis without committing the ecological fallacy.
   2. Internal and external validityDefine internal validity and its main threats, define external validity and generalization, and explain the trade-off between them across research designs.
   3. Selecting cases and unitsCompare random and purposive selection, explain why selecting on the dependent variable biases results as King, Keohane and Verba argue, and recognize survivorship bias.
4. 4

   Experiments

   3 lessons

   1. The logic of experimentsDefine potential outcomes, state the fundamental problem of causal inference, and show how random assignment makes treatment and control groups comparable so that the difference in means estimates the average treatment effect.
   2. Lab, field and survey experimentsDistinguish laboratory, field and survey experiments, describe Gerber and Green's get-out-the-vote field experiments, and explain how list experiments and conjoint designs measure sensitive attitudes and multi-attribute choices.
   3. Noncompliance and attritionDistinguish the intention-to-treat effect from the complier average causal effect under one-sided noncompliance, compute the CACE as ITT divided by the compliance rate, and recognize attrition, spillovers and limits on the external validity of experiments.
5. 5

   Surveys and sampling

   3 lessons

   1. Sampling methodsDistinguish population, sampling frame and sample, compare simple random, stratified and cluster sampling, and explain nonprobability samples, nonresponse and weighting.
   2. Sampling error and margin of errorDescribe the sampling distribution of a proportion, compute the standard error sqrt(p(1-p)/n) and the 95 percent margin of error 1.96 times the standard error, and work the poll of 1,000 at 50 percent, which gives about 3 points.
   3. Questionnaire designIdentify problems of question wording, including double-barreled and leading questions, explain response-order and question-order effects and social desirability bias, and describe how likely voter screens work in election polling.
6. 6

   Case studies and process tracing

   3 lessons

   1. Types of case studiesDistinguish typical, deviant, crucial, most-likely and least-likely cases in the schemes of Gerring and Eckstein, and say what each type can establish about a theory.
   2. Process tracing testsDefine process tracing, and use Van Evera's and Collier's four tests, straw in the wind, hoop, smoking gun and doubly decisive, in a two-by-two table of necessity and sufficiency to judge evidence for a causal claim within a case.
   3. Interviews, archives and fieldworkDescribe elite and semi-structured interviews, archival research and ethnography, explain how to choose informants and triangulate sources, and discuss positionality in qualitative fieldwork.
7. 7

   Descriptive statistics

   3 lessons

   1. Distributions and central tendencyDescribe a political variable with a frequency table and a histogram, compute the mean, median and mode, and explain how skew, as in income, pulls the mean away from the median.
   2. Measures of dispersionCompute the range, interquartile range, variance and standard deviation step by step on a small dataset, and read the boxplot as a picture of spread.
   3. Cross-tabulationBuild a crosstab of two categorical variables, compute column percentages correctly with the independent variable in the columns, and describe the relationship in words.
8. 8

   Statistical inference

   4 lessons

   1. Probability and the normal distributionUse the normal curve, z-scores and the 68-95-99.7 rule to find probabilities, and explain with the central limit theorem why sample means are approximately normal even when the data are not.
   2. Confidence intervalsConstruct a 95 percent confidence interval for a mean, as the mean plus or minus 1.96 standard errors, and for a proportion, and say what the interval does and does not mean.
   3. Hypothesis testingState null and alternative hypotheses, compute a test statistic and p-value, use the 0.05 significance level, distinguish Type I and Type II errors, and work a difference-of-means t-test by hand.
   4. The chi-square testCarry out the chi-square test of independence for a crosstab, computing expected counts, the statistic the sum of (O minus E) squared over E, the degrees of freedom and the p-value.
9. 9

   Correlation and regression

   4 lessons

   1. CorrelationRead scatterplots, compute covariance and the Pearson correlation coefficient r on a small dataset, interpret r from minus 1 to 1, and explain why correlation is not causation.
   2. Bivariate regressionFit the ordinary least squares line Y = a + bX by minimizing squared residuals, interpret the slope and intercept, and read R-squared, using an example of GDP and democracy.
   3. Multiple regressionAdd control variables to a regression, interpret coefficients as holding other variables constant, use dummy variables and interactions, and explain omitted variable bias and its sign formula.
   4. Reading a regression tableRead standard errors, t-statistics, p-values and confidence intervals for regression coefficients in a published table, and take a first look at logistic regression for binary outcomes such as voting.
10. 10

    Causal inference

    4 lessons

    1. Confounding and causal diagramsDraw directed acyclic graphs showing confounders, mediators and colliders, and explain why controlling for a confounder helps while controlling for a mediator or a collider misleads.
    2. Natural experimentsExplain natural experiments and as-if random assignment, regression discontinuity around close elections as a test of the incumbency advantage, and instrumental variables with the exclusion restriction.
    3. Difference in differencesWork the difference-in-differences estimator from a two-by-two table of treated and control groups before and after, explain the parallel trends assumption, and describe panel fixed effects.
    4. Matching and its limitsExplain matching on observed covariates and on propensity scores to approximate an experiment, check balance, and say why selection on unobservables means matching cannot remove all confounding.
11. 11

    Research ethics and transparency

    3 lessons

    1. Human subjects protectionState the Belmont Report's three principles, respect for persons, beneficence and justice, explain institutional review boards and informed consent, and use the Milgram and Tuskegee cases to show what the rules respond to.
    2. Ethics of field experimentsExplain the ethical issues of political field experiments, deception, risk of harm and consent, apply them to the 2014 Montana voter mailer controversy, and describe how to protect confidentiality and data privacy.
    3. Transparency and replicationExplain the replication crisis, p-hacking and the garden of forking paths, publication bias, preregistration and pre-analysis plans, and data access and research transparency (DA-RT).