Advanced Bayesian Statistics for the Social Sciences

Day 1: Review

The first day reviews the material commonly included in introductory courses of Bayesian statistics for social sciences. These are the basics of Bayesian inference, the differences between frequentist and Bayesian statistics, basics of Bayesian computation using Markov chain Monte Carlo, Bayesian Normal linear regression, Bayesian Binomial logistic regression, Bayesian hierarchical linear model, and basics of Bayesian model evaluation. The topics will be reviewed with the emphasis on the use of the models with JAGS. The session is not a substitute for a first introduction to Bayesian statistics.

Day 2: GLM Special Topics
Normal linear regression and Binomial logistic regression are workhorses of social science research. However, in some contexts other members of the GLM family might be more appealing. Such contexts include count, limited, or zero-inflated left-hand-side (outcome/dependent) variables. The flexibility of Bayesian computation as implemented in JAGS or Stan makes easily accessible to applied researchers a wide variety of sampling distributions and link functions that allow to handle these issues in GLM/MLM. For example, in the linear-model context it might be desirable to reduce the influence of 'outlier' observations, i.e., a robust alternative to the Normal sampling distribution might be desirable. The t, Laplace, and Logistic distributions can be used to this aim. For another example, in some contexts the outcome of interest is a proportion limited to [0%, 100%], and the data contain considerable number of 0% and 100%. The Beta and Binomial distributions can be jointly used to handle it. For a final example, due to substantive or operational concerns a linear model that minimizes the largest absolute residual might be desirable, which can be achieved using the Uniform sampling distribution. 

Day 3: Informative Priors and Regularization

Introductions to Bayesian GLM often rely on priors that lead to results numerically close to those under MLE. Other priors are preferable in many contexts. Extra-data information such as expert judgment or qualitative case studies can be used to formulate priors. This is especially useful when the data is weak in the sense that it offers only limited information on the parameters of interest due to its small size or noisiness. Also, various deals on the bias-variance trade-off are available through priors. First, weakly informative priors can be used to handle linear separation in logistic models, or in other instances where MLE estimates are known to be biased. Second, through the use of priors are available Bayesian ridge and lasso, which offer to exchange an increase in bias for a substantial reduction of variance. The ridge can be used to handle multicollinearity. The lasso provides variable selection, which can appeal in settings with many right-hand-side (predictor/independent) variables. Furthermore, the 'horseshoe' prior offers an additional Bayesian alternative to the lasso.
 

Day 4: Mixture Models
In social science research, it is often an easy assumption that the observations differ in important but unobserved ways. In some contexts, it is appealing to assume that the population is composed of a limited number of groups that differ in a characterstic such as the association between two variables of interest. Mixture models can be used to handle such unobserved heterogeneity. The flexibility of MCMC as implemented in JAGS allows for an easy use of a variety of such models. Furthermore, from the Bayesian perspective it can be appealing to think of the groups in terms of a hierarchical model under which the memberships in upper-level units are not observed. Finally, mixture models can be used to evaluate competing theories using models in which each of the mixture components represents one of the theories.

Day 5: Latent Variable Models

In social science research, latent variable models are used to a variety of aims, including the measurement of concepts which are not directly observable. An example of such application is scaling  two-way contingency tables to obtain estimates of legislator ideal points or voter ideology. Bayesian Latent Variable Models allow for straightforward modeling of hierarchical structures in such data as well as for the use of extra-data information.

• Understanding of the basics of Bayesian inference
• Understanding of the differences between frequentist and Bayesian inference
• Experience with the Generalized Linear Model (GLM), e.g. OLS, logit/probit

• Experience with Multi-Level Models (MLM) is helpful, but not necessary
• Comfortable with R
• Some experience with at least one of the following WinBUGS/OpenBUGS/JAGS/Stan recommended, but not necessary