This is the companion notebook to the course on Bayesian inference. We start by loading the packages and the data.

library(tidyverse)
# Then change your working directory
if (!require("expm")) install.packages("expm") # Only installs if missing
if (!require("markovchain")) install.packages("markovchain") # Only installs if missing
load("anes.RData")

First analysis: Beta conjugates

The simplest case when inferring proportions. Combining binomial likelihood with beta prior is appealing, given the form of the densities. The posterior distribution follows a \[B(a+y,b+n-y)\] distribution, where n is the sample size, y is the number of ‘correct’ occurrences and a and b are the prior Beta parameters.

anes_1996 <- anes %>% filter(Date == 1996)          # We keep a year with low number of respondents
y <- sum(anes_1996$Party_simple == "Republican")    # Number of Republicans
n <- nrow(anes_1996)                                # Number of observations
a <- 15                                             # Initial Beta parameter
b <- 30                                             # Initial Beta parameter
ggplot(data.frame(x = c(0,0.6)), aes(x = x)) + 
  stat_function(fun = dbeta, args = list(shape1 = a, shape2 = b), aes(color = "PRIOR")) +
  stat_function(fun = dbeta, args = list(shape1 = y , shape2 = n - y ), aes(color = "LIKELIHOOD")) +
  stat_function(fun = dbeta, args = list(shape1 = y + a, shape2 = n - y + b), aes(color = "POSTERIOR"))

# CASE when the prior is strongly concentrated around 1/3
a <- 150 # Initial Beta parameter
b <- 300 # Initial Beta parameter
ggplot(data.frame(x = c(0,0.6)), aes(x = x)) + 
  stat_function(fun = dbeta, args = list(shape1 = a, shape2 = b), aes(color = "PRIOR")) +
  stat_function(fun = dbeta, args = list(shape1 = y , shape2 = n - y ), aes(color = "LIKELIHOOD")) +
  stat_function(fun = dbeta, args = list(shape1 = y + a, shape2 = n - y + b), aes(color = "POSTERIOR"))

In the first case, the prior is diluted because the sample is already large (346 observations). One way around that is to increase the magnitude of prior parameters. This increases their relative importance in the final output.

Second analysis: non-conjugage priors on grids

A more flexible approach is to approximate the posterior distribution by means of discretisation. Below, we show what this approximation can look like if the grid is coarse.

# Illustration first: approximative rectangles via sampling
sample <- rnorm(10^5) 
sample %>% data.frame() %>% ggplot(aes(x = sample)) + geom_histogram(aes(y = ..density..)) + 
  stat_function(fun = dnorm, args = list(mean = 0, sd = 1), color = "red")