The Metropolis Hastings Algorithm

Prerequisites

You should be familiar with the concept of a Markov chain and its stationary distribution.

Introduction

The Metropolis Hastings algorithm is a beautifully simple algorithm for producing samples from distributions that may otherwise be difficult to sample from.

Suppose we want to sample from a distribution \(\pi\), which we will call the “target” distribution. For simplicity we assume that \(\pi\) is a one-dimensional distribution on the real line, although it is easy to extend to more than one dimension (see below).

The MH algorithm works by simulating a Markov Chain, whose stationary distribution is \(\pi\). This means that, in the long run, the samples from the Markov chain look like the samples from \(\pi\). As we will see, the algorithm is incredibly simple and flexible. Its main limitation is that, for difficult problems, “in the long run” may mean after a very long time. However, for simple problems the algorithm can work well.

The MH algorithm

Toy Example

To help understand the MH algorithm we now do a simple example: we implement the algorithm to sample from an exponential distribution: \[\pi(x) = \exp(-x) \qquad (x \geq 0).\] Of course it would be much easier to sample from an exponential distribution in other ways; we are just using this to illustrate the algorithm.

Remember that \(\pi\) is called the “target” distribution, so we call our function to compute \(\pi\) target:

target = function(x){
  return(ifelse(x<0,0,exp(-x)))
}

Now we implement the MH algorithm, using the simple normal random walk transition kernel \(Q\) mentioned above. Since this \(Q\) is symmetric the Hastings ratio is 1, and we get the simpler form for the acceptance probability \(A\) in the Metropolis algorithm.

Here is the code:

x = rep(0,10000)
x[1] = 3     #initialize; I've set arbitrarily set this to 3
for(i in 2:10000){
  current_x = x[i-1]
  proposed_x = current_x + rnorm(1,mean=0,sd=1)
  A = target(proposed_x)/target(current_x) 
  if(runif(1)<A){
    x[i] = proposed_x       # accept move with probabily min(1,A)
  } else {
    x[i] = current_x        # otherwise "reject" move, and stay where we are
  }
}

Having run this code we can plot the locations visited by the Markov chain \(x\) (sometimes called a trace plot).

plot(x,main="values of x visited by the MH algorithm")

Remember that we designed this algorithm to sample from an exponential distribution. This means that (provided we ran the algorithm for long enough!) the histogram of \(x\) should look like an exponential distribution. Here we check this:

hist(x,xlim=c(0,10),probability = TRUE, main="Histogram of values of x visited by MH algorithm")
xx = seq(0,10,length=100)
lines(xx,target(xx),col="red")

So we see that, indeed, the histogram of the values in \(x\) indeed provides a close fit to an exponential distribution.

Closing remarks

One particularly useful feature of the MH algorithm is that it can be implemented even when \(\pi\) is known only up to a constant: that is, \(\pi(x) = c f(x)\) for some known \(f\), but unknown constant \(c\). This is because the algorithm depends on \(\pi\) only through the ratio \(\pi(y)/\pi(x_t)=cf(y)/cf(x_t) = f(y)/f(x_t)\).

This issue arises in Bayesian applications, where the posterior distribution is proportional to the prior times the likelihood, but the constant of proportionality is often unknown. So the MH algorithm is particularly useful for sampling from posterior distributions to perform analytically-intractible Bayesian calculations.

Session information

sessionInfo()

R version 3.5.2 (2018-12-20)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.1

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] workflowr_1.2.0 Rcpp_1.0.0      digest_0.6.18   rprojroot_1.3-2
 [5] backports_1.1.3 git2r_0.24.0    magrittr_1.5    evaluate_0.12  
 [9] stringi_1.2.4   fs_1.2.6        whisker_0.3-2   rmarkdown_1.11 
[13] tools_3.5.2     stringr_1.3.1   glue_1.3.0      xfun_0.4       
[17] yaml_2.2.0      compiler_3.5.2  htmltools_0.3.6 knitr_1.21