Gibbs Sampler

The Gibbs sampler is a popular method used for MCMC (Markov chain Monte Carlo) analyses. It provides an elegant way for sampling from the joint distributions of multiple variables by applying the notion that: to sample from a joint distribution just sample repeatedly from its one-dimensional conditionals given whatever you've seen at the time (this notion is clearly implemented in the example STATISTICA Visual Basic program provided below).

For example, the values from the joint distribution of two random variables, X and Y, can be easily simulated by the Gibbs sampler that uses their conditional distributions rather than their joint distribution. Starting with an arbitrary choice of X and Y, X is simulated from the conditional distribution of X, given Y, and Y is simulated from conditional distribution of Y, given X. Alternating between two conditional distributions, in the subsequent steps, generates a sample from the correct joint distribution of X and Y; the approximation gets better and better as the length of the Gibbs sampler path increases.

STATISTICA Visual Basic provides an excellent environment for performing these types of analyses, because it 1) includes an efficient and high quality random number generator, 2) can quickly generate results spreadsheets and various graphs from the simulated data, and 3) allows convenient access to external (compiled) routines for very fast data processing.

The following simple STATISTICA Visual Basic example program illustrates the application of the Gibbs sampler to generate bivariate normal random variables. An introductory discussion of the Gibbs sampler, MCMC, and this particular example can be found  in detail in Murdoch (2000).

' See Chance Magazine; Vol. 13, No. 4, Fall 2000;

' Murdoch, Duncan J. Markov Chain Monte Carlo.

Option Base 1

Option Explicit

Sub Main

Dim GibbsSample As New Spreadsheet

Dim X As Double

Dim Y As Double

Dim StdDev As Double

Dim i As Integer

Dim Steps As Integer

Steps = 1000

GibbsSample.SetSize(Steps,2)

X = 0

Y = 0

StdDev = 1

For i=1 To Steps

' Conditional distributions:

X = 0.9*Y + RndNormal(StdDev)

Y = 0.9*X + RndNormal(StdDev)

GibbsSample.Value(i,1) = X

GibbsSample.Value(i,2) = Y

Next i

' make scatterplot with Histograms

Dim newanalysis As Analysis

Set newanalysis = Analysis (scScatterplotWithHistograms,GibbsSample)

newanalysis.Dialog.IntervalsX.Variable = "1"

newanalysis.Dialog.IntervalsY.Variable = "2"

newanalysis.Dialog.Graphs.Visible = True

End Sub

The values generated by the Gibbs sampler using this program are plotted in the following illustration.

As expected, the generated values follow a bivariate normal distribution.