Metropolis-Hastings-Sampling
Metropolis-Hastings-Sampling
Metropolis-Hastings Random-Walk Sampling (MHRW)
We present a tutorial introduction of the sampling process which is inspired by Lynch (2007, ch.5.2). His example demonstrates exceptional clear, that the MHRW is – under some apriori assumptions - reducible to a simple greedy stochastic optimization procedure working on a likelihood-ratio. We present the various steps of designing and coding the sampler in R. We vectorized some parts of Lynch’s code so that there is a considerable speed-up. Furthermore we added the deletion of the burn-in period and a possibility to thin the Markow chain to get rid of autocorrelations in the sampling process.
Initially the chain consisted of T=100000 sample steps. Then we recomputed the estimates after deleting a 10%-burn-in period. After that we determined the autocorrelation function ACF and the maximal lag (here 25) for the nonzero part of the ACF. This lag was used for thinning the posterior samples. The effective sample sizes shrunk to (4% of 90000) T=3600. Because the ACF did not reveal any significant autocorrelations these samples can be regarded as i.i.d. samples.
The estimates are very close to those reported by Lynch, though he used T=5000 samples without deleting the burn-in period and without thinning the time-series of MH-samples to eliminate autocorrelations. The percentage of acceptance decisions was 67.69 % and acceptance rate alpha (mean of the acceptance probabilities; Robert & Casella, 2010, p.171) was 0.87.
![Page01](/f/2/_processed_/5/d/csm_page01_4f4d7ffd84.png)
![Page02](/f/2/_processed_/7/0/csm_page02_fa4efbac4a.png)
![Page 03](/f/2/_processed_/1/0/csm_page_03_abfc26bde7.png)
![Page 04](/f/2/_processed_/5/b/csm_page_04_ad38b3911e.png)
![Page 05](/f/2/_processed_/7/b/csm_page_05_2dae94c148.png)
![Page 06](/f/2/_processed_/c/2/csm_page_06_5d4f9e7b46.png)
![Page 07](/f/2/_processed_/8/8/csm_page_07_8009727506.png)
![Page 08](/f/2/_processed_/d/8/csm_page_08_56a7586b0d.png)
![Page 09](/f/2/_processed_/b/7/csm_page_09_9ef6f4d448.png)
![Page 10](/f/2/_processed_/2/4/csm_page_10_99eaa0cdf6.png)
![Page 11](/f/2/_processed_/8/d/csm_page_11_f6b2735cd5.png)
![Page 12](/f/2/_processed_/7/2/csm_page_12_e2410721f0.png)
![Page 13](/f/2/_processed_/c/1/csm_page_13_9411f55b19.png)
![Page 14](/f/2/_processed_/b/3/csm_page_14_2d824e0dd0.png)
![Page 15](/f/2/_processed_/b/f/csm_page_15_40562097bf.png)
![Page 16](/f/2/_processed_/6/7/csm_page_16_f2e423be24.png)
![Page 17](/f/2/_processed_/c/2/csm_page_17_39c22f40e3.png)
![Page 18](/f/2/_processed_/a/c/csm_page_18_5c75cb8fb5.png)
![Page 19](/f/2/_processed_/2/0/csm_page_19_569aa73748.png)
MacKay's Rule of Thumb: Lower Bound on Number of Iterations of a Metropolis Method
David MacKay's "Information Theory, Inference, and Learning Algorithms" (2003, p.367f) contains a chapter Demonstration of the Metropolis-Hastings (MH) method. This chapter deals with the important question how many steps should I run the MH simulation to get the next independent sample. The rule and its motivation is discussed below.
![Rule Of Thumb01](/f/2/_processed_/c/6/csm_RuleOfThumb01_c1d78524b8.png)
![Rule Of Thumb02](/f/2/_processed_/1/0/csm_RuleOfThumb02_78eb79c366.png)
![Rule Of Thumb03](/f/2/_processed_/a/c/csm_RuleOfThumb03_8bd57a9dbb.png)
![Rule Of Thumb04](/f/2/_processed_/8/5/csm_RuleOfThumb04_a083f94647.png)
![Rule Of Thumb05](/f/2/_processed_/8/d/csm_RuleOfThumb05_8bdd5631ee.png)
![Rule Of Thumb06](/f/2/_processed_/3/7/csm_RuleOfThumb06_69f1ffe731.png)
![Rule Of Thumb07](/f/2/_processed_/8/7/csm_RuleOfThumb07_1c7e77197d.png)
![Rule Of Thumb08](/f/2/_processed_/c/d/csm_RuleOfThumb08_3cb4a0920c.png)
![Rule Of Thumb09](/f/2/_processed_/9/8/csm_RuleOfThumb09_a78f6f1a30.png)
![Rule Of Thumb10](/f/2/_processed_/5/7/csm_RuleOfThumb10_6385e70379.png)
![Rule Of Thumb11](/f/2/_processed_/9/c/csm_RuleOfThumb11_7a6a2d0b51.png)
![Rule Of Thumb12](/f/2/_processed_/6/8/csm_RuleOfThumb12_c719dde918.png)
![Rule Of Thumb13](/f/2/_processed_/2/3/csm_RuleOfThumb13_02101a5173.png)
![Rule Of Thumb14](/f/2/_processed_/8/2/csm_RuleOfThumb14_d27a1004a9.png)