Monte Carlo assessments of goodness-of-fit for ecological simulation models

Authors

Lance A. Waller, Emory University
David Smith, University of Maryland
James E. Childs, Centers for Disease Control and Prevention, AtlantaFollow
Leslie A. Real, Emory University

Date of this Version

2003

Comments

Published in Ecological Modelling 164 (2003) 49–63

Abstract

One often develops stochastic ecologic simulation models based on local interactions between individuals or groups and bases systemic conclusions on trends summarized over multiple data sets generated from the model. In many cases, such models generate data sets (“realizations”) each violating the usual assumptions associated with traditional statistical tests of goodness-of-fit, most notably that of independent observations. Monte Carlo hypothesis tests applied to multiple realizations from such models provide appropriate goodness-of-fit tests regardless of within-model peculiarities. The Monte Carlo tests address the question “Do the observed data appear consistent with the model?” in contrast to the usual question “Does the model appear consistent with the observed data?”. In addition, such tests can make use of the same data sets used to draw systemic inference (i.e. the tests require no additional simulation runs). We illustrate the concept using Pearson’s chi-square statistic with correlated data. We also consider the behavior of a similar statistic and of “modeling efficiency” in assessing the fit of a simulation model for the spatial spread of raccoon rabies in Connecticut.