| Title: | Pattern Oriented Modelling Information Criterion |
|---|---|
| Description: | Calculations of an information criterion are proposed to check the quality of simulations results of Agent-based models (ABM/IBM) or other non-linear rule-based models. The POMDEV measure (Pattern Oriented Modelling DEViance) is based on the Kullback-Leibler divergence and likelihood theory. It basically indicates the deviance of simulation results from field observations. Once POMDEV scores and metropolis-hasting sampling on different model versions are effectuated, POMIC scores (Pattern Oriented Modelling Information Criterion) can be calculated. This method could be further developed to incorporate multiple patterns assessment. Piou C, U Berger and V Grimm (2009) <doi:10.1016/j.ecolmodel.2009.05.003>. |
| Authors: | Cyril Piou [aut, cre] |
| Maintainer: | Cyril Piou <[email protected]> |
| License: | GPL-2 |
| Version: | 1.0.4 |
| Built: | 2024-11-14 03:12:38 UTC |
| Source: | https://github.com/cran/Pomic |
Calculations of an information criterion are proposed to check the quality of simulations results of Agent-based models (ABM/IBM) or other non-linear rule-based models. The POMDEV measure (Pattern Oriented Modelling DEViance) is based on the Kullback-Leibler divergence and likelihood theory. It basically indicates the deviance of simulation results from field observations. Once POMDEV scores and metropolis-hasting sampling on different model versions are effectuated, POMIC scores (Pattern Oriented Modelling Information Criterion) can be calculated. This method could be further developed to incorporate multiple patterns assessment.
| Package: | Pomic |
| Type: | Package |
| Version: | 1.0.4 |
| Date: | 2018-02-09 |
| License: | GPL-2 |
The pomdev function is for distribution patterns. It is computed as in Piou et al. (2009) It corresponds to twice the sum of the log of an approximated likelihood given by the approximating function of density from the simulation results applied on the field data.
The pomdev.extra function is to obtain POMDEV scores as well as other goodness of fit indicators.
The pomdev.corrected function is to obtain raw approximation of KL divergence.
The pomdev.ts function is for time series patterns. It is still under evaluation.
Type demo(demoPomic) for a demonstration of POMIC calculation as in appendix B of Piou et al. (2009).
And type demo{verification} for the verification of the first demonstration (as in appendix C of Piou et al. (2009)).
Cyril Piou
[email protected]
Piou 2007, Patterns and individual-based modeling of spatial competition within two main components of Neotropical mangrove ecosystems. PhD thesis, University of Bremen. http://elib.suub.uni-bremen.de/diss/docs/00010671.pdf
Piou, C., U. Berger, and V. Grimm. 2009. Proposing an information criterion for individual-based models developed in a pattern-oriented modelling framework. Ecological Modelling 220:1957-1967. http://dx.doi.org/10.1016/j.ecolmodel.2009.05.003
pomdev pomdev.ts pomdev.extra pomdev.corrected
#example of field pattern distribution: fieldpattern<-rnorm(100,10,5) #model results: model1runs<- rnorm(100*5,10.5,6) model2runs<- rnorm(100*5,9.5,2) #POMDEV measurements: pomdev.extra(fieldpattern,model1runs,eps=10^-20,plotting=TRUE) pomdev.extra(fieldpattern,model2runs,eps=10^-20,plotting=TRUE) pomdev(fieldpattern,model2runs,eps=10^-20) pomdev.corrected(fieldpattern,model1runs,eps=10^-20) pomdev.corrected(fieldpattern,model2runs,eps=10^-20) #other examples of POMDEV measures: pomdev(x<-rnorm(100),y<-rnorm(100,0.1)) #POMDEV value for a model creating a normal distribution #with identical variance as the field data but a mean 0.1 #away from the field data par(mfrow=c(2,2)) hist(x,freq=FALSE) #plot the field data as histogram lines(density(y))#check the simulation results distribution as line pomdev(x<-rnorm(100),y<-rnorm(100,0.2)) #idem 0.2 away from the field data hist(x,freq=FALSE); lines(density(y)) #plot pomdev(x<-rnorm(100),y<-rnorm(100,1)) #idem 1 away from the field data hist(x,freq=FALSE); lines(density(y)) #plot pomdev(x<-rnorm(100),y<-rnorm(100,3)) #idem 3 away from the field data hist(x,freq=FALSE); lines(density(y)) #plot ## Not run: demo(demoPomic) ## Not run: demo(verification)#example of field pattern distribution: fieldpattern<-rnorm(100,10,5) #model results: model1runs<- rnorm(100*5,10.5,6) model2runs<- rnorm(100*5,9.5,2) #POMDEV measurements: pomdev.extra(fieldpattern,model1runs,eps=10^-20,plotting=TRUE) pomdev.extra(fieldpattern,model2runs,eps=10^-20,plotting=TRUE) pomdev(fieldpattern,model2runs,eps=10^-20) pomdev.corrected(fieldpattern,model1runs,eps=10^-20) pomdev.corrected(fieldpattern,model2runs,eps=10^-20) #other examples of POMDEV measures: pomdev(x<-rnorm(100),y<-rnorm(100,0.1)) #POMDEV value for a model creating a normal distribution #with identical variance as the field data but a mean 0.1 #away from the field data par(mfrow=c(2,2)) hist(x,freq=FALSE) #plot the field data as histogram lines(density(y))#check the simulation results distribution as line pomdev(x<-rnorm(100),y<-rnorm(100,0.2)) #idem 0.2 away from the field data hist(x,freq=FALSE); lines(density(y)) #plot pomdev(x<-rnorm(100),y<-rnorm(100,1)) #idem 1 away from the field data hist(x,freq=FALSE); lines(density(y)) #plot pomdev(x<-rnorm(100),y<-rnorm(100,3)) #idem 3 away from the field data hist(x,freq=FALSE); lines(density(y)) #plot ## Not run: demo(demoPomic) ## Not run: demo(verification)
This function calculate the POMDEV value of simulations results compared to a vector taken as a field pattern.
pomdev(object1, object2, eps=10^-30, nrange=1000)pomdev(object1, object2, eps=10^-30, nrange=1000)
object1 |
Numerical, vector of the field pattern |
object2 |
Numerical, vector of simulations results |
eps |
Numerical, value to use when the likelihood = 0 for mathematical stability |
nrange |
Numerical, number of interval to consider for the probability density function creation and integration |
This function returns the POMDEV value as computed in Piou et al. It corresponds to twice the sum of the log of an approximated likelihood given by the approximating function of density from the simulation results applied on the field data.
Return a value corresponding to the pomdev score of model results to reproduce the pattern.
Cyril Piou
[email protected]
pomdev(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20)pomdev(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20)
This function calculate the KL divergence of simulations results compared to a vector taken as a field pattern with identical techniques as POMDEV.
pomdev.corrected(object1, object2, eps=10^-30, nrange=1000)pomdev.corrected(object1, object2, eps=10^-30, nrange=1000)
object1 |
Numerical, vector of the field pattern |
object2 |
Numerical, vector of simulations results |
eps |
Numerical, value to use when the likelihood = 0 for mathematical stability |
nrange |
Numerical, number of interval to consider for the probability density function creation and integration |
This function returns 1/2 the POMDEV value + correction of the constant in KL equation corresponding to the entropy of the field pattern. Thus, this value should be close to the KL divergence using the same techniques of approximation than pomdev.
Return a value corresponding to the KL divergence between the model results and the pattern
Cyril Piou
[email protected]
pomdev.corrected(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20)pomdev.corrected(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20)
This function calculate the POMDEV value of simulations results compared to a vector taken as a field pattern and compute also other goodness of fit indicators.
pomdev.extra(object1, object2, eps=10^-30, nrange=1000, fullmsd=FALSE, plotting=FALSE,...)pomdev.extra(object1, object2, eps=10^-30, nrange=1000, fullmsd=FALSE, plotting=FALSE,...)
object1 |
Numerical, vector of the field pattern |
object2 |
Numerical, vector of simulations results |
eps |
Numerical, value to use when the likelihood = 0 for mathematical stability |
nrange |
Numerical, number of interval to consider for the probability density function creation and integration |
fullmsd |
Boolean, should the full cross MSD be calculated as well |
plotting |
Boolean, should a plot illustrating the calculation be created |
... |
eventual information to give to the last plot (if plotting=T)... |
The KL divergence is only calculated comparing the simulations to the field vector, as if it is the truth (different from the flexmix KLdiv calculations)
Return a list containing:
field_data |
the field data |
sim_data |
the simulation data |
kernel_estimator |
information about the kernel estimator used |
result |
a table of 4 or 5 rows (depending if fullmsd=TRUE or not) with: |
- POMDEV |
pomdev score of model results to reproduce the pattern |
- overlap |
indicates if the model results overlap the range of the field data |
- KLdiv |
KL divergence of the model results from the field pattern |
- MSD |
|
- CrossMSD |
if fullmsd is true, calculate the mean square deviation among each pair of simulation and field data possible |
Cyril Piou
[email protected]
pomdev.extra(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20) pomdev.extra(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20,fullmsd=TRUE,plotting=TRUE)pomdev.extra(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20) pomdev.extra(rnorm(100,10,5),rnorm(100*5,10.5,6),eps=10^-20,fullmsd=TRUE,plotting=TRUE)
This function calculate the POMDEV value of time series simulations results compared to a vector taken as a field pattern.
pomdev.ts(object1, object2, eps=10^-30, nrange=1000)pomdev.ts(object1, object2, eps=10^-30, nrange=1000)
object1 |
Numerical, vector of the field pattern |
object2 |
Numerical, matrix of simulations results with nrow = length(object1) and ncol > 2 |
eps |
Numerical, value to use when the likelihood = 0 for mathematical stability |
nrange |
Numerical, number of interval to consider for the probability density function creation and integration |
This function is still under evaluation and should give a possibility of deviance measurement for temporal patterns. It corresponds to the sum of twice the log of approximated likelihoods given by approximating function of density from simulation results for each field data point.
Return a value corresponding to the pomdev score of model results to reproduce the pattern.
Cyril Piou
[email protected]
pomdev pomdev.extra pomdev.corrected
pomdev.ts(rnorm(100,10,5),matrix(rnorm(100*5,10.5,6),ncol=5),eps=10^-20)pomdev.ts(rnorm(100,10,5),matrix(rnorm(100*5,10.5,6),ncol=5),eps=10^-20)