Abstract #4

# 4
Model evaluation: Part II (exercises).
E. Kebreab*1, 1University of California, Davis, Davis, CA.

The objective of the model evaluation exercise is to familiarize users on various tools used to evaluate models. The exercise will use the R statistical software due to its relatively straightforward use, which is also freely available on the internet. A data set containing observed and predicted data will be made available to the participants. Based on principles covered, the participants will be asked to calculate the mean square error of prediction (MSEP) and its square root (RMSEP), which are one of the most commonly used methods of model evaluation. Furthermore, the exercise includes calculated the MSEP decomposition into (1) error due to overall bias of prediction, (2) error due to deviation of the regression slope from unity, and (3) error due to the disturbance. The participants will be asked to calculate another model evaluation category, which is the concordance correlation coefficient (CCC). The participants are expected to express CCC as a product of 2 components: a correlation coefficient estimate that measures precision (range 0 to 1, where 1 = perfect fit) and a bias correction factor that indicates how far the regression line deviates from the line of unity (range from 0 to 1 and 1 indicates that no deviation from the line of unity has occurred). Finally, participants will be asked to compare results from the 2 different categories of model evaluation.

Key Words: model performance, modeling, prediction accuracy