Abstract #3

# 3
Model evaluation: Part I (lecture).
Ermias Kebreab*1, 1University of California, Davis, Davis, CA.

Model evaluation indicates the level of accuracy and precision of model performance by assessing the credibility and reliability of a model in comparison to measured observations. Quantitative statistical model evaluation methods can be classified into 3 types including (1) standard regression statistics, which determines strength of linear relationship, (2) error index, which quantifies deviation in observed units, and (3) relative model evaluation that are dimensionless. Within the first category, analysis of residuals involves regressing residuals against predicted or other model variables. In this method, the model is unbiased if residuals are not correlated with predictions and the slope is not significantly different from zero. Predicted values can also be centered making the slope and intercept estimates in the regression orthogonal and thus, independent. This allows for mean biases to be assessed using the intercepts of the regression equations, and the slopes to determine the presence of linear biases. Mean square error of prediction (MSEP) and its square root (RMSEP) are commonly used methods of error index type of evaluation. In general, RMSEP values less than half of observed SD may be considered having a good performance. The MSEP can be decomposed into (1) error due to overall bias of prediction, (2) error due to deviation of the regression slope from unity, and (3) error due to the disturbance. Examples of the third category include the concordance correlation coefficient (CCC). The CCC can be represented as a product of 2 components: a correlation coefficient estimate that measures precision (range 0 to 1, where 1 = perfect fit) and a bias correction factor that indicates how far the regression line deviates from the line of unity (range from 0 to 1 and 1 indicates that no deviation from the line of unity has occurred). During model evaluation, a combination of the methods described above should be used to gain insight on model performance.

Key Words: evaluation, model