Model evaluation, as a critical process of model advancement, is necessary to identify adequacy and consistency of model predictions. The objectives of this study were 1) to evaluate the accuracy of the Molly cow model predictions of ruminal metabolism and nutrient digestion when simulating dairy and beef cattle diets; and 2) to identify deficiencies in the representations of biology that could be used to direct further model improvement. A total of 229 studies (n = 938 treatments) including dairy and beef cattle data, published from 1972 through 2016, were collected from the literature. Root mean square errors (RMSE) and concordance correlation coefficients (CCC) were calculated to assess model accuracy and precision. Ruminal pH was very poorly represented in the model with a RMSE of 4.6% and a CCC of −0.004. Although VFA concentrations had negligible mean and slope bias, the CCC was 0.28 implying that further modifications with respect to VFA production and absorption are required to improve model precision. The RMSE was greater than 50% for ruminal ammonia and blood urea-N concentrations with high proportions of error as slope bias indicating that mechanisms driving ruminal urea N recycling were not properly simulated in the model. Only slight mean and slope bias were exhibited for ruminal outflow of NDF, starch, lipid, total N, and non-ammonia N, and for fecal output of protein, NDF, lipid, and starch, indicating the mechanisms encoded in the model relative to ruminal and total-tract nutrient digestion were properly represented. Except for ruminal outflow of non-ammonia, non-microbial N, all variables related to ruminal metabolism and nutrient digestion were more accurately predicted for dairy diets than for beef diets. Overall, ruminal pH was poorly simulated and contributed to problems in ruminal nutrient degradation and VFA production predictions. Residual analyses suggest ruminal ammonia concentrations need to be considered in the ruminal pH equation, and therefore the inaccuracies in predicting ruminal urea N recycling must also be addressed. These modifications to model structure will likely improve model performance across a wider array of dietary inputs and cattle categories.