There is increased demand for herd-level animal welfare evaluations but little consensus on how best to sample for this purpose. Our objective was to evaluate the effect of different sample size approaches on farm classifications relative to thresholds of acceptability for animal-based measures. We predicted more accurate classifications when more cows were sampled and when selecting from all lactating cows compared with only the high-producing pen. On 38 freestall farms, we assessed all 12,375 lactating cows for lameness, injuries on the tarsal and carpal joints, and BCS and then compared these herd prevalence measures with 9 sampling strategies (using precision of 15, 10, or 5% applied to the high pen, all lactating cows, or the entire herd). For each sampling strategy, we selected cows randomly in 10,000 replicates, calculating the prevalence for each replicate and classifying farms as meeting (below) or failing to meet (above) 16 thresholds for cow-based measures. For each threshold, we determined how many farms were classified correctly in ≥95% of sample replicates. Across thresholds, the number of farms meeting the ≥95% target increased with the number of cows sampled (i.e., when using narrower precision values and when applying the formula to lactating cows rather than the total herd size). Contrary to predictions, sampling from the high pen resulted in similar accuracy to selecting from among all lactating cows. For example, with a threshold of < 10% severely lame cows, 33 farms met the ≥95% target when calculating sample size using 10% precision applied to lactating cows. This decreased to 32 or 26 farms when the formula was applied to the high pen or the whole herd, respectively. Using a precision of 15 or 5% changed the number of farms meeting the target by –4 to –7 and 4 to 8, respectively. Narrower precision greatly increases sample size requirements (e.g., by up to 183 cows when using 5 vs. 10% precision), and assessment programs will need to balance misclassification with feasibility constraints. Our findings suggest sampling high-producing cows can serve as a practical proxy for the larger population of lactating cows.