A welfare assessment tool was developed with the aim to remotely assess welfare status of dairy herds. This tool will help producers benchmark their own performance relative to others, adopt tailored strategies, and monitor whether improvements are effective. The welfare assessment tool was based on 14 indicators, pre-recorded through routinely collected milking records and herd data. Indicators were selected based on advice of dairy experts covering longevity, nutrition, production, health, reproduction, and young stock. All values were normalized and percentile ranks were calculated. Ranks were averaged and agglomerated to a composite herd welfare index (HWI). The percentile rank of each indicator allows us to evaluate herd strengths and weaknesses, and the HWI situates the herd within the population of herds. Based on this approach, a HWI was calculated for 4463 dairy herds in Quebec, Canada. In addition, animal-based measures of herd welfare status (prevalence of lameness and lesions) were collected on 86 tie and free stall herds during summer 2017 to validate the selected indicators and the HWI. Pearson correlation coefficients between the HWI and animal-based measures were −0.41 (lameness), −0.41 (hock lesions), and −0.60 (knee lesions) for tie stall herds (n = 64; P < 0.05) but were not significant (P > 0.05) for freestall herds (n = 22). Based on the HWI, 5 tie stall herds among the 86 herds were situated within the worst 10% of the entire population of herds in Quebec, and showed deficiencies (among worst 25%) in knee lesions (4 herds), hock lesions (3 herds) and lameness (2 herds). In addition, 5 tie stall herds were within the best 10% in Quebec and excelled (best 25%) in lameness (4 herds), and hock and knee lesions (3 herds, respectively). The welfare assessment tool described in this study allows remote assessment and monitoring of the welfare status of dairy herds based on pre-recorded data and can be a powerful tool for producers to evaluate their herd relative to peers and to highlight opportunities for improvements.