Nearly 70% of energy and 60–85% of protein requirements of the dairy cow are met from microbial fermentation, indicating a critical need for maximizing rumen function and describing rumen microbiota. However, it is still not known how diet and microbes interact to enhance milk yield in dairy cows. The purpose of this study is to compare the ruminal bacterial composition in the high and low yielding dairy cows within and between 2 dairy herds. Eighty 5 Holstein dairy cows in mid-lactation (80–180 d in milk) were selected from 2 farms: Farm 12 (M305 = 12,300 kg; n = 47; 24 primiparous cows, 23 multiparous cows) and Farm 9 (M305 = 9,700 kg; n = 38; 19 primiparous cows, 19 multiparous cows). Each study cow was sampled once using the stomach tube method and processed for 16S rRNA amplicon sequencing using the Ion Torrent (PGM) platform. Differences in bacterial communities between farms were greater (Adonis: R² = 0.16; P < 0.05) than within farms. Five bacterial lineages, namely Prevotella (48–51%), Bacteroidales (10–12%), unclassified bacteria (5–8%), Succinivibrionaceae (1.4–6.6%) and unclassified Prevotellaceae (3.8–4.7%) were observed to differentiate the community clustering patterns between the farms. A notable finding is the greater (P < 0.05) contribution of Succinivibrionaceae in Farm 12 compared with Farm 9. Furthermore, in Farm 12, this bacterial population was higher (P < 0.05) in the higher yielding cows compared with the lower yielding cows in both primiparous and multiparous groups. Prevotella, S24–7, and Succinivibrionaceae were found in greater abundance on Farm 12 and were positively correlated with milk yield. Differences in rumen bacterial populations observed between the 2 farms can be attributed to dietary composition, particularly differences in forage type and proportion in diets. A combination of corn silage and alfalfa silage may have contributed to the increased proportion of Proteobacteria. It was concluded that Farm 12 had a greater proportion of specialist bacteria that have the potential to support enhanced rumen function.