As demand for organic dairy products grows, the industry must focus on feeding strategies that optimize milk production. Dietary crude protein (CP) is variable across farms, and affects animal health and production during the grazing season. As a result of diet CP variability, milk urea nitrogen (MUN) profiles of grazing herds are often outside of the optimal range (9–12 mg/dL). This study evaluated the impact of altering the CP content of dietary supplements on milk production and the MUN profile of grazing cattle. Six Vermont organic dairy farms participated in a 6-week trial during the 2018 summer grazing season. Farms were paired by their 2017 summer MUN profile, and farms within each pair were assigned to either 1) continuation of their regular supplements (n = 3, CON), or 2) a 16% CP (% of DM) supplement formulated using an organic barley and roasted soybean mix (n = 3, TRT). All farms were maintained on their regular supplements during the first 2 weeks of the trial (baseline), and then continued on either the CON or TRT supplements for the last 4 weeks of the trial (experimental period). Each week, milk samples were collected at 2 consecutive milkings. Individual milk yields were recorded at each sampling, and samples were collected and commercially analyzed for protein, fat, and MUN content. All parameters were analyzed using PROC MIXED of SAS, and effects of treatment, week, and treatment x week were determined. Milk weight, fat percent, protein percent, and MUN content were all affected by treatment x week (P < 0.0001). Across the experimental period, milk yield was higher in the TRT group versus the CON group (11.89 kg vs 9.92 kg, respectively). Fat percent was higher in the CON group versus TRT group (4.20% vs 3.71%), as was protein percent (3.15% vs 3.07%). The MUN was higher in the TRT versus the CON group (13.24 mg/dL vs 11.62 mg/dL). Modifying the supplement CP effectively increased milk yield, indicating that protein intake was limiting milk production. However, further formulation must be done to ensure that MUN remains within the optimal range and that milk fat and protein are not negatively affected as observed in the TRT group in this study.