There is still a rising demand for fermented milks like Greek yogurt and skyr. Nutrition-conscious consumers appreciate the high protein content up to 10%, whereas products commonly contain no fat. The consistency is creamy and thick without the need for stabilizers. Typically, the milk is acidified in tanks forming a gel that is finally broken up by stirring and concentrated by separation. This process generates large amounts of acid whey that is undesired due to environmental concerns. A novel approach to avoid acid whey is to concentrate the milk before fermentation, however, resultant gels are firm so that stirring in the vat and further processing is difficult. It is also challenging to produce a smooth texture. We hypothesize that power ultrasound during fermentation is a tool to soften the gel as sound waves cause cavitation and strong shear forces in the fluid. Skim milk was concentrated to a protein content of 10% by reverse osmosis and heated to 95°C for 5 min. Milk was fermented in jars (140 g) at 44°C until pH 4.6. During acidification, samples were treated with a sonotrode (20 kHz, 200 W). The sonication was performed at different pH (5.5–5.1) for 100 s in pulsation to prevent temperature rise. Immediately after fermentation, gels were stirred using a rheometer with a vane geometry. The maximum torque required to break the set gel was recorded. Gel firmness was measured by penetration tests. Afterward, gels were processed into stirred products and analyzed (water-holding capacity, rheology, particle size). Short-term sonication at pH 5.2 reduced the maximum torque required to break the gel and the firmness by 21% (P < 0.001) and 26% (P < 0.001), respectively. Treatments at higher pH values did not result in a softening of the gel. Samples sonicated at pH ≤ 5.2 exhibited a reduced water-holding capacity. Furthermore, the Sauter mean diameter was decreased from 40.8 ± 2.0 µm (control) to 30.1 ± 1.8 µm due to sonication at pH 5.2, and the maximum shear stress was lowered by 14%. We conclude that power ultrasound affects the network structure by softening the gel and offers the potential to improve Greek yogurt production.