Wheat yields in Ethiopia need to increase considerably to reduce import dependency and keep up with the expected increase in population and dietary changes. Despite the yield progress observed in recent years, wheat yield gaps remain large. Here, we decompose wheat yield gaps in Ethiopia into efficiency, resource, and technology yield gaps and relate those yield gaps to broader farm(ing) systems aspects. To do so, stochastic frontier analysis was applied to a nationally representative panel dataset covering the Meher seasons of 2009 and 2013 and crop modelling was used to simulate the water-limited yield (Yw) in the same years. Farming systems analysis was conducted to describe crop area shares and the availability of land, labour, and capital in contrasting administrative zones. Wheat yield in farmers’ fields averaged 1.9 t ha− 1 corresponding to ca. 20% of Yw. Most of the yield gap was attributed to the technology yield gap (> 50% of Yw) but narrowing efficiency (ca. 10% of Yw) and resource yield gaps (ca. 15% of Yw) with current technologies can nearly double actual yields and contribute to achieve wheat self-sufficiency in Ethiopia. There were small differences in the relative contribution of the intermediate yield gaps to the overall yield gap across agro-ecological zones, administrative zones, and farming systems. At farm level, oxen ownership was positively associated with the wheat cultivated area in zones with relatively large cultivated areas per household (West Arsi and North Showa) while no relationship was found between oxen ownership and the amount of inputs used per hectare of wheat in the zones studied. This is the first thorough yield gap decomposition for wheat in Ethiopia and our results suggest government policies aiming to increase wheat production should prioritise accessibility and affordability of inputs and dissemination of technologies that allow for precise use of these inputs.