A broad range of potential chemical compositions makes difficult design of novel bulk metallic glasses (BMGs) without performing expensive experimentations. To overcome this problem, it is very important to establish predictive models based on artificial intelligence. In this work, a machine learning (ML) approach was proposed for predicting glass formation in numerous alloying compositions and designing novel glassy alloys. The results showed that our ML model accurately predicted the glass formation and critical thickness of MGs. As a case study, the ternary Fe–B–Co system was selected and effects of minor additions of Cr, Nb and Y with different atomic percentages were evaluated. It was found that the minor addition of Nb and Y leads to the significant improvement of glass-forming ability (GFA) in the Fe–B–Co system; however, a shift in the optimized alloying composition was occurred. The experimental results on selective alloying compositions also confirmed the capability of our ML model for designing novel Fe-based BMGs. © 2021, King Fahd University of Petroleum & Minerals.