Scientific analysis of oil painting color by combining machine learning technology has become an important direction for the integration of art and technology, which provides new technical support for improving the quality of art teaching. This study combines the Gradient Boosted Decision Tree (GBDT) algorithm to deeply analyze the color features of oil paintings and explore its practical application in art teaching. Methodologically, the oil painting color prediction model based on GBDT is constructed, and the key factors affecting oil painting color are ranked in importance and visualized by SHAP interpretation method. At the same time, UV light-curing ink printing experiments were carried out on corrugated paper and gray-backed white board paper to establish the CIELAB color spatial relationship model for different paper substrates. The results show that the GBDT model outperforms the other six mainstream algorithms in terms of prediction accuracy, with an accuracy of 0.851 and an AUC value of 0.933. The SHAP analysis indicates that the color particle size, color type, canvas texture, pigment layer thickness, and the surrounding environment color are the top five key factors affecting the color of oil paintings. Printing experiments confirmed that there is a significant difference in the color rendering effect of ink on corrugated paper and gray background whiteboard paper, corrugated paper brightness is only 13% while the color deviation value is as high as 20%. The research results provide scientific support for the teaching of oil painting color in university art majors, and through the introduction of multimedia teaching methods and curriculum system reform, students’ color perception and creative ability are effectively improved.