AI technology promotes the intelligent development of music conducting art. This paper focuses on the innovative application of AI technology in music conducting, and proposes dynamic gesture semantic classification technology based on dynamic temporal regularization (DTW), which utilizes the scale invariance of the conductor’s gesture trajectory to complete modeling and recognition. Through normalized data processing and trajectory point downsampling method, the spatial deviation is eliminated, and the gesture template matching system is constructed by combining the DTW algorithm. Experiments show that the localization accuracy of the five gesture trajectory points of the proposed model exceeds 0.95, with good localization ability. The average recognition accuracy reaches 99.31%, close to 100.00%, higher than 92.76% of the comparison model. The gesture classification accuracy is 0.902, precision is 0.936, recall is 0.954, and F1 value is 0.967, which is better than the comparison model. The model has the best performance when the number of templates is 7.