In this paper, a scientific and effective scoring method plays an important role in the improvement of students’ English speaking level in the teaching of spoken English. In this paper, we design an English speaking scoring method that integrates spectral clustering algorithm and speech data. The spectral clustering algorithm effectively integrates the feature information in the speech data by constructing a similarity matrix, and divides the students’ spoken English samples into different categories. The categorized data are inputted into the scoring algorithm for speech matching, and the reference spoken English speech data are used as the standard, and the difference between the two is calculated by the dynamic time regularization algorithm, reflecting the difference between the students’ spoken English speech and the reference speech, and scoring the students’ spoken English performance. The spectral clustering algorithm in this paper is able to achieve a higher degree of accuracy and reduction in the classification of students’ spoken English samples compared with comparative algorithms such as K-mean clustering. And based on this paper’s automatic English speaking scoring algorithm, the mean difference between algorithmic scoring and manual scoring is only 0.2698 points, and there is no significant difference in the scoring level between the two. The application in English teaching can reduce teachers’ workload while improving students’ English speaking learning effect, which provides a more intelligent method for English speaking teaching.
