This paper organically integrates the acoustic model topology, BIC and PSO, and improves the ASR system by optimizing the HMM acoustic model topology. The improved ASR system is used as an auxiliary teaching tool to optimize English listening teaching in colleges and universities by adopting TTS technology. Design the caMST model based on Rasch model to realize the accurate assessment of students’ English listening ability. Four English major classes of a key university in city A with a total of 120 students were selected for experiments to build an ASR system with optimal acoustic topology. The unidimensionality calibration of the Rasch model is incorporated to ensure the quality of the ca-MST question bank. Putting the proposed system into use, the results of the teaching experiment showed that the mean value of the listening score of the experimental class increased from 18.085 to 22.189 after the experiment, and the t-value of the change in the score was 2.874, whose significance (two-tailed) was 0.003, which was lower than the significance level of 0.05. The control group’s achievement improvement is small, and the listening ability does not show significant improvement, which in turn proves that using the system in this paper to assist teaching helps students’ listening ability, and provides more choices for the optimization of English listening teaching in colleges and universities.
