With the development of natural language processing technology, traditional models face the problems of inefficiency and information loss when processing long sequential texts. The low-dimensional embedding space combined with the self-attention mechanism provides a new direction for optimizing the model, which can capture the text semantic information more efficiently, reduce the computational complexity, and at the same time improve the model’s generalization ability and classification accuracy, so as to provide a more efficient solution for natural language processing. In this study, we propose a natural language processing model based on the self-attention mechanism, which combines PCA and VSM to construct a low-dimensional embedding space, aiming to solve the efficiency problem of traditional models for processing long sequential texts. Methodologically, the model adopts principal component analysis for dimensionality reduction of word vectors, uses BiLSTM network to extract text features, and introduces the self-attention mechanism to give different weights to the text. The experiments are carried out on MRD and SST datasets, and the results show that: the training time on the two datasets using the PCA-VSM model is 107s and 104s respectively, which is much better than other models; the model has the highest accuracy when the cumulative contribution rate of the feature values is 90%; under the optimal parameter configurations, the BLEU metrics of this paper’s model on the MRD and SST datasets respectively reach 0.369 and 0.381, and the Rouge-L metrics are 0.489 and 0.488 respectively, which are significantly better than the other compared models. It is shown that the self-attention mechanism model based on low-dimensional embedding space can effectively improve the performance of natural language processing tasks.
