On this page

Research on Efficient Migration Learning Algorithms Based on Large Models

By: Kezhi Zhen1, Zheng Qi1, Xu Li1, Xin Yin1, Jifu Wang1, Jing Chen1
1China Tobacco Guizhou Industrial Co., LTD. Guiyang, Guizhou, 550001, China

Abstract

Natural language processing and computer vision technologies have greatly contributed to the development of large language models. In this paper, we focus on the introduction of Adaptive Module method in the pre-trained model to realize the efficient migration of the model and improve the performance of the model. In the applied research in the field of natural language processing, the Adapters module is introduced into the ALBERTBiLSTM-CRF model to tune the overall model. The adapter mechanism is utilized to improve the representation ability in the visual Transformer model. The results show that, through the comparative analysis of a large number of transfer learning methods, it can be seen that Adapters achieved a high average performance, with a tuning parameter number of only 0.23%. Therefore, Adapters is selected for the case study.The average number of parameters in the ALBERT-BiLSTM-CRF model with the addition of Adapters module is only 30M with an F1 value of 94.41%.The Adapters adapter component mechanism is capable of adapting to a wide range of downstream tasks and obtaining a better image representation.