Based on the systematic collection and classification of highly representative ethnic element patterns, this paper applies computer image recognition and extraction technology to complete the task of extracting the representative elements of the patterns, so as to provide pre-study basic materials for the subsequent modern housing design. The extracted color features, texture features and organization features are applied to the modern housing design, and the design effect is evaluated with the help of hierarchical analysis algorithm and fuzzy comprehensive evaluation. The weight of livability A1 is 0.2708, and its subordinate secondary indicators exist ethnic characteristics space layout B3 (0.111) > living comfort B2 (0.0908) > B4 cultural atmosphere creation (0.04) > B1 living comfort (0.029), and the results of the evaluation of the effect of the housing design are (4.8791, 1.5293, 1.3133, 1.5134, and 0.7649), and based on the principle of maximum affiliation, it is concluded that the housing design effect of integrating ethnic art elements in a neighborhood is excellent. This study provides reference for traditional elements and modern housing design, and has theoretical guidance value for sustainable development of architecture.