Navigating the Taxonomy of Large Language Models: A Comparative Exploration through Data Manipulation and Evaluation Techniques

Abstract

As the landscape of research on Large Language Models (LLMs) rapidly evolves, understanding the taxonomy of these models becomes increasingly crucial for researchers. This study explores the automatic classification of survey papers related to LLMs, utilizing graph representation learning and various data manipulation techniques. By collecting and analyzing metadata from 144 literature reviews, we construct co-category graphs to evaluate and compare the effectiveness of different classification paradigms, including pre-trained language models and graph neural networks. Our findings indicate that leveraging graph structures significantly enhances taxonomy classification performance compared to traditional language models. Additionally, we demonstrate the advantages of using weak labels generated from smaller models, revealing new insights into weak-to-strong generalization. This research not only contributes to the understanding of LLM taxonomy but also provides a framework for future explorations in the field, highlighting the importance of innovative evaluation methods in navigating the complexities of LLM research.