회귀 최적화 기반 관련도와 중복도를 이용한 다중 레이블 특징 선별 기법 = Multi-label Feature Selection Using Redundancy and Relevancy based on Regression Optimization

다국어 초록 (Multilingual Abstract)

High-dimensional data causes difficulties in machine learning due to high time consumption and large memory requirements. In particular, in a multi-label environment, higher complexity is required as much as the number of labels. This paper proposes a feature selection method to improve classification performance in multi-label settings. The method considers three types of relationships: between features, between features and labels, and between labels themselves. To achieve this, a regression-based objective function is designed.
This objective function calculates the linear relationships between features and labels and uses mutual information to compute relationships between features and between labels. By minimizing this objective function, the optimal weights for feature selection are found. To optimize the objective function, a gradient descent method is applied to develop a fast-converging algorithm. The experimental results on six multi-label datasets show that the proposed method outperforms existing multi-label feature selection techniques. The classification performance of the proposed method, averaged over six datasets, showed a Hamming loss of 0.1285, a ranking loss of 0.1811, and a multi-label accuracy of 0.6416. Compared to the AMI(Approximating Mutual Information) algorithm, the performance was better by 0.0148, 0.0435, and 0.0852, respectively.

참고문헌 (Reference)

1 E. Smirni, "Workload-Aware Load Balancing for Cluster Web Servers" 16 (16): 219-232, 2005

2 E. Elhamifar, "Sparse subspace clustering : Algorithm, theory, and applications" 35 (35): 2765-2781, 2013

3 S. Sharmin, "Simultaneous feature selection and discretization based on mutual information" 91 : 162-174, 2019

4 J. Lee, "SCLS : Multi-label feature selection based on scalable criterion for large label set" 66 : 342-352, 2017

5 G. Tsoumakas, "Random k-labelsets : an ensemble method for multilabel classification" 406-417, 2007

6 S. Diplaris, "Protein Classification with Multiple Algorithms" 448-456, 2005

7 H. Lim, "Optimization approach for feature selection in multi-label classification" 89 : 25-30, 2017

8 J. Lee, "Mutual information-based multi-label feature selection using interaction information" 42 : 2013-2025, 2015

9 K. Trohidis, "Multilabel classification of music into emotions" 325-330, 2008

10 M. Faraji, "Multi-label feature selection with global and local label correlation" 246 : 123198-, 2024

11 Y. Li, "Multi-label feature selection via robust flexible sparse regularization" 134 : 109074-, 2023

12 J. Dai, "Multi-label feature selection by strongly relevant label gain and label mutual aid" 145 : 109945-, 2024

13 Y. Yang, "Multi-label feature selection based on stable label relevance and label-specific features" 648 : 119525-, 2023

14 Y. Lin, "Multi-label feature selection based on max-dependency and min-redundancy" 168 : 92-103, 2015

15 Y. Fan, "Multi-label feature selection based on label correlations and feature redundancy" 241 : 108256-, 2022

16 Z. He, "Multi-label feature selection based on correlation label enhancement" 647 : 119526-, 2023

17 M. -L. Zhang, "ML-KNN : A lazy learning approach to multi-label learning" 40 : 2038-2048, 2007

18 P. Zhang, "MFSJMI : Multi-label feature selection considering join mutual information and interaction weight" 138 : 109378-, 2023

19 M. R. Boutell, "Learning multi-labelscene classiffication" 37 : 1757-1771, 2004

20 Y. Fan, "Learning correlation information for multi-label feature selection" 145 : 109899-, 2024

21 Y. Li, "Label correlations variation for robust multi-label feature selection" 609 : 1075-1097, 2022

22 A. Clare, "Knowledge discovery in multi-label phenotype data" 42-53, 2001

23 L. Hu, "Feature-specific mutual information variation for multi-label feature selection" 593 : 449-471, 2022

24 M. L. Zhang, "Feature selection for multi-label naive Bayes classification" 179 : 3218-3229, 2009

25 J. Lee, "Feature selection for multi-label classification using multivariate mutual information" 34 : 349-357, 2013

26 H. Peng, "Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy" 27 : 1226-1238, 2005

27 J. Lee, "Fast multi-label feature selection based on information-theoretic feature ranking" 48 : 2761-2771, 2015

28 T. Klonecki, "Cost-constrained feature selection in multilabel classification using an information-theoretic approach" 141 : 1-18, 2023

29 R. B. Pereira, "Categorizing feature selection methods for multi-label classification" 49 : 57-78, 2018

30 J. Lee, "Approximating mutual information for multi-label feature selection" 48 : 1-2, 2012

31 J. P. Pestian, "A shared task involving multi-label classification of clinical free text" 97-104, 2007

32 J. Read, "A pruned problem transformation method for multi-label classification" 143-150, 2008

33 A. Elisseeff, "A kernel method for multi-labelled classification" 2001