Multiclass Least Squares Twin Support Vector Machine for Pattern Classification

Divya Tomar,Sonali Agarwal 보안공학연구지원센터 2015 International Journal of Database Theory and Appli Vol.8 No.6

This paper proposes a Multiclass Least Squares Twin Support Vector Machine (MLSTSVM) classifier for multi-class classification problems. The formulation of MLSTSVM is obtained by extending the formulation of recently proposed binary Least Squares Twin Support Vector Machine (LSTSVM) classifier. For M-class classification problem, the proposed classifier seeks M-non parallel hyper-planes, one for each class, by solving M-linear equations. A regularization term is also added to improve the generalization ability. MLSTSVM works well for both linear and non-linear type of datasets. It is relatively simple and fast algorithm as compared to the other existing approaches. The performance of proposed approach has been evaluated on twelve benchmark datasets. The experimental result demonstrates the validity of proposed MLSTSVM classifier as compared to the typical multi-classifiers based on ‘Support Vector Machine’ and ‘Twin Support Vector Machine’. Statistical analysis of the proposed classifier with existing classifiers is also performed by using Friedman’s Test statistic and Nemenyi post hoc techniques.

희박한 최소 절대 편차 지지벡터기계

정강모(Kang-Mo Jung) 한국자료분석학회 2023 Journal of the Korean Data Analysis Society Vol.25 No.5

고전적인 지지기계벡터가 어떤 부등식 제약 조건에서 최적화 문제의 해를 구하는 것에 비해 최소 제곱 지지기계벡터는 이 부등식 제약 조건을 등식 제약 조건으로 변환하여 문제의 해를 구한다. 따라서 최소 제곱 지지기계벡터는 행렬을 이용하여 정확 해를 구할 수 있어 회귀와 분류문제의 많은 분야에서 탁월한 성과를 이뤘다. 그러나 최소 제곱 지지기계벡터에서 구한 해는 이상치에 민감하고, 고전적인 지지기계벡터의 장점인 희박한 지지벡터를 제공하지 못한다는 단점이 있다. 이를 해결하기 위해 본 논문에서는 최소 절댓값 손실함수를 이용함으로써 이상치에 강건한 최소 절대 편차 지지기계벡터의 해를 구한다. 또한, 지지벡터의 희박성을 위해 재귀적 축소 최소 제곱 지지기계벡터를 이용하는 방법을 제시하고자 한다. 최소 절댓값 손실함수의 최적화 문제를 해결하기 위해 분리-브레그만 반복 방법을 사용하여 정확한 해를 구하였다. 본 논문에서 제시한 방법은 기존의 최소 제곱 지지기계벡터가 가지는 단점을 극복하는 효율적인 방법으로 간단한 수치 자료와 벤치마크 자료의 분석 결과가 해의 강건성과 희박성 측면에서 기존 결과와 비교할 만한 수준을 보였다. The support vector machine solves a quadratic programming problem with linear inequality and equality constraints. However, it is not trivial to solve the quadratic problem. The least squares support vector machine(LS-SVM) solves a linear system by equality constraints instead of inequality constraints. LS-SVM is a popular method in regression and classification problems, because it effectively solves simple linear systems. There are two issues with the LS-SVM solution : the lack of robustness to outliers and the absence of sparseness. In this paper, we propose a sparse and robust support vector machine for regression problems using the least absolute deviation support vector machine (LAD-SVM) and recursive reduced LS-SVM (RR-LS-SVM). The split-Bregman iteration gives the exact solution for the LAD-SVM problem, while RR-LS-SVM gives a sparse solution with a much smaller number of all support vectors. Numerical experiments with simulation and benchmark data demonstrate that the proposed algorithm can achieve comparable performance to other methods in terms of robustness and sparseness.

A transductive least squares support vector machine with the difference convex algorithm

Jooyong Shim,Kyungha Seok 한국데이터정보과학회 2014 한국데이터정보과학회지 Vol.25 No.2

Unlabeled examples are easier and less expensive to obtain than labeled exam-ples. Semisupervised approaches are used to utilize such exam-ples in an effort to boost the predictive performance. This paper proposes a novel semisupervised classification method named transductive least squares support vector machine (TLS-SVM), which is based on the least squares support vector machine. The proposed method utilizes the difference convex algorithm to derive nonconvex minimization solutions for the TLS-SVM. A generalized cross validation method is also developed to choose the hy-perparameters that affect the performance of the TLS-SVM. The experimental results confirm the successful performance of the proposed TLS-SVM.

A transductive least squares support vector machine with the difference convex algorithm

Shim, Jooyong,Seok, Kyungha The Korean Data and Information Science Society 2014 한국데이터정보과학회지 Vol.25 No.2

Unlabeled examples are easier and less expensive to obtain than labeled examples. Semisupervised approaches are used to utilize such examples in an eort to boost the predictive performance. This paper proposes a novel semisupervised classication method named transductive least squares support vector machine (TLS-SVM), which is based on the least squares support vector machine. The proposed method utilizes the dierence convex algorithm to derive nonconvex minimization solutions for the TLS-SVM. A generalized cross validation method is also developed to choose the hyperparameters that aect the performance of the TLS-SVM. The experimental results conrm the successful performance of the proposed TLS-SVM.

A transductive least squares support vector machine with the difference convex algorithm

심주용,석경하 한국데이터정보과학회 2014 한국데이터정보과학회지 Vol.25 No.2

Unlabeled examples are easier and less expensive to obtain than labeled examples. Semisupervised approaches are used to utilize such examples in an effort to boost the predictive performance. This paper proposes a novel semisupervised classification method named transductive least squares support vector machine (TLS-SVM), whichis based on the least squares support vector machine. The proposed method utilizesthe difference convex algorithm to derive nonconvex minimization solutions for the TLS-SVM. A generalized cross validation method is also developed to choose the hyperparameters that affect the performance of the TLS-SVM. The experimental results confirm the successful performance of the proposed TLS-SVM.

Least-Squares Support Vector Machine for Regression Model with Crisp Inputs-Gaussian Fuzzy Output

Chang Ha Hwang 한국데이터정보과학회 2004 한국데이터정보과학회지 Vol.15 No.2

Least-squares support vector machine (LS-SVM) has been very successful in pattern recognition and function estimation problems for crisp data. In this paper, we propose LS-SVM approach to evaluating fuzzy regression model with multiple crisp inputs and a Gaussian fuzzy output. The proposed algorithm here is model-free method in the sense that we do not need assume the underlying model function. Experimental result is then presented which indicate the performance of this algorithm.

Least-squares support vector machine for regression model with crisp inputs-gaussian fuzzy output

황창하 한국데이터정보과학회 2004 한국데이터정보과학회지 Vol.15 No.2

Least-squares support vector machine (LS-SVM) has been very successful in pattern recognition and function estimation problems for crisp data. In this paper, we propose LS-SVM approach to evaluating fuzzy regression model with multiple crisp inputs and a Gaussian fuzzy output. The proposed algorithm here is model-free method in the sense that we do not need assume the underlying model function. Experimental result is then presented which indicate the performance of this algorithm.

A Study on Support Vectors of Least Squares Support Vector Machine

Seok, Kyungha,Cho, Daehyun 한국통계학회 2003 Communications for statistical applications and me Vol.10 No.3

LS-SVM(Least-Squares Support Vector Machine) has been used as a promising method for regression as well as classification. Suykens et al.(2000) used only the magnitude of residuals to obtain SVs(Support Vectors). Suykens' method behaves well for homogeneous model. But in a heteroscedastic model, the method shows a poor behavior. The present paper proposes a new method to get SVs. The proposed method uses the variance of noise as well as the magnitude of residuals to obtain support vectors. Through the simulation study we justified excellence of our proposed method.

Least-Squares Support Vector Machine for Regression Model with Crisp Inputs-Gaussian Fuzzy Output

Hwang, Chang-Ha Korean Data and Information Science Society 2004 한국데이터정보과학회지 Vol.15 No.2

Least-squares support vector machine (LS-SVM) has been very successful in pattern recognition and function estimation problems for crisp data. In this paper, we propose LS-SVM approach to evaluating fuzzy regression model with multiple crisp inputs and a Gaussian fuzzy output. The proposed algorithm here is model-free method in the sense that we do not need assume the underlying model function. Experimental result is then presented which indicate the performance of this algorithm.

Adaptive States Feedback Control of Unknown Dynamics Systems Using Support Vector Machines

Wang, Fa-Guang,Kim, Min-Chan,Park, Seung-Kyu,Kwak, Gun-Pyong The Korea Institute of Information and Commucation 2008 Journal of information and communication convergen Vol.6 No.3

This paper proposes a very novel method which makes it possible that state feedback controller can be designed for unknown dynamic system with measurable states. This novel method uses the support vector machines (SVM) with its function approximation property. It works together with RLS (Recursive least-squares) algorithm. The RLS algorithm is used for the identification of input-output relationship. A virtual state space representation is derived from the relationship and the SVM makes the relationship between actual states and virtual states. A state feedback controller can be designed based on the virtual system and the SVM makes the controller with actual states. The results of this paper can give many opportunities that the state feedback control can be applied for unknown dynamic systems.