Accuracy Estimation of a Classifier Based on the Differences in Samples

Min Zhang,Shengbo Yu 보안공학연구지원센터 2016 International Journal of Signal Processing, Image Vol.9 No.11

The classification accuracy is an important standard to measure the quality of the classifier. Usually, the classification accuracy is assessed later, not during the classification process. Problems such as classification accuracy drops cannot be timely and effectively found. It is necessary that marking test samples when estimating classification accuracy. It is a problem that we care about that how much is the classification accuracy when a group of new samples obtained. The problem must be concerned when using and improving the classifier in the case of growing data. To solve this problem, this paper put forward different estimates from different perspectives which based on the difference between samples. One estimate is based on the difference in samples distribution, which is from the Bayesian criterion. Another estimate is based on the difference in each sample instance, which is from the K nearest neighbor classification. Classification accuracy is also estimated by using the artificial neural networks, which combine the characteristics of the above two methods. And results show the proposed methods have good effects.

훈련 자료의 임의 선택과 다중 분류자를 이용한 원격탐사 자료의 분류

박노욱 ( No Wook Park ),유희영 ( Hee Young Yoo ),김이현 ( Yi Hyun Kim ),홍석영 ( Suk Young Hong ) 대한원격탐사학회 2012 大韓遠隔探査學會誌 Vol.28 No.5

이 논문에서는 원격탐사 자료의 분류를 목적으로 서로 다른 훈련 집단들과 분류자들로부터 생성된 분류 결과들을 결합하는 분류 틀을 제안하였다. 제안 분류 틀의 핵심 부분은 서로 다른 훈련 집단과 분류자들을 이용함으로써 분류 결과 사이의 다양성을 증가시켜서 결과적으로 분류 정확도를 향상시키는데 있다. 제안 분류 틀에서는 우선 서로 다른 샘플링 밀도를 가지는 서로 다른 훈련 집단들을 생성한 후에, 이들을 서로 다른 구분 능력을 나타내는 분류자들의 입력 훈련 자료로 사용한다. 그리고 초기 분류 결과들에 다수결규칙을 적용하여 최종 분류 결과를 얻게 된다. 다중 시기 ENVISAT ASAR 자료를 이용한 토지 피복 분류사례 연구를 통해 제안 방법론의 적용 가능성을 검토하였다. 사례 연구에서 3개의 훈련 집단과 최대우도 분류자, 다층 퍼셉트론 분류자, support vector machine 등과 같은 3개의 분류자를 이용한 9개의 분류 결과를 결합하였다. 사례 연구 결과, 제안 분류 틀 안에서 토지 피복 구분에 관한 상호 보완적인 정보의 이용이 가능해져서 가장 높은 분류 정확도를 나타내었다. 서로 다른 결합들을 비교하였을 때, 다양성이 크지 않은 분류 결과들을 결합한 경우에는 분류 정확도의 향상이 나타나지 않았다. 따라서 다중 분류 시스템의 설계시 분류자들의 다양성을 확보하는 것이 중요함을 확인할 수 있었다. In this paper, a classifier ensemble framework for remote sensing data classification is presented that combines classification results generated from both different training sets and different classifiers. A core part of the presented framework is to increase a diversity between classification results by using both different training sets and classifiers to improve classification accuracy, First, different training sets that have different sampling densities are generated and used as inputs for supervised classification using different classifiers that show different discrimination capabilities. Then several preliminary classification results are combined via a majority voting scheme to generate a final classification result. A case study of land-cover classification using multi-temporal ENVISAT ASAR data sets is carried out to illustrate the potential of the presented classification framework. In the case study, nine classification results were combined that were generated by using three different training sets and three different classifiers including maximum likelihood classifier, multi-layer perceptron classifier, and support vector machine. The case study results showed that complementary information on the discrimination of land-cover classes of interest would be extracted within the proposed framework and the best classification accuracy was obtained. When comparing different combinations, to combine any classification results where the diversity of the classifiers is not great didn`t show an improvement of classification accuracy. Thus, it is recommended to ensure the greater diversity between classifiers in the design of multiple classifier systems.

QuickBird 위성영상을 이용한 수종분류에서 픽셀과 분할기반 분류방법의 정확도 비교

정상영 ( Sang Young Chung ),임종수 ( Jong Su Yim ),신만용 ( Man Yong Shin ) 한국산림과학회 2011 한국산림과학회지 Vol.100 No.4

This study was conducted to compare classification accuracy by tree species using QuickBird imagery for pixel- and segment-based classifications that have been mostly applied to classify land covers. A total of 398 points was used as training and reference data. Based on this points, the points were classified into fourteen land cover classes: four coniferous and seven deciduous tree species in forest classes, and three non-forested classes. In pixel-based classification, three images obtained by using raw spectral values, three tasseled indices, and three components from principal component analysis were produced. For the both classification processes, the maximum likelihood method was applied. In the pixel-based classification, it was resulted that the classification accuracy with raw spectral values was better than those by the other band combinations. As resulted that, the segment-based classification with a scale factor of 50% provided the most accurate classification (overall accuracy:76% and value:0.74) compared to the other scale factors and pixel-based classification.

DINA 모형 하에서 MLE, MAP, EAP 능력 추정 방법의 분류 정확도 및 합치도 연구

강태훈(Taehoon Kang),심혜진(Hye-Jin Shim) 한국교육방법학회 2017 교육방법연구 Vol.29 No.1

본 연구에서는 DINA 모형 하에서 피험자 능력 모수를 추정할 수 있는 방법인 MLE, MAP, EAP 각각의 분류 정확도 및 상호 간의 합치도를 모의실험을 통하여 조사하였다. 선행연구와의 차별성을 확보하기 위하여 문항의 질, 검사 길이, 능력 정도, 인지요소 간 상관 정도, 그리고 검사 성격 등의 다양한 모의실험 요인을 고려하였다. 연구 결과, 개별 인지요소를 추정할 때의 정확성은 어느 방법을 사용하여도 모든 조건에서 약 75% 이상인 것으로 나타났다. 하지만 전체 인지요소를 모두 함께 정확 하게 추정하는 것은 상대적으로 어려운 것으로 나타났으며 특히 문항의 질이 좋지 않고 문항 수가적을 때에는 50% 초반까지 분류 정확도가 하락하였다. 능력추정 방법 간 결과를 비교해 보면, 개별인지요소를 추정할 때에는 EAP를 사용하는 것이 바람직하지만 전체 인지요소 모두를 정확하게 추정하는 것이 더 중요하다면 MLE/MAP를 선택하는 것이 나았다. 다음, 검사별 Q-matrix 구성에 따라서 능력 추정의 정확성에 영향을 미치는 것을 알 수 있었는데 검사 문항 수에 비하여 상대적으로 많은 인지요소를 고려할 경우 분류 정확도가 낮아짐을 확인하였다. 문항 수가 많을수록 그리고 문항의 질이 좋을수록 분류 정확성이 높아짐을 알 수 있었다. 흥미롭게도 특정 인지요소만을 측정하는 문항이 5문항 이상인 경우 문항의 질이 다소 하락하여도 높은 분류 정확도를 구할 수 있었다. 또한 인지요소 간의 상관 정도는 능력 추정의 정확도와 무관한 것으로 나타났다. In this simulation study, the classification and agreement of MLE, MAP, and EAP which are methods to estimate examinees’ ability parameters under the DINA model. For the purpose of ensuring differentiation from the previous studies, various simulation factors were considered such as quality of test items, test lengths, ability distributions, correlations among cognitive attributes, and natures of test. The results showed that the accuracy of estimating each individual cognitive attribute was higher than about 75% in all conditions using any of three methods. It was relatively difficult, however, to accurately estimate all the cognitive attributes together. Especially, when the number of test items is small, the classification accuracy has decreased to the early 50%. It was better to use EAP when estimating individual cognitive attribute. If it was more important to accurately estimate mastery pattern, however, it is better to choose MLE/MAP. Next, the Q-matrix configuration appeared to affect the accuracy of ability estimation. As the number of cognitive attributes came to be relatively larger compared to the number of related test items, the classification accuracy tended to be lower. The higher the number of items and the better the quality of items, the higher the classification accuracy. Interestingly, if the number of items that measure a specific cognitive attribute were more than 5, the ability estimation accuracy could be kept relatively high even though the quality of items dropped somewhat. Also, the amounts of correlations among cognitive attributes appeared not to be related to the accuracy of ability estimation.

Comparison of Visual Interpretation and Image Classification of Satellite Data

Lee, In-Soo,Shin, Dong-Hoon,Ahn, Seung-Mahn,Lee, Kyoo-Seock,Jeon, Seong-Woo The Korean Society of Remote Sensing 2002 大韓遠隔探査學會誌 Vol.18 No.3

The land uses of Korean peninsula are very complicated and high-density. Therefore, the image classification using coarse resolution satellite images may not provide good results for the land cover classification. The purpose of this paper is to compare the classification accuracy of visual interpretation with that of digital image classification of satellite remote sensing data such as 20m SPOT and 30m TM. In this study, hybrid classification was used. Classification accuracy was assessed by comparing each classification result with reference data obtained from KOMPSAT-1 EOC imagery, air photos, and field surveys.

대규모 준거참조평가에서 성취수준의 분류 일치도와 정확도 분석

송미영,김성숙,박인용 한국교육평가학회 2013 교육평가연구 Vol.26 No.2

이 연구는 우리나라의 대표적 대규모 준거참조평가인 국가수준 학업성취도 평가의 성취수준 분류에 대한 일치도와 정확도를 분석하였다. 초·중·고등학생들의 교과별 성취 수준은, 변형된 앙고프 방법을 활용하여 설정된 세 개의 분할점수를 기준으로 우수학력, 보통학력, 기초학력, 기초학력미달의 네 수준으로 분류된다. 베타-이항분포, 문항반응이론, 일반화가능도이론 등의 세 가지 방법을 적용하여 성취수준 분류의 신뢰도를 검토한 결과, 네 단계의 성취수준을 모두 고려할 때 분류 일치도 계수는 0.625~0.825, 우연에 의한 확률을 제거한 후 분류결정이 일치될 확률인 K 계수는 0.429~0.680, 분류 정확도 계수는 0.719~0.892로 학생들의 성취수준은 일관성 있고 정확하게 분류된 것으로 나타났다. 이와 같이 국가수준 학업성취도 평가의 신뢰도를 검증하고, 이러한 방법을 적용하여 성취수준을 여러 단계로 분류하는 준거참조평가의 신뢰도를 검증하고자 하는 경우에 고려해야 할 사항들에 대해 제언하였다. The study described procedures for estimating various indices of classification consistency and accuracy for multiple category classifications using data from a single test administration. The estimates of the classification consistency and accuracy indices were compared under beta binomial model, three-parameter logistic IRT model, and G theory. Using real data sets from the National Assessment of Educational Achievement, the reliability of criterion-referenced assessment was examined. The consistency coefficients, p and k ranged 0.625~0.825, and 0.429~0.680, respectively, and the accuracy coefficients did 0.719~0.892 for multiple classification. This results showed that the classification of the examinee were consistent and accurate.

Comparison of Visual Interpretation and Image Classification of Satellite Data

In Soo Lee,Dong Hoon Shin,Seung Mahn Ahn,Kyoo Seock Lee,Seong Woo Jeon 大韓遠隔探査學會 2002 大韓遠隔探査學會誌 Vol.18 No.3

The land uses of Korean peninsula are very complicated and high-density. Therefore, the image classification using coarse resolution satellite images may not provide good results for the land cover classification. The purpose of this paper is to compare the classification accuracy of visual interpretation with that of digital image classification of satellite remote sensing data such as 20m SPOT and 30m TM. In this study, hybrid classification was used. Classification accuracy was assessed by comparing each classification result with reference data obtained from KOMPSAT-1 EOC imagery, air photos, and field surveys.

Improving Classification Accuracy Using Missing Data Filling Algorithms for the Criminal Dataset

Cuicui Sun,Chunlong Yao,Lan Shen,Xiaoqiang Yu 보안공학연구지원센터 2016 International Journal of Hybrid Information Techno Vol.9 No.4

Predicting crime types by using classification algorithms can help to find factors affecting crimes and prevent crimes. Due to various reasons in the process of data collection, there are often a large number of missing values in actual criminal dataset, which seriously affects the classification accuracy. Therefore, based on mutual KNNI (K nearest neighbor imputation) algorithm and combined with GRA (Grey Relational Analysis) theory, a novel data filling algorithm called GMKNN is proposed in order to improve the classification accuracy. The algorithm replaces the Euclidean distance formula used in mutual KNNI algorithm with the Grey relational grade formula to eliminate the effect of noise from the nearest neighbors and effectively deal with the discrete attributes. By comparing with several popular data filling algorithms based on a real criminal dataset with lots of missing values, higher classification accuracy can be obtained by using GMKNN algorithm, which is up to 77.837%.

웨이브릿 기반 텍스처 융합 영상을 이용한 위성영상 자료의 분류 정확도 향상 연구

황화정 ( Hwa Jeong Hwang ),류희영 ( Hee Young Yoo ),이기원 ( Ki Won Lee ),권병두 ( Byung Doo Kwon ) 大韓遠隔探査學會 2007 大韓遠隔探査學會誌 Vol.23 No.2

지금까지 위성영상 정보 처리 분야에서는 분광정보를 이용한 영상분석과 시각적 해석 및 자동 분류에 대한 연구가 주로 수행되었으나, 최근에는 영상자료에서 시각적으로 나타나지 않는 특성이나 공간정보의 추출을 위한 여러 시도가 이루어지고 있다. 본 연구에서는 영상정보의 특성 추출기법인 텍스처 영상 생성기법과 웨이브릿 변환을 연계하여 웨이브릿 기반 텍스처 융합 영상에 대한 연구를 수행하였다. 또한 이러한 영상이 분류 정확도에 어떻게 기여하는 가를 분석하기 위한 적용 사례로 도심지 공간분석과 칼데라 주변지역의 지질학적 구조분석을 수행하였다. 영상 분석 시 공간정보 활용을 위한 텍스처 영상 생성기법과 웨이브릿 기반 텍스처 융합 영상 생성기법을 시용하면 원본영상만을 사용하였을 때보다 높은 분류정확도를 보였다. 고해상도 영상을 사용한 도심지의 경우 원본영상에 텍스처영상과 웨이브릿 기반 텍스처 융합 영상을 모두 활용한 경우의 분류정확도가 가장 높은 값을 보였다. 이는 상세화소의 변화가 매우 중요한 도심지의 특성상, 세밀한 공간정보가 최대로 활용되었기 때문으로 해석되어진다. 또한 중 저해상도 영상을 사용한 지질학적 구조분석의 경우 원본영상에 텍스처 영상만을 활용한 경우가 가장 높은 분류정확도를 보였다. 이는 칼데라를 중심으로 한 비교적 크기가 큰 지질학적 구조 분석 시 고도변화와 지열분포 등의 정보가 적당히 단순화 될 필요가 있었기 때문인 것으로 해석된다. 따라서 이러한 기법들을 실제 연구에 적용하기 위해서는 연구의 목적과 위성영상의 해상도 등의 정보를 모두 고려하여 적절한 기법을 잘 적용하는 것이 중요하다. The spectral information based image analysis, visual interpretation and automatic classification have been widely carried out so far for remote sensing data processing. Yet recently, many researchers have tried to extract the spatial information which cannot be expressed directly in the image itself. Using the texture and wavelet scheme, we made a wavelet-based texture fusion image which includes the advantages of each scheme. Moreover, using these schemes, we carried out image classification for the urban spatial analysis and the geological structure analysis around the caldera area. These two case studies showed that image classification accuracy of texture image and wavelet-based texture fusion image is better than that of using only raw image. In case of the urban area using high resolution image, as both texture and wavelet based texture fusion image are added to the original image, the classification accuracy is the highest. Because detailed spatial information is applied to the urban area where detail pixel variation is very significant. In case of the geological structure analysis using middle and low resolution image, the images added by only texture image showed the highest classification accuracy. It is interpreted to be necessary to simplify the information such as elevation variation, thermal distribution, on the occasion of analyzing the relatively larger geological structure like a caldera. Therefore, in the image analysis using spatial information, each spatial information analysis method should be carefully selected by considering the characteristics of the satellite images and the purpose of study.

Integration of Multi-spectral Remote Sensing Images and GIS Thematic Data for Supervised Land Cover Classification

Dong Ho Jang,Chang Jo Chung 大韓遠隔探査學會 2004 大韓遠隔探査學會誌 Vol.20 No.5

Nowadays, interests in land cover classification using not only multi-sensor images but also thematic GIS information are increasing. Often, although useful GIS information for the classification is available, the traditional MLE (maximum likelihood estimation techniques) does not allow us to use the information, due to the fact that it cannot handle the GIS data properly. This paper propose two extended MLE algorithms that can integrate both remote sensing images and GIS thematic data for land-over classification. They include modified MLE and Bayesian predictive likelihood estimation technique (BPLE) techniques that can handle both categorical GIS thematic data and remote sensing images in an integrated manner. The proposed algorithms were evaluated through supervised land-cover classification with Landsat ETM+ images and an existing land-use map in the Gongju area, Korea. As a result, the proposed method showed considerable improvements in classification accuracy, when compared with other multi-spectral classification techniques. The integration of remote sensing images and the land-use nap showed that overall accuracy indicated an improvement in classification accuracy of 10.8% when using MLE, and 9.6% for the BPLE. The case study also showed that the proposed algorithms enable the extraction of the area with land-cover change. In conclusion, land cover classification results produced through the integration of various GIS spatial data and multi-spectral images, will be useful to involve complementary data to make more accurate decisions.