Distortion-free predictive streaming time-series matching

Loh, W.K.,Moon, Y.S.,Srivastava, J. North-Holland [etc ; Elsevier Science Ltd 2010 Information sciences Vol.180 No.8

Efficient processing of streaming time-series generated by remote sensors and mobile devices has become an important research area. As in traditional time-series applications, similarity matching on streaming time-series is also an essential research issue. To obtain more accurate similarity search results in many time-series applications, preprocessing is performed on the time-series before they are compared. The preprocessing removes distortions such as offset translation, amplitude scaling, linear trends, and noise inherent in time-series. In this paper, we propose an algorithm for distortion-free predictive streaming time-series matching. Similarity matching on streaming time-series is saliently different from traditional time-series in that it is not feasible to directly apply the traditional algorithms for streaming time-series. Our algorithm is distortion-free in the sense that it performs preprocessing on streaming time-series to remove offset translation and amplitude scaling distortions at the same time. Our algorithm is also predictive, since it performs streaming time-series matching against the predicted most recent subsequences in the near future, and thus improves search performance. To the best of our knowledge, no streaming time-series matching algorithm currently performs preprocessing and predicts future search results simultaneously.

Korean National Income Based on a Chain Index: 1953~2010

박창귀 한국개발연구원 2012 KDI Journal of Economic Policy (KDI JEP) Vol.34 No.3

Korea’s national income statistics have been compiled by the Bank of Korea since 1953. However, there is a break in the time series. The current time series (1970 onward) is based on the ‘1993 SNA (System of National Accounts)’ suggested by the UN, and the previous time series (1953~1970) was based on the ‘1953 SNA’. The difference between the previous and current time series is 4.8% in 1970 when the two series overlap. The difference is even greater in terms of comparisons across industries. In addition, it has now become even more difficult to connect the current and the previous time series because, in 2009, the Bank of Korea introduced a chain weighted method for calculating the current time series (1970 onward). Under the chain weighted method, the time series underwent substantial modification; for instance, the economic growth rate during 1970~2005 is 0.9%p higher than the rate under the general method. This paper applies chain weighted values and the ‘1993 SNA’ to the previous time series (1953~1970) by utilizing various national account manuals published by the UN and previous Korean input-output tables in order to calculate a long term time series from 1953 to 2010 based on the same criteria as the current time series (1970 onward). In the revised time series, it appears that 1953 GDP at current basic prices is 3.5% higher and the growth rate for the period of 1953~1970 is 1.5%p higher each year than under the previous time series. Under the revised time series the size of the Korean economy as of 2010 is 50-fold bigger than that of 1953. In terms of industries, manufacturing and SOC show significant expansion whereas the extent of that of the service industry is relatively small.

서울 프라임 오피스 이산적 가격시계열 보간에 관한 연구

노상윤 한국감정원 2017 부동산분석 Vol.3 No.2

This study presents an objective method to generate a stable and continuous time series by interpolating and predicting the Seoul prime office price time series. The transaction price time series is important for calculating ROC and capitalization rate in the real estate investment market. It is also essential information for monitoring the market, to establish an investment strategy or creating a benchmark index for rational performance evaluation. The linear interpolation was found to be the most efficient method for interpolating the price time series. The results of analysis using an ETS model showed that the price time series varied with determinants such as level, trend, and seasonality; thus the time series trends of transaction prices for the three major business districts and the other district could be analyzed and forecast by the time series of the decomposed factors. In addition, based on the time series of calculated transaction prices for the three major business districts and the other district, a time series of the average price of Seoul prime offices could be generated. As a result of decomposing this time series through the ETS model, it was found that the transaction price time series rose sharply until 2008 and then turned into a pattern degressive increase. The price time series shows a fixed seasonality in four-year intervals, and it is expected that the time series will enter a new cycle after 3Q of 2015 and that the prices of Seoul prime offices will be adjusted for the next two to three years. 본 연구는 서울 프라임 오피스 매매가격 시계열을 보간하고 추정하여 안정적이고 연속적인 시계열을 생성하는 객관적인 방법을 제시한 연구이다. 매매가격시계열은 투자시장의 자본수익률과 자본환원율을 산출하는 데에 중요한 정보이다. 또한 시장을 모니터하여 투자전략을 수립하거나 합리적인 성과평가를 위한 벤치마크 지수를 생성하는 데에도 꼭 필요한 정보이다. 매매가격 시계열의 보간에는 선형보간법이 가장 효율적이었다. ETS모형을 활용하여 분석한 결과 매매가격 시계열은 수준, 추세, 그리고 계절성 등의 결정요인들에 의해 변화됨을 확인할 수 있었다. 따라서 분해되어진 이러한 요인 시계열들을 통해 3대 권역과 기타(ETC) 권역의 매매가격 시계열 추이를 분석하고 전망할 수 있었다. 아울러 3대 권역 및 기타 권역의 산출된 매매가격 시계열을 기초로 서울 프라임 오피스 평균매매가격 시계열을 생성할 수 있었다. 이 시계열을 ETS모형을 통해 분해한 결과 서울 매매가격 시계열은 2008년까지 급격하게 상승하다가 이후 체감적인 상승패턴으로 변화되었다. 4년을 주기로 일정한 계절성을 보이고 있었는데, 2015년 3분기 이후 새로운 순환주기에 진입한 것으로 분석되어 향후 2~3년간 서울 프라임 오피스 매매가격은 일정한 조정을 받을 것으로 전망되었다.

Issues Related to the Use of Time Series in Model Building and Analysis: Review Article

Wei, William W.S. The Korean Statistical Society 2015 Communications for statistical applications and me Vol.22 No.3

Time series are used in many studies for model building and analysis. We must be very careful to understand the kind of time series data used in the analysis. In this review article, we will begin with some issues related to the use of aggregate and systematic sampling time series. Since several time series are often used in a study of the relationship of variables, we will also consider vector time series modeling and analysis. Although the basic procedures of model building between univariate time series and vector time series are the same, there are some important phenomena which are unique to vector time series. Therefore, we will also discuss some issues related to vector time models. Understanding these issues is important when we use time series data in modeling and analysis, regardless of whether it is a univariate or multivariate time series.

Clustering Algorithm for Time Series with Similar Shapes

( Jungyu Ahn ),( Ju-hong Lee ) 한국인터넷정보학회 2018 KSII Transactions on Internet and Information Syst Vol.12 No.7

Since time series clustering is performed without prior information, it is used for exploratory data analysis. In particular, clusters of time series with similar shapes can be used in various fields, such as business, medicine, finance, and communications. However, existing time series clustering algorithms have a problem in that time series with different shapes are included in the clusters. The reason for such a problem is that the existing algorithms do not consider the limitations on the size of the generated clusters, and use a dimension reduction method in which the information loss is large. In this paper, we propose a method to alleviate the disadvantages of existing methods and to find a better quality of cluster containing similarly shaped time series. In the data preprocessing step, we normalize the time series using z-transformation. Then, we use piecewise aggregate approximation (PAA) to reduce the dimension of the time series. In the clustering step, we use density-based spatial clustering of applications with noise (DBSCAN) to create a precluster. We then use a modified K-means algorithm to refine the preclusters containing differently shaped time series into subclusters containing only similarly shaped time series. In our experiments, our method showed better results than the existing method.

이상탐지 기반의 효율적인 시계열 유사도 측정 및 순위화

최지현 ( Ji-hyun Choi ),안현 ( Hyun Ahn ) 한국인터넷정보학회 2024 인터넷정보학회논문지 Vol.25 No.2

시계열 분석은 시간 순서로 정렬된 데이터로부터 다양한 정보와 인사이트를 발견하기 위한 방법으로 많은 조직에서 비즈니스 문제 해결을 위해 적용하고 있다. 그중에서 시계열 유사도 측정은 패턴이 비슷한 시계열들을 식별하기 위한 단계로서 시계열 검색 및 군집화와 같은 시계열 분석 응용에서 매우 중요하다. 본 연구에서는 전체 시계열이 아닌 이상치들을 중심으로 시계열 유사도 측정을 계산효율적으로 수행하는 방법을 제안한다. 이와 관련하여 이상탐지를 통해 추출된 서브시퀀스 집합에 대한 유사도 측정 결과와 시계열 전체에 대한 유사도 측정 결과 사이의 순위 상관관계를 측정 및 분석하여 제안 방법을 검증한다. 실험 결과로써, 주식 종목 시계열 데이터에 이상치 비율 10%을 적용한 유사도 측정으로부터 최대 0.9 이상의 스피어만 순위 상관계수를 확인하였다. 결론적으로 제안 방법을 통해 시계열 유사도 측정에 소요되는 계산량을 유의미하게 절감하는 동시에 신뢰 가능한 시계열 검색 및 군집화 결과를 기대할 수 있다. Time series analysis is widely employed by many organizations to solve business problems, as it extracts various information and insights from chronologically ordered data. Among its applications, measuring time series similarity is a step to identify time series with similar patterns, which is very important in time series analysis applications such as time series search and clustering. In this study, we propose an efficient method for measuring time series similarity that focuses on anomalies rather than the entire series. In this regard, we validate the proposed method by measuring and analyzing the rank correlation between the similarity measure for the set of subsets extracted by anomaly detection and the similarity measure for the whole time series. Experimental results, especially with stock time series data and an anomaly proportion of 10%, demonstrate a Spearman’s rank correlation coefficient of up to 0.9. In conclusion, the proposed method can significantly reduce computation cost of measuring time series similarity, while providing reliable time series search and clustering results.

동조화 관계를 갖는 시계열을 위한 군집화 알고리즘

안준규,이주홍 한국지능시스템학회 2017 한국지능시스템학회논문지 Vol.27 No.6

Existing time series clustering algorithms are not suitable for finding co-movement relations in time series. In general, systems generating time series should take this into account for time series co-movement analysis because the state variables of the system change over time. In this paper, we propose a Co-movement time series clustering (CTC) algorithm to find time-series clusters with co-movement relations. The algorithm defines the WeightedDist function to consider the importance of time series data over time in co-movement analysis. In addition, the CTC algorithm includes a refinement process so that clusters generated by the algorithm do not include noise data that is significantly out of Co-movement. Experiments have shown that time-series clusters with Co-movement relationships are better found than comparative algorithms. 기존의 시계열 군집화 알고리즘들은 시계열의 동조화 관계를 찾는데 있어 적합하지 못하다. 일반적으로 시계열을생성하는 시스템은 시간의 흐름에 따라 시스템의 상태변수들이 변하기 때문에 시계열의 동조화분석에 이를 고려해야한다. 본 논문에서는 동조화 관계를 갖는 시계열 군집을 찾기 위해 CTC(Co-movement Time series Clustering)알고리즘을제안한다. 해당 알고리즘은 시간의 흐름에 따른 시계열 데이터의 중요도를 동조화분석에 고려하기 위하여 가중거리함수를정의하였다. 또한 CTC알고리즘에 정제과정을 포함하여 알고리즘에 의하여 생성된 군집에 동조화정도가 현저히 벗어나는노이즈 데이터가 포함되지 않도록 하였다. 실험을 통하여 동조화 관계를 갖는 시계열 군집를 비교 알고리즘들 보다 더 잘찾아주는 것을 보였다.

QP-DTW: Upgrading Dynamic Time Warping to Handle Quasi Periodic Time Series Alignment

( Imen Boulnemour ),( Bachir Boucheham ) 한국정보처리학회 2018 Journal of information processing systems Vol.14 No.4

Dynamic time warping (DTW) is the main algorithms for time series alignment. However, it is unsuitable for quasi-periodic time series. In the current situation, except the recently published the shape exchange algorithm (SEA) method and its derivatives, no other technique is able to handle alignment of this type of very complex time series. In this work, we propose a novel algorithm that combines the advantages of the SEA and the DTW methods. Our main contribution consists in the elevation of the DTW power of alignment from the lowest level (Class A, non-periodic time series) to the highest level (Class C, multiple-periods time series containing different number of periods each), according to the recent classification of time series alignment methods proposed by Boucheham (Int J Mach Learn Cybern, vol. 4, no. 5, pp. 537-550, 2013). The new method (quasi-periodic dynamic time warping [QP-DTW]) was compared to both SEA and DTW methods on electrocardiogram (ECG) time series, selected from the Massachusetts Institute of Technology - Beth Israel Hospital (MIT-BIH) public database and from the PTB Diagnostic ECG Database. Results show that the proposed algorithm is more effective than DTW and SEA in terms of alignment accuracy on both qualitative and quantitative levels. Therefore, QP-DTW would potentially be more suitable for many applications related to time series (e.g., data mining, pattern recognition, search/retrieval, motif discovery, classification, etc.).

Research on data augmentation algorithm for time series based on deep learning

Shiyu Liu,Hongyan Qiao,Lianhong Yuan,Yuan Yuan,Jun Liu 한국인터넷정보학회 2023 KSII Transactions on Internet and Information Syst Vol.17 No.6

Data monitoring is an important foundation of modern science. In most cases, the monitoring data is time-series data, which has high application value. The deep learning algorithm has a strong nonlinear fitting capability, which enables the recognition of time series by capturing anomalous information in time series. At present, the research of time series recognition based on deep learning is especially important for data monitoring. Deep learning algorithms require a large amount of data for training. However, abnormal sample is a small sample in time series, which means the number of abnormal time series can seriously affect the accuracy of recognition algorithm because of class imbalance. In order to increase the number of abnormal sample, a data augmentation method called GANBATS (GAN-based Bi-LSTM and Attention for Time Series) is proposed. In GANBATS, Bi-LSTM is introduced to extract the timing features and then transfer features to the generator network of GANBATS.GANBATS also modifies the discriminator network by adding an attention mechanism to achieve global attention for time series. At the end of discriminator, GANBATS is adding averagepooling layer, which merges temporal features to boost the operational efficiency. In this paper, four time series datasets and five data augmentation algorithms are used for comparison experiments. The generated data are measured by PRD(Percent Root Mean Square Difference) and DTW(Dynamic Time Warping). The experimental results show that GANBATS reduces up to 26.22 in PRD metric and 9.45 in DTW metric. In addition, this paper uses different algorithms to reconstruct the datasets and compare them by classification accuracy. The classification accuracy is improved by 6.44%-12.96% on four time series datasets.

정규화 변환을 지원하는 스트리밍 시계열 매칭 알고리즘

노웅기(Woong-Kee Loh),문양세(Yang-Sae Moon),김영국(Young-Kuk Kim) 한국정보과학회 2006 정보과학회논문지 : 데이타베이스 Vol.33 No.6

최근에 센서 및 모바일 장비들의 발전으로 인하여 이러한 장비들로부터 생성된 대량의 데이타 스트림(data stream)의 처리가 중요한 연구 과제가 되고 있다. 데이타 스트림 중에서 연속되는 시점에 얻어진 실수 값들의 스트림을 스트리밍 시계열(streaming time-series)이라 한다. 스트리밍 시계열에 대한 유사성 매칭은 여러 가지 고유 특성에 의하여 기존의 시계열 데이타와는 다르게 처리되어야 한다. 본 논문에서는 정규화 변환(normalization transform)을 지원하는 스트리밍 시계열 매칭 문제를 해결하기 위한 효율적인 알고리즘을 제안한다. 기존에는 스트리밍 시계열을 아무런 변환 없이 비교하였으나, 본 논문에서는 정규화 변환된 스트리밍 시계열을 비교한다. 정규화 변환은 절대적인 값은 달라도 유사한 변동 경향을 가지는 시계열 데이타를 찾기 위하여 유용하다. 본 논문의 공헌은 다음과 같다. (1) 기존의 정규화 변환을 지원하는 서브시퀀스 매칭 알고리즘[4]에서 제시된 정리(theorem)를 이용하여 정규화 변환을 지원하는 스트리밍 시계열 매칭 문제를 풀기 위한 간단한 알고리즘을 제안한다. (2) 검색 성능을 향상시키기 위하여 간단한 알고리즘을 k (≥ 1) 개의 인덱스를 이용하는 알고리즘으로 확장한다. (3) 주어진 k에 대하여, 확장된 알고리즘의 검색 성능을 최대화하기 위해 k 개의 인덱스를 생성할 최적의 윈도우 길이를 선택하기 위한 근사 방법(approximation)을 제시한다. (4) 스트리밍 시계열의 연속성(continuity) 개념[8]에 기반하여, 현재 시점 t?에서의 스트리밍 서브시퀀스에 대한 검색과 동시에 미래 시점 (t? + m - 1) (m ≥ 1)까지의 검색 결과를 한번의 인덱스 검색으로 구할 수 있도록 재차 확장한 알고리즘을 제안한다. (5) 일련의 실험을 통하여 본 논문에서 제안된 알고리즘들 간의 성능을 비교하고, k 및 m 값의 변화에 따라 제안된 알고리즘들의 검색 성능 변화를 보인다. 본 논문에서 제시한 정규화 변환 스트리밍 시계열 매칭 문제에 대한 연구는 이전에 수행된 적이 없으므로 순차 검색(sequential scan) 알고리즘과 성능을 비교한다. 실험결과, 제안된 알고리즘은 순차 검색에 비하여 최대 13.2배까지 성능이 향상되었으며, 인덱스의 개수 k가 증가함에 따라 검색 성능도 함께 증가하였다. According to recent technical advances on sensors and mobile devices, processing of data streams generated by the devices is becoming an important research issue. The data stream of real values obtained at continuous time points is called streaming time-series. Due to the unique features of streaming time-series that are different from those of traditional time-series, similarity matching problem on the streaming time-series should be solved in a new way. In this paper, we propose an efficient algorithm for streaming time-series matching problem that supports normalization transform. While the existing algorithms compare streaming time-series without any transform, the algorithm proposed in the paper compares them after they are normalization-transformed. The normalization transform is useful for finding time-series that have similar fluctuation trends even though they consist of distant element values. The major contributions of this paper are as follows. (1) By using a theorem presented in the context of subsequence matching that supports normalization transform[4], we propose a simple algorithm for solving the problem. (2) For improving search performance, we extend the simple algorithm to use k ( ≥ 1) indexes. (3) For a given k, for achieving optimal search performance of the extended algorithm, we present an approximation method for choosing k window sizes to construct k indexes. (4) Based on the notion of continuity[8] on streaming time-series, we further extend our algorithm so that it can simultaneously obtain the search results for m ( ≥ 1) time points from present t? to a time point (t? + m - 1) in the near future by retrieving the index only once. (5) Through a series of experiments, we compare search performances of the algorithms proposed in this paper, and show their performance trends according to k and m values. To the best of our knowledge, since there has been no algorithm that solves the same problem presented in this paper, we compare search performances of our algorithms with the sequential scan algorithm. The experiment result showed that our algorithms outperformed the sequential scan algorithm by up to 13.2 times. The performances of our algorithms should be more improved, as k is increased.