CS6220: DATA MINING TECHNIQUES Mining Time Series Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 30, 2015
Announcement • No class next week and see you on Dec. 14. • The final report and presentation guideline is going to be released soon. • Office hour: • Tuesday: 3:30-5:00pm • Friday: 2:30-4:00pm 2
Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification Decision Tree; HMM Label Neural Naïve Bayes; Propagation* Network Logistic Regression SVM; kNN Clustering K-means; PLSA SCAN*; hierarchical Spectral clustering; DBSCAN; Clustering* Mixture Models; kernel k-means* Apriori; GSP; Frequent FP-growth PrefixSpan Pattern Mining Prediction Linear Regression Autoregression Similarity DTW P-PageRank Search PageRank Ranking 3
Mining Time Series Data • Basic Concepts • Time Series Prediction and Forecasting • Time Series Similarity Search • Summary 4
Example: Inflation Rate Time Series 5
Example: Unemployment Rate Time Series 6
Example: Stock 7
Example: Product Sale 8
Time Series • A time series is a sequence of numerical data points, measured typically at successive times, spaced at (often uniform) time intervals • Random variables for a time series are Represented as: • 𝑍 = 𝑍 1 , 𝑍 2 , … , 𝑝𝑠 • 𝑍 = 𝑍 𝑢 : 𝑢 ∈ 𝑈 , 𝑥ℎ𝑓𝑠𝑓 𝑈 𝑗𝑡 𝑢ℎ𝑓 𝑗𝑜𝑒𝑓𝑦 𝑡𝑓𝑢 • An observation of a time series with length N is represent as: • 𝑍 = {𝑧 1 , 𝑧 2 , … , 𝑧 𝑂 } 9
Mining Time Series Data • Basic Concepts • Time Series Prediction and Forecasting • Time Series Similarity Search • Summary 10
Categories of Time-Series Movements • Categories of Time-Series Movements (T, C, S, I) • Long-term or trend movements (trend curve): general direction in which a time series is moving over a long interval of time • Cyclic movements or cycle variations: long term oscillations about a trend line or curve • e.g., business cycles, may or may not be periodic • Seasonal movements or seasonal variations • E.g., almost identical patterns that a time series appears to follow during corresponding months of successive years. • Irregular or random movements 11
12
Lag, Difference • The first lag of 𝑍 𝑢 is 𝑍 𝑢−1 ; the jth lag of 𝑍 𝑢 is 𝑍 𝑢−𝑘 • The first difference of a time series, Δ𝑍 𝑢 = 𝑍 𝑢 − 𝑍 𝑢−1 • Sometimes difference in logarithm is used Δln(𝑍 𝑢 ) = ln(𝑍 𝑢 ) − ln(𝑍 𝑢−1 ) 13
Example: First Lag and First Difference 14
Autocorrelation • Autocorrelation: the correlation between a time series and its lagged values • The first autocorrelation 𝜍 1 • The jth autocorrelation 𝜍 𝑘 Autocovariance 15
Sample Autocorrelation Calculation • The jth sample autocorrelation 𝑑𝑝𝑤(𝑍 𝑢 ,𝑍 𝑢−𝑘 ) • 𝜍 𝑘 = 𝑍 𝑍 𝑢 𝑢−𝑘 𝑤𝑏𝑠(𝑍 𝑢 ) 𝑧 𝑘+1 𝑧 1 • Where 𝑑𝑝𝑤(𝑍 𝑢 , 𝑍 𝑢−𝑘 ) is calculated as: 𝑧 𝑘+2 𝑧 2 ⋮ ⋮ 𝑧 𝑈−1 𝑧 𝑈−𝑘−1 𝑧 𝑈 𝑧 𝑈−𝑘 • i.e., considering two time series: Y(1,…,T -j) and Y(j+1,…,T) 16
Example of Autocorrelation • For inflation and its change 𝝇 𝟐 = 𝟏. 𝟗𝟔 , very high: Last quarter’s inflation rate contains much information about this quarter’s inflation rate 17
Focus on Stationary Time Series • Stationary is key for time series regression: Future is similar to the past in terms of distribution 18
Autoregression • Use past values 𝑍 𝑢−1, 𝑍 𝑢−2 , … to predict 𝑍 𝑢 • An autore toregressi gression on is a regression model in which Y t is regressed against its own lagged values. • The number of lags used as regressors is called the or order er of the autoregression. • In a first order autoregression , Y t is regressed against Y t – 1 • In a p th order autoregression , Y t is regressed against Y t – 1 , Y t – 2 ,…, Y t – p 19
The First Order Autoregression Model AR(1) • AR(1) model: • The AR(1) model can be estimated by OLS regression of Y t against Y t – 1 • Testing β 1 = 0 vs. β 1 ≠ 0 provides a test of the hypothesis that Y t – 1 is not useful for forecasting Y t 20
Prediction vs. Forecast • A predicted value refers to the value of Y predicted (using a regression) for an observation in the sample used to estimate the regression – this is the usual definition • Predicted values are “in sample” • A forecast refers to the value of Y forecasted for an observation not in the sample used to estimate the regression. • Forecasts are forecasts of the future – which cannot have been used to estimate the regression. 21
Time Series Regression with Additional Predictors • So far we have considered forecasting models that use only past values of Y • It makes sense to add other variables ( X ) that might be useful predictors of Y , above and beyond the predictive value of lagged values of Y : • 22
Mining Time Series Data • Basic Concepts • Time Series Prediction and Forecasting • Time Series Similarity Search • Summary 23
Why Similarity Search? • Wide applications • Find a time period with similar inflation rate and unemployment time series? • Find a similar stock to Facebook? • Find a similar product to a query one according to sale time series? • … 24
Example VanEck International Fund Fidelity Selective Precious Metal and Mineral Fund Two similar mutual funds in the different fund group 25
Similarity Search for Time Series Data • Time Series Similarity Search • Euclidean distances and 𝑀 𝑞 norms • Dynamic Time Warping (DTW) • Time Domain vs. Frequency Domain 26
Euclidean Distance and Lp Norms • Given two time series with equal length n • 𝐷 = 𝑑 1 , 𝑑 2 , … , 𝑑 𝑜 • 𝑅 = 𝑟 1 , 𝑟 2 , … , 𝑟 𝑜 • 𝑒 𝐷, 𝑅 = ∑|𝑑 𝑗 − 𝑟 𝑗 | 𝑞 1/𝑞 • When p=2, it is Euclidean distance 27
Enhanced Lp Norm-based Distance • Issues with Lp Norm: cannot deal with offset and scaling in the Y-axis • Solution: normalizing the time series ′ = 𝑑 𝑗 −𝜈(𝐷) • 𝑑 𝑗 𝜏(𝐷) 28
Dynamic Time Warping (DTW) • For two sequences that do not line up well in X-axis, but share roughly similar shape • We need to warp the time axis to make better alignment 29
Goal of DTW • Given • Two sequences (with possible different lengths): • 𝑌 = {𝑦 1 , 𝑦 2 , … , 𝑦 𝑂 } • 𝑍 = {𝑧 1 , 𝑧 2 , … , 𝑧 𝑁 } • A local distance (cost) measure between 𝑦 𝑜 and 𝑧 𝑛 • Goal: • Find an alignment between X and Y, such that, the overall cost is minimized 30
Cost Matrix of Two Time Series 31
Represent an Alignment by Warping Path • An (N,M)-warping path is a sequence 𝑞 = (𝑞 1 , 𝑞 2 , … , 𝑞 𝑀 ) with 𝑞 𝑚 = (𝑜 𝑚 , 𝑛 𝑚 ) , satisfying the three conditions: • Boundary condition: 𝑞 1 = 1,1 , 𝑞 𝑀 = 𝑂, 𝑁 • Starting from the first point and ending at last point • Monotonicity condition: 𝑜 𝑚 and 𝑛 𝑚 are non- decreasing with 𝑚 • Step size condition: 𝑞 𝑚+1 − 𝑞 𝑚 ∈ 0,1 , 1,0 , 1,1 • Move one step right, up, or up-right 32
Q: Which Path is a Warping Path? 33
Optimal Warping Path • The total cost given a warping path p • 𝑑 𝑞 𝑌, 𝑍 = ∑ 𝑚 𝑑(𝑦 𝑜 𝑚 , 𝑧 𝑛 𝑚 ) • The optimal warping path p* • 𝑑 𝑞 ∗ 𝑌, 𝑍 = min 𝑑 𝑞 𝑌, 𝑍 𝑞 𝑗𝑡 𝑏𝑜 𝑂, 𝑁 − 𝑥𝑏𝑠𝑞𝑗𝑜 𝑞𝑏𝑢ℎ • DTW distance between X and Y is defined as: • the optimal cost 𝑑 𝑞 ∗ 𝑌, 𝑍 34
How to Find p*? • Naïve solution: • Enumerate all the possible warping path • Exponential in N and M! 35
Dynamic Programming for DTW • Dynamic programming: • Let D(n,m) denote the DTW distance between X(1,…,n) and Y(1,…,m ) • D is called accumulative cost matrix • Note D(N,M) = DTW(X,Y) • Recursively calculate D(n,m) • 𝐸 𝑜, 𝑛 = min 𝐸 𝑜 − 1, 𝑛 , 𝐸 𝑜, 𝑛 − 1 , 𝐸 𝑜 − 1, 𝑛 − 1 + 𝑑(𝑦 𝑜 , 𝑧 𝑛 ) • When m or n = 1 • 𝐸 𝑜, 1 = ∑ 𝑙=1:𝑜 𝑑 𝑦 𝑙 , 1 ; Time complexity: O(MN) • 𝐸 1, 𝑛 = ∑ 𝑙=1:𝑛 𝑑 1, 𝑧 𝑙 ; 36
Trace back to Get p* from D 37
Example 38
Time Domain vs. Frequency Domain • Many techniques for signal analysis require the data to be in the frequency domain • Usually data-independent transformations are used • The transformation matrix is determined a priori • discrete Fourier transform (DFT) • discrete wavelet transform (DWT) • The distance between two signals in the time domain is the same as their Euclidean distance in the frequency domain 39
Example of DFT 40
41
Example of DWT (with Harr Wavelet) 42
43
Recommend
More recommend
