Seeking Gold in Sand Applying Random Matrix Theory to Separation of - PowerPoint PPT Presentation

Seeking Gold in Sand Applying Random Matrix Theory to Separation of Signals from Noise in Stock Market Data Mike S. Wang Department of Chemistry University of Cambridge August, 2016 Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Preview of the talk What is Random Matrix Theory? 1 Project investigations: Seeking Gold in Sand 2 Looking beyond Financial Applications 3 Q&A Time 4 Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

What is Random Matrix Theory? Random Matrix Theory (RMT) is the study of matrices with random variable entries , e.g.   X 11 X 12 · · · X 21 X 22 · · ·     . . . ... . . . . In particular, we are interested in the emergent behaviours of random matrices in the asymptotic limit. Introduced by Wishart (1928), RMT gained prominence when Wigner (1950) applied the theory in nuclear physics. Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Project overview We will • investigate the correlation matrix of stock market data (e.g. S&P 500); • given a ‘crude’ prediction, build improved analytic predictions for the purpose of signals/noise detection. Improving a ‘crude’ prediction mode analysis + clustering analysis ⇓ new correlation matrix model ⇓ RMT better analytic predictions for signal and noise separation ⇓ more profitable investment portfolios Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Preview of data: obtaining correlation matrix Price indices → logarithmic returns → de-meaned and normalised data: P stocks       126 . 8 30 . 5 · · · − 0 . 9 1 . 7 · · · 1 . 0 0 . 2 · · · 126 . 3 30 . 7 · · ·   1 . 5 0 . 3 · · · 0 . 2 1 . 0 · · ·     �→ �→ T days       . . ... . . . . ... ... . . . . . . . . . . . . � �� � � �� � � �� � raw data standardised data X correlation matrix E If X is the standardised data matrix, the correlation matrix E is calculated as E = X t X . T Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Visualisation and features of the correlation matrix 1 0.06 1.2 50 normalised distribution density 0.04 100 market mode 1 0.02 150 0.5 sector clustering 0.8 200 0 20 40 60 80 100 0.6 250 0 300 0.4 350 0.2 400 -0.5 0 450 0 2 4 6 8 10 100 200 300 400 eigenvalues (a) Eigenvalues → different modes (b) Heatmap → suggests clustering ⇓ Study of the structure Mode analysis : localisation of modes. Clustering analysis : hierarchical clustering structure. Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

A ‘crude’ prediction for signals and noise If the stock prices are independent and purely random, then whatever the distribution , then 0.9 √ 0.8 ( λ + − x )( x − λ ) 1 ν ( x ) = 2 π cx   0.7 c = P 1 T 0.6 λ ± = σ 2 (1 ± √ c ) 2 0.5 ν ( x ) ...   ⇒ 0.4   0.3 1 0.2 All noise!! 0.1 � �� � 0 0 0.5 1 1.5 2 2.5 3 x underlying correlation matrix Marˇ cenko-Pastur (M-P) distribution In reality, the stock prices of course cannot be completely independent and purely random, and we see that 1.4 histogram of observed eigenvalues 1.2 Marchenko-Pastur law normalised distribution density 1 0.8 ? ⇒ 0.6 0.4 signals 0.2 incorrect noise band 0 0 1 2 3 4 5 eigenvalues Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Mode analysis We have studied localisation of specific modes . 1st mode; eigenvalue = 99.1247; IPR = 0.0029781 452-th mode; eigenvalue = 0.059557; IPR = 0.14855 0.08 0.6 Materials Cons. D Cons. S Financials Industrials Materials Cons. D Cons. S Healthcare Industrials Energy Healthcare Telecom Energy IT Telecom Utilities 0.07 Financials IT Utilities 0.4 0.06 0.2 0.05 0.04 0 0.03 -0.2 0.02 -0.4 0.01 0 -0.6 0 50 100 150 200 250 300 350 400 450 0 50 100 150 200 250 300 350 400 450 (b) The lowest mode: localised. (a) The market mode: uniform. Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Sector analysis Having removed the market mode, the internal structure of the stock market is • revealed by the hierarchical clustering method; • confirmed by the minimum spanning tree . COH ORLY AZO TIF SPLS ROST RSH BBBY JWN HD BBY JCP KSS WMT TGT FDO COST LTD TJX GPS ANF WAG HCBK CVS 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 (a) A hierarchical dendrogram. (b) The minimum spanning tree. Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

A better prediction A new model for the underlying correlation matrix structure: What next? a model for underlying correlation matrix ⇓ better analytic predictions for noise bands ⇓ more profitable investment portfolios � ! Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Beyond financial applications Many other mathematical and scientific fields: • statistics and numerical analysis; • number theory; • theoretical neuroscience; • optimisation and control; • ... . . . and biochemistry! Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Application of RMT in biochemistry Covariance analysis of protein sequence alignments can be used to infer protein structure and function. Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Q&A time Thanks for listening! Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Challenges ahead We have made many simplifications in deriving a new analytic prediction. In particular, we could further investigate: • what is the optimal number of layers of hierarchy in the correlation matrix structure model? • how could we take time evolution of the market into account? • . . . Mike S. Wang Seeking Gold in Sand with Random Matrix Theory

Effect of the market mode: a visualisation 1 0.9 20 20 0.8 0.8 0.7 40 40 0.6 0.6 60 60 0.5 0.4 70 0.4 80 80 0.2 0.3 100 0.2 100 0 0.1 120 120 0 -0.2 20 40 60 70 80 100 120 20 40 60 80 100 120 (a) market mode unremoved (b) market mode removed Mike S. Wang Seeking Gold in Sand with Random Matrix Theory