How to Detect How to Actually . . . Linear Dependence Resulting - PowerPoint PPT Presentation

Linear Dependencies . . . How Linear . . . Detecting Linear . . . Detecting Linear . . . How to Detect How to Actually . . . Linear Dependence Resulting Algorithm Heavy-Tailed . . . on the Copula Level? Utility: Reminder Case of Approximate . . . Vladik Kreinovich 1 , Hung T. Nguyen 2 , 3 , Home Page and Songsak Sriboonchitta 3 Title Page 1 Department of Computer Science, University of Texas at El Paso ◭◭ ◮◮ 500 W. University, El Paso, TX 79968, USA, vladik@utep.edu ◭ ◮ 2 Department of Mathematical Sciences, New Mexico State University Page 1 of 13 Las Cruces, New Mexico 88003, USA, hunguyen@nmsu.edu Go Back 3 Department of Economics, Chiang Mai University Full Screen Chiang Mai, Thailand, songsak@econ.chiangmai.ac.th Close Quit

Linear Dependencies . . . How Linear . . . 1. Linear Dependencies Are Ubiquitous Detecting Linear . . . • Dependencies between quantities are often described Detecting Linear . . . by smooth (even analytical) functions y = f ( x 1 , . . . , x n ): How to Actually . . . n n n Resulting Algorithm c i · ( x i − x (0) c ij · ( x i − x (0) i ) · ( x j − x (0) y = f ( x (0) )+ � � � i )+ j )+ . . . Heavy-Tailed . . . i =1 i =1 j =1 Utility: Reminder • For values x i close to x (0) i , we can safely ignore terms Case of Approximate . . . which are quadratic in x i − x (0) (or of higher order): Home Page i n Title Page c i · ( x i − x (0) y ≈ f ( x (0) ) + � i ) . ◭◭ ◮◮ i =1 ◭ ◮ • Linear dependencies often extend beyond local. Page 2 of 13 • Linear dependencies make computations easier; e.g.: Go Back – systems of linear equations can be efficiently solved, – systems of non-linear equations are NP-hard. Full Screen • It is thus important to detect linear dependencies. Close Quit

Linear Dependencies . . . How Linear . . . 2. How Linear Dependence is Detected Now Detecting Linear . . . • Exact linear dependence can be detected by whether Detecting Linear . . . the corr. system of linear equations has a solution: How to Actually . . . Resulting Algorithm n y ( k ) = f ( x (0) ) + � � x ( k ) − x (0) � c i · k = 1 , . . . , K. , Heavy-Tailed . . . i i i =1 Utility: Reminder Case of Approximate . . . • There exist efficient algorithms for checking solvability Home Page of such a linear system. Title Page • Approximate linearity can be gauged by computing the C XY ◭◭ ◮◮ Pearson’s correlation coefficient r ( F ) = . σ X · σ Y ◭ ◮ • In the case of an exact linear dependence Y = c 0 + c 1 · X , Page 3 of 13 we have r ( F ) = 1 if c 1 > 0 and r F = − 1 if c 1 < 0. Go Back • The square R 2 = ( r ( F )) 2 is a measure of fit with the linear model: the closer R 2 to 1, the better the fit. Full Screen Close Quit

Linear Dependencies . . . How Linear . . . 3. Detecting Linear Dependence Based on a Cop- Detecting Linear . . . ula: 1st Problem Detecting Linear . . . • A joint distribution of X and Y can be described by How to Actually . . . def the cdf F ( x, y ) = Prob ( X ≤ x & Y ≤ y ). Resulting Algorithm Heavy-Tailed . . . • This description depends on the units in which we de- Utility: Reminder scribe x and y : m to cm, log scale, etc. Case of Approximate . . . • A unit-independent description is known as a copula : Home Page def Title Page C ( a, b ) = F ( x, y ) for x, y s.t. a = F X ( x ) and b = F Y ( y ) . ◭◭ ◮◮ • Here, F ( x, y ) = C ( F X ( x ) , F Y ( y )), i.e., ◭ ◮ C ( a, b ) = F ( F − 1 X ( a ) , F − 1 Y ( b )) . Page 4 of 13 • How can we detect linear dependence between the quan- Go Back tities x and y based only on the copula C ( a, b )? Full Screen Close Quit

Linear Dependencies . . . How Linear . . . 4. Detecting Linear Dependence Based on a Cop- Detecting Linear . . . ula: Main Idea and the Resulting Definition Detecting Linear . . . • For a given copula C ( a, b ), possible cdfs are of the form How to Actually . . . Resulting Algorithm F ( x, y ) = C ( F X ( x ) , F Y ( y )) . Heavy-Tailed . . . • A dependence is linear if r = ± 1 for some marginals Utility: Reminder F X ( x ) and F Y ( y ). Case of Approximate . . . • So, we define measures of linearity as Home Page L − def Title Page = F X ( x ) ,F Y ( y ) r ( C ( F X ( x ) , F Y ( y )); min ◭◭ ◮◮ L + def = F X ( x ) ,F Y ( y ) r ( C ( F X ( x ) , F Y ( y )) . max ◭ ◮ • The values L − and L + depend only on the copula. Page 5 of 13 • As a measure of fit, we can use M = max(( L − ) 2 , ( L + ) 2 ) . Go Back Full Screen • M = 1 if and only if the dependence is linear for some marginals. Close Quit

Linear Dependencies . . . How to Actually Compute L − and L + : Towards How Linear . . . 5. Detecting Linear . . . an Algorithm Detecting Linear . . . • For any X and Y corr. to the copula, all others are How to Actually . . . obtained by re-scaling X ′ = A ( X ) and Y ′ = B ( Y ). Resulting Algorithm • Thus, L − is min and L + is max of the expression Heavy-Tailed . . . def Utility: Reminder = r ( C ( A ( x ) , B ( y )) over all possible A ( x ) and B ( y ). L Case of Approximate . . . δL δL Home Page • Min, max are attained when δA ( x ) = δB ( y ) = 0, so: Title Page A ( x ) = a 1 + a 2 · E [ B ( Y ) | X = x ]; B ( y ) = b 1 + b 2 · E [ A ( X ) | Y = y ] . ◭◭ ◮◮ • W.l.o.g., we can take A (0) = B (0), A (1) = B (1) = 1; ◭ ◮ then: Page 6 of 13 A ( x ) = E [ B ( Y )) | X = x ] − E [ B ( Y ) | X = 0] E [ B ( Y ) | X = 1] − E [ B ( Y ) | X = 0] ; Go Back B ( y ) = E [ A ( X ) | Y = y ] − E [ A ( X ) | Y = 0] Full Screen E [ A ( X ) | Y = 1] − E [ A ( X ) | Y = 0] . Close Quit

Linear Dependencies . . . How Linear . . . 6. Resulting Algorithm Detecting Linear . . . • We start with some initial functions A (0) ( x ) and B (0) ( y ). Detecting Linear . . . How to Actually . . . • For example, we can take A (0) ( x ) = x and B (0) ( y ) = y . Resulting Algorithm • Once we know A ( k ) ( x ) and B ( k ) ( y ), we compute: Heavy-Tailed . . . A ( k +1) ( x ) = E [ B ( k ) ( Y )) | X = x ] − E [ B ( k ) ( Y ) | X = 0] Utility: Reminder E [ B ( k ) ( Y ) | X = 1] − E [ B ( k ) ( Y ) | X = 0] , Case of Approximate . . . Home Page B ( k +1) ( y ) = E [ A ( k ) ( X ) | Y = y ] − E [ A ( k ) ( X ) | Y = 0] E [ A ( k ) ( X ) | Y = 1] − E [ A ( k ) ( X ) | Y = 0] . Title Page ◭◭ ◮◮ • We stop when ◭ ◮ | A ( k +1) ( x ) − A ( k +1) ( x ) | ≤ ε ; | B ( k +1) ( y ) − B ( k +1) ( y ) | ≤ ε. Page 7 of 13 • We then compute L ± = r ( A ( k +1) ( x ) , B ( k +1) ( y )). Go Back • Testing: for jointly distributed Gaussian variables, this Full Screen indeed leads to Pearson’s correlation r ( F ). Close Quit

Linear Dependencies . . . How Linear . . . 7. Two Important Mathematical Subtleties Detecting Linear . . . 1. X is not well correlated with Y = X when X ≥ 0 and Detecting Linear . . . Y = Z � = X for X < 0. How to Actually . . . Resulting Algorithm • However, L ( A ( X ) , B ( Y )) = 1 when A ( x ) = x and Heavy-Tailed . . . B ( y ) = y for x, y ≥ 0 and A ( x ) = B ( y ) = 0 else. Utility: Reminder • We thus need to make sure that A ( x ) and B ( y ) are Case of Approximate . . . never constant. Home Page • So, for some δ > 0, we require that A ′ ( x ) ≥ δ and Title Page B ′ ( y ) ≥ δ for all x and y . ◭◭ ◮◮ 2. Max, min are always attained only on a compact set. ◭ ◮ • Due to Ascoli-Arzela theorem, compactness means that Page 8 of 13 functions should be uniformly continuous. Go Back • So, we select M > 0 and require that A ′ ( x ) ≤ M and B ′ ( y ) ≤ M for all x and y . Full Screen Close Under these requirements, the definition works. Quit

Linear Dependencies . . . How Linear . . . 8. Heavy-Tailed Distribution: 2nd Problem Detecting Linear . . . C XY Detecting Linear . . . • Pearson’s correlation r ( F ) = assumes that the σ X · σ Y How to Actually . . . marginal distributions have finite variance. Resulting Algorithm • In reality, however, many econometric-related distribu- Heavy-Tailed . . . tions are heavy-tailed , with infinite variance. Utility: Reminder Case of Approximate . . . • In the 1960s, B. Mandelbrot showed that large-scale Home Page fluctuations have pdf ρ ( y ) = A · y − α , with α ≈ 2 . 7. Title Page • For this distribution, variance is infinite. ◭◭ ◮◮ • Since then, similar heavy-tailed distributions have been ◭ ◮ empirically found in other financial situations. Page 9 of 13 • We thus need to extend our definitions to the heavy- tailed case. Go Back Full Screen Close Quit

Linear Dependencies . . . How Linear . . . 9. Utility: Reminder Detecting Linear . . . • People’s behavior is determined by their preferences. Detecting Linear . . . How to Actually . . . • A standard way to describe preferences of a decision Resulting Algorithm maker is to use the notion of utility u . Heavy-Tailed . . . • A user prefers an alternative for which the expected Utility: Reminder n � value p i · u i of the utility is the largest possible. Case of Approximate . . . i =1 Home Page • Alternative, we can say that the expected value Title Page n def � p i · U i of the disutility U = − u is the smallest. ◭◭ ◮◮ i =1 • For a random variable, we select an estimate m that ◭ ◮ minimizes expected disutility: Page 10 of 13 � def Go Back d U ( X ) = min m E [ U ( Y − m )] = min U ( y − m ) · ρ ( y ) dy. m Full Screen Close Quit