The algebra of integrated partial belief systems Manuele Leonelli 1 , - PowerPoint PPT Presentation

The algebra of integrated partial belief systems Manuele Leonelli 1 , Eva Riccomagno 2 , James Q. Smith 1 1 Department of Statistics, The University of Warwick 2 Dipartimento di Matematica, Universit´ a degli Studi di Genova Algebraic Statistics, Genova June 10, 2015 Research funded by the Department of Statistics, The University of Warwick, and EPRSC grant EP/K039628/1.

Modelling big systems Current decision support systems address complex domains: e.g. nuclear emergency management, food security; decision making = ⇒ Bayesian subjective probabilities; single agents systems are well established, but no clear extension to multi-agent; distributed and exact (symbolic) computations are vital;

Notation random vector Y = ( Y T i ) i ∈ [ m ] , [ m ] = { 1 , . . . , m } ; panels of experts { G 1 , . . . , G m } , where G i is responsible for Y i ; decision space d ∈ D ; θ i parametrizes f i the density of Y i | ( θ i , d ) ; π i is the density over θ i | d ; d ∗ optimal policy maximizing the expected utility � ¯ ¯ u ( d ) = u ( d | θ ) π ( θ | d ) d θ Θ where � ¯ u ( d | θ ) = u ( y , d ) f ( y | θ , d ) d y Y is the conditional expected utility (CEU).

Notation random vector Y = ( Y T i ) i ∈ [ m ] , [ m ] = { 1 , . . . , m } ; panels of experts { G 1 , . . . , G m } , where G i is responsible for Y i ; decision space d ∈ D ; θ i parametrizes f i the density of Y i | ( θ i , d ) ; π i is the density over θ i | d ; d ∗ optimal policy maximizing the expected utility � ¯ ¯ u ( d ) = u ( d | θ ) π ( θ | d ) d θ Θ where � ¯ u ( d | θ ) = u ( y , d ) f ( y | θ , d ) d y Y is the conditional expected utility (CEU).

Utility theory Utility u : Y × D → R such that ( y , d ) � ( y ′ , d ′ ) ⇐ ⇒ u ( y , d ) ≤ u ( y ′ , d ′ ) Panel separable factorization � � u ( y , d ) = k I u i ( y i , d ) , I ∈P 0 ([ m ]) i ∈ I with P 0 the power set without empty set. Polynomial marginal utility (univariate) of degree n i , ρ ij ∈ R , � ρ ij ( d ) y j u ( y i , d ) = i . j ∈ [ n i ]

Integrated partial belief systems Panels agrees on: a decision space D ; a family of utility functions U ; a dependence structure between various functions of Y , θ and d ; to delegate quantifications to the most informed panel. Definition An IPBS is adequate if ¯ u ( d ) , for each d ∈ D and u ∈ U , can be computed from the beliefs of G i , i ∈ [ m ] .

Integrated partial belief systems Panels agrees on: a decision space D ; a family of utility functions U ; a dependence structure between various functions of Y , θ and d ; to delegate quantifications to the most informed panel. Definition An IPBS is adequate if ¯ u ( d ) , for each d ∈ D and u ∈ U , can be computed from the beliefs of G i , i ∈ [ m ] .

Algebraic expected utility ¯ u ( d | θ ) is called algebraic in the panels if, for each d ∈ D , there exist λ i ( θ i , d ) such that ¯ u ( d | θ ) is a square-free polynomial q d of the λ i ¯ u ( d | θ ) = q d ( λ 1 ( θ 1 , d ) , · · · , λ m ( θ m , d )) . Let λ i ( θ i , d ) = ( λ ji ( θ i , d )) j ∈ [ s i ] , λ 0 i ( θ i , d ) = 1 and B = × i ∈ [ m ] { 0 , . . . , s i } . For a given b ∈ B let b j , i = 0 if j � = b i , b j , i = 1 if j = b i , b 0 , i = 1. Definition ¯ u ( d | θ ) is called algebraic if, for each d ∈ D , q d is a square-free polynomial of the λ ji such that � q d ( λ 1 ( θ 1 , d ) , . . . , λ m ( θ m , d )) = k b , d λ b ( θ , d ) , b ∈ B � � λ ji ( θ i , d ) b j , i . λ b ( θ , d ) = i ∈ [ m ] j ∈ [ s i ] 0

Algebraic expected utility ¯ u ( d | θ ) is called algebraic in the panels if, for each d ∈ D , there exist λ i ( θ i , d ) such that ¯ u ( d | θ ) is a square-free polynomial q d of the λ i ¯ u ( d | θ ) = q d ( λ 1 ( θ 1 , d ) , · · · , λ m ( θ m , d )) . Let λ i ( θ i , d ) = ( λ ji ( θ i , d )) j ∈ [ s i ] , λ 0 i ( θ i , d ) = 1 and B = × i ∈ [ m ] { 0 , . . . , s i } . For a given b ∈ B let b j , i = 0 if j � = b i , b j , i = 1 if j = b i , b 0 , i = 1. Definition ¯ u ( d | θ ) is called algebraic if, for each d ∈ D , q d is a square-free polynomial of the λ ji such that � q d ( λ 1 ( θ 1 , d ) , . . . , λ m ( θ m , d )) = k b , d λ b ( θ , d ) , b ∈ B � � λ ji ( θ i , d ) b j , i . λ b ( θ , d ) = i ∈ [ m ] j ∈ [ s i ] 0

Score separability For a given b ∈ B , let � � λ ji ( θ i , d ) b j , i � � µ ji ( d ) = E , µ i ( d ) = µ ji ( d ) j ∈ [ s i ] . Definition Call an IPBS score separable if, for all d ∈ D and all b ∈ B such that k b , d � = 0, � � E ( λ b ( θ , d )) = µ ji ( d ) . i ∈ [ m ] j ∈ [ s i ] ∪{ 0 } Lemma Suppose panel G i delivers µ i ( d ) , i ∈ [ m ] , d ∈ D . Then, assuming a CEU is algebraic, if the IPBS is score separable then it is adequate.

New independence conditions Definition (Quasi independence) An IPBS is called quasi independent if E ( q d ( λ 1 ( θ 1 , d ) , . . . , λ m ( θ m , d ))) = q d ( E ( λ 1 ( θ 1 , d )) , . . . , E ( λ m ( θ m , d ))) . Let θ = θ 1 · · · θ n , a , c ∈ Z n ≥ 0 . Definition (Moment independence) θ entertains moment independence of order c if for any a ≤ lex c E ( θ a ) = � E ( θ a i i ) . i ∈ [ n ]

New independence conditions Definition (Quasi independence) An IPBS is called quasi independent if E ( q d ( λ 1 ( θ 1 , d ) , . . . , λ m ( θ m , d ))) = q d ( E ( λ 1 ( θ 1 , d )) , . . . , E ( λ m ( θ m , d ))) . Let θ = θ 1 · · · θ n , a , c ∈ Z n ≥ 0 . Definition (Moment independence) θ entertains moment independence of order c if for any a ≤ lex c E ( θ a ) = � E ( θ a i i ) . i ∈ [ n ]

Moment independence Consider two parameters θ 1 and θ 2 and suppose moment independence of order ( 2 , 2 ) . 2 ) = E ( θ 1 ) 2 E ( θ 2 ) 2 + V ( θ 1 ) E ( θ 2 ) 2 E ( θ 2 1 θ 2 2 ) = E ( θ 2 1 ) E ( θ 2 + E ( θ 1 ) 2 V ( θ 2 ) + V ( θ 1 ) V ( θ 2 ) If θ 1 ⊥ ⊥ θ 2 , 2 ) = E ( θ 1 θ 2 ) 2 + V ( θ 1 θ 2 ) E ( θ 2 1 θ 2 = E ( θ 1 E ( θ 2 )) 2 + V ( θ 1 E ( θ 2 )) + E ( θ 2 1 V ( θ 2 )) = E ( θ 1 ) 2 E ( θ 2 ) 2 + V ( θ 1 ) E ( θ 2 ) 2 + E ( θ 2 1 ) V ( θ 2 ) = E ( θ 1 ) 2 E ( θ 2 ) 2 + V ( θ 1 ) E ( θ 2 ) 2 + E ( θ 1 ) 2 V ( θ 2 ) + V ( θ 1 ) V ( θ 2 )

Moment independence Consider two parameters θ 1 and θ 2 and suppose moment independence of order ( 2 , 2 ) . 2 ) = E ( θ 1 ) 2 E ( θ 2 ) 2 + V ( θ 1 ) E ( θ 2 ) 2 E ( θ 2 1 θ 2 2 ) = E ( θ 2 1 ) E ( θ 2 + E ( θ 1 ) 2 V ( θ 2 ) + V ( θ 1 ) V ( θ 2 ) If θ 1 ⊥ ⊥ θ 2 , 2 ) = E ( θ 1 θ 2 ) 2 + V ( θ 1 θ 2 ) E ( θ 2 1 θ 2 = E ( θ 1 E ( θ 2 )) 2 + V ( θ 1 E ( θ 2 )) + E ( θ 2 1 V ( θ 2 )) = E ( θ 1 ) 2 E ( θ 2 ) 2 + V ( θ 1 ) E ( θ 2 ) 2 + E ( θ 2 1 ) V ( θ 2 ) = E ( θ 1 ) 2 E ( θ 2 ) 2 + V ( θ 1 ) E ( θ 2 ) 2 + E ( θ 1 ) 2 V ( θ 2 ) + V ( θ 1 ) V ( θ 2 )

Some results Theorem Under quasi independence, an algebraic CEU is score separable. Corollary Let a ji i , a ji ∈ Z s i λ ji ( θ i , d ) = θ ≥ 0 ; a ∗ i = ( a ∗ ji ) j ∈ [ s i ] , where a ∗ ji = max { a ji | j ∈ [ s i ] } ; i ) moment independent of order a ∗ = ( a ∗ T ) ; θ = ( θ T i an algebraic CEU is score separable.

Polynomial SEMs A polynomial structural equation model (SEM) over Y = ( Y i ) i ∈ [ m ] is defined as � θ i a i Y a i Y i = [ i − 1 ] + ε i , a i ∈ A i where A i ⊂ Z i − 1 ≥ 0 , ε i ∼ ( 0 , ψ i ) , Y [ i − 1 ] = Y 1 · · · Y i − 1 . Theorem Assume a polynomial SEM; a panel separable utility; a marginal polynomial utility; The IPBS is score separable under quasi independence.

Polynomial SEMs A polynomial structural equation model (SEM) over Y = ( Y i ) i ∈ [ m ] is defined as � θ i a i Y a i Y i = [ i − 1 ] + ε i , a i ∈ A i where A i ⊂ Z i − 1 ≥ 0 , ε i ∼ ( 0 , ψ i ) , Y [ i − 1 ] = Y 1 · · · Y i − 1 . Theorem Assume a polynomial SEM; a panel separable utility; a marginal polynomial utility; The IPBS is score separable under quasi independence.

� � � � Bayesian networks Definition (Linear SEM) A BN over a DAG G , V ( G ) = { 1 , . . . , m } , is a linear SEM if � Y i = θ 0 i + θ ji Y j + ε i , j ∈ Π i Π i parent set of Y i , ε i ∼ ( 0 , ψ i ) and θ 0 i , θ ji ∈ R . Y 1 = θ 01 + ε 1 4 2 Y 2 = θ 02 + θ 12 Y 1 + ε 2 Y 3 = θ 03 + θ 13 Y 1 + θ 23 Y 2 + ε 3 1 3 Y 4 = θ 04 + ε 4

� � � � � � � � � � � � � � � � Rooted paths A rooted path P from i 1 to j m is a sequence ( i 1 , ( i 1 , j 1 ) , . . . , ( i k , j k ) , ( i k + 1 , j k + 1 ) , . . . , ( i m , j m )) , where j k = i k + 1 . � P i is the set of rooted paths ending in i . 4 2 4 4 2 4 2 2 1 1 3 3 3 3 1 1 � P 3 = { ( 3 ) , ( 2 , ( 2 , 3 )) , ( 1 , ( 1 , 3 )) , ( 1 , ( 1 , 2 ) , ( 2 , 3 )) }

� � � � � � � � � � � � � � � � Rooted paths A rooted path P from i 1 to j m is a sequence ( i 1 , ( i 1 , j 1 ) , . . . , ( i k , j k ) , ( i k + 1 , j k + 1 ) , . . . , ( i m , j m )) , where j k = i k + 1 . � P i is the set of rooted paths ending in i . 4 2 4 4 2 4 2 2 1 1 3 3 3 3 1 1 � P 3 = { ( 3 ) , ( 2 , ( 2 , 3 )) , ( 1 , ( 1 , 3 )) , ( 1 , ( 1 , 2 ) , ( 2 , 3 )) }