From probabilistic inference to Bayesian unfolding (passing through - PowerPoint PPT Presentation

Uncertainties in measurements Having to perform a measurement: Which numbers shall come out from our device? Having performed a measurement: What have we learned about the value of the quantity of interest? How to quantify these kinds of uncertainty? G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 9

Uncertainties in measurements Having to perform a measurement: Which numbers shall come out from our device? Having performed a measurement: What have we learned about the value of the quantity of interest? How to quantify these kinds of uncertainty? Under well controlled conditions (calibration) we can make use of past frequencies to evaluate ‘somehow’ the detector response P ( x | µ ) . G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 9

Uncertainties in measurements Having to perform a measurement: Which numbers shall come out from our device? Having performed a measurement: What have we learned about the value of the quantity of interest? How to quantify these kinds of uncertainty? Under well controlled conditions (calibration) we can make use of past frequencies to evaluate ‘somehow’ the detector response P ( x | µ ) . There is (in most cases) no way to get directly hints about P ( µ | x ) . G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 9

Uncertainties in measurements Μ 0 Experimental response ? x P ( x | µ ) experimentally accessible (though ’model filtered’) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 10

Uncertainties in measurements ? Μ Inference x x 0 P ( µ | x ) experimentally inaccessible G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 10

Uncertainties in measurements ? Μ Inference x x 0 P ( µ | x ) experimentally inaccessible but logically accessible! → we need to learn how to do it G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 10

Uncertainties in measurements Μ given x Μ x given Μ x x 0 Symmetry in reasoning! G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 10

Uncertainty and probability We, as physicists, consider absolutely natural and meaningful statements of the following kind ◦ P ( − 10 < ǫ ′ /ǫ × 10 4 < 50) >> P ( ǫ ′ /ǫ × 10 4 > 100) ◦ P (170 ≤ m top / GeV ≤ 180) ≈ 70% ◦ P ( M H < 200 GeV ) > P ( M H > 200 GeV ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 11

Uncertainty and probability We, as physicists, consider absolutely natural and meaningful statements of the following kind ◦ P ( − 10 < ǫ ′ /ǫ × 10 4 < 50) >> P ( ǫ ′ /ǫ × 10 4 > 100) ◦ P (170 ≤ m top / GeV ≤ 180) ≈ 70% ◦ P ( M H < 200 GeV ) > P ( M H > 200 GeV ) . . . although, such statements are considered blaspheme to statistics gurus G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 11

Uncertainty and probability We, as physicists, consider absolutely natural and meaningful statements of the following kind ◦ P ( − 10 < ǫ ′ /ǫ × 10 4 < 50) >> P ( ǫ ′ /ǫ × 10 4 > 100) ◦ P (170 ≤ m top / GeV ≤ 180) ≈ 70% ◦ P ( M H < 200 GeV ) > P ( M H > 200 GeV ) . . . although, such statements are considered blaspheme to statistics gurus I stick to common sense (and physicists common sense) and assume that probabilities of causes, probabilities of of hypotheses, probabilities of the numerical values of physics quantities, etc. are sensible concepts that match the mind categories of human beings (see D. Hume, C. Darwin + modern researches) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 11

The six box problem H 0 H 3 H 4 H 1 H 2 H 5 Let us take randomly one of the boxes. G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 12

The six box problem H 0 H 3 H 4 H 1 H 2 H 5 Let us take randomly one of the boxes. We are in a state of uncertainty concerning several events , the most important of which correspond to the following questions: (a) Which box have we chosen, H 0 , H 1 , . . . , H 5 ? (b) If we extract randomly a ball from the chosen box, will we observe a white ( E W ≡ E 1 ) or black ( E B ≡ E 2 ) ball? ∪ 5 Our certainty: j =0 H j = Ω ∪ 2 i =1 E i = Ω . G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 12

The six box problem H 0 H 3 H 4 H 1 H 2 H 5 Let us take randomly one of the boxes. We are in a state of uncertainty concerning several events , the most important of which correspond to the following questions: (a) Which box have we chosen, H 0 , H 1 , . . . , H 5 ? (b) If we extract randomly a ball from the chosen box, will we observe a white ( E W ≡ E 1 ) or black ( E B ≡ E 2 ) ball? • What happens after we have extracted one ball and looked its color? ◦ Intuitively we now how to roughly change our opinion. ◦ Can we do it quantitatively, in an objective way? G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 12

The six box problem H 0 H 3 H 4 H 1 H 2 H 5 Let us take randomly one of the boxes. We are in a state of uncertainty concerning several events , the most important of which correspond to the following questions: (a) Which box have we chosen, H 0 , H 1 , . . . , H 5 ? (b) If we extract randomly a ball from the chosen box, will we observe a white ( E W ≡ E 1 ) or black ( E B ≡ E 2 ) ball? • What happens after we have extracted one ball and looked its color? ◦ Intuitively we now how to roughly change our opinion. ◦ Can we do it quantitatively, in an objective way? • And after a sequence of extractions? G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 12

Predicting sequences Side remark/exercise Imagine the four possible sequences resulting from the first two extractions from the misterious box: BB, BW, WB and WW • How likely do you consider them to occur? [ → If you could win a prize associated with the occurrence of one of them, on which sequence(s) would you bet?] G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 13

Predicting sequences Side remark/exercise Imagine the four possible sequences resulting from the first two extractions from the misterious box: BB, BW, WB and WW • How likely do you consider them to occur? [ → If you could win a prize associated with the occurrence of one of them, on which sequence(s) would you bet?] • Or do you consider them equally likelly? G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 13

Predicting sequences Side remark/exercise Imagine the four possible sequences resulting from the first two extractions from the misterious box: BB, BW, WB and WW • How likely do you consider them to occur? [ → If you could win a prize associated with the occurrence of one of them, on which sequence(s) would you bet?] • Or do you consider them equally likelly? • ? G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 13

Predicting sequences Side remark/exercise Imagine the four possible sequences resulting from the first two extractions from the misterious box: BB, BW, WB and WW • How likely do you consider them to occur? [ → If you could win a prize associated with the occurrence of one of them, on which sequence(s) would you bet?] • Or do you consider them equally likelly? • ? • No, they are not equally likelly! G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 13

Predicting sequences Side remark/exercise Imagine the four possible sequences resulting from the first two extractions from the misterious box: BB, BW, WB and WW • How likely do you consider them to occur? [ → If you could win a prize associated with the occurrence of one of them, on which sequence(s) would you bet?] • Or do you consider them equally likelly? • ? • No, they are not equally likelly! Laplace new perfectly why → If our logical abilities have regressed it is not a good sign! (Remember Leibnitz/Hume quote) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 13

The toy inferential experiment The aim of the experiment will be to guess the content of the box without looking inside it, only extracting a ball, recording its color and reintroducing it into the box G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 14

The toy inferential experiment The aim of the experiment will be to guess the content of the box without looking inside it, only extracting a ball, recording its color and reintroducing it into the box This toy experiment is conceptually very close to what we do in Physics • try to guess what we cannot see (the electron mass, a branching ratio, etc) . . . from what we can see (somehow) with our senses. The rule of the game is that we are not allowed to watch inside the box! (As we cannot open and electron and read its properties, like we read the MAC address of a PC interface) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 14

Cause-effect representation box content → observed color G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 15

Cause-effect representation box content → observed color An effect might be the cause of another effect − → G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 15

A network of causes and effects G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 16

A network of causes and effects A report ( R i ) might not correspond exactly to what really happened ( O i ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 16

A network of causes and effects Of crucial interest in Science! ⇒ Our devices seldom tell us ’the truth’. G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 16

A network of causes and effects ⇒ Belief Networks (Bayesian Networks) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 16

From causes to effects and back Our original problem: Causes C1 C2 C3 C4 Effects E1 E2 E3 E4 G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 17

From causes to effects and back Our original problem: Causes C1 C2 C3 C4 Effects E1 E2 E3 E4 Our conditional view of probabilistic causation P ( E i | C j ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 17

From causes to effects and back Our original problem: Causes C1 C2 C3 C4 Effects E1 E2 E3 E4 Our conditional view of probabilistic causation P ( E i | C j ) Our conditional view of probabilistic inference P ( C j | E i ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 17

From causes to effects and back Our original problem: Causes C1 C2 C3 C4 Effects E1 E2 E3 E4 Our conditional view of probabilistic causation P ( E i | C j ) Our conditional view of probabilistic inference P ( C j | E i ) The fourth basic rule of probability: P ( C j , E i ) = P ( E i | C j ) P ( C j ) = P ( C j | E i ) P ( E i ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 17

Symmetric conditioning Let us take basic rule 4, written in terms of hypotheses H j and effects E i , and rewrite it this way: P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) “The condition on E i changes in percentage the probability of H j as the probability of E i is changed in percentage by the condition H j .” G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 18

Symmetric conditioning Let us take basic rule 4, written in terms of hypotheses H j and effects E i , and rewrite it this way: P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) “The condition on E i changes in percentage the probability of H j as the probability of E i is changed in percentage by the condition H j .” It follows P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 18

Symmetric conditioning Let us take basic rule 4, written in terms of hypotheses H j and effects E i , and rewrite it this way: P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) “The condition on E i changes in percentage the probability of H j as the probability of E i is changed in percentage by the condition H j .” It follows P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) Got ‘after’ Calculated ‘before’ (where ‘before’ and ‘after’ refer to the knowledge that E i is true.) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 18

Symmetric conditioning Let us take basic rule 4, written in terms of hypotheses H j and effects E i , and rewrite it this way: P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) “The condition on E i changes in percentage the probability of H j as the probability of E i is changed in percentage by the condition H j .” It follows P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) ”post illa observationes” “ante illa observationes” (Gauss) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 18

Symmetric conditioning Let us take basic rule 4, written in terms of hypotheses H j and effects E i , and rewrite it this way: P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) “The condition on E i changes in percentage the probability of H j as the probability of E i is changed in percentage by the condition H j .” It follows P ( H j | E i ) = P ( E i | H j ) P ( H j ) P ( E i ) ”post illa observationes” “ante illa observationes” (Gauss) ⇒ Bayes theorem G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 18

Application to the six box problem H 0 H 3 H 4 H 1 H 2 H 5 Remind: • E 1 = White • E 2 = Black G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 19

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 Our prior belief about H j G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 Probability of E i under a well defined hypothesis H j It corresponds to the ‘response of the apparatus in measurements. → likelihood (traditional, rather confusing name!) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 Probability of E i taking account all possible H j → How much we are confident that E i will occur. G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 Probability of E i taking account all possible H j → How much we are confident that E i will occur. Easy in this case, because of the symmetry of the problem. But already after the first extraction of a ball our opinion about the box content will change, and symmetry will break. G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 But it easy to prove that P ( E i | I ) is related to the other ingredients, usually easier to ‘measure’ or to assess somehow, though vaguely G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( H j | E i , I ) = P ( E i | H j , I ) P ( H j | I ) P ( E i | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = 1 / 2 • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 But it easy to prove that P ( E i | I ) is related to the other ingredients, usually easier to ‘measure’ or to assess somehow, though vaguely ‘decomposition law’: P ( E i | I ) = � j P ( E i | H j , I ) · P ( H j | I ) ( → Easy to check that it gives P ( E i | I ) = 1 / 2 in our case). G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

Collecting the pieces of information we need Our tool: P ( E i | H j , I ) · P ( H j | I ) P ( H j | E i , I ) = P j P ( E i | H j , I ) · P ( H j | I ) • P ( H j | I ) = 1 / 6 • P ( E i | I ) = � j P ( E i | H j , I ) · P ( H j | I ) • P ( E i | H j , I ) : P ( E 1 | H j , I ) = j/ 5 P ( E 2 | H j , I ) = (5 − j ) / 5 We are ready − → Let’s play! G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 20

A different way to view fit issues θ µ x i µ y i y i x i [ for each i ] • Determistic link µ x ’s to µ y ’s • Probabilistic links µ x → x , µ y → y ⇒ aim of fit: { x , y } → θ ⇒ f ( θ | { x , y } ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 21

Parametric inference Vs unfolding f ( θ | { x , y } ) : G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 22

Parametric inference Vs unfolding f ( θ | { x , y } ) : probabilistic parametric inference ⇒ it relies on the kind of functions parametrized by θ µ y = µ y ( µ x ; θ ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 22

Parametric inference Vs unfolding f ( θ | { x , y } ) : probabilistic parametric inference ⇒ it relies on the kind of functions parametrized by θ µ y = µ y ( µ x ; θ ) ⇒ data distilled into θ ; BUT sometimes we wish to interpret the data as little as possible ⇒ just public ‘something equivalent’ to an experimental distribution, with the bin contents fluctuating according to an underlying multinomial distribution, but having possibly got rid of physical and instrumental distortions, as well as of background. G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 22

Parametric inference Vs unfolding f ( θ | { x , y } ) : probabilistic parametric inference ⇒ it relies on the kind of functions parametrized by θ µ y = µ y ( µ x ; θ ) ⇒ data distilled into θ ; BUT sometimes we wish to interpret the data as little as possible ⇒ just public ‘something equivalent’ to an experimental distribution, with the bin contents fluctuating according to an underlying multinomial distribution, but having possibly got rid of physical and instrumental distortions, as well as of background. ⇒ Unfolding (deconvolution) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 22

Smearing matrix → unfolding matrix Invert smearing matrix? G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 23

Smearing matrix → unfolding matrix Invert smearing matrix? In general is a bad idea: not a rotational problem but an inferential problem! G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 23

Smearing matrix → unfolding matrix � � � � 0 . 8 0 . 2 1 . 33 − 0 . 33 : → U = S − 1 = Imagine S = 0 . 2 0 . 8 − 0 . 33 1 . 33 � � � � 10 8 Let the true be s t = : → s m = S · s t = ; 0 2 � � � � √ 8 10 → S − 1 · s m = If we measure s m = 2 0 G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 23

Smearing matrix → unfolding matrix � � � � 0 . 8 0 . 2 1 . 33 − 0 . 33 : → U = S − 1 = Imagine S = 0 . 2 0 . 8 − 0 . 33 1 . 33 � � � � 10 8 Let the true be s t = : → s m = S · s t = ; 0 2 � � � � √ 8 10 → S − 1 · s m = If we measure s m = 2 0 BUT � � � � 9 11 . 7 → S − 1 · s m = if we had measured 1 − 1 . 7 � � � � 10 13 . 3 → S − 1 · s m = if we had measured 0 − 3 . 3 G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 23

Smearing matrix → unfolding matrix � � � � 0 . 8 0 . 2 1 . 33 − 0 . 33 : → U = S − 1 = Imagine S = 0 . 2 0 . 8 − 0 . 33 1 . 33 � � � � 10 8 Let the true be s t = : → s m = S · s t = ; 0 2 � � � � √ 8 10 → S − 1 · s m = If we measure s m = 2 0 Indeed, matrix inversion is recognized to producing ‘crazy spectra’ and even negative values (unless such large numbers in bins such fluctuations around expectations are negligeable) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 23

Bin to bin? En passant: • OK if the are no migrations: → each bin is an ‘independent issue’, treated with a binomial process, given some efficiencies. G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 24

Bin to bin? En passant: • OK if the are no migrations: → each bin is an ‘independent issue’, treated with a binomial process, given some efficiencies. • Otherwise ◦ ’error analysis’ troublesome (just imagine e.g. that a bin has an ‘efficiency’ > 1 , because of migrations from other bins); ◦ iteration is important (efficiencies depend on ‘true distribution’) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 24

Bin to bin? En passant: • OK if the are no migrations: → each bin is an ‘independent issue’, treated with a binomial process, given some efficiencies. • Otherwise ◦ ’error analysis’ troublesome (just imagine e.g. that a bin has an ‘efficiency’ > 1 , because of migrations from other bins); ◦ iteration is important (efficiencies depend on ‘true distribution’) [Anyway, one might set up a procedure for a specific problem, test it with simulations and apply it to real data (the frequentistic way – if ther is the way . . . )] G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 24

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) x C : true spectrum (nr of events in cause bins) x E : observed spectrum (nr of events in effect bins) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) x C : true spectrum (nr of events in cause bins) x E : observed spectrum (nr of events in effect bins) Our aim: • not to find the true spectrum • but, more modestly, rank in beliefs all possible spectra that might have caused the observed one: ⇒ P ( x C | x E , I ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) • P ( x C | x E , I ) depends on the knowledge of smearing matrix Λ , with λ ji ≡ P ( E j | C i , I ) . G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) • P ( x C | x E , I ) depends on the knowledge of smearing matrix Λ , with λ ji ≡ P ( E j | C i , I ) . • but Λ is itself uncertain, because inferred from MC simulation: ⇒ f (Λ | I ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) • P ( x C | x E , I ) depends on the knowledge of smearing matrix Λ , with λ ji ≡ P ( E j | C i , I ) . • but Λ is itself uncertain, because inferred from MC simulation: ⇒ f (Λ | I ) • for each possible Λ we have a pdf of spectra: → P ( x C | x E , Λ , I ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) • P ( x C | x E , I ) depends on the knowledge of smearing matrix Λ , with λ ji ≡ P ( E j | C i , I ) . • but Λ is itself uncertain, because inferred from MC simulation: ⇒ f (Λ | I ) • for each possible Λ we have a pdf of spectra: → P ( x C | x E , Λ , I ) � ⇒ P ( x C | x E , I ) = P ( x C | x E , Λ , I ) f (Λ | I ) d Λ [by MC!] G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) • Bayes theorem: P ( x C | x E , Λ , I ) ∝ P ( x E | x C , Λ , I ) · P ( x C | I ) . G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

Discretized unfolding C n C C 1 C 2 C i E j E n E E 1 E 2 T ( T : ‘trash’) • Bayes theorem: P ( x C | x E , Λ , I ) ∝ P ( x E | x C , Λ , I ) · P ( x C | I ) . • Indifference w.r.t. all possible spectra P ( x C | x E , Λ , I ) ∝ P ( x E | x C , Λ , I ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 25

P ( x E | x C i , Λ , I ) C n C C 1 C 2 C i E j E n E E 1 E 2 T Given a certain number of events in a cause-bin x ( C i ) , the number of events in the effect-bins, included the ‘trash’ one, is described by a multinomial distribution: x E | x ( C i ) ∼ Mult [ x ( C i ) , λ i ] , with = { λ 1 ,i , λ 2 ,i , . . . , λ n E +1 ,i } λ i = { P ( E 1 | C i , I ) , P ( E 2 | C i , I ) , . . . , P ( E n E +1 ,i | C i , I ) } G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 26

P ( x E | x C , Λ , I ) C n C C 1 C 2 C i E j E n E E 1 E 2 T x E | x ( C i ) multinomial random vector, ⇒ x E | x ( C ) sum of several multinomials. G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 27

P ( x E | x C , Λ , I ) C n C C 1 C 2 C i E j E n E E 1 E 2 T x E | x ( C i ) multinomial random vector, ⇒ x E | x ( C ) sum of several multinomials. BUT no ‘easy’ expression for P ( x E | x C , Λ , I ) G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 27

P ( x E | x C , Λ , I ) C n C C 1 C 2 C i E j E n E E 1 E 2 T x E | x ( C i ) multinomial random vector, ⇒ x E | x ( C ) sum of several multinomials. BUT no ‘easy’ expression for P ( x E | x C , Λ , I ) ⇒ STUCK! G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 27

P ( x E | x C , Λ , I ) C n C C 1 C 2 C i E j E n E E 1 E 2 T x E | x ( C i ) multinomial random vector, ⇒ x E | x ( C ) sum of several multinomials. BUT no ‘easy’ expression for P ( x E | x C , Λ , I ) ⇒ STUCK! ⇒ Change strategy G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 27

The rescue trick Instead of using the original probability inversion (applied directly) to spectra P ( x C | x E , Λ , I ) ∝ P ( x E | x C , Λ , I ) · P ( x C | I ) , we restart from P ( C i | E j , I ) ∝ P ( E j | C i , I ) · P ( C i | I ) . G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 28

The rescue trick Instead of using the original probability inversion (applied directly) to spectra P ( x C | x E , Λ , I ) ∝ P ( x E | x C , Λ , I ) · P ( x C | I ) , we restart from P ( C i | E j , I ) ∝ P ( E j | C i , I ) · P ( C i | I ) . Consequences: 1. the sharing of observed events among the cause bins needs to be performed ‘by hand’; G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 28

The rescue trick Instead of using the original probability inversion (applied directly) to spectra P ( x C | x E , Λ , I ) ∝ P ( x E | x C , Λ , I ) · P ( x C | I ) , we restart from P ( C i | E j , I ) ∝ P ( E j | C i , I ) · P ( C i | I ) . Consequences: 1. the sharing of observed events among the cause bins needs to be performed ‘by hand’; 2. a uniform prior P ( C i | I ) = k does not mean indifference over all possible spectra. ⇒ P ( C i | I ) = k is a well precise spectrum (in most cases far from the physical one) ⇒ VERY STRONG prior that biases the result! G. D’Agostini, Probabilistic inference and unfolding, G¨ ottingen, 19 October 2010 – p. 28