In honor of Professor Amari Information geometry associated with - - PowerPoint PPT Presentation

▶
in honor of professor amari
SMART_READER_LITE
LIVE PREVIEW

In honor of Professor Amari Information geometry associated with - - PowerPoint PPT Presentation

Nov 09, 2022 312 likes •673 views

Information Geometry and its Applications IV June 12-17, 2016, Liblice, Czech In honor of Professor Amari Information geometry associated with two generalized means Shinto Eguchi Institute of Statistical Mathematics A joint work with Osamu

slide-1
SLIDE 1

Information geometry associated with two generalized means

Shinto Eguchi Institute of Statistical Mathematics

Information Geometry and its Applications IV June 12-17, 2016, Liblice, Czech

A joint work with Osamu Komori and Atsumi Ohara University of Fukui

In honor of Professor Amari

slide-2
SLIDE 2

2

Outline

Information geometry Generalized information geometry

slide-3
SLIDE 3

3

The core of information geometry

(Amari-Nagaoka, 2001)

Pythagoras

slide-4
SLIDE 4

4

Metric and connections

Information metric m-connection e-connection

Rao (1945), Dawid (1975), Amari (1982)

Conjugate

slide-5
SLIDE 5

5

Exponential model

Exponential model Mean parameter Remark 2

Amari (1982)

Remark 1

slide-6
SLIDE 6

6

Minimum KL leaf

Mean equal space Exponential model

slide-7
SLIDE 7

7

Pythagoras foliation

slide-8
SLIDE 8

8

KL divergence (revisited)

The normalizing constant We observe

KL divergence is induced by e-geodesic

  • Cf. the canonical divergence, Amari-Nagaoka (2001)
  • Cf. Ay-Amari (2015)
slide-9
SLIDE 9

9

(log , exp)

KL-divergence (log, exp) Pythagoras identity exponential model mean equal space e-geodesic m-geodesic Pythagoras foliation

slide-10
SLIDE 10

10

( log, exp) (, )

￥

slide-11
SLIDE 11

11

Kolmogorov-Nagumo mean

  • Cf. Kolmogorov(1930), Nagumo (1930), Naudts (2009)
slide-12
SLIDE 12

12

Generalized e-geodesic

Remark

slide-13
SLIDE 13

13

Generalized KL-divergence

slide-14
SLIDE 14

14

Generalized m-geodesic

Remark

slide-15
SLIDE 15

15

Metric and connections by D()

Riemannian metric Generalized m-connection Generalized e-connection Remark

slide-16
SLIDE 16

16

Generalized two geodesic curves

slide-17
SLIDE 17

17

Pythagorean theorem

Pythagoras theorem

slide-18
SLIDE 18

18

Power-log function

slide-19
SLIDE 19

19

Generalized exponential model

G-exponential model Mean parameter Remark 2

  • Cf. Matsuzoe-Henmi (2014)

Remark 1

  • Cf. Naudts (2010)
slide-20
SLIDE 20

20

Minimum GKL leaf

Mean equal space G-exponential model

slide-21
SLIDE 21

21

Generalized Pythagoras foliation

slide-22
SLIDE 22

22

( log , exp ) (

  • Cf. Newton (2012)
slide-23
SLIDE 23

23

Expected  density

Consider Let Then Therefore

slide-24
SLIDE 24

24

Quasi divergence

Remark

slide-25
SLIDE 25

25

Another generalization of KL-divergence

The other adjustment One adjustment

slide-26
SLIDE 26

26

Gamma divergence

Fujisawa-Eguchi (2008)

Remark

slide-27
SLIDE 27

27 27

-geometry

Pythagoras G-e-geodesic m-geodesic

slide-28
SLIDE 28

28

Metric and connections by ()

Riemannian metric Generalized m-connection Generalized e-connection Remark

slide-29
SLIDE 29

29 29

Minimum  estimation

Data set Loss function Proposed estimator Estimation selection, Robustness, Spontaneous data learning Expected loss consistency Model

slide-30
SLIDE 30

Non-convex learning

30

Selection of  Loss function Model

slide-31
SLIDE 31

31 31

Spontaneous data learning

The consistent class of estimators Redescending Influence function Super robustness Multi modal detection Difference of convex functions

Estimation selection than model selection

two local minima

slide-32
SLIDE 32

32

Concluding remarks

(

slide-33
SLIDE 33



33

Welcome to any comments: eguchi@ism.ac.jp

slide-34
SLIDE 34

34

U-divergence

U-cross-entropy U-entropy U-divergence Note Exm

slide-35
SLIDE 35

35

Generalized exponential model (ver. 2)

G-exponential model Mean parameter Remark 2 Remark 1