Deep learning to diagnose the neutron star Kenji Fukushima The - PowerPoint PPT Presentation

Deep learning to diagnose 
 the neutron star Kenji Fukushima The University of Tokyo Based on work with Yuki Fujimoto, Koichi Murase — Deep Learning and Physics 2018 — June 2, 2018 @ Osaka � 1

Disclaimer I am a “user” of the deep learning… June 2, 2018 @ Osaka � 2

From the point of view of physics users… Sounds fancy is not enough… Useful ? Advantageous? � 3

Conventional Physics Approach MODEL Input Data Very Uncertain Easy Hard Nonlinear Mapping Compare… Exclude? Output Data Limited Information Exp. EXPERIMENT Theory June 2, 2018 @ Osaka � 4

One another to infinity, so what ? � 5

“Model Independent” Analysis Not unique… Input Data What is the 
 “most likely” Easy Hard one? Nonlinear Mapping Output Data Inverse Problem Limited Information Exp. EXPERIMENT June 2, 2018 @ Osaka � 6

� 7

Neutron Star EoS 30 a 20 Demorest et al. (2010) 10 0 –10 Precise determination of 
 –20 –30 NS mass using Shapiro delay –40 GR MS0 MPA1 2.5 1.928(17) M sun (J1614-2230) P < ∞ AP3 PAL1 Causality ENG AP4 MS2 (slightly changed in 2016) 2.0 J1614-2230 SQM3 MS1 FSU J1903+0327 Mass ( M ( ) SQM1 GM3 PAL6 1.5 J1909-3744 GS1 Double neutron star sy Double neutron s sy systems Antoniadis et al. (2013) 1.0 2.01(4) M sun (PSRJ0348+0432) Rotation 0.5 0.0 7 8 9 10 11 12 13 14 15 Radius (km) June 2, 2018 @ Osaka � 8

Neutron Star EoS Equation of State Input Data Nonlinear Mapping Pressure : p Tolman-Oppenheimer- 
 p = p ( ρ ) Mass density : r -Volkoff (TOV) Eqs (Energy density : ε = ρ c 2 ) pressure diff M - R Relation Output Data M = M ( ρ max ) NS mass : M gravity NS radius : R R = R ( ρ max ) Mathematically one-to-one correspondence June 2, 2018 @ Osaka � 9

Neutron Star EoS Lindblom (1992) Brute-force solution of the inverse problem Test data put by hand Thanks to Y. Fujimoto Mass Solve TOV Pressure Reconstructed Density Radius The answer exists! June 2, 2018 @ Osaka � 10

No magic box… Only “solvable” problem can be solved… � 11

Neutron Star EoS (Side Remark) R is fixed by TOV with p ( R )=0 (“surface” condition) dp/dr ( r = R ) = 0 Very uncertain 
 “by definition” d 2 p/dr 2 ( r = R ) ∝ M 2 /R 2 Determination of R is very uncertain, and on top of that, R itself is anyway very uncertain… People do not care assuming that NS mass > 1.2 M sun June 2, 2018 @ Osaka � 12

Model Independent Analysis Bayesian Analysis B : M - R Observation A : EoS Parameters Normalization (Bayes’ theorem) P ( A | B ) P ( B ) = P ( B | A ) P ( A ) Want to know Likelihood prior Model Calculable by TOV Model must be assumed. EoS parametrization must be introduced. Integration in parameter space must be defined. If infinite observations, prior dependence should be gone. June 2, 2018 @ Osaka � 13

Model Independent Analysis Raithel-Ozel-Psaltis (2017) Mock data (SLy + Noises) Prior 
 Black curve Dep. True EoS Magenta curve Guessed EoS Gray band 68% credibility June 2, 2018 @ Osaka � 14

Model Independent Analysis Bayesian Analysis Supervised Learning Several M - R 
 Nonlinear Several parameters 
 observation points 
 to characterize EoS Mapping with errors { M i , R i } { P i } { P i } = F ( { M i , R i } ) ~ 15 Points (observations) ~ 5 Points Too precise parametrization of EoS is useless 
 (beyond the uncertainty from observations) June 2, 2018 @ Osaka � 15

Deep Learning (2) x 1 (0) ( L ) =y 1 x 1 =x 1 x 1 (2) x 2 (0) ( L ) =y 2 (2) x 2 =x 2 x 2 x 3 { M i , R i } { P i } (2) x 4 (0) ( L ) =y 3 x 3 =x 3 x 3 ~ 15 ~ 5 (0) ( L ) =y M x N =x N x M N k ! x ( k +1) W ( k +1) x ( k ) + a ( k +1) X = σ ( k +1) i ij j i i =1 Parameters to be tuned Backpropagation sigmoid func. tanh ReLU ons” and the typical clude the sigmoid f ( e + 1), the ReLU on σ ( x ) = 1 / ( e x + 1), LU σ ( x ) = max { 0 , x } , h t σ ( x ) = tanh( x ), June 2, 2018 @ Osaka � 16

Deep Learning Our Neural Network Design Layer index Nodes Activation 1 30 N/A 2 60 ReLU 3 40 ReLU 4 40 ReLU 5 5 tanh Probably we don’t need such many hidden layers 
 and such many nodes… anyway, this is one 
 working example… June 2, 2018 @ Osaka � 17

Deep Learning For good learning, the “textbook” choice is important… Training data (200000 sets in total) Randomly generate 5 sound velocities → EoS × 2000 sets Solve TOV to identify the corresponding M - R curve Randomly pick up 15 observation points × ( n s = 100) sets (with ∆ M = 0 . 1 M � , ∆ R = 0 . 5 km) The machine learns the M - R data have error fluctuations Validation data (200 sets) Generate independently of the training data June 2, 2018 @ Osaka � 18

Deep Learning 0.18 “Loss Function" 
 n s = 100 0.16 n s = 1 = deviation from the 
 0.14 Loss function (msle) true answers ( msle ) 0.12 0.1 0.08 Monotonically decrease 
 0.06 for the training data, but 
 0.04 not necessarily so for 
 0.02 the validation data 0 1 10 100 1000 10000 Epochs With fluctuations in the training data, the learning goes quickly Once the over-fitting occurs, 
 the model becomes more stupid… June 2, 2018 @ Osaka � 19

Deep Learning Test with the validation data 
 Fujimoto-Fukushima-Murase (2017) (parameters not optimized to fit the validation data) Two Typical Examples (not biased choice) 10 0 3 p / ρ c 2 10 -1 M / M ⊙ 2 1 10 -2 0 10 12 14 16 0.1 0.2 0.5 1 R [km] ρ c 2 [GeV/fm 3 ] : randomly generated original EoS : reconstructed EoS and associated M - R June 2, 2018 @ Osaka � 20

Deep Learning Overall performance test Mass ( M � ) 0.6 0.8 1.0 1.2 1.4 1.6 1.8 RMS (km) 0.16 0.12 0.10 0.099 0.11 0.11 0.12 (with ∆ M = 0 . 1 M � , ∆ R = 0 . 5 km) Very promising! Credibility estimate has not been done for simplicity, but 
 it can be included in the learning process. June 2, 2018 @ Osaka � 21

Usefulness confirmed, easy implementation but advantageous ? Bayesian or NN, which to choose? � 22

<latexit sha1_base64="tX4yBQlgRSFv2wqlUxTw0zZmEAI=">ACjnichVG/TxRBFP5YUfH8waKNCc3GCwayxti4CAxEqWgPMADEpZcZpeB2zD7I7tzl5zr/QM2lBRWkhBjqGmxsPEfsOBPMJaY2Fj4du+MsUDfZOZ9873vXkz4yU6yAzRxYh1bfT6jZtjtyq379y9N25P3N/I4k7q6Yf6zjd8mSmdBCpgmMVltJqmToabXpHbwo4ptdlWZBHL0vUTthHI/CvYCXxqmWvZM7qah0j7064X692sF7LXdNWRvZfu6E0bV/qfLk/07KrVCMiIYRTADE/RwWFuqzou6IsRWxdAasf0eLnYRw0cHIRQiGMYaEhmPbQgQEuZ2kDOXMgrKuEIfFdZ2OEtxhmT2gNd93m0P2Yj3Rc2sVPt8iuaZstLBFH2hD3RJn+mUvtLPK2vlZY2ilx57b6BVSWv8zcP1H/9VhewN2n9U/+zZYA/1steAe09KpriFP9B3Xx1dri+uTeWP6Zi+cf/v6I+8Q2i7nf/ZFWtvUWFP+D3KztXg43ZmqCaWH1SXo+/IoxTOIRpvm957GEFTQ5HMPcYZzfLRsa856aj0bpFojQ80D/GXWyi+WfJrJ</latexit> <latexit sha1_base64="tX4yBQlgRSFv2wqlUxTw0zZmEAI=">ACjnichVG/TxRBFP5YUfH8waKNCc3GCwayxti4CAxEqWgPMADEpZcZpeB2zD7I7tzl5zr/QM2lBRWkhBjqGmxsPEfsOBPMJaY2Fj4du+MsUDfZOZ9873vXkz4yU6yAzRxYh1bfT6jZtjtyq379y9N25P3N/I4k7q6Yf6zjd8mSmdBCpgmMVltJqmToabXpHbwo4ptdlWZBHL0vUTthHI/CvYCXxqmWvZM7qah0j7064X692sF7LXdNWRvZfu6E0bV/qfLk/07KrVCMiIYRTADE/RwWFuqzou6IsRWxdAasf0eLnYRw0cHIRQiGMYaEhmPbQgQEuZ2kDOXMgrKuEIfFdZ2OEtxhmT2gNd93m0P2Yj3Rc2sVPt8iuaZstLBFH2hD3RJn+mUvtLPK2vlZY2ilx57b6BVSWv8zcP1H/9VhewN2n9U/+zZYA/1steAe09KpriFP9B3Xx1dri+uTeWP6Zi+cf/v6I+8Q2i7nf/ZFWtvUWFP+D3KztXg43ZmqCaWH1SXo+/IoxTOIRpvm957GEFTQ5HMPcYZzfLRsa856aj0bpFojQ80D/GXWyi+WfJrJ</latexit> <latexit sha1_base64="tX4yBQlgRSFv2wqlUxTw0zZmEAI=">ACjnichVG/TxRBFP5YUfH8waKNCc3GCwayxti4CAxEqWgPMADEpZcZpeB2zD7I7tzl5zr/QM2lBRWkhBjqGmxsPEfsOBPMJaY2Fj4du+MsUDfZOZ9873vXkz4yU6yAzRxYh1bfT6jZtjtyq379y9N25P3N/I4k7q6Yf6zjd8mSmdBCpgmMVltJqmToabXpHbwo4ptdlWZBHL0vUTthHI/CvYCXxqmWvZM7qah0j7064X692sF7LXdNWRvZfu6E0bV/qfLk/07KrVCMiIYRTADE/RwWFuqzou6IsRWxdAasf0eLnYRw0cHIRQiGMYaEhmPbQgQEuZ2kDOXMgrKuEIfFdZ2OEtxhmT2gNd93m0P2Yj3Rc2sVPt8iuaZstLBFH2hD3RJn+mUvtLPK2vlZY2ilx57b6BVSWv8zcP1H/9VhewN2n9U/+zZYA/1steAe09KpriFP9B3Xx1dri+uTeWP6Zi+cf/v6I+8Q2i7nf/ZFWtvUWFP+D3KztXg43ZmqCaWH1SXo+/IoxTOIRpvm957GEFTQ5HMPcYZzfLRsa856aj0bpFojQ80D/GXWyi+WfJrJ</latexit> <latexit sha1_base64="tX4yBQlgRSFv2wqlUxTw0zZmEAI=">ACjnichVG/TxRBFP5YUfH8waKNCc3GCwayxti4CAxEqWgPMADEpZcZpeB2zD7I7tzl5zr/QM2lBRWkhBjqGmxsPEfsOBPMJaY2Fj4du+MsUDfZOZ9873vXkz4yU6yAzRxYh1bfT6jZtjtyq379y9N25P3N/I4k7q6Yf6zjd8mSmdBCpgmMVltJqmToabXpHbwo4ptdlWZBHL0vUTthHI/CvYCXxqmWvZM7qah0j7064X692sF7LXdNWRvZfu6E0bV/qfLk/07KrVCMiIYRTADE/RwWFuqzou6IsRWxdAasf0eLnYRw0cHIRQiGMYaEhmPbQgQEuZ2kDOXMgrKuEIfFdZ2OEtxhmT2gNd93m0P2Yj3Rc2sVPt8iuaZstLBFH2hD3RJn+mUvtLPK2vlZY2ilx57b6BVSWv8zcP1H/9VhewN2n9U/+zZYA/1steAe09KpriFP9B3Xx1dri+uTeWP6Zi+cf/v6I+8Q2i7nf/ZFWtvUWFP+D3KztXg43ZmqCaWH1SXo+/IoxTOIRpvm957GEFTQ5HMPcYZzfLRsa856aj0bpFojQ80D/GXWyi+WfJrJ</latexit> Baysian vs NN y θ := { c 2 EoS Obs. D = { ( M i , R i ) } s,i } D| θ ) for each EoS. Bayesian f MAP ( D ) = arg max [Pr( θ ) Pr( D| θ )] θ Pr( θ |D ) Z NN minimizes h ` [ f ] i = d θ d D Pr( θ ) Pr( D| θ ) ` ( θ , f ( D )) Approximated estimated → Baysian NN allows for more general choice of loss functions. Baysian assumes parametrized likelihood functions. June 2, 2018 @ Osaka � 23

Conclusion Yes, useful! Maybe, less biased ? Developing a toolkit for real data like not discrete data with error, 
 but regions of credibility Error analysis (credibility estimate) in the output side June 2, 2018 @ Osaka � 24