Warp-and-Project Tomography for Rapidly Deforming Objects Guangming - PowerPoint PPT Presentation

Warp-and-Project Tomography for Rapidly Deforming Objects Guangming Zang, Ramzi Idoughi, Ran Tao, Gilles Lubineau, Peter Wonka, Wolfgang Heidrich, King Abdullah University of Science And Technology

Dynamic scene reconstruction [Wang et al. 2009] [Li et al. 2013] [Zheng et al. 2017] [Innmann et al. 2016] [Dou et al. 2016] 2

Dynamic X-ray tomography ECG gating Classical reconstruction credit: RTK credit: austincc.edu [Chen et al. 2012] [Mory et al. 2016] 3

Dynamic scene reconstruction Optical means X-ray tomography Cameras/ sensors One or more One Resolution Low High Reconstruction Surface Surface + internal structures Capture Speed Fast Slow Deformation type General Periodic or with Pattern Application fields General Medical, Security, Industry 4

Dynamic scene reconstruction Optical means X-ray tomography Cameras/ sensors One or more One Resolution Low High Reconstruction Surface Surface + internal structures Capture Speed Fast Slow Deformation type General Periodic or with Pattern Application fields General Medical, Security, Industry High quality surface and internal reconstruction for fast deforming objects with general motion 5

Motivation Is it possible to scan rapidly deforming objects with internal details? Pills dissolving Hydro-gel balls 6

Motivation Space time tomography [Zang et al. SIGGRAPH18] Shortcomings ?  Assumption : slow and smooth motion fields  Trade-off : spatial VS. temporal reconstruction quality  Sampling : costly uniform temporal sampling 7

Motivation SART-ROF ST-Tomography Warp-and-Project [Getreuer 2012] [Zang et al. 2018] [Ours] 8

Image formation model : X-ray path 9

Image formation model : X-ray path : Unknown field 10

Image formation model : X-ray path : Unknown field : Measurement 11

Image formation model : X-ray path : Unknown field : Measurement 12

Image formation model : X-ray path : Unknown field : Measurement Each projection image has its own time stamp! 13

Linear system • Sparse system • Memory consuming • Ill-posed problem Radon Transform Frames Measurements 14

Linear system Warp and project tomography  Non-parametric and matrix-free • Sparse system  No assumption of the motion • Memory consuming  A non-uniform temporal up-sampling • Ill-posed problem Radon Transform Frames Measurements 15

Objective function 16

Objective function Data fitting (Forward / backward warping) 17

Objective function Data fitting (Forward / backward warping) Volume correlation 18

Objective function Data fitting (Forward / backward warping) Volume correlation Volume smoothness 19

Objective function Data fitting (Forward / backward warping) Volume correlation Volume smoothness Deformation field smoothness 20

Optimization framework Simulated plume data Volume size: 100x150x100 Time frames: 300 21

Warping operator ? 22

Warping operator ? 23

Warping operator 24

Warping operator 25

Optimization framework t 1 X-Ray source 26

Optimization framework t 1 t 2 X-Ray source 27

Optimization framework t 1 t 2 t 3 X-Ray source 28

Optimization framework t 1 t 2 t 3 t 4 X-Ray source 29

Optimization framework t 1 t 2 t 3 t 4 t 5 X-Ray source 30

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 X-Ray source 31

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 X-Ray source 32

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 X-Ray source 33

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 X-Ray source 34

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 35

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 36

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 Backward warping Forward warping 37

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 38

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 39

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 Forward warping Backward warping 40

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 Forward warping Backward warping 41

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 42

Optimization framework t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 43

Material deformation analysis Controlled compression of a copper foam X-ray source Sensor Compression stage 44

Material deformation analysis Ground truth reconstruction Stop-motion capture - Controlled compression Before - 192 intermediate states - 60 projections for each state Reconstruction - Ground truth: 192*60 projections - Other reconstructions: only 192 projections After 45

Material deformation analysis SART-ROF ST-Tomography Warp-and-Project Ground truth [Getreuer 2012] [Zang et al. 2018] [Ours] 46

Material deformation analysis SART-ROF ST-Tomography Warp-and-Project Ground truth [Getreuer 2012] [Zang et al. 2018] [Ours] Absolute error 47

Rock porosity characterization Before After 48

Rock porosity characterization 49

Fungus re-hydration Before After Capture duration: 38min # projections: 600 50

Fungus re-hydration Temporal sampling SART-ROF ST-Tomography Warp-and-Project [Getreuer 2012] [Zang et al. 2018] [Ours] 64 key frames 30 key frames 30 key frames 30 key frames 128 key frames 51

Hydro-gel balls Before After Capture duration: 43min # projections: 640 52

Hydro-gel balls SART-ROF ST-Tomography Warp-and-Project [Getreuer 2012] [Zang et al. 2018] [Ours] Frame 05 Frame 10 Frame 15 53

Hydro-gel balls 54

Summary • A new 4D tomographic reconstruction for rapidly deforming objects • Well suited to graphics, and several scientific applications Material deformation analysis Porosity characterization 55

What is next ? • Software engineering (memory, computation speed, etc.) • Combination with other tomography techniques (e.g. phase contrast tomography) • Extension to other imaging modalities (e.g. electron microscopy) 56

What is next ? Applications: High speed 3D soot imaging Camera 2 Camera 2 Camera 3 Camera 1 Laser 57

Thank you ! Code and data Project webpage: available soon Sponsorship: KAUST as part of VCC Center Competitive Funding. 58