Shane Transue, Phuc Nguyen, Tam Vu, and Min-Hyung Choi IEEE Conference on Connected Health: Applications, Systems and Engineering Technologies
Introduction: Real-time Tidal Volume Estimation Tidal-volume estimation Goal: Monitor a patient’s tidal volume remotely Evaluate medical conditions: Chronic Pulmonary Disease (COPD), Cystic Fibrosis Tidal-volume Estimation Methodologies Accelerometers, pressure, etc. [A. Fekr et al ., 2015] Invasive deployment, expensive Proposed Camera-based Volume Estimation Proposed: Two phase real-time tidal Input: Depth-image, Estimated skeletal posture volume estimation using depth imaging Training: Spirometer + Camera monitoring Output: Real-time tidal volume waveform Camera-based Vision Challenges Occlusion, clothing, monitoring distance, depth-image error Assumptions: Correct posture, form-fitting clothing, limited movement 2/24
Methodologies and Related Work Camera-based Respiratory Monitoring Chest Surface Reconstructions Respiration rate [vs] tidal volume estimation Recent developments for respiration rate monitoring Remote infrared using Kinect [A. Loblaw et al., 2013] [Y. Mizobe et al ., 2006] Real-time Vision-based Monitoring [K. Tan et al .,2010] Recent developments within tidal volume estimation Chest surface monitoring [M.-C. Yu et al., 2012] Emerging: Fine-grain tidal volume estimation ( continuous ) [H. Aoki et al ., 2012] Surface Reconstruction Non-restraint pulmonary test [Y. Mizobe et al ., 2006] Non-contact measurement (structured light) [H. Aoki et al ., 2012] [Implemented Reconstruction] 3/24
Chest Volume Extraction Methodology 2 Phase Monitoring System Initial Training (1) Spirometer + Camera-based monitoring Correlation: Deformation to Volume Models surface-to-volume correlation Defines per-patient breathing profile Real-time Monitoring: After training, patient can breath freely (1.0 – 2.0[m]) Real-time Monitoring (2) Camera-based Monitoring (no spiro) Allows patient to breath naturally Patient Guidelines: Patient movement should be minimized Correct posture should be maintained Input: Depth-cloud and Skeletal Structure 4/24
Omni-directional Deformation Model Deformation Model: Chest volume displacement surface modeling Orthogonal Models (current methodologies) 2D distance lattice as chest surface ( depth surface) Inaccurate representation of lung displacement Lungs act as balloons not as a planar surface Boundary curvature information is distorted Omni-directional Model Represents natural chest displacements (lung expansion) Cross-sectional orthogonal model view Unique to each patients breathing characteristics of prior methods (top), introduced Retains boundary curvature omni-directional model (bottom) Challenge: Patient movement 5/24
Chest Volume Extraction Methodology (1) Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal 6/24
Chest Volume Extraction Methodology (2) Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal Clipping Regions 7/24
Chest Volume Extraction Methodology (3) Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal Clipping Regions Chest Depth Cloud 8/24
Chest Volume Extraction Methodology (4) Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal Clipping Regions Chest Depth Cloud Chest Surface 9/24
Chest Volume Extraction Methodology (F) Proposed Chest Reconstruction: All operations are performed per-frame Depth + Skeletal Clipping Regions Chest Depth Cloud Chest Surface Real-time Chest Mesh Volume 10/24
Chest Surface Acquisition Focus region: Patient’s Chest using Skeletal Tracking Depth-image Segmentation Bounded region (cylinder) to clip chest region Based on skeletal and depth data Stable Depth Chest Sampling: Bit-history of stable points (only saturated points included in reconstruction) Depth + skeletal data (left), cylinder bounding region (center), depth bit-history (right) 11/24
Surface Normal Estimation Volumetric Requirements (1) Surface boundary definition ( characteristic function ) (2) Surface orientation (surface normals) Stencil-based Surface Normal Estimation Defined as a spatial filter (neighboring concentric squares) Chest-cloud with Normals Efficient computation (for real-time monitoring) 12/24
Chest-region Surface Filling Clip Region Hole Filling Clip regions require synthetic data to enclose the chest volume Planar uniform surfaces introduced to fill holes in the surface Chest points projected onto back-plane to fill back Planar hole filling algorithm Skeletal joints (shoulders, neck, waist), with depth edges Generated Clip-regions: Shoulders, 2D Convex-hull of projected edge points back, neck, and waist Generate synthetic grid to close hole (a) Chest depth-cloud neck edge points, (b) planar hole fill algorithm applied to (a) 13/24
Chest Surface Reconstruction Chest-deformation to Tidal Volume Estimation Observation: Chest deformation over time indirectly correlates with tidal volume Objective: Infer tidal volume from enclosed iso-surface and spirometer-based training Chest-based Iso-surface Reconstruction and Volume Iso-surface reconstruction from oriented points (MC Variant) [M. Kazhdan, 2005] Signed tetrahedral volume [C. Zhang and T. Chen, 2001] 14/24
Tidal Volume Estimation Chest Volume to tidal Volume Correlation Iso-volume Chest Region to Tidal Volume Estimation Mapping Per-patient Spirometer-based Training Chest deformations used to infer changes in tidal volume Quantifies relationship between chest deformations and tidal volume (per-patient) Patient Requirement: 30 second initial training period (with spirometer) 15/24
Tidal Volume Estimation Chest surface encloses arbitrary volume, does not represent tidal volume Influenced by body shape, clothing, posture, etc. Per-patient training establishes predictive model to estimate future tidal volume Volume Correlation: Bayesian Back-propagation Neural Network Training Spirometer directly measures tidal volume Direct chest volume measured as change dV of the patient’s chest Inhale/Exhale Deformation Data processed with simple smoothing filters (windowed zero mean, band-pass) 16/24
Real-time Tidal Volume Estimation (Video) 17/24
Initial Trial: Tidal Volume Estimation Results (Top) Raw Data: Chest mesh volume and spirometer tidal volume (Center) Result: Correlated Result (estimated tidal volume) (Bottom) Error: Between spirometer and estimated volumes 18/24
Initial Trial: Tidal Volume Estimation Results Real-time Tidal Volume Monitoring Results Illustration of four tidal volume waveforms (unique to each patient) Initial trial with limited patient count based on the proposed training and real-time monitoring system 19/24
Tidal Volume Estimation Error Sources Patient Related Sources Patient movement (omni-directional model) Clothing (occluding chest surface) Patient distance results in depth-image density changes: Closer Distances: Higher depth-image resolution, higher frame time, lower error Further Distances: Lower depth-image resolution, lower frame time, higher error 20/24
Tidal Volume Estimation Error Sources Device and Methodology Related Sources Depth-image distance measurement errors Patient chest region clipping (cylindrical volume) Distance-based processing time (decreases sampling) Closer Distances: Higher depth-image resolution, high depth accuracy, longer frame time Further Distances: Lower depth-image resolution, low depth accuracy, shorter frame time 21/24
Conclusion Novel Omni-directional deformation model Mimics omni-directional lung deformations Incorporates patients unique deformations Monitors surface deformation patterns Provides a complete 3D iso-surface Tidal Volume Estimation Training: Introduces patient-specific breathing characteristics and monitoring Enabled non-contact tidal-volume estimation in real-time (with visualization) 92.2% - 94.19% Accuracy compared to spirometer ground-truth values 22/24
Future Work Continued Challenges Video-based monitoring (occlusion, clothing, measurement errors) Body-shape, clothing interference Non-linear deformation to tidal volume correlation Air is compressible (error within mesh to volume correlation) Per-patient waveform characteristic experimentation Patient Requirements Curvature Analysis Limit impact of movement (signal fluctuations) Relax posture requirements (especially arm segmentation) Simplification of training procedure Chest Segmentation 23/24
Recommend
More recommend
