Interpretable and Accurate Fine-grained Recognition via Region - PowerPoint PPT Presentation

Interpretable and Accurate Fine-grained Recognition via Region Grouping Zixuan Huang 1 , Yin Li 2,1 1. Department of Computer Sciences 2. Department of Biostatistics & Medical Informatics

Interpretation = part segmentation + part attribution Yellow-headed blackbird Input Part segmentation Part attribution Only image-level label required! 2 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Related work Zhou et al., CVPR’16 Selvaraju et al., ICCV’17 Feng & Vedaldi, ICCV’17 Zhang et al., CVPR’18 Chen et al., NeurIPS’19 Brendel et al., ICLR’19 3 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Interpretation = part segmentation + part attribution Yellow-headed blackbird Input Part segmentation 4 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Part segmentation via region grouping Region Part • Assign feature vectors features dictionary to different centers • Encode each part into one vector Input Feature Part Feature Part image map segmentation vectors assignment 5 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Interpretation = part segmentation + part attribution Yellow-headed blackbird Part segmentation Part attribution 6 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Part attribution via region attention Attention selects important regions for classification • Generate region-based Cliff swallow Region features attention • Attention-guided classification Part Attention segmentation map 7 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Learning with image-level labels How does an object part occur in natural images? U-shaped distribution 8 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Regularization by part occurrence Max-pooling of part assignment Match the empirical distribution to as a part detector prior using Earth-Mover distance 0.98 Calculate Earth-Mover distance Max 0.04 Empirical 0.92 distribution … Part assignment 0.13 for bird head Minibatch of N samples 9 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Results - CUB200 Bird species classification Bird landmark localization (accuracy) (interpretability) Localization Error (%) Accuracy (%) 25 90 20 85 15 80 10 75 5 0 70 N N L N N DFF SCOPS Ours l 1 s e r R 0 N S T n N u 1 E S r C A O C e t K e T K - - A L N F M s D e R 10 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Qualitative results Input Assignment Attention 11 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Results - iNaturalist2017 Pointing game using attention Species classification (interpretability) (accuracy) Accuracy (%) Pointing Error (%) 70 15 65 10 60 5 55 0 50 CAM / Guided Ours SSN TASN ResNet101 Ours Grad-CAM Grad-CAM See our paper for more results on iNaturalist and CelebA datasets 12 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping

Conclusion • An interpretable and accurate model for fine-grained classification • Region grouping + attention = interpretability • A novel prior as regularization • Strong performance over challenging datasets Thank you! Project website: https://www.biostat.wisc.edu/~yli/cvpr2020-interp/ 13 CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping