Robust texture image representation by scale selective local binary - PowerPoint PPT Presentation

Robust texture image representation by scale selective local binary patterns (TIP2016) 郭振华 清华大学深圳研究生院 zhenhua.guo@sz.tsinghua.edu.cn

清华大学深圳研究生院  清华大学深圳研究生院成立于 2001 年。为清华大学唯一的异 地办学机构。同一学校，同一品牌，秉承同一文化传统。  有在校全日制研究生 3000 余人，其中博士生 380 余人。  有专职教师 150 余人，博士后 80 余人，双基地教师 280 余人， 兼职教师 40 余人。 2 http://www.sz.tsinghua.edu.cn

清华大学深圳研究生院  设生命与健康、能源与环境、信息科学与技术、物流与交通 、先进制造、海洋科学与技术、社会科学与管理七个学部。  着力发展信息、先进制造、网络与媒体技术、环境、材料、 新能源、物流、海洋等学科。  校园建筑面积 10 万平米，创新基地规划 10 万平米。 3

Outline  Texture definition and challenge  Texton (Statistical vs. Binary)  Overview of LBP and CLBP  Proposed SSLBP  Experimental Results and Discussion 4

Texture Definition  Definition Wiki Dictionary: The feel or shape of a surface or substance; the smoothness, roughness, softness, etc. of something. In fact, the definition of texture is still an open issue.

Texture is everywhere 天空 ( 纹理 ) 楼房 ( 纹理 ) 树林 ( 纹理 ) 草地 ( 纹理 ) 6

Wide Application Desert vs Mountain Normal vs Abnormal Defect Detection 7

Common issues  Lightness, rotation and scale 8

Structural Approach Structural approach: a set of texels in some regular or repeated pattern 9

Limitation of Structure Approach How do you decide what is a texel? grass leaves What/Where are the texels? 10

Statistical Texton 11

Binary Texton  Two advantages: Fast Insensitive to training set 12

LBP in spatial domain T. Ojala, M. Pietik ä inen, and T. T. M ä enp ää . Multiresolution gray-scale and rotation invariant texture classification 13 with Local Binary Pattern. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), pp. 971-987, 2002.

LBP and contrast operators 14

Circle-LBP 15

Multiscale LBP 16

An example of LBP image and histogram 17

Rotation Invariance (ri) 18

Rotation Invariance 19

Completed LBP Central pixel and its P circularly and evenly spaced neighbours with radius R . 24 (a) (b) (c) (d) (a) A 3  3 sample block; (b) the local differences; (c) the sign and (d) magnitude components.

Completed LBP CLBP_S S Local Difference LDSMT CLBP M CLBP_M CLBP Map Original Histogram Image Center Gray Level CLBP_C Classifier 21

Representation Example Histogram of CLBP_S of a sample. Histogram of CLBP_M of the sample. Histogram of CLBP_S_M. Histogram of CLBP_S/M. Zhenhua Guo, Lei Zhang, David Zhang, A Completed Modeling of Local Binary Pattern Operator for Texture Classification, IEEE Transactions on Image Processing , vol. 19, no. 6, pp. 1657-1663, 2010.

Scale Variation  Scale invariance is more difficult.  Two popular ways: Local scale invariance Global scale invariance 23

Scale Invariance (I)  Local scale invariance Detect a Harris or Laplacian Region->Normalize the region->Feature Extraction 24

Scale Invariance (I)  Local scale invariance Estimating local scale or extracting local fractal feature 25

Scale Invariance (II)  Global scale invariance Global fractal feature 26

Scale Invariance (II)  Global scale invariance Polar transform 27

Scale Invariance (II)  Global scale invariance Scale shift matching 28

Assumption  Statistical dominant local patterns provide discriminant information for texture classification.  When an image changes scale, percentage of dominant patterns does not change. S. Liao, M. W. K. Law, and A. C. S. Chung. Dominant local binary patterns for texture classification. IEEE 29 Transactions on Image Processing 18(5), pp. 1107-1118, 2009.

Algorithm 1: Feature Learning  Step 1: for one training sample, build a scale space by a 2D  Gaussian filter;  f , =1 l   i s   l   s g , 1< l L, "*" is the convolution operator   l 1  Step 2: compute local pattern histogram for each image;  Step 3: only maximal frequency among different scale is f s s s kept; H ( )=max( k H ( ), k H ( ), ..., k H ( )) k i 1 2 L CLBP _ S / C CLBP _ S / C CLBP _ S / C CLBP _ S / C  Step 4: compute average frequency for the whole training set; f H ( ) k i CLBP _ S / C T T H ( )= k H ( )+ k CLBP _ S / C CLBP _ S / C N  Step 5: dominant patterns with high frequency are learnt. 30

Algorithm 2: Feature Extraction  Step 1: for one test sample, build a scale space by a 2D  Gaussian filter;  I , =1 l   s   l   s g , 1< l L, "*" is the convolution operator   l 1  Step 2: compute histogram for selected patterns by algorithm 1;  Step 3: only maximal frequency among different scale is kept. I s s s DPH ( )=max( k DPH ( ), k DPH ( ), ..., k DPH ( )) k 1 2 L CLBP _ S / C CLBP _ S / C CLBP _ S / C CLBP _ S / C 31

Scale selective LBP (SSLBP) I s 1 : Original Image Feature Size: K T Feature frequency of DP ri CLBP _ M / C P ,R 0 14 . 0 01 . 0 02 . 0 05 . 0 0 . Feature Size: K 0 05 . 0 1 . 0 08 . 0 03 . 0 02 . Feature frequency T of DP ri CLBP _ S / C P ,R Convolved by 2D Gaussian function g  I s 2 T Feature frequency of DP ri Feature CLBP _ M / C P ,R 0 09 . 0 02 . 0 04 . 0 03 . 0 02 . Image Scale Scale Space 0 06 . 0 07 . 0 06 . 0 05 . 0 0 . Space Feature frequency . T of DP ri CLBP _ S / C . P ,R . I s L T Feature frequency of DP ri CLBP _ M / C P ,R 0 13 . 0 01 . 0 07 . 0 01 . 0 01 . 0 15 . 0 01 . 0 02 . 0 01 . 0 04 . Feature frequency T of DP ri CLBP _ S / C P ,R Max operation Max operation Output feature 0 15 . 0 1 . 0 08 . 0 05 . 0 04 . 0 14 . 0 02 . 0 07 . 0 05 . 0 02 . for image I 32 K  Feature Size: 2

An example 0.06 0.06 0.05 0.05 0.04 0.04 Frequency(%) Frequency(%) 0.03 0.03 0.02 0.02 0.35 0.01 0.01 0 0 5 10 15 20 25 30 35 40 45 50 5 10 15 20 25 30 35 40 45 50 Pattern Index Pattern Index 0.08 0.08 0.07 0.07 0.06 0.06 0.05 0.05 Frequency(%) Frequency(%) 0.04 0.04 0.03 0.03 0.02 0.02 0.17 0.01 0.01 33 0 0 5 10 15 20 25 30 35 40 45 50 5 10 15 20 25 30 35 40 45 50 Pattern Index Pattern Index

5 Dominant Patterns 34

Scale Estimation 2.1 2.05 Scale parameter of KTH-TIPS. 2 Average Feature Scale  1.95 FS I 1.9    I IS , 1,2,...,9 AFS z z z 1.85 90 1.8 K  ( ) ScaleIndex k I 1.75   k 1 FS I 1.7 K 1 2 3 4 5 6 7 8 9 35 Image Scale

Texture Databases Texture Number Number Database Imaging property Image Size of of Name classes samples The images are captured under different illumination and CUReT Fixed, 200*200 61 92 viewing directions. It extends CUReT by imaging new samples of ten of the KTH- Varied, CUReT textures at a subset of the viewing and lighting 10 81 TIPS 196*201 angles used in CUReT but also over a range of scales. Textures are acquired under significant scale and viewpoint changes, arbitrary rotations, and uncontrolled illumination UIUC Fixed, 640*480 25 40 conditions, even including textures with non-rigid deformation. It has been designed in a similar way as UIUC, while the Fixed, 1280* UMD 25 40 image resolution is 4 times of UIUC. 960 It is systematically collected with varied viewing angles, A LOT Varied, 1536* illumination angles, and illumination colors for each 250 100 891 material. 36

CUReT Database 37

KTH-TIPS Database 38

UIUC Database 39

UMD Database 40

ALOT Database 41

Parameters  Scale Space: 4  2D Gaussian filter: 2 0.25  Radius: 3, 9  Neighbor: 24  Feature extractor: CLBP_S/C, CLBP_M/C  Feature Length: 2400  NNC: NNC+Chi-square distance  Feature preprocessing:   H sqrt H ( ), k 1,2,..., K k k 42

Nearest Subspace Classifier (NSC)  There are C classes of textures.  n training samples in each class, a set of     histograms for one class: H h , h ,..., h  c c ,1 c ,2 c n ,  project h y into the subspace spanned by H c :    T 1 T y ( H H ) H h c c c c  The projection residuals is computed as:   H  y err h c c c 2 K. Lee, J. Ho, and D. Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE 43 Transactions on Pattern Analysis and Machine Intelligence 27(5), pp. 684 – 698, 2005.