Top-Down Connections and its Applications to 3D Object Recognition - PowerPoint PPT Presentation

Topographic Class Grouping through Top-Down Connections and its Applications to 3D Object Recognition Matthew D. Luciw and Juyang Weng Embodied Intelligence Laboratory Department of Computer Science Michigan State University East Lansing MI 48824 USA Email: {luciwmat, weng}@cse.msu.edu

Some Background  Roles of top-down connections in cerebral cortex are unclear  Found some brain areas where neurons grouped by class  E.g., fusiform face area (FFA) and parahippocampal place area (PPA)  Neurons in inferotemporal cortex: object recognition with some invariance  How do such abstract meaning carrying neurons develop? Michigan State University 2

Introduction Novelty: Topographic Class Grouping (TCG) mechanism.  TCG: Top-down connections can enable cortical neurons  to group by class. Understanding how cerebral cortex develops  Discriminating features  Invariance  Motor-initiated abstraction  Application  Can apply to any pattern recognition application  This study: 3D object recognition of center-normalized,  background controlled objects. TCG enabled  Significant reduction of the recognition errors.  Increased neuronal purity  Decreased class-response scatter  Michigan State University 3

Potential Towards Human Level AI  Human-level intelligence must be developed [1]  Model cortical developmental principals  Not modality specific  Not application specific  Architecture capable of brain-like development + Experience  Intelligence development  This work: Invariance & Abstraction  When output is label: invariance  But motor output does not have to be class label  General framework for motor-biased abstraction via supervision from downstream [1]: Weng , McClelland, Pentland, Sporns, Stockman, Sur and Thelen, Autonomous mental development by robots and animals, Science, 2001 Michigan State University 4

Multilevel In-Place Learning Networks (MILN) Biologically inspired developmental  networks Hebbian-based firing rules on every layer  develops feature detectors (neurons) Hierarchical: from sensors (e.g., cameras)  to motors (e.g., class label) Three types of projections:  Bottom-up  Lateral  Top-down  Here we examine the effect of the top-down  connections Advantages:  Lowest complexity  Deals with high dimensional raw input  Fast and parallelizable  No local minima problem (does not use  backpropagation for training) Weng et al. IJHR 2007

Related Work  SOM (Kohenen, 1997)  Idea of using expanded input with input & output vectors  HDR (Zhang et al, 2005)  Used expanded input  AGREL (Roelfsema & Van Ooyen 2005)  Top-down connections for attention  LISSOM (Sit & Mikkulainen 2006)  Top-down connections for correlations (no motors)  MILN (Weng & Luciw, 2006, Weng et al. 2007)  Top-down connections to generate invariance Michigan State University 6

3D Object Recognition MSU Dataset: 25 Rotating Objects (25 classes) 200 images from each class 4/5 for training Used grayscale NORB (5 classes) [2] – Left: training objects, Right: testing objects variations of elevation, lighting, rotation [2]: Y. LeCun, F.J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. CVPR , 2004.

Two-Layer Architecture Michigan State University 8

Layer Development  Three types of projections: Bottom-up, Lateral, Top-down  Layer-one arranged in 2D plane 1. Neuron’s pre -response • Similarity of bottom-up weight to bottom-up excitation • Similarity of top-down weight to top-down excitation • Bottom-up vs. Top-down weighted by Beta

Layer Development (Cont.) 2. Lateral Inhibition (Competition) Lateral inhibition Top-k firing neurons not inhibited  3. Lateral Excitation (Smoothing) Non-inhibited neurons boost  local firing rates (e.g., neighbors ) 3 x 3 neighbors fire and update  Lateral excitation, Do For each winner neuron j :

Layer Self-Organization (Cont.) 4. Hebbian Weight Adjustment Optimal adaptation of winners  Plasticity enabled  Hebbian adaptation (LCA):  Lobe component i: the principal component of the region Ri  Partition the input space  Input space is top-down boosted if Beta nonzero

Two Causes of TCG Lateral Excitation No Yes Top-Down Connections No No TCG No TCG Yes No TCG TCG Both lateral excitation and top-down connections are required! Michigan State University 12

Top Down Projections Lead to Class- Discriminating Features The top-down connections boost the variations in the neuronal between class directions during the training phase, leading to class discriminating features.

Lateral Excitation Leads to Class ``Groups’’ to Grow and Compete For Space Lateral excitation ``pulls’’ nearby neurons in the neural plane to represent nearby features in the input space, which is boosted by the top-down connections.

Example With Toy Data Two classes, two manifolds, high energy along noise direction, black squares are location of neurons’ bottom -up weight, lines are neighbor connections. (a): After neuron initialization (no training) (b): After training without top-down connections: lateral excitation pulled between manifolds (c): After training with top-down connections: fewer between manifold neighbors and most neurons lie on a class manifold.

Experiment Setup  Two parameters tested  Weight of top-down connections (Beta)  Zero or 0.3  Size of neuronal map on layer-one  25 Objects: 20 x 20, 30 x 30, 40 x 40  NORB: 40 x 40, 60 x 60  5 trials for each set of parameters  Average values reported  Each trial trained for 50,000 image/label pairs  Error: disjoint testing samples  Other metrics: used training data Michigan State University 16

Visualization of TCG  (a): No Top-Down Excitation (Beta = 0)  (b): Used Top-Down Connections (Beta = 0.3)  Shows % of neuronal updating history for highest class

Motor-Initiated Abstraction Invariance and abstraction are analogous  From top-down connections  Within-class variation is disregarded 1. Between-class variation is kept 2. Abstract class neurons are manifested in the representation.  Develops meaning: the features are tuned & organized from top-down to solve  the imposed problem (here: classification, but could be something else) No abstraction With abstraction

Error Results – 25 Objects Neural Avg. error Avg. error Error without TCG difference plane size with TCG 20 x 20 8.13% 3.03% 5.1% 30 x 30 2.62% 0.83% 1.79% 40 x 40 0.63% 0.33% 0.30% Michigan State University 19

Error Results – NORB Method Resource Disjoint Test Error KNN+L2 24,000 18.4 No TCG TCG Diff MILN 1,600 26.5 17.7 8.8 3,600 26.2 15.7 10.5 MILN + Top-Down Connections achieves better performance than KNN using 1,600 elements compared to 24,000 Michigan State University 20

Developmental Purity  Neuron purity: measured statistically by the average entropy of the neurons’ development.  Use probability that neuron updated for each class  Purer neurons are more “abstract,” -- characterizing class-specific (or motor-specific) input information, resulting in better classification rates.  If purity is one, neuron i developed using samples from a single class. Michigan State University 21

Example Entropy Maps  Entropy is (1-purity)  (a): Without Top-Down  (b): Using Top-Down Michigan State University 22

Class-Response Scatter  Neurons that respond to the same class become relatively nearer.  Measured statistically by a smaller within-class scatter of responses when the neuronal plane has a fixed size.  c: # classes, n: # neurons, A: neuron probability matrix, N: neuron position (2D), M: mean class firing positions Michigan State University 23

Example of Weights and Firing  (a): Eight images from a class  (b): Bottom-up weight without TCG  (c): Top-responding neuron for each image without TCG  High class-response scatter  (d): Bottom-up weight with TCG  (e): Top-responding neuron for each image with TCG  Low class response scatter Michigan State University 24

Grouping Results – 25 Objects Developmental Class- Top-down Neural purity response Connectedness* plane size ( ¯ ) scatter No (0) 20 x 20 0.775 49.48 1.7 Yes (0.3) 0.814 6.58 1 No (0) 30 x 30 0.785 174.84 2.3 Yes (0.3) 0.855 10.84 1 No (0) 40 x 40 0.799 310.84 3.2 Yes (0.3) 0.919 15.08 1 • Connectedness of one means there is a single area for each class. See paper for details. Michigan State University 25

Grouping Results – NORB Top-down Edge Neural Purity Scatter Connected plane ( ¯ ) Wrap size No (0) N 0.5 270.6 8.1 Yes (0.3) N 40 x 40 0.87 102.59 1 Yes (0.3) Y 0.93 171.01 1.24 No (0) N 0.51 635.84 13.8 Yes (0.3) N 60 x 60 0.89 174.79 1 Yes (0.3) Y 0.91 386.88 1.23 Michigan State University 26