Sum-Product Networks for Probabilistic Semantic Maps Kaiyu Zheng , - - PowerPoint PPT Presentation

▶
sum product networks for
SMART_READER_LITE
LIVE PREVIEW

Sum-Product Networks for Probabilistic Semantic Maps Kaiyu Zheng , - - PowerPoint PPT Presentation

Feb 20, 2023 23 likes •307 views

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps Kaiyu Zheng , Andrzej Pronobis, Rajesh Rao University of Washington AAAI 2018 Motivation (Robotics) Mobile Robots in Indoor Spaces Learning Graph-Structured

slide-1
SLIDE 1

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Kaiyu Zheng, Andrzej Pronobis, Rajesh Rao University of Washington

AAAI 2018

slide-2
SLIDE 2

Mobile Robots in Indoor Spaces

Motivation (Robotics)

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

slide-3
SLIDE 3

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Semantic Maps

Motivation (Robotics)

slide-4
SLIDE 4

Semantic Mapping

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Motivation (Robotics)

places spatial relations Local evidence Inferred distributions

  • f latent variables

(semantic attributes) (entities)

slide-5
SLIDE 5

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Motivation (Robotics)

places spatial relations

  • Model semantic

map as a whole

  • This is Structured

Prediction (SP)

Problelm:

Learn general spatial relations between things in the world Estimate semantic attributes in specific environment?

slide-6
SLIDE 6

Probabilistic Graphical Models

Pros:

  • Probabilistic
  • Generative
  • Interpretable

Motivation (Machine Learning)

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Cons:

  • Intractable exact

inference Examples: Bayesian Network, Markov Random Field, Chain Graph [Pronobis&Jensfelt ICRA’12]

slide-7
SLIDE 7
  • End-to-end
  • Remarkable results for visual data

Motivation (Machine Learning)

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Figure from [Shelhamer et. al. PAMI’16]

[Schwing & Urtasun, ICML’15, Belanger & McCallum, ICML’16, Shelhamer et. al. PAMI’16]

Recent Deep Structured Prediction Approaches

slide-8
SLIDE 8
  • But…
  • Strict constraints on

variable interactions

  • Fixed number of variables
  • Static global structure
  • Often not probabilistic

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Motivation (Machine Learning)

[Schwing & Urtasun, ICML’15, Belanger & McCallum, ICML’16, Shelhamer et. al. PAMI’16]

Recent Deep Structured Prediction Approaches

slide-9
SLIDE 9

Sum-Product Networks

  • Viewed in 2 ways:
  • Deep architecture
  • Graphical model
  • Structure semantics:
  • Hierarchical mixture of parts

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps Latent Variable Input Variables

slide-10
SLIDE 10

Sum-Product Networks

[Poon & Domingos, UAI’11, Friesen & Domingos, ICML’16]

Naïve Bayes Mixture Model

  • 3 components
  • 2 binary variables

X1 X2 X2 X1

0.3 0.2 0.5

0.2 0.8 0.3 0.7 0.5 0.5 0.6 0.4

Sum (Mixture Model) Weights (Priors) Product (Compositions of Parts) Low-level Features Input Variables

P(X1, X2)

slide-11
SLIDE 11
  • Learn conditional or joint distributions
  • Tractable partition function, exact inference

Sum-Product Networks

[Poon & Domingos, UAI’11, Friesen & Domingos, ICML’16]

slide-12
SLIDE 12
  • Template-based approach
  • Defined as a set of template SPN models
  • Template models represent general,

higher-order relations between latent variables

  • Applied to form a single distribution for a

specific structured problem for inference

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Proposed Method

Graph-Structured Sum-Product Networks

slide-13
SLIDE 13

GraphSPN

Graph-Structured Sum-Product Networks

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Learning General Knowledge

Sub-graphs Annotated training data (graph-structured) Partition Template 1 Template N Train Template 1 Template N Template SPNs

slide-14
SLIDE 14

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Graph-Structured Sum-Product Networks

Instantiation for Specific Problem

slide-15
SLIDE 15

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

slide-16
SLIDE 16

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

slide-17
SLIDE 17

GraphSPN for Semantic Mapping

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Experiments

Observed local evidence

slide-18
SLIDE 18

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Observed local evidence

slide-19
SLIDE 19

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Observed local evidence Inferred distribution

  • f latent variables

(Semantic place categories)

P(Yi)

slide-20
SLIDE 20

Dataset

  • 99 semantic maps of 11 floors in 3 buildings

in different cities

  • Cross-validation:
  • Trained on graphs from 2 buildings
  • Tested on graphs from remaining building

Learning Graph-Structured Sum-Product Network for Probabilistic Semantic Maps

Experiments

slide-21
SLIDE 21

Infer Latent Semantics based on Noisy Evidence

Learning Graph-Structured Sum-Product Network for Probabilistic Semantic Maps

Experiment 1

Node associated with incorrect evidence (20%) Node associated with correct evidence (80%) correct class incorrect class correct class local evidence local evidence noise

slide-22
SLIDE 22

Infer Latent Semantics based on Noisy Evidence

Learning Graph-Structured Sum-Product Network for Probabilistic Semantic Maps

Experiment 1

Correction of incorrect information (20%) Strengthen correct information (80%) correct class incorrect class correct class

  • utput
  • utput
slide-23
SLIDE 23

Noisified GraphSPN Groundtruth

Results: Inference Behavior

Experiment 1 Similar results even without local evidence for some places

slide-24
SLIDE 24

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Noise Level → Accuracy (%) →

Freiburg Saarbrücken Stockholm

GraphSPN MRF (order 2)

Results: Increasing Noise

Experiment 1

MRF (order 3)

slide-25
SLIDE 25

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

See paper for more details

Regular Novel

  • ffice

doorway corridor

  • ffice

doorway corridor

Novelty Detection

Experiment 2

slide-26
SLIDE 26

Conclusions

  • Introduced GraphSPNs
  • Leverages Sum-Product Networks
  • Applied GraphSPNs to model semantic maps

Learning Graph-Structured Sum-Product Network for Probabilistic Semantic Maps

General approach to model arbitrary dynamic graphs Complex, noisy variable dependencies Inference based

  • n instantiaion of

template models

slide-27
SLIDE 27

Unified Model

Unified Model for Spatial Knowledge

Learning Graph-Structured Sum-Product Network for Probabilistic Semantic Maps

Ongoing Work

Sensory information Local place semantics Global topology Semantics in global context DGSM (Pronobis and Rao, IROS 2017) GraphSPN This work!

slide-28
SLIDE 28

Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps

Kaiyu Zheng, Andrzej Pronobis, Rajesh Rao University of Washington

http://www.kaiyuzh.me http://www.pronobis.pro

Thank you