An introduction to Sum-Product Networks (SPNs): A new deep - PowerPoint PPT Presentation

An introduction to Sum-Product Networks (SPNs): A new deep probabilistic architecture Felix McGregor Prof Johan du Preez

What are SPNs? • Poon and Domingos, (UAI 2011 Best Paper) • Acyclic directed graphs os sums and products

Two views of f SPNs 1. Deep architecture • Product node as activation function • Clear semantics • Reason meaningfully about relationships between variables as we are calculating probabilities with respect to some features

Bayesian Networks Two vie iews of f SPNs 2. Probabilistic graphical models • Tractable inference can calculate partition • Inference in linear time to the size of the Markov Random Fields network

Probabilistic inference • An SPN represents a joint distribution over a set of variables • P( 𝑌 1 = 1, 𝑌 2 = 0) ?

Marginal Inference • P( 𝑌 1 = 1) ?

MPE Inference • What is the most likely state?

Conditions for a valid SPN • Complete: children of sum are of the same scope (Mixtures of distributions) • Decomposable: children of a product node are of different scopes (Distributions that factorise)

What does this mean? Gens ICML 2013

Parameter estimation • Lends itself naturally to backpropagation • Vanishing gradient / gradient diffusion • “Hard” gradient DiMauro 2016

Parameter estimation Gens & Domingos 2012

Structure learning: LearnSPN Split variables on Cluster similar approximate instances independence

Cool applications: Face completion Poon & Domingos 2011

Cool applications • 83.96% on CIFAR 10 Discriminative Learning of Sum-Product Networks (NIPS 2012) • Satellite image classification

Resources • Best place for all things SPN • https://github.com/arranger1044/awesome-spn • Some video lecture sites • http://techtalks.tv/ • http://videolectures.net/ • My Github • https://github.com/felixmcgregor/Sum-Product-Networks

An introduction to Sum-Product Networks (SPNs): A new deep - PowerPoint PPT Presentation

An introduction to Sum-Product Networks (SPNs): A new deep probabilistic architecture Felix McGregor Prof Johan du Preez What are SPNs? Poon and Domingos, (UAI 2011 Best Paper) Acyclic directed graphs os sums and products Two views

Product Section Product Section New Product Introduction New Product Introduction Product

ex Addition: 1-bit half adder A + Sum B Carry out Carry A B Sum out 0 0 A 0 1 Sum

Sum Product Networks What is a Sum Product Network? 1. It is a tractable probabilistic model

Basic Ruby Syntax sum = 0 Newline is statement separator i = 1 while i <= 10 do sum += i*i

ex start small with a 1-bit (half) adder A B Carry out Sum A 0 0 Sum 0 1 B 1 0 1 1

Chapter 6 Methods 1 Opening Problem Find the sum of integers from 1 to 10, from 20 to 30, and

The Dot Product and Orthogonal Vectors The Dot Product Defn. The dot product (or inner product )

Horizontal Water Source Heat Pump Horizontal Water Source Heat Pump Product Training Product

McQuay Product Presentation McQuay Product Presentation McQuay Product Presentation TI TLE :

Basic Ruby Syntax No variable declarations sum = 0 Newline is statement separator i = 1 while

Nutrition Education and other Enrichm ent Activities for Sum m er Meal Program s Thursday May 5

The ML Language end; Lisp Algol 60 (We will use Standard ML.)

More on Verilog Sign extension: Example 1 wire [3:0] c, d reg [4:0] sum; always @ (c or d)

Game Theory : Zero-Sum Games, The Minimax Theorem CSC304 - Nisarg Shah 1 Zero-Sum Games

Part I bers, t - target number Question: Is there a subset of X such the sum of its elements is t ?

W4231: Analysis of Algorithms Subset Sum The Subset Sum problem is defined as follows: 11/30/99

4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

Sums & 1 + x + x 2 + + x n G = n x - x 2 - - x n - x n + 1 Money xG = n

Sum-of-products Evaluation Schemes with Fixed-Point arithmetic, and their application to IIR

Some subsystems of constant-depth Frege with parity Michal Garl k Polytechnic University of

Set-theoretic remarks on a possible definition of elementary -topos Giulio Lo Monaco Masaryk

How Simulations and Databases Play Nicely Alex Szalay, JHU Gerard Lemson, MPA Thursday,

Information Retrieval CS276: Information Retrieval and Web Search

Electromagnetic Form Factors of Electromagnetic Form Factors of Electromagnetic Form Factors of

Explore More Topics

Sambuz

Useful Links

Newsletter

Mail Us

An introduction to Sum-Product Networks (SPNs): A new deep - PowerPoint PPT Presentation

An introduction to Sum-Product Networks (SPNs): A new deep probabilistic architecture Felix McGregor Prof Johan du Preez What are SPNs? Poon and Domingos, (UAI 2011 Best Paper) Acyclic directed graphs os sums and products Two views

Product Section Product Section New Product Introduction New Product Introduction Product

ex Addition: 1-bit half adder A + Sum B Carry out Carry A B Sum out 0 0 A 0 1 Sum

Sum Product Networks What is a Sum Product Network? 1. It is a tractable probabilistic model

Basic Ruby Syntax sum = 0 Newline is statement separator i = 1 while i &lt;= 10 do sum += i*i

ex start small with a 1-bit (half) adder A B Carry out Sum A 0 0 Sum 0 1 B 1 0 1 1

Chapter 6 Methods 1 Opening Problem Find the sum of integers from 1 to 10, from 20 to 30, and

The Dot Product and Orthogonal Vectors The Dot Product Defn. The dot product (or inner product )

Horizontal Water Source Heat Pump Horizontal Water Source Heat Pump Product Training Product

McQuay Product Presentation McQuay Product Presentation McQuay Product Presentation TI TLE :

Basic Ruby Syntax No variable declarations sum = 0 Newline is statement separator i = 1 while

Nutrition Education and other Enrichm ent Activities for Sum m er Meal Program s Thursday May 5

The ML Language end; Lisp Algol 60 (We will use Standard ML.)

More on Verilog Sign extension: Example 1 wire [3:0] c, d reg [4:0] sum; always @ (c or d)

Game Theory : Zero-Sum Games, The Minimax Theorem CSC304 - Nisarg Shah 1 Zero-Sum Games

Part I bers, t - target number Question: Is there a subset of X such the sum of its elements is t ?

W4231: Analysis of Algorithms Subset Sum The Subset Sum problem is defined as follows: 11/30/99

4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

Sums &amp; 1 + x + x 2 + + x n G = n x - x 2 - - x n - x n + 1 Money xG = n

Sum-of-products Evaluation Schemes with Fixed-Point arithmetic, and their application to IIR

Some subsystems of constant-depth Frege with parity Michal Garl k Polytechnic University of

Set-theoretic remarks on a possible definition of elementary -topos Giulio Lo Monaco Masaryk

How Simulations and Databases Play Nicely Alex Szalay, JHU Gerard Lemson, MPA Thursday,

Information Retrieval CS276: Information Retrieval and Web Search

Electromagnetic Form Factors of Electromagnetic Form Factors of Electromagnetic Form Factors of

Explore More Topics

Sambuz

Useful Links

Newsletter

Mail Us

Basic Ruby Syntax sum = 0 Newline is statement separator i = 1 while i <= 10 do sum += i*i

Sums & 1 + x + x 2 + + x n G = n x - x 2 - - x n - x n + 1 Money xG = n