spectral graph theory and clustering linear algebra reminder Real - - PowerPoint PPT Presentation

▶

Sep 08, 2023 110 likes •270 views

spectral graph theory and clustering linear algebra reminder Real symmetric matrices have real eigenvalues and eigenvectors. = 2 1 3 1 1 1 = 1 2 1 = 0 0 = (1) 0 3 1 2 1

SLIDE 1

spectral graph theory and clustering

SLIDE 2

linear algebra reminder

Real symmetric matrices have real eigenvalues and eigenvectors.

𝐵 = 2 −1 3 −1 −2 −1 3 −1 2

eigenvalues: eigenvectors: 1 7.2749 1

𝐵𝑗𝑘 = 𝐵𝑘𝑗

−2.2749 −1 −1 1 5.2749 1 −0.2749 1 −1 1 = 1 −1 = (−1) −1 1

SLIDE 3

heat flow

SLIDE 4

heat flow

SLIDE 5

a version in discrete time and space An undirected graph 𝐻 = (𝑊, 𝐹) For now, assume that 𝐻 is 𝒆-regular for some number 𝑒.

SLIDE 6

a version in discrete time and space An undirected graph 𝐻 = (𝑊, 𝐹) 𝑊 = 1, 2, … , 𝑜 𝑣 = 𝑣1, 𝑣2, … , 𝑣𝑜 ∈ ℝ𝑜 Random walk matrix: 𝑋 is an 𝑜 × 𝑜 real symmetric matrix. 𝑋

𝑗𝑗 = 1

2 𝑋

𝑗𝑘 = 1

2𝑒

{𝑗, 𝑘} an edge

𝑋

𝑗𝑘 = 0

{𝑗, 𝑘} not an edge

𝑋𝑣 𝑗 = 1 2 𝑣𝑗 + 1 2 1 𝑒

𝑘∶ 𝑗,𝑘 ∈𝐹

𝑣𝑘

SLIDE 7

heat dispersion on a graph

SLIDE 8

evolution of the random walk / heat flow 𝑣 = 𝑣1, 𝑣2, … , 𝑣𝑜

𝑋𝑣 =

𝑗=1 𝑜

𝑋

1,𝑗𝑣𝑗 , 𝑗=1 𝑜

𝑋

2,𝑗 𝑣𝑗, … , 𝑗=1 𝑜

𝑋

𝑜,𝑗 𝑣𝑗

𝑋2𝑣 =

𝑗,𝑘=1 𝑜

𝑋

1,𝑘𝑋 𝑘,𝑗𝑣𝑗 , … , 𝑗,𝑘=1 𝑜

𝑋

𝑜,𝑘 𝑋 𝑘,𝑗 𝑣𝑗

𝜈1 𝑤1 𝜈2 𝑤2 𝜈𝑜 𝑤𝑜

eigenvalues/ eigenvectors of 𝑋

⋯ 𝑣 = 𝛽1𝑤1 + 𝛽2𝑤2 + ⋯ + 𝛽𝑜𝑤𝑜 𝑋𝑣 = 𝜈1𝛽1𝑤1 + 𝜈2𝛽2𝑤2 + ⋯ + 𝜈𝑜𝛽𝑜𝑤𝑜 𝑋2𝑣 = 𝜈1

2𝛽1𝑤1 + 𝜈2 2𝛽2𝑤2 + ⋯ + 𝜈𝑜 2𝛽𝑜𝑤𝑜

𝑋𝑙𝑣 = 𝜈1

𝑙𝛽1𝑤1 + 𝜈2 𝑙𝛽2𝑤2 + ⋯ + 𝜈𝑜 𝑙𝛽𝑜𝑤𝑜

𝜈1 = 1 𝑤1 = 1 𝑜 , … , 1 𝑜

SLIDE 9

𝑋𝑙𝑣 = 𝛽1 𝑤1 + 𝜈2

𝑙𝛽2𝑤2 + ⋯ + 𝜈𝑜 𝑙𝛽𝑜𝑤𝑜

evolution of the random walk / heat flow 𝑣 = 𝑣1, 𝑣2, … , 𝑣𝑜

𝑋𝑣 =

𝑗=1 𝑜

𝑋

1,𝑗𝑣𝑗 , 𝑗=1 𝑜

𝑋

2,𝑗 𝑣𝑗, … , 𝑗=1 𝑜

𝑋

𝑜,𝑗 𝑣𝑗

𝑋2𝑣 =

𝑗,𝑘=1 𝑜

𝑋

1,𝑘𝑋 𝑘,𝑗𝑣𝑗 , … , 𝑗,𝑘=1 𝑜

𝑋

𝑜,𝑘 𝑋 𝑘,𝑗 𝑣𝑗

𝜈1 𝑤1 𝜈2 𝑤2 𝜈𝑜 𝑤𝑜

eigenvalues/ eigenvectors of 𝑋

⋯ 𝑣 = 𝛽1𝑤1 + 𝛽2𝑤2 + ⋯ + 𝛽𝑜𝑤𝑜 𝑋𝑣 = 𝜈1𝛽1𝑤1 + 𝜈2𝛽2𝑤2 + ⋯ + 𝜈𝑜𝛽𝑜𝑤𝑜 𝑋2𝑣 = 𝜈1

2𝛽1𝑤1 + 𝜈2 2𝛽2𝑤2 + ⋯ + 𝜈𝑜 2𝛽𝑜𝑤𝑜

𝜈1 = 1 𝑤1 = 1 𝑜 , … , 1 𝑜

SLIDE 10

𝜈2 𝑤2 𝑋𝑙𝑣 = 𝛽1 𝑤1 + 𝜈2

𝑙𝛽2𝑤2 + ⋯ + 𝜈𝑜 𝑙𝛽𝑜𝑤𝑜

evolution of the random walk / heat flow 𝑣 = 𝑣1, 𝑣2, … , 𝑣𝑜

𝑋𝑣 =

𝑗=1 𝑜

𝑋

1,𝑗𝑣𝑗 , 𝑗=1 𝑜

𝑋

2,𝑗 𝑣𝑗, … , 𝑗=1 𝑜

𝑋

𝑜,𝑗 𝑣𝑗

𝑋2𝑣 =

𝑗,𝑘=1 𝑜

𝑋

1,𝑘𝑋 𝑘,𝑗𝑣𝑗 , … , 𝑗,𝑘=1 𝑜

𝑋

𝑜,𝑘 𝑋 𝑘,𝑗 𝑣𝑗

𝜈1 𝑤1 𝜈𝑜 𝑤𝑜

eigenvalues/ eigenvectors of 𝑋

⋯ 𝑣 = 𝛽1𝑤1 + 𝛽2𝑤2 + ⋯ + 𝛽𝑜𝑤𝑜 𝑋𝑣 = 𝜈1𝛽1𝑤1 + 𝜈2𝛽2𝑤2 + ⋯ + 𝜈𝑜𝛽𝑜𝑤𝑜 𝑋2𝑣 = 𝜈1

2𝛽1𝑤1 + 𝜈2 2𝛽2𝑤2 + ⋯ + 𝜈𝑜 2𝛽𝑜𝑤𝑜

𝜈1 = 1 𝑤1 = 1 𝑜 , … , 1 𝑜

SLIDE 11

spectral embedding 𝑤2

SLIDE 12

bottlenecks 𝐻 = (𝑊, 𝐹)

𝑇

Φ 𝑇 = 𝐹 𝑇 𝑇 Φ∗ 𝐻 = min

𝑇 ≤𝑜 2

Φ 𝑇

SLIDE 13

PCA cannot find non-linear structure

SLIDE 14

spectral partitioning can...

[photo credit: Ma-Wu-Luo-Feng 2011]

SLIDE 15

spectral partitioning can...

[photo credit: Sidi, et. al. 2011]