machine learning and rendering
play

Machine Learning and Rendering Alex Keller, Director of Research - PowerPoint PPT Presentation

Nov 20, 2023 •419 likes •1.5k views

East Building, Ballroom BC nvidia.com/siggraph2018 Machine Learning and Rendering Alex Keller, Director of Research Machine Learning and Rendering Course web page at https://sites.google.com/site/mlandrendering/ 14:00 From Machine Learning

  1. Modern Path Tracing Numerical integro-approximation ⌅ Monte Carlo methods r r r r r r r r r r r r r r r r n [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 r Z r r r r ∑ r r g ( y ) = f ( y , x i ) r r r r r r r r r n r i = 1 r r r r r r r r r r r r r r rr r r r – uniform, independent, unpredictable random samples x i r r r r r r r r r r – simulated by pseudo-random numbers r r r r r r r r r r r ⌅ quasi-Monte Carlo methods r r r r r n [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 Z r r ∑ g ( y ) = f ( y , x i ) r n r i = 1 r r r r – much more uniform correlated samples x i r r r r – realized by low-discrepancy sequences, which are progressive Latin-hypercube samples 5

  2. Modern Path Tracing Numerical integro-approximation ⌅ Monte Carlo methods r r r r r r r r r r r r r r r r n [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 r Z r r r r ∑ r r g ( y ) = f ( y , x i ) r r r r r r r r r n r i = 1 r r r r r r r r r r r r r r rr r r r – uniform, independent, unpredictable random samples x i r r r r r r r r r r – simulated by pseudo-random numbers r r r r r r r r r r r ⌅ quasi-Monte Carlo methods r r r r r n [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 Z r r ∑ g ( y ) = f ( y , x i ) r n r i = 1 r r r r – much more uniform correlated samples x i r r r r – realized by low-discrepancy sequences, which are progressive Latin-hypercube samples 5

  3. Modern Path Tracing Numerical integro-approximation ⌅ Monte Carlo methods r r r r r r r r r r r r r r r r n [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 r Z r r r r ∑ r r g ( y ) = f ( y , x i ) r r r r r r r r r n r i = 1 r r r r r r r r r r r r r r rr r r r – uniform, independent, unpredictable random samples x i r r r r r r r r r r – simulated by pseudo-random numbers r r r r r r r r r r r r r ⌅ quasi-Monte Carlo methods r r r r r r r r r n r [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 Z r r r ∑ g ( y ) = f ( y , x i ) r r n r r r i = 1 r r r r r r r r – much more uniform correlated samples x i r r r r r r r – realized by low-discrepancy sequences, which are progressive Latin-hypercube samples 5

  4. Modern Path Tracing Numerical integro-approximation ⌅ Monte Carlo methods r r r r r r r r r r r r r r r r n [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 r Z r r r r ∑ r r g ( y ) = f ( y , x i ) r r r r r r r r r n r i = 1 r r r r r r r r r r r r r r rr r r r – uniform, independent, unpredictable random samples x i r r r r r r r r r r – simulated by pseudo-random numbers r r r r r r r r r r r r r r r ⌅ quasi-Monte Carlo methods r r r r r r r r r r r r r n r r [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 Z r r r r ∑ g ( y ) = f ( y , x i ) r r r r n r r r r i = 1 r r r r r r r r r r r r r – much more uniform correlated samples x i r r r r r r r r r – realized by low-discrepancy sequences, which are progressive Latin-hypercube samples 5

  5. Modern Path Tracing Numerical integro-approximation ⌅ Monte Carlo methods r r r r r r r r r r r r r r r r n [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 r Z r r r r ∑ r r g ( y ) = f ( y , x i ) r r r r r r r r r n r i = 1 r r r r r r r r r r r r r r rr r r r – uniform, independent, unpredictable random samples x i r r r r r r r r r r – simulated by pseudo-random numbers r r r r r r r r r r r r r r r r r ⌅ quasi-Monte Carlo methods r r r r r r r r r r r r r r r r r n r r [ 0 , 1 ) s f ( y , x ) dx ⇡ 1 r Z r r r r r ∑ g ( y ) = f ( y , x i ) r r r r r r n r r r r r i = 1 r r r r r r r r r r r r r r r r r – much more uniform correlated samples x i r r r r r r r r r r r r – realized by low-discrepancy sequences, which are progressive Latin-hypercube samples 5

  6. Modern Path Tracing Pushbutton paradigm ⌅ deterministic – may improve speed of convergence – reproducible and simple to parallelize 6

  7. Modern Path Tracing Pushbutton paradigm ⌅ deterministic – may improve speed of convergence – reproducible and simple to parallelize ⌅ unbiased – zero difference between expectation and mathematical object – not sufficient for convergence 6

  8. Modern Path Tracing Pushbutton paradigm ⌅ deterministic – may improve speed of convergence – reproducible and simple to parallelize ⌅ biased – allows for ameliorating the problem of insufficient techniques – can tremendously increase efficiency 6

  9. Modern Path Tracing Pushbutton paradigm ⌅ deterministic – may improve speed of convergence – reproducible and simple to parallelize ⌅ biased – allows for ameliorating the problem of insufficient techniques – can tremendously increase efficiency ⌅ consistent – error vanishes with increasing set of samples – no persistent artifacts introduced by algorithm I Quasi-Monte Carlo image synthesis in a nutshell I The Iray light transport simulation and rendering system 6

  10. Modern Path Tracing Pushbutton paradigm ⌅ deterministic – may improve speed of convergence – reproducible and simple to parallelize ⌅ biased – allows for ameliorating the problem of insufficient techniques – can tremendously increase efficiency ⌅ consistent – error vanishes with increasing set of samples – no persistent artifacts introduced by algorithm I Quasi-Monte Carlo image synthesis in a nutshell I The Iray light transport simulation and rendering system 6

  11. Reconstruction from noisy input: Massively parallel path space filtering (link)

  12. From Machine Learning to Graphics

  13. Machine Learning Taxonomy ⌅ unsupervised learning from unlabeled data – examples: clustering, auto-encoder networks 9

  14. Machine Learning Taxonomy ⌅ unsupervised learning from unlabeled data – examples: clustering, auto-encoder networks ⌅ semi-supervised learning by rewards – example: reinforcement learning 9

  15. Machine Learning Taxonomy ⌅ unsupervised learning from unlabeled data – examples: clustering, auto-encoder networks ⌅ semi-supervised learning by rewards – example: reinforcement learning ⌅ supervised learning from labeled data – examples: support vector machines, decision trees, artificial neural networks 9

  16. Reinforcement Learning Goal: maximize reward ⌅ state transition yields reward Agent s t r t + 1 ( a t | s t ) 2 R s t + 1 a t r t + 1 ( a t | s t ) Environment 10

  17. Reinforcement Learning Goal: maximize reward ⌅ state transition yields reward Agent s t r t + 1 ( a t | s t ) 2 R s t + 1 a t r t + 1 ( a t | s t ) ⌅ learn a policy π t – to select an action a t 2 A ( s t ) Environment – given the current state s t 2 S 10

  18. Reinforcement Learning Goal: maximize reward ⌅ state transition yields reward Agent s t r t + 1 ( a t | s t ) 2 R s t + 1 a t r t + 1 ( a t | s t ) ⌅ learn a policy π t – to select an action a t 2 A ( s t ) Environment – given the current state s t 2 S ⌅ maximizing the discounted cumulative reward ∞ γ k · r t + 1 + k ( a t + k | s t + k ) , where 0 < γ < 1 ∑ V ( s t ) ⌘ k = 0 10

  19. Reinforcement Learning Q-Learning [Watkins 1989] ⌅ learns optimal action selection policy for any given Markov decision process Q 0 ( s , a ) r ( s , a )+ γ · V ( s 0 ) � � = ( 1 � α ) · Q ( s , a )+ α · for a learning rate α 2 [ 0 , 1 ] 11

  20. Reinforcement Learning Q-Learning [Watkins 1989] ⌅ learns optimal action selection policy for any given Markov decision process Q 0 ( s , a ) r ( s , a )+ γ · V ( s 0 ) � � = ( 1 � α ) · Q ( s , a )+ α · for a learning rate α 2 [ 0 , 1 ] with the following options for the discounted cumulative reward 8 max a 0 2 A Q ( s 0 , a 0 ) consider best action in next state s 0 > > > > > > < V ( s 0 ) ⌘ > > > > > > : 11

  21. Reinforcement Learning Q-Learning [Watkins 1989] ⌅ learns optimal action selection policy for any given Markov decision process Q 0 ( s , a ) r ( s , a )+ γ · V ( s 0 ) � � = ( 1 � α ) · Q ( s , a )+ α · for a learning rate α 2 [ 0 , 1 ] with the following options for the discounted cumulative reward 8 max a 0 2 A Q ( s 0 , a 0 ) consider best action in next state s 0 > > > > > > < V ( s 0 ) ⌘ ∑ a 0 2 A π ( s 0 , a 0 ) Q ( s 0 , a 0 ) policy weighted average over discrete action space > > > > > > : 11

  22. Reinforcement Learning Q-Learning [Watkins 1989] ⌅ learns optimal action selection policy for any given Markov decision process Q 0 ( s , a ) r ( s , a )+ γ · V ( s 0 ) � � = ( 1 � α ) · Q ( s , a )+ α · for a learning rate α 2 [ 0 , 1 ] with the following options for the discounted cumulative reward 8 max a 0 2 A Q ( s 0 , a 0 ) consider best action in next state s 0 > > > > > > < V ( s 0 ) ⌘ ∑ a 0 2 A π ( s 0 , a 0 ) Q ( s 0 , a 0 ) policy weighted average over discrete action space > > > > > A π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 > R policy weighted average over continuous action space : 11

  23. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ radiance integral equation L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) 12

  24. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ structural equivalence of integral equation and Q -learning L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) ✓ ◆ Q 0 ( s , a ) π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 = ( 1 � α ) Q ( s , a )+ α r ( s , a ) + γ R A 12

  25. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ structural equivalence of integral equation and Q -learning L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) ✓ ◆ Q 0 ( s , a ) π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 = ( 1 � α ) Q ( s , a )+ α r ( s , a ) + γ R A 12

  26. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ structural equivalence of integral equation and Q -learning L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) ✓ ◆ Q 0 ( s , a ) π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 = ( 1 � α ) Q ( s , a )+ α r ( s , a ) + γ R A 12

  27. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ structural equivalence of integral equation and Q -learning L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) ✓ ◆ Q 0 ( s , a ) π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 = ( 1 � α ) Q ( s , a )+ α r ( s , a ) + γ R A 12

  28. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ structural equivalence of integral equation and Q -learning L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) ✓ ◆ Q 0 ( s , a ) π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 = ( 1 � α ) Q ( s , a )+ α r ( s , a ) + γ R A 12

  29. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ structural equivalence of integral equation and Q -learning L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) ✓ ◆ Q 0 ( s , a ) π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 = ( 1 � α ) Q ( s , a )+ α r ( s , a ) + γ R A ⌅ graphics example: learning the incident radiance ✓ ◆ Z Q 0 ( x , ω ) = ( 1 � α ) Q ( x , ω )+ α L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 12

  30. Reinforcement Learning Maximize reward by learning importance sampling online ⌅ structural equivalence of integral equation and Q -learning L ( x , ω ) = L e ( x , ω ) + R f s ( ω i , x , ω )cos θ i L ( h ( x , ω i ) , � ω i ) d ω i S 2 + ( x ) ✓ ◆ Q 0 ( s , a ) π ( s 0 , a 0 ) Q ( s 0 , a 0 ) da 0 = ( 1 � α ) Q ( s , a )+ α r ( s , a ) + γ R A ⌅ graphics example: learning the incident radiance ✓ ◆ Z Q 0 ( x , ω ) = ( 1 � α ) Q ( x , ω )+ α L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 to be used as a policy for selecting an action ω in state x to reach the next state y := h ( x , ω ) – the learning rate α is the only parameter left I Technical Note: Q -Learning 12

  31. Reinforcement Learning Online algorithm for guiding light transport paths Function pathTrace(camera , scene) throughput 1 ray setupPrimaryRay( camera ) for i 0 to ∞ do y , n intersect( scene , ray ) if isEnvironment(y) then return throughput · getRadianceFromEnvironment( ray , y ) else if isAreaLight(y) return throughput · getRadianceFromAreaLight( ray , y ) ω , p ω , f s sampleBsdf( y , n ) throughput throughput · f s · cos( n , ω ) / p ω ray y , ω 13

  32. Reinforcement Learning Online algorithm for guiding light transport paths Function pathTrace(camera , scene) throughput 1 ray setupPrimaryRay( camera ) for i 0 to ∞ do y , n intersect( scene , ray ) if i > 0 then ✓ ◆ Q 0 ( x , ω ) = ( 1 � α ) Q ( x , ω )+ α R L e ( y , � ω )+ +( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 if isEnvironment(y) then return throughput · getRadianceFromEnvironment( ray , y ) else if isAreaLight(y) return throughput · getRadianceFromAreaLight( ray , y ) ω , p ω , f s sampleScatteringDirectionProportionalToQ( y ) throughput throughput · f s · cos( n , ω ) / p ω ray y , ω 13

  33. approximate solution Q stored on discretized hemispheres across scene surface

  34. 2048 paths traced with BRDF importance sampling in a scene with challenging visibility

  35. Path tracing with online reinforcement learning at the same number of paths

  36. Metropolis light transport at the same number of paths

  37. Reinforcement Learning Guiding paths to where the value Q comes from ⌅ shorter expected path length ⌅ dramatically reduced number of paths with zero contribution ⌅ very efficient online learning by learning Q from Q 18

  38. Reinforcement Learning Guiding paths to where the value Q comes from ⌅ shorter expected path length ⌅ dramatically reduced number of paths with zero contribution ⌅ very efficient online learning by learning Q from Q ⌅ directions for research – representation of value Q : data structures from games – importance sampling proportional to the integrand, i.e. the product of policy γ · π times value Q I On-line learning of parametric mixture models for light transport simulation I Product importance sampling for light transport path guiding I Fast product importance sampling of environment maps I Learning light transport the reinforced way I Practical path guiding for efficient light-transport simulation 18

  39. From Graphics back to Machine Learning

  40. Artificial Neural Networks in a Nutshell Supervised learning of high dimensional function approximation ⌅ input layer a 0 , L � 1 hidden layers, and output layer a L a 0 , 0 a L , 0 a 1 , 0 a 2 , 0 a 0 , 1 a L , 1 a 1 , 1 a 2 , 1 a 0 , 2 a L , 2 a 1 , 2 a 2 , 2 . . . . . . . . . . . . a 0 , n 0 � 1 a L , n L � 1 a 1 , n 1 � 1 a 2 , n 2 � 1 20

  41. Artificial Neural Networks in a Nutshell Supervised learning of high dimensional function approximation ⌅ input layer a 0 , L � 1 hidden layers, and output layer a L a 0 , 0 a L , 0 a 1 , 0 a 2 , 0 a 0 , 1 a L , 1 a 1 , 1 a 2 , 1 a 2 , 1 a 0 , 2 a L , 2 a 1 , 2 a 2 , 2 . . . . . . . . . . . . a 0 , n 0 � 1 a L , n L � 1 a 1 , n 1 � 1 a 2 , n 2 � 1 – n l rectified linear units (ReLU) a l , i = max { 0 , ∑ w l , j , i a l � 1 , j } in layer l 20

  42. Artificial Neural Networks in a Nutshell Supervised learning of high dimensional function approximation ⌅ input layer a 0 , L � 1 hidden layers, and output layer a L a 0 , 0 a L , 0 a 1 , 0 a 2 , 0 a 0 , 1 a L , 1 a 1 , 1 a 2 , 1 a 2 , 1 a 0 , 2 a L , 2 a 1 , 2 a 2 , 2 . . . . . . . . . . . . a 0 , n 0 � 1 a L , n L � 1 a 1 , n 1 � 1 a 2 , n 2 � 1 – n l rectified linear units (ReLU) a l , i = max { 0 , ∑ w l , j , i a l � 1 , j } in layer l – backpropagating the error δ l � 1 , i = ∑ a l , j > 0 δ l , j w l , j , i 20

  43. Artificial Neural Networks in a Nutshell Supervised learning of high dimensional function approximation ⌅ input layer a 0 , L � 1 hidden layers, and output layer a L a 0 , 0 a L , 0 a 1 , 0 a 2 , 0 a 0 , 1 a L , 1 a 1 , 1 a 2 , 1 a 2 , 1 a 0 , 2 a L , 2 a 1 , 2 a 2 , 2 . . . . . . . . . . . . a 0 , n 0 � 1 a L , n L � 1 a 1 , n 1 � 1 a 2 , n 2 � 1 – n l rectified linear units (ReLU) a l , i = max { 0 , ∑ w l , j , i a l � 1 , j } in layer l – backpropagating the error δ l � 1 , i = ∑ a l , j > 0 δ l , j w l , j , i , update weights w 0 l , j , i = w l , j , i � λδ l , j a l � 1 , i if a l , j > 0 20

  44. Artificial Neural Networks in a Nutshell Supervised learning of high dimensional function approximation ⌅ example architectures classifier I Multilayer feedforward networks are universal approximators I Approximation capabilities of multilayer feedforward networks I Universal approximation bounds for superpositions of a sigmoidal function 21

  45. Artificial Neural Networks in a Nutshell Supervised learning of high dimensional function approximation ⌅ example architectures classifier generator I Multilayer feedforward networks are universal approximators I Approximation capabilities of multilayer feedforward networks I Universal approximation bounds for superpositions of a sigmoidal function 21

  46. Artificial Neural Networks in a Nutshell Supervised learning of high dimensional function approximation ⌅ example architectures classifier generator auto-encoder I Multilayer feedforward networks are universal approximators I Approximation capabilities of multilayer feedforward networks I Universal approximation bounds for superpositions of a sigmoidal function 21

  47. Efficient Training of Artificial Neural Networks Using an integral equation for supervised learning ⌅ Q -learning ✓ ◆ Z Q 0 ( x , ω ) = ( 1 � α ) Q ( x , ω )+ α L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 22

  48. Efficient Training of Artificial Neural Networks Using an integral equation for supervised learning ⌅ Q -learning ✓ ◆ Z Q 0 ( x , ω ) = ( 1 � α ) Q ( x , ω )+ α L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 for α = 1 yields the residual, i.e. loss ✓ ◆ Z ∆ Q := Q ( x , ω ) � L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 22

  49. Efficient Training of Artificial Neural Networks Using an integral equation for supervised learning ⌅ Q -learning ✓ ◆ Z Q 0 ( x , ω ) = ( 1 � α ) Q ( x , ω )+ α L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 for α = 1 yields the residual, i.e. loss ✓ ◆ Z ∆ Q := Q ( x , ω ) � L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 ⌅ supervised learning algorithm – light transport paths generated by a low discrepancy sequence for online training 22

  50. Efficient Training of Artificial Neural Networks Using an integral equation for supervised learning ⌅ Q -learning ✓ ◆ Z Q 0 ( x , ω ) = ( 1 � α ) Q ( x , ω )+ α L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 for α = 1 yields the residual, i.e. loss ✓ ◆ Z ∆ Q := Q ( x , ω ) � L e ( y , � ω )+ + ( y ) f s ( ω i , y , � ω )cos θ i Q ( y , ω i ) d ω i S 2 ⌅ supervised learning algorithm – light transport paths generated by a low discrepancy sequence for online training – learn weights of an artificial neural network for Q ( x , n ) by back-propagating loss of each path I A machine learning driven sky model I Global illumination with radiance regression Functions I Machine learning and integral equations I Neural importance sampling 22

  51. Efficient Training of Artificial Neural Networks Learning from noisy/sampled labeled data ⌅ find set of weights θ of an artificial neural network f to minimize summed loss L – using clean targets y i and data ˆ x i distributed according to ˆ x ⇠ p (ˆ x | y i ) argmin θ ∑ L ( f θ (ˆ x i ) , y i ) i 23

  52. Efficient Training of Artificial Neural Networks Learning from noisy/sampled labeled data ⌅ find set of weights θ of an artificial neural network f to minimize summed loss L – using clean targets y i and data ˆ x i distributed according to ˆ x ⇠ p (ˆ x | y i ) argmin θ ∑ L ( f θ (ˆ x i ) , y i ) i – using targets ˆ y i distributed according to ˆ y ⇠ p (ˆ y ) instead argmin θ ∑ L ( f θ (ˆ x i ) , ˆ y i ) i 23

  53. Efficient Training of Artificial Neural Networks Learning from noisy/sampled labeled data ⌅ find set of weights θ of an artificial neural network f to minimize summed loss L – using clean targets y i and data ˆ x i distributed according to ˆ x ⇠ p (ˆ x | y i ) argmin θ ∑ L ( f θ (ˆ x i ) , y i ) i – using targets ˆ y i distributed according to ˆ y ⇠ p (ˆ y ) instead argmin θ ∑ L ( f θ (ˆ x i ) , ˆ y i ) i ⇧ allows for much faster training of artificial neural networks used in simulations ⌅ amounts to learning integration and integro-approximation I Noise2Noise: Learning image restoration without clean data 23

  54. Example Applications of Artificial Neural Networks in Rendering Learning from noisy/sampled labeled data ⌅ denoising quasi-Monte Carlo rendered images – noisy targets computed 2000 ⇥ faster than clean targets 24

  55. Example Applications of Artificial Neural Networks in Rendering Sampling according to a distribution given by observed data ⌅ generative adversarial network (GAN) I image source I Tutorial on GANs 25

  56. Example Applications of Artificial Neural Networks in Rendering Sampling according to a distribution given by observed data ⌅ generative adversarial network (GAN) I image source I Tutorial on GANs 25

  57. Example Applications of Artificial Neural Networks in Rendering Sampling according to a distribution given by observed data ⌅ generative adversarial network (GAN) I image source I Tutorial on GANs 25

  58. Example Applications of Artificial Neural Networks in Rendering Sampling according to a distribution given by observed data ⌅ generative adversarial network (GAN) ∇ θ g ∑ m – update generator G using i = 1 log( 1 � D ( G ( ξ i ))) I image source I Tutorial on GANs 25

  59. Example Applications of Artificial Neural Networks in Rendering Sampling according to a distribution given by observed data ⌅ generative adversarial network (GAN) – update discriminator D (k times) using ∇ θ d m ∑ m 1 i = 1 [log D ( x i )+log( 1 � D ( G ( ξ i )))] ∇ θ g ∑ m – update generator G using i = 1 log( 1 � D ( G ( ξ i ))) I image source I Tutorial on GANs 25

  60. Example Applications of Artificial Neural Networks in Rendering Sampling according to a distribution given by observed data ⌅ Celebrity GAN I Progressive growing of GANs for improved quality, stability, and variation 26

  61. Example Applications of Artificial Neural Networks in Rendering Replacing simulations by learned predictions for more efficiency ⌅ much faster simulation of participating media – hierarchical stencil of volume densities as input to the neural network I Deep scattering: Rendering atmospheric clouds with radiance-predicting neural networks I Learning particle physics by example: Accelerating science with generative adversarial networks 27

  62. Neural Networks linear in Time and Space

  63. Neural Networks linear in Time and Space Complexity ⌅ the brain – about 10 11 nerve cells with to up to 10 4 connections to others 29

  64. Neural Networks linear in Time and Space Complexity ⌅ the brain – about 10 11 nerve cells with to up to 10 4 connections to others ⌅ artificial neural networks – number of neural units L ∑ n = n l where n l is the number of neurons in layer l l = 1 29

  65. Neural Networks linear in Time and Space Complexity ⌅ the brain – about 10 11 nerve cells with to up to 10 4 connections to others ⌅ artificial neural networks – number of neural units L ∑ n = n l where n l is the number of neurons in layer l l = 1 – number of weights L ∑ n w = n l � 1 · n l l = 1 29

  66. Neural Networks linear in Time and Space Complexity ⌅ the brain – about 10 11 nerve cells with to up to 10 4 connections to others ⌅ artificial neural networks – number of neural units L ∑ n = n l where n l is the number of neurons in layer l l = 1 – number of weights L ∑ n w = c · n l l = 1 – constrain to constant number c of weights per neuron 29

  67. Neural Networks linear in Time and Space Complexity ⌅ the brain – about 10 11 nerve cells with to up to 10 4 connections to others ⌅ artificial neural networks – number of neural units L ∑ n = n l where n l is the number of neurons in layer l l = 1 – number of weights L ∑ n w = c · n l = c · n l = 1 – constrain to constant number c of weights per neuron to reach complexity linear in n 29

  68. Neural Networks linear in Time and Space Sampling proportional to the weights of the trained neural units ⌅ partition of unit interval by sums P k := ∑ k j = 1 | w j | of normalized absolute weights w 1 w 2 w m 0 1 P 0 P 1 P 2 P m � 1 P m 30

  69. Neural Networks linear in Time and Space Sampling proportional to the weights of the trained neural units ⌅ partition of unit interval by sums P k := ∑ k j = 1 | w j | of normalized absolute weights w 1 w 2 w m 0 1 P 0 P 1 P 2 P m � 1 P m – using a uniform random variable ξ 2 [ 0 , 1 ) to select input i , P i � 1  ξ < P i satisfying Prob ( { P i � 1  ξ < P i } ) = | w i | 30

  70. Neural Networks linear in Time and Space Sampling proportional to the weights of the trained neural units ⌅ partition of unit interval by sums P k := ∑ k j = 1 | w j | of normalized absolute weights w 1 w 2 w m 0 1 P 0 P 1 P 2 P m � 1 P m – using a uniform random variable ξ 2 [ 0 , 1 ) to select input i , P i � 1  ξ < P i satisfying Prob ( { P i � 1  ξ < P i } ) = | w i | ⌅ in fact derivation of quantization to ternary weights in { � 1 , 0 , + 1 } – integer weights result from neurons referenced more than once – relation to drop connect and drop out 30

  71. Neural Networks linear in Time and Space Sampling proportional to the weights of the trained neural units 1 0 . 8 Test Accuracy 0 . 6 LeNet on MNIST 0 . 4 LeNet on CIFAR-10 AlexNet on CIFAR-10 0 . 2 Top-5 Accuracy AlexNet on ILSVRC12 Top-1 Accuracy AlexNet on ILSVRC12 0 0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1 Percent of fully connected layers sampled 31

  72. Neural Networks linear in Time and Space Sampling paths through networks ⌅ complexity bounded by number of paths times depth L of network 32

  73. Neural Networks linear in Time and Space Sampling paths through networks ⌅ complexity bounded by number of paths times depth L of network ⌅ application after training – backwards random walks using sampling proportional to the weights of a neuron – compression and quantization by importance sampling 32

  74. Neural Networks linear in Time and Space Sampling paths through networks ⌅ complexity bounded by number of paths times depth L of network ⌅ application after training – backwards random walks using sampling proportional to the weights of a neuron – compression and quantization by importance sampling ⌅ application before training – uniform (bidirectional) random walks to connect inputs and outputs – sparse from scratch 32

  75. Neural Networks linear in Time and Space Sampling paths through networks ⌅ sparse from scratch a 0 , 0 a L , 0 a 1 , 0 a 2 , 0 a 0 , 1 a L , 1 a 1 , 1 a 2 , 1 a 0 , 2 a L , 2 a 1 , 2 a 2 , 2 . . . . . . . . . . . . a 0 , n 0 � 1 a L , n L � 1 a 1 , n 1 � 1 a 2 , n 2 � 1 33

  76. Neural Networks linear in Time and Space Sampling paths through networks ⌅ sparse from scratch a 0 , 0 a L , 0 a 1 , 0 a 2 , 0 a 0 , 1 a L , 1 a 1 , 1 a 2 , 1 a 2 , 1 a 0 , 2 a L , 2 a 1 , 2 a 2 , 2 . . . . . . . . . . . . a 0 , n 0 � 1 a L , n L � 1 a 1 , n 1 � 1 a 2 , n 2 � 1 33

Recommend

from Image Processing to Machine Learning Dr. Yu-Kun Lai, Cardiff University GAMES webinar

from Image Processing to Machine Learning Dr. Yu-Kun Lai, Cardiff University GAMES webinar

Non-photorealistic Rendering of Images: from Image Processing to Machine Learning Dr. Yu-Kun Lai, Cardiff University GAMES webinar 4/7/2019 Non-Photorealistic Rendering A well researched topic that mimics artistic styles using computer

891 views • 42 slides
An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning http://www.cs.iastate.edu/~cs573x/bbsilab.html Machine Learning Software Preparing Data Building Classifiers Interpreting Results Machine Learning Software

496 views • 26 slides
Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by which computers can be trained through observation, rather than being explicitly programmed. Basically, a program is given input and adjusts its

559 views • 17 slides
Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: Readings: What is machine learning? The Discipline of ML Decision tree learning Mitchell, Chapter 3

872 views • 29 slides
Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning Pipeline in Classification Tasks A set of hand-designed audio features are selected

401 views • 26 slides
CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

1/22/19 CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine Learning? A Simple Task: Recognize Obama How do you program a computer to Recognize faces? Recommend movies? Decide which web

582 views • 5 slides
Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their performance We use machine learning in situations where it is very challenging (or impossible) to define the rules by hand: e.g. face detection

1.04k views • 28 slides
Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Introduction to Machine Learning 1 / 12 Outline 1 Nearest Neighbor Prediction 2 Complexity Considerations 3 The Voronoi Diagram 4 Overfitting

565 views • 12 slides
1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

What is Learning? Herbert Simon: Learning is any process by which a system improves performance from CS 391L: Machine Learning experience. Introduction What is the task? Classification Problem solving / planning /

603 views • 6 slides
MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

JACOPO DI IORIO, MATTEO FONTANA, DANIELE OCCHIUTO, CLAUDIA VOLPETTI MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING STATISTICAL LEARNING Separate evolutions, but with shared methodologies MERGE

565 views • 55 slides
Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine Learning? What is Machine Learning? Examples of machine learning applications: Facial recognition software E-mail spam detectors Automated

302 views • 17 slides
Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning 11.1 Learning agents 11.2 Inductive learning 11.3 Deep learning 11.4 Statistical learning 11.5 Reinforcement learning 11.6 Transfer learning

1.7k views • 159 slides
Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine learning pipeline Temporal Frame-level Unsupervised Classifier Summary

613 views • 27 slides
Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine learning refers to any system where the performance of a machine in performing a task improves by gaining more experience in performing that task .

394 views • 27 slides
MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING 2012 MACHINE LEARNING MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How to separate the red class from the grey class? x 2 360 r x 1 Polar coordinates Data

1.04k views • 44 slides
BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

photo:@rewardyfahmi // Unsplash BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest Neighbor Classifier Aykut Erdem // Hacettepe University // Fall 2019 When Do We Use Machine Learning? ML is

1.33k views • 110 slides
Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a

Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a

Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a dataset, identify features which may help predict information about the data Identify common types of machine learning algorithms 2 Machine

553 views • 27 slides
Machine Learning @ Amazon Ralf Herbrich Amazon 6/29/17 1 Overview Machine Learning in

Machine Learning @ Amazon Ralf Herbrich Amazon 6/29/17 1 Overview Machine Learning in

Machine Learning @ Amazon Ralf Herbrich Amazon 6/29/17 1 Overview Machine Learning in Practise Probabilities Finite Resource Machine Learning @ Amazon Forecasting Machine Translation Visual Systems Conclusions

348 views • 34 slides
Softmax Classifier + SGD Todays Class Intro to Machine Learning What is Machine Learning?

Softmax Classifier + SGD Todays Class Intro to Machine Learning What is Machine Learning?

CS6501: Deep Learning for Visual Recognition Softmax Classifier + SGD Todays Class Intro to Machine Learning What is Machine Learning? Supervised Learning: Classification with K-nearest neighbors Unsupervised Learning: Clustering with

1.36k views • 91 slides
Neural Networks for Machine Learning Lecture 1a Why do we need machine learning? Geoffrey Hinton

Neural Networks for Machine Learning Lecture 1a Why do we need machine learning? Geoffrey Hinton

Neural Networks for Machine Learning Lecture 1a Why do we need machine learning? Geoffrey Hinton with Nitish Srivastava Kevin Swersky What is Machine Learning? It is very hard to write programs that solve problems like recognizing a

514 views • 48 slides
VCD: Rendering Objective: To understand, and be able to use, tonal rendering to communicate and

VCD: Rendering Objective: To understand, and be able to use, tonal rendering to communicate and

VCD: Rendering Objective: To understand, and be able to use, tonal rendering to communicate and represent our design ideas What is Rendering? Rendering is the use of tone (light and dark) to show 3D form and texture. There are a range of

460 views • 14 slides
MACHINE LEARNING Overview 1 MACHINE LEARNING Oral Presentations of Projects Start at 9h15 am

MACHINE LEARNING Overview 1 MACHINE LEARNING Oral Presentations of Projects Start at 9h15 am

MACHINE LEARNING MACHINE LEARNING Overview 1 MACHINE LEARNING Oral Presentations of Projects Start at 9h15 am and last until 12h30 2 MACHINE LEARNING Exam Format The exam lasts a total of 40 minutes: -Upon entering the room, you pick at

919 views • 57 slides
Machine Learning for Music: Intro Juhan Nam Definition of Machine Learning Tom M. Mitchell

Machine Learning for Music: Intro Juhan Nam Definition of Machine Learning Tom M. Mitchell

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Machine Learning for Music: Intro Juhan Nam Definition of Machine Learning Tom M. Mitchell provided a widely accepted definition: A computer program is said to learn from

595 views • 24 slides
Welcome to the Machine Learning Toolbox! Machine Learning Toolbox Supervised learning caret

Welcome to the Machine Learning Toolbox! Machine Learning Toolbox Supervised learning caret

MACHINE LEARNING TOOLBOX Welcome to the Machine Learning Toolbox! Machine Learning Toolbox Supervised learning caret R package Automates supervised learning (a.k.a. predictive modeling ) Target variable Machine Learning Toolbox

634 views • 16 slides

More recommend