Non-Parametric Methods; Simulations March 6, 2020 Data Science - PowerPoint PPT Presentation

Non-Parametric Methods; Simulations March 6, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie Pavlick HTAs: Josh Levin, Diane Mutako, Sol Zitter

Announcements

Today • Non-Parametric Methods • Simulations (example using Gaussian Mixture Models)

Today • Non-Parametric Methods • Simulations (example using Gaussian Mixture Models)

Parametric vs. Non- Parametric cholesterol Given x, predict y eucalyptus

Parametric vs. Non- Parametric cholesterol Given x, predict y y = mx + b + e eucalyptus

Parametric vs. Non- Parametric cholesterol Given x, predict y y = mx + b + e eucalyptus

Clicker Question!

Parametric vs. Non- Parametric cholesterol Given x, predict y y = mx + b + e Thoughts? eucalyptus

Parametric vs. Non- Parametric cholesterol Given x, predict y y = mx + b + e Nearest Neighbors! eucalyptus

Parametric vs. Non- Parametric cholesterol Given x, predict y y = mx + b + e Nearest Neighbors! eucalyptus

Parametric vs. Non- Parametric cholesterol Given x, predict y y = mx + b + e Nearest Neighbors! eucalyptus

Parametric vs. Non- Parametric cholesterol Given x, predict y y = mx + b + e Nearest Neighbors! eucalyptus

Clicker Question!

Non-Parametric Models • “Non-parametric” models: No assumptions about the number of parameters in the model or the particular form of the model • Pros: • Can work well with small data • Or when you have very complex distributions and you aren’t sure what assumptions can be made • Cons: • Size of model can increase with size of data • Slow to compute (randomized/interative processes) • Fewer assumptions -> weaker conclusions (higher p-values)

Non-Parametric Models • “Non-parametric” models: No assumptions about the number of parameters in the model or the particular form of the model • Pros: • Can work well with small data • Or when you have very complex distributions and you aren’t sure what assumptions can be made • Cons: • Size of model can increase with size of data • Slow to compute (randomized/interative processes) • Fewer assumptions -> weaker conclusions (higher p-values)

Non-Parametric Models • “Non-parametric” models: No assumptions about the number of parameters in the model or the particular form of the model • Pros: • Can work well with small data • Or when you have very complex distributions and you aren’t sure what assumptions can be made • Cons: • Size of model can increase with size of data • Slow to compute (randomized/interative processes) • Fewer assumptions -> weaker conclusions (higher p-values)

Non-Parametric Models • “Non-parametric” models: No assumptions about the number of parameters in the model or the particular form of the model • Pros: • Can work well with small data • Or when you have very complex distributions and you aren’t sure what assumptions can be made • Cons: • Size of model can increase with size of data • Slow to compute (randomized/iterative processes) • Fewer assumptions -> weaker conclusions (higher p-values)

<latexit sha1_base64="8/FnER2VbVMPSWj2wlp3l3vNu4=">ACQXicbVBNS8QwE39XOtX1aOX4CIowtKoBdh0YvHFVwtbJeSZlMNpmlJpmIp/Wte/AfevHvxoIhXL2Z3C34+CPN4b4bJvCgTXIPrPloTk1PTM7ONOXt+YXFp2VlZPdprijr0lSkyo+IZoJL1gUOgvmZYiSJBLuIro+H/sUNU5qn8gyKjPUTcil5zCkBI4WOH0RElX4VSnyIg1gRWnpVKastP/TwDg4GKWhT/VBu4yCwv7oDSHGQ5KayWygx0bjCtcplDEXoN2WOwL+S7yaNFGNTug8mGU0T5gEKojWPc/NoF8SBZwKVtlBrlG6DW5ZD1DJUmY7pejBCq8aZQBjlNlngQ8Ur9PlCTRukgi05kQuNK/vaH4n9fLIT7ol1xmOTBJx4viXGBz5zBOPOCKURCFIYQqbv6K6RUxMYIJ3TYheL9P/kvOd1ue2/JO95rtozqOBlpHG2gLeWgftdEJ6qAuougOPaEX9GrdW8/Wm/U+bp2w6pk19APWxycnx65E</latexit> <latexit sha1_base64="8/FnER2VbVMPSWj2wlp3l3vNu4=">ACQXicbVBNS8QwE39XOtX1aOX4CIowtKoBdh0YvHFVwtbJeSZlMNpmlJpmIp/Wte/AfevHvxoIhXL2Z3C34+CPN4b4bJvCgTXIPrPloTk1PTM7ONOXt+YXFp2VlZPdprijr0lSkyo+IZoJL1gUOgvmZYiSJBLuIro+H/sUNU5qn8gyKjPUTcil5zCkBI4WOH0RElX4VSnyIg1gRWnpVKastP/TwDg4GKWhT/VBu4yCwv7oDSHGQ5KayWygx0bjCtcplDEXoN2WOwL+S7yaNFGNTug8mGU0T5gEKojWPc/NoF8SBZwKVtlBrlG6DW5ZD1DJUmY7pejBCq8aZQBjlNlngQ8Ur9PlCTRukgi05kQuNK/vaH4n9fLIT7ol1xmOTBJx4viXGBz5zBOPOCKURCFIYQqbv6K6RUxMYIJ3TYheL9P/kvOd1ue2/JO95rtozqOBlpHG2gLeWgftdEJ6qAuougOPaEX9GrdW8/Wm/U+bp2w6pk19APWxycnx65E</latexit> <latexit sha1_base64="8/FnER2VbVMPSWj2wlp3l3vNu4=">ACQXicbVBNS8QwE39XOtX1aOX4CIowtKoBdh0YvHFVwtbJeSZlMNpmlJpmIp/Wte/AfevHvxoIhXL2Z3C34+CPN4b4bJvCgTXIPrPloTk1PTM7ONOXt+YXFp2VlZPdprijr0lSkyo+IZoJL1gUOgvmZYiSJBLuIro+H/sUNU5qn8gyKjPUTcil5zCkBI4WOH0RElX4VSnyIg1gRWnpVKastP/TwDg4GKWhT/VBu4yCwv7oDSHGQ5KayWygx0bjCtcplDEXoN2WOwL+S7yaNFGNTug8mGU0T5gEKojWPc/NoF8SBZwKVtlBrlG6DW5ZD1DJUmY7pejBCq8aZQBjlNlngQ8Ur9PlCTRukgi05kQuNK/vaH4n9fLIT7ol1xmOTBJx4viXGBz5zBOPOCKURCFIYQqbv6K6RUxMYIJ3TYheL9P/kvOd1ue2/JO95rtozqOBlpHG2gLeWgftdEJ6qAuougOPaEX9GrdW8/Wm/U+bp2w6pk19APWxycnx65E</latexit> <latexit sha1_base64="8/FnER2VbVMPSWj2wlp3l3vNu4=">ACQXicbVBNS8QwE39XOtX1aOX4CIowtKoBdh0YvHFVwtbJeSZlMNpmlJpmIp/Wte/AfevHvxoIhXL2Z3C34+CPN4b4bJvCgTXIPrPloTk1PTM7ONOXt+YXFp2VlZPdprijr0lSkyo+IZoJL1gUOgvmZYiSJBLuIro+H/sUNU5qn8gyKjPUTcil5zCkBI4WOH0RElX4VSnyIg1gRWnpVKastP/TwDg4GKWhT/VBu4yCwv7oDSHGQ5KayWygx0bjCtcplDEXoN2WOwL+S7yaNFGNTug8mGU0T5gEKojWPc/NoF8SBZwKVtlBrlG6DW5ZD1DJUmY7pejBCq8aZQBjlNlngQ8Ur9PlCTRukgi05kQuNK/vaH4n9fLIT7ol1xmOTBJx4viXGBz5zBOPOCKURCFIYQqbv6K6RUxMYIJ3TYheL9P/kvOd1ue2/JO95rtozqOBlpHG2gLeWgftdEJ6qAuougOPaEX9GrdW8/Wm/U+bp2w6pk19APWxycnx65E</latexit> Law of Large Numbers • If you perform the same experiment a large number times, the average will converge to the expected value • Assumes that errors are “random” and uncorrelated, so will balance out over time X n = 1 ¯ n ( X 1 + · · · + X n ) ¯ X n → µ as n → ∞ https://en.wikipedia.org/wiki/Law_of_large_numbers