NAS-Bench-101 : Towards Reproducible Neural Architecture Search - PowerPoint PPT Presentation

May 20, 2023 •130 likes •217 views

NAS-Bench-101 : Towards Reproducible Neural Architecture Search Chris Ying 1 , Aaron Klein 2 , Esteban Real 1 , Eric Christiansen 1 , Kevin Murphy 1 , Frank Hutter 2 1 Google Brain, 2 University of Freiburg * equal contribution ICML 2019

NAS-Bench-101 : Towards Reproducible Neural Architecture Search Chris Ying *1 , Aaron Klein *2 , Esteban Real 1 , Eric Christiansen 1 , Kevin Murphy 1 , Frank Hutter 2 1 Google Brain, 2 University of Freiburg * equal contribution ICML 2019
Motivation Neural architecture search (NAS) methods are notoriously difficult to reproduce and compare: Different search spaces and training procedures 1. Implicit biases imposed by search space and training, different NAS methods optimized ○ for different setups Cannot separate benefit of NAS from the careful design of the search space and training ○ procedures
Motivation Neural architecture search (NAS) methods are notoriously difficult to reproduce and compare: Different search spaces and training procedures 1. Implicit biases imposed by search space and training, different NAS methods optimized ○ for different setups Cannot separate benefit of NAS from the careful design of the search space and training ○ procedures Compute cost limits number of trials and makes methods inaccessible to 2. most researchers
NAS-Bench-101 General search space of directed ● acyclic graphs for cell-based NAS methods Exhaustively trained & evaluated ● all models on CIFAR-10 to create a queryable dataset ~423K unique cells * 4 epoch budgets * 3 repeats = ~5M total models trained
NAS-Bench-101 Enables: Studying the landscape of a neural architecture search space as a discrete 1) optimization space Efficient benchmarking of NAS methods by separating the process of 2) searching for models (cheap) from evaluating the models (expensive)
Aggregate Analysis of Search Space Search space exhibits locality : ● similar architectures often have similar performance Randomly selecting top model is ● extremely unlikely, but many models within short edit-distance away
Benchmarking Querying dataset enables running ● entire NAS experiments in seconds Can investigate the robustness of ● NAS methods across random repeats Results suggest that conclusions ● may generalize to larger spaces
Pacific Ballroom Poster #12 Dataset and code available at: https://github.com/google-research/nasbench

Recommend

NAS-Bench-1Shot1: Benchmarking and Dissecting One-Shot Neural Architecture Search

NAS-Bench-1Shot1: Benchmarking and Dissecting One-Shot Neural Architecture Search Albert-Ludwigs-Universitt Freiburg DeToL 07.11.2019 Julien Siems, Arbr Zela and Frank Hutter Under review as a conference paper at ICLR 2020 Motivation

1.43k views • 24 slides

Neural Architecture Search Yu Cao What is Neural Architecture Search (NAS) Selecting the optimal

Neural Architecture Search Yu Cao What is Neural Architecture Search (NAS) Selecting the optimal network architecture automatically via machine instead of design it manually. It is an important aspect of AutoML. NAS search space 1.

3.43k views • 37 slides

A Neural Network Architecture for Detec2ng Gramma2cal Errors in SMT A Neural Network Architecture

A Neural Network Architecture for Detec2ng Gramma2cal Errors in SMT A Neural Network Architecture for Detec2ng Gramma2cal Errors in SMT Morpho-Syntac-c features outperform word embeddings on this task Syntac-c n-grams improve the

1.18k views • 38 slides

A STEP TOWARD QUANTIFYING INDEPENDENTLY REPRODUCIBLE MACHINE LEARNING RESEARCH Edward Raff

A STEP TOWARD QUANTIFYING INDEPENDENTLY REPRODUCIBLE MACHINE LEARNING RESEARCH Edward Raff 12/2019, NEURAL INFORMATION PROCESSING SYSTEMS REPRODUCIBLE MACHINE LEARNING The machine learning community is rightfully putting a greater emphasis

702 views • 7 slides

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document

1.48k views • 18 slides

Reproducible Research with Stata using version control, GitHub, and MarkDoc E. F. Haghish Nov.

Reproducible Research with Stata Reproducible Research with Stata using version control, GitHub, and MarkDoc E. F. Haghish Nov. 17th, 2016 Reproducible Research with Stata Reproducible Analysis Overview Definition Figure 1: Reproducible

1.25k views • 42 slides

Mayfly Reproducible Research in Minutes Reproducible Research is

Mayfly Reproducible Research in Minutes Reproducible Research is the new paradigm Step 1: Configure Environment And more graphics should be interac@ve

329 views • 5 slides

Neural Architecture Search in a Proxy Validation Loss Landscape Yanxi Li 1 , Minjing Dong 1 ,

Neural Architecture Search in a Proxy Validation Loss Landscape Yanxi Li 1 , Minjing Dong 1 , Yunhe Wang 2 , Chang Xu 1 1 University of Sydney 2 Huawei Noah's Ark Lab. Aim Improve the efficiency of Neural Architecture Search (NAS) via learning a

580 views • 26 slides

Gaps between the bedside and the bench: Perspectives from the bench University of Oregon

Gaps between the bedside and the bench: Perspectives from the bench University of Oregon University of Tbingen Bernardo Blanco-Snchez Antje Bernd Aurlie Clment Eberhart Zrenner Javier Fierro John

477 views • 32 slides

Bench Decorum Bench Decorum Definition Appropriate conduct and sportsmanlike behaviour on the

Bench Decorum Bench Decorum Definition Appropriate conduct and sportsmanlike behaviour on the floor and bench Bench Decorum Coach Communication & Conduct A coach shall address the officials in a business-like manner A coach

199 views • 4 slides

Meta-Learning Neural Bloom Filters Jack Rae Sergey Bartunov Tim Lillicrap Architecture

Meta-Learning Neural Bloom Filters Jack Rae Sergey Bartunov Tim Lillicrap Architecture Interested in neural networks with compressive, distributed memories. Problem Trend in the use of neural networks to replace classical data-structures.

784 views • 9 slides

Reproducible, Reusable, and Robust Reinforcement Learning Joelle Pineau Facebook AI Research,

Reproducible, Reusable, and Robust Reinforcement Learning Joelle Pineau Facebook AI Research, Montreal School of Computer Science, McGill University Neural Information Processing Systems (NeurIPS) December 5, 2018 Reproducibility refers to

979 views • 58 slides

Are Labels Necessary for Neural Architecture Search? Chenxi Liu , Piotr Dollr, Kaiming He, Ross

Are Labels Necessary for Neural Architecture Search? Chenxi Liu , Piotr Dollr, Kaiming He, Ross Girshick, Alan Yuille, Saining Xie Spotlight @ECCV 2020 (Short Summary) Designing neural architectures ( , 6) (

185 views • 6 slides

Benches of NCLT and NCLAT (2) State of Uttarakhand. 4. National Company Law Tribunal, Bengaluru

Title of the Bench Territorial Jurisdiction of the Bench 1. New Delhi (1) State of Haryana. a) National Company Law Tribunal, Principal Bench. (2) State of Rajasthan. b) National Company Law Tribunal, New Delhi Bench. (3) Union territory of

546 views • 23 slides

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks These are the commonest type of neural

534 views • 32 slides

Reproducible Quantum Chemistry in JupyterLab Chris Harris (Kitware) @openchem Overview

Reproducible Quantum Chemistry in JupyterLab Chris Harris (Kitware) @openchem Overview Scientific Use Case Why Jupyter? Approach Demo Architecture - Backend - Frontend Deployment Future Project and Team

654 views • 23 slides

Bench Grinder BEST PRACTICES 2018 Uses of Bench Grinder Sharpening / Maintaining metal tools

Bench Grinder BEST PRACTICES 2018 Uses of Bench Grinder Sharpening / Maintaining metal tools Specifically air hammer bits Grinding Wheel On/Off Switch Best Practice - Tooling Inspection Set-up Operation Storage Inspect Before

287 views • 12 slides

Passive Treatment of Mining Influenced Water: From Bench Scale to O & M From Bench Scale to O

Passive Treatment of Mining Influenced Water: From Bench Scale to O & M From Bench Scale to O & M BIOCHEMICAL REACTOR CONSTRUCTION, MINE POOL CHEMISTRY CHANGES, & O & M , GOLINSKY MINE, CALIFORNIA Jim Gusek, Sovereign

740 views • 59 slides

HIIT BENCH T H E G Y M I N A B E N C H 1 2 WHERE FUNCTION MEETS FITNESS in less than a

PRESENTS HIIT BENCH T H E G Y M I N A B E N C H 1 2 WHERE FUNCTION MEETS FITNESS in less than a square metre HIIT BENCH organizes and makes functional training easy. Space saving and compact, it provides storage for all your needs both for

558 views • 9 slides

Reproducible research in practice ifgi Institute for Geoinformatics University of Mnster

Reproducible research in practice ifgi Institute for Geoinformatics University of Mnster Edzer Pebesma Reproducible Research Workshop, UZH, Sep 13-14, 2016 1 / 23 Overview 1. Who am I? 2. What is reproducible research? What is

465 views • 25 slides

Efficient Neural Architecture Search via Parameter Sharing ICML 2018 Authors: Hieu Pham, Melody

Efficient Neural Architecture Search via Parameter Sharing ICML 2018 Authors: Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, Jeff Dean Presented By: Bhavya Goyal, Nils Palumbo, Abrar Majeedi Why Network Architecture Design? Model

766 views • 60 slides

Feedforward neural nets CSE 250B Outline 1 Architecture 2 Expressivity 3 Learning The

Feedforward neural nets CSE 250B Outline 1 Architecture 2 Expressivity 3 Learning The architecture y h ( ` ) . . . h (2) h (1) x The value at a hidden unit h z 1 z 2 z m How is h computed from z 1 , . . . , z m ? The value at a

821 views • 54 slides

Progressive Neural Architecture Search Chenxi Liu , Barret Zoph, Maxim Neumann, Jonathon Shlens,

Progressive Neural Architecture Search Chenxi Liu , Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy 09/10/2018 @ECCV 1 Outline Introduction and Architecture Progressive

562 views • 55 slides

Bit Fu Bi Fusion on Bit-Level Dynamically Composable Architecture for Deep Neural Networks

Bit Fu Bi Fusion on Bit-Level Dynamically Composable Architecture for Deep Neural Networks Hardik Sharma Georgia Institute of Technology Jongse Park Arm, Inc. Naveen Suda Liangzhen Lai University of California, San Diego

275 views • 24 slides