Predicting Deep Zero-Shot Convolutional Neural Networks using - PowerPoint PPT Presentation

Sep 02, 2023 •36 likes •156 views

Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions Jimmy Lei Ba, Kevin Swersky, Sanja Fidler, Ruslan Salakhutdinov ICCV 2015 Presenter: Fartash Faghri Zero-shot Learning Classify images of an unseen class

Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions Jimmy Lei Ba, Kevin Swersky, Sanja Fidler, Ruslan Salakhutdinov ICCV 2015 Presenter: Fartash Faghri
Zero-shot Learning • Classify images of an unseen class given semantically or visually similar classes at training time. • Shared knowledge between Antol et al. [1] classes can be given in various forms, such as attributes or class descriptions.
Contributions • The main contribution is the convolutional classifier. The rest of the contributions are shared with [2]. • Predicts visual classes using text corpus, in particular, the encyclopedia corpus. This overcomes the difficulty of hand-crafted attributes. • The key difference with the most related work is that image and text features are transformed into a joint embedding space.
Classifier • Image feature vectors: • Text feature vectors: • A linear classifier: • Image transformation: • Text transformation:
Convolutional Classifier • Text can describe attributes (low) or objects (high). • Classifier on fully connected features: • Classifier on convolutional features: • Joint classifier: • is a global pooling function.
Learning • Binary Cross Entropy: • Hinge Loss: • Euclidean Distance between and
Loss Comparison Produced by WolframAlpha
Experiments • DA: the model is similar to the hinge loss form • DA+GP: in that model multiple text descriptions can be given for a class, GP part gives p(c|t), a prior. • fc baseline feat.: features from [2], HOG, GIST, etc • ROC: true positive rate vs false positive rate
Results
Results (cont.)
References • [1] Antol, Stanislaw, C. Lawrence Zitnick, and Devi Parikh. "Zero-shot learning via visual abstraction." European Conference on Computer Vision. Springer International Publishing, 2014. • [2] Elhoseiny, Mohamed, Babak Saleh, and Ahmed Elgammal. "Write a classifier: Zero-shot learning using purely textual descriptions." Proceedings of the IEEE International Conference on Computer Vision. 2013.

Recommend

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional Neural Networks

648 views • 26 slides

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan Naveed Won Mo (Andy) Jung Lassonde School of Engineering, York University Canada Objective Overview of Convolutional Neural Network

567 views • 17 slides

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira Quiz coming up Next Monday (2/5) 30 minutes Topics: Optimization Basic neural networks Neural Network

619 views • 41 slides

MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks

Run 3493 Event 41075, Oct. 23rd, 2015 CONVOLUTIONAL NEURAL NETWORKS IN MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks (CNNs) are a type of deep, feed-forward neural networks that have been

831 views • 43 slides

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides

Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs

Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs Convolutions Pooling, Padding and Stride Convolutional Neural Networks (LeNet) Deep ConvNets (AlexNet) Networks using Blocks (VGG)

1.91k views • 89 slides

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides

Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem

Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem November 2016 Hacettepe University Administrative Assignment 3 is out! It is due November 30, 2016 You will implement a 2-layer Neural

1.28k views • 114 slides

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides

Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12

Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12 Convolutional Networks Scale up neural networks to process very large images / video sequences Sparse connections Parameter sharing

552 views • 27 slides

Evaluations of Deep Convolutional Neural Networks for Automatic Identification of Malaria Infected

Evaluations of Deep Convolutional Neural Networks for Automatic Identification of Malaria Infected Cells Yuhang Dong, Zhuocheng Jiang, Hongda Shen, W. David Pan Dept. of Electrical & Computer Engineering, Univ. of Alabama in Huntsville (UAH)

370 views • 14 slides

Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks

Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks Based on Decompressed Images Yuhang Dong, Zhuocheng Jiang, Hongda Shen, W. David Pan Dept. of Electrical & Computer Engineering, Univ. of Alabama

660 views • 18 slides

RECOGNIZING PORNOGRAPHIC IMAGES USING DEEP CONVOLUTIONAL NEURAL NETWORKS Olarik Surinta and

RECOGNIZING PORNOGRAPHIC IMAGES USING DEEP CONVOLUTIONAL NEURAL NETWORKS Olarik Surinta and Thananchai Khamket Faculty of Informatics Mahasarakham University th Present at the 4 International Conference on Digital Arts, Media and

618 views • 22 slides

High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks

High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks Krzysztof J. Geras Joint work with Kyunghyun Cho , Linda Moy , Gene Kim , Stacey Wolfson and Artie Shen . GTC 2017 W HERE DEEP LEARNING IS USEFUL amount

772 views • 42 slides

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides

Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine

Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Deep Convolutional Neural Nets 1 / 25 Outline 1 Why Neural Networks? 2 Circuits 3 Neurons, Layers, and Networks 4 Correlation and Convolution 5

287 views • 25 slides

Predicting Temporal Sets with Deep Neural Networks Le Yu, Leilei Sun*, Bowen Du, Chuanren Liu, Hui

Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 20), August 23 27, 2020, Virtual Event, CA, USA. Predicting Temporal Sets with Deep Neural Networks Le Yu, Leilei Sun*, Bowen Du, Chuanren Liu, Hui

440 views • 18 slides

Lecture 18: Concluding Convolutional Neural Networks, Graphical Models as Foundation for Recurrent

Lecture 18: Concluding Convolutional Neural Networks, Graphical Models as Foundation for Recurrent Neural Networks and Bayesian Networks Reference: We will be referring to sections etc of Deep Learning by Yoshua Bengio, Ian J. Goodfellow

974 views • 54 slides

ImageNet Classification with Deep Convolutional Neural Networks Krizhevsky et. all Outline

ImageNet Classification with Deep Convolutional Neural Networks Krizhevsky et. all Outline Introduction DataSet Architecture of the Network Reducing overfitting Learning Results Discussion Introduction

1.3k views • 31 slides

Augmentation Introduction ImageNet Classification with Deep Convolutional Neural Networks,

Day 2 Lecture 2 Augmentation Introduction ImageNet Classification with Deep Convolutional Neural Networks, Krizhevsky A., 2012 ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) 1.2 million training images , 50,000 validation images, and

537 views • 14 slides

Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic

Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences Seonguk Seo* 1 Paul Hongsuck Seo* 1,2 Bohyung Han 1 Overconfidence Issues Overconfidence to unseen examples 99.9+% sure for the

759 views • 31 slides

Convolutional Neural Networks (Application in Object and Scene

Convolutional Neural Networks (Application in Object and Scene Recognition) Harsh Agrawal (Sept 8 th , 2015) ECE: 6504, Deep Learning For Perception Contents Y. LeCun, L.

732 views • 54 slides