Deep Learning: Trends and Challenges DAVIDE BACCIU DIPARTIMENTO DI INFORMATICA UNIVERSITÀ DI PISA
Trends BioMedical Challenges IoT Applications Structured* HPC Knowledge Transfer*
AI Deep Prediction Learning Trainable predictor Hard-coded expert reasoning Learned Expert- Learned features designed feature features hierarchy ML Input
Neural Net Machinery in 1 Slide Learning Ground-truth predictions in training data can be used to adapt the Network synaptic weights of all neurons w 1 input Network prediction … w n Neuron Activation Weighted input f summation + Synaptic weights thresholding function Free parameters of the model (often differentiable and nonlinear)
Trends BioMedical Challenges IoT Applications Structured* HPC Knowledge Transfer*
Structured Data Compound information whose atomic components provide informative content when considered in their surrounding context Sequences Trees Graphs
ML@UNIPI (since 1993) Learning with Structured Data Learning from a population where each individual is a fixed-size vector Structured dataset Vectorial dataset Learning from a population where each individual is a variable size graph (vectorial information as node labels)
Recursive Neural Networks Prediction A neural model for the that can unfold on whole the structure of the structure sample a a b Neural b c c encoding of the nodes d c c d
From Image to Graph Convolutions Learn hidden neurons responsive to visual patterns Image Learning hidden neurons responsive to structural patterns • Node labels … … • Connectivity Graph
Kipf & Welling, Semi-Supervised Classification with Graph Convolutional Networks, ICLR 2017 Community Detection Community detection in social graphs
Trends BioMedical Challenges IoT Applications Structured* HPC Knowledge Transfer*
Source: query on Scopus abstracts on Sept. 2017 The Rise of Deep Learning... 3500 …and biomedical 3000 applications slowly 2500 starting to catch 2000 up 1500 1000 some 50 review 500 papers 0 2005 2007 2009 2011 2013 2015 2017 Deep Learning Deep Learning + Life Sciences
Alipanahi, Babak, et al. "Predicting the sequence specificities of DNA-and RNA- binding proteins by deep learning." Nature biotechnology 33.8 (2015): 831-838. http://tools.genes.toronto.edu/deepbind/ CNN for DNA/RNA Sequences (DeepBind) … T A G A C A T C T 1D convolutions on the … input sequence train to 927 CNN models predicting a binding score for respond to task-specific transcription factors and RNA-binding motifs proteins
ML@UNIPI CNN for DNA Sequences Digits Deep learning visual training system designed for machine vision applications GPU accelerated CNN training
ML@UNIPI Triplet vocabulary cag gcc taa cac atg caa gtc gaa cgg taa nag att gat agc ttg cta tca atg ctg acg anc ggc Triplet ID gga cgg gtg agt aat gcc tgg gaa tat acc ctg aaa 1 atg tgg gg gat aac tat tgg aaa cga tag cta aac 2 ata … … … taa 59 … … ttt 64 Use ID as graylevel of the corresponding pixel Convolutions have to be 1D 500K DNA sequences from 18 bacteria even if it is an image! species transformed into images
ML@UNIPI Testing Deep Learning Acceleration CNN Training Time 5000 Dell PowerEdge C4130 4500 3d.3h • 4xP100 16Gb PCIE 4000 • 2 Xeon E5-2690v4 3500 • 256GB RAM 3000 2500 2000 Dell PowerEdge C4130 • 4xM40 12Gb 1500 • 2 Xeon E5-2670v3 1000 500 • 128GB RAM 3h.30m 0 Dna100K Dna500K P100 M40
ML@UNIPI Exploiting Clonal Diversity for Personalized Cancer Treatment Predicting the effect of chemioterapic drugs from patients clonal trees Allele primary tumor frequency information Non-Isomorph tree transduction Metastasis 1 Metastasis 2
Trends BioMedical Challenges IoT Applications Structured* HPC Knowledge Transfer*
Internet of Streams Enormous amounts of heterogeneous sequential data + Adding actuation calls for increased adaptivity
Cloud Intelligence Deep learning for sequences (LSTM,GRU,…) Do we really need: • To transfer all our data to the could for analytics • Complex DL models for all our tasks
ML@UNIPI Edge Intelligence • Learning models that scale from tiny (8KB) to large (or deep) • Reservoir computing and randomized networks
ML@UNIPI Distributed Intelligence as an IoT Service Embedded learning, management and over-the-air deployment Multiple learning primitives within the same neural machinery Automating • Supervised, anomaly detection medical & feature selection screening (from tuning to Identifying 30mins to anomalies/novelties normality 10secs)
Trends BioMedical Challenges IoT Applications Structured* HPC Knowledge Transfer*
Are We Really Building Adaptive Applications? Probably yes.. if we consider agents and reinforcement learning Predictor created at development time Otherwise we use pre- programmed adaptation
The Adaptivity Challenge Learning as a Standardization primitive & Protocols Learning Automation
Trends BioMedical Challenges IoT Applications Structured* HPC Knowledge Transfer*
Different Forms of Parallelism? Structures are irregular and Current deep learning require synchronization accelerations based on Branch&bound? stream/data parallelism
Trends BioMedical Challenges IoT Applications Structured* HPC Knowledge Transfer*
Sharing Learned Knowledge A scalable approach for IoT applications Impacting also biomedical applications Reusing trained models Hidden neural representation as a unifying language?
Deep Learning… • …or learning representations from data • Effective for the machine to perform predictions • Not necessarily helping humans understand the underlying biological process • Structured information as a means to supply relational knowledge Upcoming life-science and IoT applications Success will depend on how key challenges will be addressed
Recommend
More recommend
