3/18/2020 Tensorflow 1.x API Graphs and Sessions Keras provides a high-level API allowing to easily describe Tensorflow describes the computation to be performed by complex DNNs by hiding some low-level details means of a data-flow graph output Tensorflow also provides a native API (not based on Keras) Relu that allows to describe DNNs in a fine-grained manner, Tensorflow variables used for training working with individual variables User provided inputs p p Add It may be useful for some specific scenarios, e.g., Computational layers reinforcement learning ( more about it in the next lecture! ) Today, we will focus on the Python APIs MatMul b w x 3 4 TF Workflow The simplest graph In Tensorflow, the workflow is separated into two phases Building the simplest graph # import libraries import tensorflow as tf # tensorflow 1 Graph definition and connection # instantiate a sum s = tf.add(10, 20) tf dd(10 20) 2 Graph execution by means of a TF Session Add 10 20 1
3/18/2020 The simplest graph Running a graph Running the simplest graph The shorter way # import libraries # import libraries import tensorflow as tf # tensorflow import tensorflow as tf # tensorflow # instantiate a sum # load the dataset s = tf add(10 20) s tf.add(10, 20) s tf.add(10, 20) s = tf add(10 20) # create a session object # create a session object sess = tf.Session() with tf.Session() as sess: # print the result # print the result print sess.run(s) # the output will be 30 print sess.run(s) # the output will be 30 # close the session sess.close() A more complex graph Constants Building and running a more complex graph Tensorflow allows to create constants as follows: # import libraries # import libraries import tensorflow as tf import tensorflow as tf Pow Add a=1, b=2, c=3, d=4 # constants definition c1 = tf.constant(1) Mul Add m = tf.multiply(a, b) c2 = tf.constant(2) s = tf.add(c, d) c1 c2 a b c d s = tf.add(a,b) pow = tf.pow(m, s) with tf.Session() as sess: with tf.Session() as sess: print sess.run(s) print sess.run(pow) Constants Constants Tensorflow allows to create constants as follows: Constants and operations can be associated with a name # import libraries # import libraries import tensorflow as tf import tensorflow as tf Add # constants definition # constants definition c1 = tf.constant(1) c1 = tf.constant(1, name =“c1”) Useful for visualize the graph with TensorBoard: more about it later! c2 = tf.constant(2) c2 = tf.constant(2, name =“c2”) c1 c2 s = tf.add(a,b) s = tf.add(a, b, name =“sum”) with tf.Session() as sess: with tf.Session() as sess: print sess.run(s) print sess.run(s) 2
3/18/2020 Variables Variables In Tensorflow, a variable is a special operation that returns Variables need to be explicitly initialized : there are different ways a handle to a persistent mutable tensor that survives Method 1: Initialize all variables at once across executions of a graph # import libraries # import libraries import tensorflow as tf import tensorflow as tf p # variables definition # variables definition v1 = tf.Variable(2, name = “scalar”) v1 = tf.Variable(2, name = “scalar”) v2 = tf.Variable([2, 3], name = “array”) v2 = tf.Variable([2, 3], name = “array”) v3 = tf.Variable([[2, 3], [1, 0]], name = “matrix”) v3 = tf.Variable([[2, 3], [1, 0]], name = “matrix”) v4 = tf.Variable(tf.zeros([784, 10]), name = “tensor”) init = tf.global_variables_initializer() with tf.Session() as sess: print sess.run(init) Variables Variables Variables need to be explicitly initialized : there are different ways Variables need to be explicitly initialized : there are different ways Method 2: Initialize only some specific variables Method 3: Initialize a single variable # import libraries # import libraries import tensorflow as tf import tensorflow as tf # variables definition # variables definition v1 = tf.Variable(2, name = “scalar”) v1 = tf.Variable(2, name = “scalar”) v2 = tf.Variable([2, 3], name = “array”) v2 = tf.Variable([2, 3], name = “array”) v3 = tf.Variable([[2, 3], [1, 0]], name = “matrix”) v3 = tf.Variable([[2, 3], [1, 0]], name = “matrix”) init2 = tf.variables_initializer([v1, v2]) with tf.Session() as sess: print sess.run(v3.initializer) with tf.Session() as sess: print sess.run(init2) What are Variables for? Variables Since variables can be modified at run-time, they are It may be useful to initialize variables with random values usually used for the weights and biases of the DNNs weights = tf.Variable(tf. random_normal ([400, 200], stddev=0.35), weights = tf.Variable(tf.zeros([400,200]),name="weights") name="weights") biases = tf.Variable(tf.zeros([200]), name= biases ) biases = tf Variable(tf zeros([200]) name="biases") bi biases = tf.Variable(tf.zeros([200]), name="biases") tf V i bl (tf ([200]) "bi ") init = tf.global_variables_initializer() init = tf.global_variables_initializer() with tf.Session() as sess: with tf.Session() as sess: sess.run(init) sess.run(init) 3
3/18/2020 Variables Variables Other common random initializations Other common random initializations tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, # random initialization with Xavier filler seed=None, name=None) V1 = tf.get_variable("V1", shape=[784, 256], initializer = tf.contrib.layers.xavier_initializer()) tf.random_uniform(shape, minval=0, maxval=None, dtype tf.float32, seed None, name None) dtype=tf.float32, seed=None, name=None) with tf.Session() as sess: # print the result tf.random_shuffle(value, seed=None, name=None) sess.run(V1.initializer) x = V1.eval() tf.random_crop(value, size, seed=None, name=None) print(x) tf.multinomial(logits, num_samples, seed=None, name=None) get_variable return the variable with name "V1“ if tf.random_gamma(shape, alpha, beta=None, dtype=tf.float32, seed=None, name=None) already defined, otherwise creates it Variables Assign a Variable Copy a value of a variable into another variable The wrong way # variables definition weights = tf.Variable(tf.random_normal([784, 200], v1 = tf.Variable(60, name = “scalar”) stddev=0.35), name="weights") v1.assign(120) weights2 = tf.Variable(weights.initialized_value(), i ht 2 tf V i bl ( i ht i iti li d l () with tf.Session() as sess: name=“weights_copy”) sess.run(v1.initializer) print v1.eval() What does it print? It prints 60 . Why? It prints 60 because assign is a Tensorflow operation that requires to be executed with sess.run() Assign a Variable Assign a Variable The correct way More complex assignments v1 = tf.Variable(100, name = “scalar”) # variables definition assign_op = v1.assign(4 * v1) v1 = tf.Variable(60, name = “scalar”) assign_op = v1.assign(120) with tf.Session() as sess: sess.run(v1.initializer) ( ) with tf.Session() as sess: sess.run( assign_op ) sess.run( assign_op ) print v1.eval() print v1.eval() sess.run( assign_op ) print v1.eval() What does it print? It prints 120 . What does it print? 400 , and then 1600 . Note that in this case we don’t need to initialize V1, the Note that in this case we need to initialize V1, because assignment is anyway performed by assign_op the assignment relies on the initial value of V1. 4
3/18/2020 Save a session Restore a session # variables definition # variables definition v1 = tf.Variable(2, name = “scalar”) v1 = tf.Variable(2, name = “scalar”) v2 = tf.Variable([[2, 3], [1, 0]], name = “matrix”) v2 = tf.Variable([[2, 3], [1, 0]], name = “matrix”) saver = tf.train.Saver() saver = tf.train.Saver() with tf.Session() as sess: with tf.Session() as sess: sess.run(init_op) # restore the variables to disk .. saver.restore(sess, "/tmp/model.ckpt") # save the variables to disk … saver.save(sess, "/tmp/model.ckpt") print("Model saved in file: /tmp/model.ckpt") More at: https://chromium.googlesource.com/external/github.com/tensorflow/tensorflow/+/r0.7/tensorflow/g3doc/how_tos/variables/index.md Interactive Session Placeholders An interactive session allows to set the current session as In the previous slides, we saw how to model constants, the default one variables (e.g., for weights and biases) As What is missing? a consequence, it is possible to avoid the Recall “with tf.Session() as sess:” command We need a way to define y sess = tf. InteractiveSession () the input variables before x = tf.constant(600.0) knowing the actual values y = tf.constant(2000.0) used during execution #This is a short way to define tf.add(a,b) z = a + b print(z. eval ()) Placeholders sess.close() Placeholders Data Types Constants, variables and placeholders are associated with a x = tf.placeholder(tf.float32, shape=[3], name=“x") data type y = tf.constant(tf.random_normal([3], stddev=0.35)), Tensorflow data types are similar (and compatible) with name=“y") those provided by the numpy library of Python c tf.add(x, y) c = tf add(x y) Some examples: S l Data Type Description with tf.Session() as sess: tf.float32 32 bits floating point print sess.run(c, {x: [1, 2, 3]} ) tf.float64 64 bits floating points tf.int32 32 bits integer tf.int64 64 bits integer tf.uint16 16 bits unsigned integer This is to set values to the placeholder tf.string Tensor of byte arrays before running the session tf.bool boolean 5
Recommend
More recommend
