CPL 2016, week 4 Inter-thread communication Oleg Batrashev Institute of Computer Science, Tartu, Estonia March 2, 2016
Overview Studies so far: 1. Inter-thread visibility: JMM 2. Inter-thread synchronization: locks and monitors 3. Thread management: executors, tasks, cancelation Today: ◮ Inter-thread communication Next week: ◮ Actor model
Inter-thread communication 75/101 Confinements - Outline Inter-thread communication Confinements Queues and messages Producer/consumer pattern Pros and cons Advanced patterns Back pressure Non-FIFO queues Channels vs streams Non-determinism Declarative concurrency
Inter-thread communication 76/101 Confinements - Confinement idea ◮ need to syncrhonize data accesses from multiple threads ◮ ensure some other way, that the data is accessed from single thread at a time ◮ use language semantics, programming pattern/protocol, design rules... ◮ if we can do that, no locking is needed! ◮ confine (bound) data to 1. single method/function, sequence of methods 2. single thread 3. single object 4. single object/thread from a group of objects dynamically ◮ i.e. data is changing “ownership” over time
Inter-thread communication 77/101 Confinements - Method confinement Within single method only executing thread can access: ◮ local variables of primitive type, ◮ new objects, which reference do not escape . An object reference escapes if: 1. it is passed as an argument to another method/constructor 2. it is returned from method invocation 3. stored to global variable (static or object field, accessible from another thread) 4. if another object escapes that can be traversed to this object
Inter-thread communication 78/101 Confinements - Hand-off and other protocols Alternative to using exclusively inside a method: ◮ pass “ownership” (read/write) of the object ( hand-off ) ◮ tail call hand-off – ensure the object is never used locally after it escapes Location loc = new Location (10 ,20); // use loc here otherMethod (loc ); // do not use loc here after } ◮ caller copies, receiver copies the object ◮ use scalar (primitive) arguments otherMethod (loc.lon , loc.lat ); ◮ trust – beleive a callee does not publish the object, so it can be used without locking after the callee returns ◮ similar problem exists for a single thread – when passing Location to a method, can we be sure it is not changed?
Inter-thread communication 79/101 Confinements - Thread confinement Make sure data is only visible from single thread: ◮ private field in Thread subclass ◮ make sure it is accessed only from the run() method or its private helper subroutines ◮ similarly, can be private field in Runnable ◮ dedicated objects are also ok if references are not escaped ◮ ThreadLocal in Java ◮ every thread has its own version of the reference stored in the ThreadLocal object ◮ often used for sessions and other contexts Thread confinement is essential part of Actor model (next lecture) and CSP (Concurrent Sequential Processes [read Google Go language]) ◮ how it can be useful if accessed from single thread? see queues and messages
Inter-thread communication 80/101 Confinements - Object confinement An object is accessed from single thread if ◮ is only visible within another (encapsulation in OOP), ◮ is only accessed from synchronized methods ◮ does not escape In other words, synchronization is delegated to the containing object ◮ remember, proper usage may require client synchronization list data is escaped through its iterators
Inter-thread communication 81/101 Confinements - Confinement within groups Consider example: 1. data is confined to an object ◮ object responsibility is to synchronize access to it 2. take() returns the data but deletes reference in the object ◮ remember collection example with 2 locks (Refining locks) 3. now data is confined to the thread ◮ only the thread can access it through its local variable 4. put data to some other object and do not use anymore in the thread 5. now data is confined to another object Variants and names: ◮ known as tokens, capabilities, resources ◮ ring communication in hardware or software to pass the token Requires careful code design.
Inter-thread communication 82/101 Queues and messages - Outline Inter-thread communication Confinements Queues and messages Producer/consumer pattern Pros and cons Advanced patterns Back pressure Non-FIFO queues Channels vs streams Non-determinism Declarative concurrency
Inter-thread communication 83/101 Queues and messages - Total thread confinement In ideal situation, data is completely confined to (two) threads if: ◮ they run on different machines, ◮ they communicate by sending messages over the network. In JVM threads may almost acheive this if: ◮ almost all data is local to threads, ◮ except queues that contain messages and reflect network communication; ◮ data is cloned (duplicated) if sent using message to another thread. No synchronization is needed, except queue access! ◮ cloning data is inefficient – consider immutable objects.
Inter-thread communication 84/101 Queues and messages - Immutable messages 1. mutable Location objects passed inside Traveler to Server.push() public class Location { public double lon , lat; ◮ end up being encoded to the network socket, ◮ modified in place by the simulation, ◮ if done from two different threads lon and lat may have values from different simulation runs; ◮ should copy the location once Server gets its own thread. 2. immutable Point objects has no such problem and may be passed to different threads without worry public class Point { public final int x, y; Messages are passed from one thread to another – it may be good idea to make them immutable.
Inter-thread communication 85/101 Queues and messages - Java Queue interfaces From Java docs of the Queue and BlockingQueue interfaces Throws Returns Blocks Times out exception special value Insert add(e) offer(e) put(e) offer(e,time,unit) Remove remove() poll() take() poll(time,unit) Examine element() peek() - - ◮ first line 4 methods to insert elements into queue ◮ second line 4 methods to remove elements from queue ◮ third line 2 methods to examine the queue without removing elements ◮ first 2 columns differ only if queue is empty or full ◮ last 2 columns are only available for BlockingQueue
Inter-thread communication 86/101 Queues and messages - Producer/consumer pattern Producer and consumer The simplest scenario: ◮ one or several threads put elements to the queue – producers ◮ one or several threads take elements from the queue and process them – consumers Queue implementation in Java are ◮ ConcurrentLinkedQueue -- unbounded; non-blocking ◮ non-blocking write – obvious because the queue is unlimited, ◮ non-blocking read – you cannot wait for an element to appear in the queue; ◮ LinkedBlockingQueue – unbounded; blocking read ◮ ArrayBlockingQueue – bounded; blocking
Inter-thread communication 87/101 Queues and messages - Producer/consumer pattern Transducers One thread may be both consumer and producer – transducer (transformer) ◮ take element from one queue ◮ transform it, e.g. ◮ throw away not needed elements ( filter ) ◮ extract useful information from each element ( map ) ◮ aggregate messages and produce compound value ( reduce ) ◮ put result into another queue while (isRunning) { e = queue1.take (); if (! isAcceptable (e)) continue; aggrValue += e.value; if (aggrValue > 100) queue2.put(new Message(aggrValue )); }
Inter-thread communication 88/101 Queues and messages - Pros and cons Deadlock free execution Consider application with producers/consumers/transducers only: Producer TransducerA queue1 TransducerB queue2 ◮ messages and their referenced objects are either immutable, copied, or ensured not to be used when sent out ◮ i.e. no synchronization on message contents are needed ◮ threads need to synchronize only on queue access, ◮ no thread ever takes two locks at a time. Hence this system is deadlock free! It may stall only because of lack of generated messages. ◮ This is essentially confinement within groups pattern. ◮ Actual idea – limit thread execution boundaries. ◮ no thread walks from network layer to GUI layer!
Inter-thread communication 89/101 Queues and messages - Pros and cons Problem with queues Not everything is so easy though... ◮ Unbounded queues may lead to the exhaustion of (memory) resources. Solutions: ◮ stop producers through back-pressure : its simplest form is bounded queue; ◮ do not let the queue grow: throw away or aggregate messages. ◮ Race conditions are still there and even more, for example: 1. Producer generates messages data and stop ; 2. TransducerA reads from queue1 , puts result to queue2 ; 3. TransducerB reads and puts transformed message to queue1 ; 4. TransducerA processes this transformed message. ◮ It may happen that stop message is put by the producer between 1 and 3, in which case the transformed message is put after the stop is processed by TransducerA. ◮ It may not happen with single thread “penetrating all layers”.
Inter-thread communication 90/101 Advanced patterns - Outline Inter-thread communication Confinements Queues and messages Producer/consumer pattern Pros and cons Advanced patterns Back pressure Non-FIFO queues Channels vs streams Non-determinism Declarative concurrency
Recommend
More recommend
