CPL 2016, week 3 Thread management: execution and shutdown Oleg - PowerPoint PPT Presentation

CPL 2016, week 3 Thread management: execution and shutdown Oleg Batrashev Institute of Computer Science, Tartu, Estonia February 22, 2016

Outline Thread management Thread pros and cons Executor framework Cancellation and shutdown Thread usage anti-patterns

Sequential web server Example from [3] section 6.1.1 class SingleThreadWebServer { public static void main(String [] args) throws IOException { ServerSocket socket = new ServerSocket (80); while (true) { Socket connection = socket.accept (); handleRequest (connection ); } } } ◮ only one client is served at a time ◮ other clients must wait ◮ possible bad CPU/memory/IO utilization

Win of threads ◮ faster execution ◮ get weather forecast in reasonable time ◮ better throughput ◮ serve more web clients ◮ better responsiveness, for example in GUI ◮ delegate long-running task to other threads ◮ hide IO latency by delegation of blocking operations ◮ better resource utilization ◮ why quad-core if using always one core? ◮ better structured apps (next lecture)

Cost of threads ◮ creating new thread is not free (see lab1) ◮ mode switch (from user to kernel mode) ◮ ask OS for the new thread context and memory ◮ switching threads is not free: ◮ CPUs/cores must be shared between threads – (context switch) ◮ synchronizing threads is not free ◮ inhibit optimizations due to visibility constraints (lecture 1) ◮ contended synchronization – many threads request the same resource and fail ◮ race conditions and deadlocks in your application!

Threading strategies ◮ have only single thread – no wins ◮ new thread for every new task ◮ pay cost of creating many threads ◮ reuse old threads – thread pools ◮ avoid costs of thread creation ◮ no resource exhaustion (too many threads) possible strategies for a server request handling: ◮ thread per client request ◮ thread per connection ◮ N connections to M threads

Thread per task web-server Example from [3] section 6.1.2 class ThreadPerTaskWebServer { public static void main(String [] args) throws IOException { ServerSocket socket = new ServerSocket (80); while (true) { final Socket connection = socket.accept (); Runnable task = new Runnable () { public void run () { handleRequest (connection ); } }; new Thread(task ). start (); } } } ◮ many clients are served ◮ cost of creating threads, danger of resource exhaustion

Outline Thread management Thread pros and cons Executor framework Cancellation and shutdown Thread usage anti-patterns

Executor interface ◮ Java5 introduced new java.util.concurrent framework public interface Executor { void execute(Runnable command ); } ◮ executor with its own strategy implements the interface ◮ Runnable command represents the task to be executed ◮ to execute a command invoke executor.execute(mycommand); ◮ separate tasks (code to run) from execution (how to run) ◮ flexibility to replace the stategy ◮ same socket server with 3 different executors (next slides) ◮ later will see task cancellation and executor shutdown

Executor: fixed size thread pool Example from [3] section 6.2.1 private static final Executor exec = Executors. newFixedThreadPool (NTHREADS ); .. while (true) { final Socket connection = socket.accept (); Runnable task = new Runnable () { public void run () { handleRequest (connection ); } }; exec.execute(task ); } ◮ requests are handled in threads concurrently up to NTHREADS at a time

Executor: thread per task and no threads Thread per task executor: public class ThreadPerTaskExecutor implements Executor { public void execute(Runnable r) { new Thread(r). start (); }; } No thread executor: public class WithinThreadExecutor implements Executor { public void execute(Runnable r) { r.run (); }; }

Standard executors Executors class factory methods: ◮ newFixedThreadPool – executor with fixed number of threads ◮ newCachedThreadPool – creates threads on demand ◮ old threads will be reused if available ◮ destroys thread on 60 seconds of inactivity ◮ newSingleThreadExecutor – one thread executes tasks ◮ processed sequentially in FIFO, LIFO, or priority order ◮ like newFixedThreadPool(1) but not reconfigurable to several threads ◮ newScheduledThreadPool – fixed size thread pool that supports delayed and periodic task execution ◮ similar to Timer but handles unchecked exceptions ◮ DelayedQueue allowes to schedule delayed messages

Result bearing tasks public interface Callable <V> { V call () throws Exception; } public interface Future <V> { boolean cancel(boolean mayInterruptIfRunning ); boolean isCancelled (); boolean isDone (); V get () throws InterruptedException , ExecutionException , CancellationException ; V get(long timeout , TimeUnit unit) throws InterruptedException , ExecutionException , CancellationException , TimeoutException ; } ◮ Callable is the task with a result V ◮ use it instead of Runnable if it returns a result ◮ Future is the interface to control your task execution ◮ provided by Executor but may be tuned if needed

Using Futures Create task, submit, and use the future to get results: Callable <Data > task = new Callable <Data >() { ... } Future <Data > future = executor.submit(task ); // task is executed in separate thread // do something else in a while Data data = future.get () // blocks until data is available ◮ should cancel it if not needed anymore ◮ could use get() with timeout to poll the result ◮ it throws exceptions depending on what happened since Java 6 ExecutorService implementations can override newTaskFor in AbstractExecutorService : ◮ default is just new FutureTask<T>(callable) ◮ own implementation allows to modify cancel procedure, e.g. close sockets or other resources on task cancellation

Completion service Imagine you have several futures that may block: Data d1 = future1.get (); process(d1); Data d2 = future2.get (); process(d2); ◮ it is not defined in which order future1 and future2 are available Executor meets blocking queue ExecutorCompletionService : ◮ finished results are put into the queue as Future s ◮ implementation is quite straightforward ◮ override FutureTask done method protected void done () { completionQueue .add(this ); } ◮ delegate future.get() to this queue

Outline Thread management Thread pros and cons Executor framework Cancellation and shutdown Thread usage anti-patterns

Cancellation reasons A task may be abstained from execution or stopped: ◮ a user requests that ◮ “cancel” button in GUI ◮ user selects another object to request and show ◮ time-limited activity ◮ timeout on server connect or some calculation ◮ errors in execution ◮ may trigger cancellation of related tasks ◮ shutdown – application is being closed

Thread interruption Forcefully stopping a thread is not a good idea: ◮ it may need some finalizing operations ◮ Java Thread.stop() method is deprecated Instead, Thread.interrupt() “asks” thread to stop ◮ thread interrupt flag is set ( interrupted state) ◮ some blocking operations (sleep,wait) throw InterruptedException ◮ interrupt flag is cleared when exception is thrown! ◮ usually must re-set after catch (see next slide why) ◮ there are no other consequences, i.e. thread continue running as if nothing happened! ◮ a thread must check for the interrupt flag regularly ◮ it may/should use its own volatile boolean isRunning ◮ socket.connect , stream.read , synchronized are not interruptible

Example: consuming interrupt state public void run () { try { while (! Thread. currentThread (). isInterrupted ()) { Element elem = queue.take (); System.out.println(elem ); } } catch ( InterruptedException consumed) { /* Allow thread to exit but consume interrupted flag. */ } } public void cancel () { interrupt (); } ◮ may be ok if you own the thread ◮ otherwise, it may be interrupt request to shutdown the executor, not only cancel the task ◮ executor code may suspend in blocking operation, because you consumed the flag ◮ use Thread.currentThread.interrupt() in catch(InterruptedException) to restore the flag

Non-interruptible blocking Some blocking operations are non-interruptible: ◮ socket in java.io – stream read and write methods ◮ closing the socket throws SocketException ◮ channels and selectors in java.nio are mostly interruptible ◮ implicit lock – not interruptible ◮ some locks in java.util.concurrent are interruptible, ◮ otherwise, there is nothing you can do about it! Example, lets stop a service by ◮ calling service’s synchronized stop() which ◮ clears service state and calls thread.interrupt() ◮ that service thread just decided to take synchronized(this) ◮ after lock is acquired it tries to use cleaned service state ◮ remember interrupt() just “asks” to stop