IPC, Threads, Races, Critical Sections Why We Wait 7C. - PDF document

4/23/2018 IPC, Threads, Races, Critical Sections Why We Wait 7C. Asynchronous Event Completion • We await completion of non-trivial operations 7D. Mutual Exclusion – data to be read from disk – a child process to be created 7E. Implementing Mutual Exclusion • We wait for important events 7F. Asynchronous completion – a request/notification from another process 7G. Implementing asynchronous completion – an out-of-band error that must be handled • We wait to ensure correct ordering – B cannot be performed until A has completed – if A precedes B, B must see the results of A IPC, Threads, Races, Critical Sections 1 Introduction to Synchronization 2 Correct Ordering Problem 2: asynchronous completion • most procedure calls are synchronous server client (3 threads) – we call them, they do their job, they return 1. shutdown send shutdown – when the call returns, the result is ready • many operations cannot happen immediately 1. process cmd 2. status log status 2. return status – waiting for a held lock to be released 3. exit process – waiting for an I/O operation to complete 3. SIGCHLD exit process – waiting for a response to a network request – delaying execution for a fixed period of time “Surely the final status message will be received and processed • we call such completions asynchronous before the SIGCHLD causes the client to shut down! “ Introduction to Synchronization 3 IPC, Threads, Races, Critical Sections 4 Approaches to Waiting Condition Variables • create a synchronization object • spinning … “busy waiting” – associate that object with a resource or request – works well if event is independent and prompt – requester blocks awaiting event on that object – wasted CPU, memory, bus bandwidth – upon completion, the event is "posted" – may actually delay the desired event – posting event to object unblocks the waiter • yield and spin … “are we there yet?” – allows other processes access to CPU wait – wasted process dispatches – works very poorly for multiple waiters signal • either may still require mutual exclusion IPC, Threads, Races, Critical Sections 5 IPC, Threads, Races, Critical Sections 6 1

4/23/2018 Awaiting Asynchronous Events The Mutual Exclusion Challenge pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; • We cannot prevent parallelism pthread_cond_t cond = PTHREAD_COND_INITIALIZER; – it is fundamental to our technology … pthread_mutex_lock(&lock); • We cannot eliminate all shared resources while (ready == 0) – increasingly important to ever more applications pthread_cond_wait(&cond, &lock); pthread_mutex_unlock(&lock) • What we can do is ... … – identify the at risk resources, and risk scenarios if (pthread_mutex_lock(&lock)) { ready = 1; – design those classes to enable protection pthread_cond_signal(&cond); – identify all of the critical sections pthread_mutex_unlock(&lock); – ensure each is correctly protected (case by case) } IPC, Threads, Races, Critical Sections 7 Mutual Exclusion and Asynchronous Completion 8 Evaluating Mutual Exclusion Approaches • Effectiveness/Correctness • Avoid shared mutable resources – the best choice … if it is an option – ensures before-or-after atomicity • Interrupt Disables • Fairness – a good tool with limited applicability – no starvation (un-bounded waits) • Spin Locks • Progress – very limited applicability – no client should wait for an available resource • Atomic Instructions – susceptibility to convoy formation, deadlock – very powerful, but difficult w/limited applicability • Performance • Mutexes – delay, instructions, CPU load, bus load – higher level, broad applicability – in contended and un-contended scenarios Mutual Exclusion and Asynchronous Completion 9 Implementing Mutual Exclusion 10 What Happens During an Interrupt? Approach: Interrupt Disables • Interrupt controller requests CPU for service • temporarily block some or all interrupts – can be done with a privileged instruction • CPU stops the executing program – side-effect of loading new Processor Status • Interrupt vector table is consulted • abilities – PC/PS of Interrupt Service Routine (ISR) – prevent Time-Slice End (timer interrupts) • ISR handles the interrupt (just like a trap) – save regs, find/call 2 nd level handler, restore regs – prevent re-entry of device driver code • dangers • Upon return, CPU state is restored – may delay important operations – code resumes w/no clue it was interrupted – a bug may leave them permanently disabled Implementing Mutual Exclusion 11 Implementing Mutual Exclusion 12 2

4/23/2018 Preventing Preemption Preventing Driver Reentrancy zz_io_startup ( struct iorq *bp ) { DLL_insert(DLL *head, DLL*element) { … int save = disableInterrupts(); save = intr_enable( ZZ_DISABLE ); DLL *last = head->prev; element->prev = last; /* program the DMA request */ element->next = head; zzSetReg(ZZ_R_ADDR, bp->buffer_start ); DLL_insert(DLL *head, DLL*element) { zz_intr_handler () { zzSetReg(ZZ_R_LEN, bp->buffer_length); last->next = element; zzSetReg(ZZ_R_BLOCK, bp->blocknum); … DLL *last = head->prev; head->prev = element; zzSetReg(ZZ_R_CMD, bp->write? /* update data read count */ element->prev = last; resid = zzGetReg(ZZ_R_LEN); ZZ_C_WRITE : ZZ_C_READ ); } element->next = head; zzSetReg(ZZ_R_CTRL, ZZ_INTR+ZZ_GO); last->next = element; /* turn off device ability to interrupt */ /* reenable interrupts */ zzSetReg(ZZ_R_CTRL, ZZ_NOINTR); head->prev = element; … intr_enable( save ); } Serious consequences could result if the interrupt handler was called while we were half-way through programming the DMA operation. restoreInterrupts(save); Implementing Mutual Exclusion 13 Implementing Mutual Exclusion 14 Preventing Driver Reentrancy Interrupts and Resource Allocation … • interrupts are usually self-disabling lock(event_list); – CPU may not deliver #2 until #1 is acknowledged add_to_queue(event_list, my_proc); – interrupt vector PS usually disables causing intr unlock(event_list); xx_interrupt: yield(); … • they are restored after servicing is complete … lock(event_list); – ISR may explicitly acknowledge the interrupt post(event_list); return; – return from ISR will restore previous (enabled) PS • drivers usually disable during critical sections – updating registers used by interrupt handlers – updating resources used by interrupt handlers Implementing Mutual Exclusion 15 Implementing Mutual Exclusion 16 Interrupts and Resource Allocation Evaluating Interrupt Disables • Effectiveness/Correctness • interrupt handlers are not allowed to block – ineffective against MP/device parallelism – only a scheduled process/thread can block – only usable by kernel mode code – interrupts are disabled until call completes • Progress • ideally they should never need to wait – deadlock risk (if ISR can block for resources) – needed resources are already allocated • Fairness – operations implemented w/lock-free code – pretty good (assuming disables are brief) • brief spins may be acceptable • Performance – wait for hardware to acknowledge a command – one instruction, much cheaper than system call – wait for a co-processor to release a lock – long disables may impact system performance Implementing Mutual Exclusion 17 Implementing Mutual Exclusion 18 3

4/23/2018 Approach: Spin Locks Atomic Instructions • loop until lock is obtained • atomic read/modify/write operations – usually done with atomic test-and-set operation – implemented by the memory bus – effective w/multi-processor or device conflicts • abilities • ordinary user-mode instructions – prevent parallel execution – wait for a lock to be released – may be supported by libraries or even compiler – limited to a few (e.g. 1-8) contiguous bytes • dangers • very expensive (e.g. 20-100x) instructions – likely to delay freeing of desired resource – bug may lead to infinite spin-waits – wait for all cores to write affected cache-line – force all cores to drop affected cache-line Implementing Mutual Exclusion 19 Implementing Mutual Exclusion 20 Atomic Instructions – Test & Set Spin Locks /* * Concept: Atomic Test-and-Set DLL_insert(DLL *head, DLL*element) { * this is implemented in hardware, not code while(TestAndSet(lock,1) == 1); */ DLL *last = head->prev; int TestAndSet( int *ptr, int new) { element->prev= last; int old = *ptr; element->next = head; *ptr = new; last->next = element; return( old ); head->prev= element; } lock = 0; } Implementing Mutual Exclusion 21 Implementing Mutual Exclusion 22 Evaluating Spin Locks What If You Don’t Get the Lock? • Effectiveness/Correctness • give up? – effective against preemption and MP parallelism – but you can’t enter your critical section – ineffective against conflicting I/O access • try again? • Progress – OK if we expect it to be released very soon – deadlock danger in ISRs • what if another process has to free the lock? • Fairness – spinning keeps that process from running – possible unbounded waits • what lock release will take a long time? • Performance – we are burning a lot of CPU w/useless spins – waiting is extremely expensive (CPU, bus, mem) Implementing Mutual Exclusion 23 Implementing Mutual Exclusion 24 4