Unlimited Event Channels David Vrabel 24 October 2013 What are - PowerPoint PPT Presentation

“Unlimited” Event Channels David Vrabel 24 October 2013

What are Event Channels? ◮ Paravirtual interrupts ◮ Edge triggered ◮ Bidirectional ◮ Directed at a single VCPU 24 October 2013 “Unlimited” Event Channels 2 / 13

How do Event Channels Work? Domain A Domain B 1 Notify Shared Memory 4 Call Handlers Xen 2 3 Set Pending Upcall 24 October 2013 “Unlimited” Event Channels 3 / 13

Original (2-Level) Design Per-VCPU Shared Memory 64 bits Selector 1 ... Shared Memory 64 bits Pending ... 0 1 0 0 ... 0 1 0 ... Masked ... ... ... 4096 bits 24 October 2013 “Unlimited” Event Channels 4 / 13

Problems ◮ Too few ◮ No priorities ◮ Unfair 24 October 2013 “Unlimited” Event Channels 5 / 13

Additional Requirements ◮ Identical ABI between 32-bit and 64-bit guests ◮ Reasonable memory usage ◮ Easily extensible 24 October 2013 “Unlimited” Event Channels 6 / 13

New (FIFO-based) Design Bit 15 1 0 Bit 31 30 29 16 0 READY 1 Port 0 P M L LINK (Reserved) 1 1 0 1 6 2 1 0 1 0 3 HEAD 4 Priority 0 5 1 1 1 0 1 3 6 ... 7 15 Up to 132,072 words ... Per-VCPU shared memory Shared memory 24 October 2013 “Unlimited” Event Channels 7 / 13

Raising an Event function raise(q, p) 1. Set pending E[p].pending = 1 2. Unmasked and not in list? if not E[p].masked and not E[p].linked Add to list E[p].linked = 1 3. Set LINK of tail if T[q] != p or linked = link(T[q], p) Set HEAD in control block else linked = false 4. Advance tail if not linked C.head[q] = p T[q] = p 24 October 2013 “Unlimited” Event Channels 8 / 13

Handling an Event function handle_one_event(q) 1. Get local head p = H[q] 2. List was empty? if p == 0 Get new HEAD from p = C.head[q] control block link = unlink(p) H[q] = link 3. Clear LINKED and LINK if E[p].pending and not E[p].masked handle(p) 4. Advance local head 5. Pending and unmasked? Handle IRQ 24 October 2013 “Unlimited” Event Channels 9 / 13

Event Channel Fairness 3000 2-Level FIFO-based 2500 2000 Latency (ns) 1500 1000 500 0 75 80 85 90 Event Channel Port 24 October 2013 “Unlimited” Event Channels 10 / 13

Current Status ◮ Xen support in 4.4 ◮ Linux support in 3.14 24 October 2013 “Unlimited” Event Channels 11 / 13

Further Work ◮ Investigate how priorities may be used ◮ Add support for limiting number of event channels to other toolstacks (XAPI, libvirt) ◮ Add support in other OSes ◮ Xen event lock scalability ◮ Linux PIRQ setup/teardown refactoring 24 October 2013 “Unlimited” Event Channels 12 / 13

Questions? ◮ Design Document http://xenbits.xenproject.org/people/dvrabel/event-channels-F.pdf 24 October 2013 “Unlimited” Event Channels 13 / 13