The Xen Credit CPU S Scheduler Emmanuel Ackaouy ack@xensource.com
Goals • Maintainability – Simple & Fast • Zero-admin good default behavior – Fairness : Weight based – I/O versus CPU intensive VCPUs – Tuning via per-VM weight & cap • Better SMP support – Work conserving – Transparent cross CPU migration – System wide accounting & fairness 9/18/06 XenSource: Open Source Enterprise Virtualization 2
Round Robin CPU Usage CPU0 VCPU 2 5 1.0 2 0 1 5 VCPU 2.0 1 0 5 VCPU 3.0 0 VCPU1 VCPU2 VCPU3 VCPU4 VCPU 4.0 9/18/06 XenSource: Open Source Enterprise Virtualization 3
Preemption Latency to Run CPU Usage 100 1 5 8 0 6 0 1 0 CPU0 4 0 5 2 0 VCPU I/O 0 0 I/O SPIN I/O VCPU SPIN 8 0 1 5 6 0 1 0 4 0 5 2 0 0 0 I/O SPIN I/O 9/18/06 XenSource: Open Source Enterprise Virtualization 4
Priorities! Run Queue CPU0 VCPU I/O VCPU I/O Priority UNDER VCPU SPIN DOM Active Domains I/O Credit 30 DOM DOM Host SPIN I/O Cap 0 9/18/06 XenSource: Open Source Enterprise Virtualization 5
CPU Usage CPU0 CPU1 2 5 2 0 VCPU VCPU 1.0 3.0 1 5 1 0 VCPU VCPU 2.0 4.0 5 0 VCPU1 VCPU2 VCPU3 VCPU4 9/18/06 XenSource: Open Source Enterprise Virtualization 6
CPU Usage CPU0 CPU1 5 0 4 0 VCPU VCPU 1.0 3.0 3 0 2 0 VCPU 1 0 2.0 0 VCPU1 VCPU2 VCPU3 9/18/06 XenSource: Open Source Enterprise Virtualization 7
Run Queues CPU0 CPU1 VCPU 3.0 VCPU VCPU 1.0 3.0 Priority OVER VCPU 2.0 DOM4 Active Domains Credit -10 Cap 0 Host DOM1 DOM2 DOM3 9/18/06 XenSource: Open Source Enterprise Virtualization 8
Self Balancing! CPU Usage CPU0 CPU1 5 0 VCPU VCPU 4 0 1.0 3.0 3 0 VCPU 2 0 2.0 1 0 0 VCPU1 VCPU2 VCPU3 CPU0 CPU1 CPU0 CPU1 3 5 3 0 VCPU VCPU VCPU VCPU 2 5 1.0 2.0 3.0 2.0 2 0 1 5 VCPU VCPU 1 0 3.0 1.0 5 0 VCPU1 VCPU2 VCPU3 9/18/06 XenSource: Open Source Enterprise Virtualization 9
Global A Accounting • Periodically recompute priorities for all active VCPUs – UNDER (below fair share usage) – OVER (above fair share) • VCPU fair shares derived from VM weight and cap in proportion to all active VMs • Accounting cost – A function of # active VCPUs in system – NOT a function of # scheduling decisions 9/18/06 XenSource: Open Source Enterprise Virtualization 10
Local C CPU S Scheduling • Per-CPU sorted runq – 2 VCPU priorities: OVER and UNDER fair share • Clock tick – Charge running VCPU • Scheduling decision – End of time slice or VCPU blocks – Common case: run next local VCPU under fair share – Otherwise: pick queued remote under fair share VCPU, local over VCPU, or remote over VCPU (in that order) • Scheduling cost – Common case: Pick next VCPU on local runq 9/18/06 XenSource: Open Source Enterprise Virtualization 11
Credit Scheduler Internals Run Queues CPU0 CPU1 CPU2 CPU3 VCPU 4.1 VCPU VCPU VCPU VCPU 1.0 1.1 1.2 1.3 Priority OVER VCPU VCPU VCPU VCPU 2.0 2.1 2.2 2.3 VCPU VCPU VCPU VCPU 3.0 3.1 3.2 3.3 VCPU VCPU VCPU 0.0 4.0 4.1 DOM4 Active Domains Credit -10 Cap 0 Host DOM0 DOM1 DOM2 DOM3 DOM4 9/18/06 XenSource: Open Source Enterprise Virtualization 12
Recommend
More recommend