Lessons Learnt from Running a Container Native Cloud Xu Wang - PowerPoint PPT Presentation

Lessons Learnt from Running a Container Native Cloud Xu Wang (@gnawux) CTO & Cofounder, Hyper.sh Reinvent IaaS with Container!

Agenda • Hyper.sh: a Container Native Cloud • Under the Hood: How We Build a Container Native Cloud • Innovation from the User Side: App Patterns in Hyper.sh • Solve the Challenges Introduced by Containers Reinvent IaaS with Container!

How we define Container Native Cloud • Container is a First-Class citizen in the cloud • Container level APIs • Launch container without virtual machine (cluster) • Container-Centric Resources • Floating IP, Security Group… • Container-Based Services • Service (Load Balancer), Cron, Function… • Charged by Container • Charged based on container life cycle rather than VMs Reinvent IaaS with Container!

Why Container Native Cloud “All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections .” ----David Wheeler Reinvent IaaS with Container!

Why Container Native Cloud (cont.) • Overhead: • Resources • Ops https://www.openstack.org/videos/boston-2017/kubernetes-on-openstack-on-kubernetes-the-infrastructure-club-sandwich Reinvent IaaS with Container!

Hyper.sh: a Container Native Cloud Reinvent IaaS with Container!

Agenda • Hyper.sh: a Container Native Cloud • Under the Hood: How We Build a Container Native Cloud • Innovation from the User Side: App Patterns in Hyper.sh • Solve the Challenges Introduced by Containers Reinvent IaaS with Container!

How we build hyper.sh Hyper func compose cron job service High level app engines engine engine engine engine kubernetes scheduler Kube- stackube (hypernetes) API apirouter openstack/stackube controller node node node Hyperd / keystone Neutron Cinder runV Distributed graph engine Reinvent IaaS with Container!

Runtime: Secure à Simple • Container Isolated by hypervisor • Secure as VM, 
 Fast as Container Reinvent IaaS with Container!

runV: Secure as VM, Fast as Container Reinvent IaaS with Container!

runV: Project Summary • https://github.com/hyperhq/runV • Version 1.0 (Sep 2017) • Compatible with OCI runtime spec 1.0 • Full Support of Kubernetes CRI Semantics • Supported arch: x86_64, aarch64, power, s390x • Supported hypervisor: kvm/qemu, xen (fv), xenpv, kvmtool • Contributors: HyperHQ, Huawei, ZJU, Intel, Alibaba, IBM • Work with Hyperd, Containerd, & CRI-O Reinvent IaaS with Container!

Stackube: K8S as the Scheduler of OpenStack Reinvent IaaS with Container!

What’s in Stackube Repo? • Keystone RBAC plugin • Neutron CNI plugin • With a k8s Network object controller • Neutron based multi-tenant k8s service proxy • k8s Cinder plugin with block device mode • Deployment scripts and guide • Documentations Reinvent IaaS with Container!

Project Status • https://github.com/openstack/stackube • 1.0-beta released • Done: Migrate existing code from Hypernetes to Stackube • Done: Keystone RBAC plugin • Done: k8s Network and tenant object controller • Done: Neutron CNI plugin • Done: Neutron based multi-tenant k8s service proxy • Done: k8s Cinder plugin with block device mode • Done: Multi-tenant k8s DNS addon • Done: containerize all plugins above • Done: multi-node deployment scripts Reinvent IaaS with Container!

Agenda • Hyper.sh: a Container Native Cloud • Under the Hood: How We Build a Container Native Cloud • Innovation from the User Side: App Patterns in Hyper.sh • Solve the Challenges Introduced by Containers Reinvent IaaS with Container!

CI on Hyper.sh • CI integration with Hyper • Buildbot on Hyper • Jenkins on Hyper (Hykins) • Drone on Hyper • Agility • Zero Preparing Time • Zero Cleanup/Waste Time • No resource pool is required anymore Reinvent IaaS with Container!

Parallel Containers & Timed Containers • Spawn containers as Jobs come • Rather than queue and schedule in a VM cluster • Or periodically spawn containers • Lower Management Cost • More Responsive Reinvent IaaS with Container!

Disaster Recovery • Containers on Hyper.sh as a backup • Start backup containers in seconds once server in private cluster down • Result • Much faster than start a VM cluster and launch Apps • For most Apps, no hot standby any more Reinvent IaaS with Container!

Summary: the Value for Customers • Eliminate the Visible Resource Pool • Shorter Preparation Time, More friendly for Parallel Jobs • Spawn on Demand, Minimize Standby Resources • Independent Lifecycle for Each Container, Simplify Management Reinvent IaaS with Container!

Agenda • Hyper.sh: a Container Native Cloud • Under the Hood: How We Build a Container Native Cloud • Innovation from the User Side: App Patterns in Hyper.sh • Solve the Challenges Introduced by Containers Reinvent IaaS with Container!

Challenges from Concurrent Launch • The fact • Much more containers than traditional VM • Shorter living time of a container • The challenges • Metric the server load – Not only by static resource • Consumption introduced by create and cleanup resources • The solutions • Scheduler plugin, not only based on static resource usage • Optimize procedure, reduce the workload of message queue • Cache and delay delete Reinvent IaaS with Container!

Challenges from High Density • The fact • More VM (containers) on a single host • More ports on a single host • The challenges • ovs agent overhead • open-vswitch device lookup consumption • The solutions • Avoid unnecessary ports • Reduce/Eliminate the agent consumption • Optimize ovs port lookup • Newer version upstream introduced many fixes Reinvent IaaS with Container!

Thank You! Reinvent IaaS with Container!