OPEN INFRASTRUCTURE SUMMIT | SHANGHAI, NOVEMBER 4-6 2019 Run Kubernetes on OpenStack and Bare Metal Fast Ramon Acedo Rodriguez 1 Senior Principal Product Manager, Red Hat
Open Hybrid Cloud Vision OPTIONAL SECTION MARKER OR TITLE 2
The Open Hybrid Cloud The 4 Footprints APP APP APP APP PRIVATE PUBLIC PHYSICAL VIRTUAL CLOUD CLOUD BARE METAL VIRTUAL PRIVATE CLOUD 3
Kubernetes-powered Open Hybrid Cloud Goal : Give developers the freedom to innovate faster across on-premises and public clouds APP APP APP APP C VM C C PUBLIC On-premises PHYSICAL VIRTUAL CLOUD BARE METAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS 4
Kubernetes on OpenStack is focused on the private cloud Containers | VMs | Serverless Apps BARE METAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS BARE METAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS 5
Kubernetes on OpenStack Introduction 6
Why Kubernetes on OpenStack? Open, scalable, managed, workloads. WORKLOAD DRIVEN DEEPLY INTEGRATED PROGRAMMATIC API DRIVEN ACROSS DATACENTRE SCALE OUT INFRASTRUCTURE SOLID FOUNDATION VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS 7
Kubernetes on OpenStack Integrations 8
Kubernetes and OpenStack Are Complementary complementary Exposition of resources Consumption of resources Provides the infrastructure layer Provides the container platform layer
Kubernetes on OpenStack Key Integration Points
Kubernetes on OpenStack Integration Example: Cinder
OpenShift on OpenStack Logical Architecture
Installation Requirements OpenShift on OpenStack 13
Requirements in your OpenStack Platform OpenStack Resources: Red Hat OpenStack Platform 13 (Queens) Floating IPs: 2 ● 3 Master nodes Security Groups: 3 ● Security Group Rules: 60 At least 16 GB RAM, 4 vCPUs and 25 GB Disk ● ● Routers: 1 ● Subnets: 1 ● At least 2 worker nodes RAM: 112 GB ● A least 8 GB RAM, 2 vCPUs and 25 GB Disk ● vCPUs: 28 ● Volume Storage: 175 GB ● Object Storage (Swift) Instances: 7 ● Swift containers: 2 ● CoreOS image Swift objects: 1 ● Available space in Swift: at least 10 MB ● https://github.com/openshift/installer/blob/master/docs/user/openstack/README.md
OpenShift Internal Load Balancing and DNS 15
Internal DNS and Load Balancing The openshift-installer will configure internal Load Balancing & DNS Load Balancing DNS HAProxy + keepalived on master nodes ● ● CoreDNS and mDNS run on all nodes Use of keepalived VIPs for: ● ○ Internal API ○ Ingress traffic to workloads ○ Internal DNS requests https://github.com/openshift/installer/blob/master/docs/design/openstack/networking-infrastructure.md
Kuryr-Kubernetes SDN SDN for OpenShift on OpenStack 17
Kuryr-Kubernetes CNI Kuryr improves the network performance of pods when running on OpenStack. SDN solution using Kubernetes Container Network Interface (CNI) and OpenStack Neutron. Provides interconnectivity between Kubernetes pods and OpenStack virtual instances Kuryr-Kubernetes Logo 18
When to use Kuryr Recommended when Requires ● Neutron tenant networks are used for OpenShift Application Nodes ● OpenStack Queens or newer ● Octavia Load Balancer Not recommended when ● Neutron Trunk Ports ● Using only OpenStack VLAN-based provider networks (Kuryr not needed) ● Many OpenShift services and few hypervisors: For each OpenShift service, Octavia will run a VM with a ○ load balancer in it 19
Kuryr Internal Architecture Kuryr integrates with Neutron and Octavia Pods are connected via Kuryr CNI to Neutron Kuryr creates Load Balancers with Octavia for the Kubernetes services 20
Kuryr Performance Comparison Performance test done with OpenShift 3.11 / Kubernetes 1.11 and OpenStack Queens Bandwidth tests between pods on the same hypervisor show slightly better performance 21
Kuryr Performance Comparison Performance test done with OpenShift 3.11 / Kubernetes 1.11 and OpenStack Queens Bandwidth tests between pods on different hypervisors show up to 9x performance improvement 22
Kuryr Performance Comparison Blog Post Performance test done with OpenShift 3.11 and OpenStack Queens https://blog.openshift.com/accelerate-your-openshift-network-performance-on-openstack-with-kuryr 23
Reference Architecture 24
OpenShift on OpenStack Reference Architecture Current: OSP 13 LTS and OCP 3.11 with Kuryr | Next: OSP 13 LTS and OCP 4.3 with Kuryr https://access.redhat.com/documentation/en-us/reference_architectures/2019/html-single/deploying_red_hat_o 25 penshift_container_platform_3.11_on_red_hat_openstack_platform_13
Ways to Install Kubernetes on OpenStack Introduction 26
Kubernetes Installation with OpenShift 4 on OpenStack Installation Experiences Full Stack Automation Pre-existing Infrastructure Simplified, opinionated User-provisioned Installer-provisioned OpenStack resources (UPI) OpenStack resources (IPI) BARE METAL VIRTUAL PRIVATE CLOUD BARE METAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS 27
Full Stack Automation Installation Deploying OpenShift on OpenStack with installer-provisioned infrastructure (IPI) Installer Provisions: OpenShift Cluster Networks Internal Load Balancers 1 Internal DNS 1 openshift-install OpenStack Instances . . . Red Hat CoreOS Red Hat OpenStack CoreOS Ignition Configs Platform OpenShift Nodes Overcloud OpenShift Cluster Resources BARE METAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS BARE METAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS 28 1 External Load Balancers (routing) and external DNS servers are provided by the user
Installing on Pre-existing Infrastructure Deploying OpenShift on OpenStack with user-provisioned infrastructure (UPI) User Provisions: OpenShift Cluster Networks openshift-install Internal Load Balancers 1 Internal DNS 1 User provisioned resources OpenStack Instances . . . Red Hat CoreOS Red Hat OpenStack CoreOS Ignition Configs Platform OpenShift Nodes Overcloud OpenShift Cluster Resources BARE METAL VIRTUAL BARE METAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS 29
Installing on OpenStack Documentation User-provisioned Infrastructure | Installer-provisioned Infrastructure USER PROVISIONED INSTRALLER- PROVISIONED INFRASTRUCTURE INFRASTRUCTURE https://github.com/openshift/installer/blob/master/docs/user/openstack/README.md https://github.com/openshift/installer/blob/master/docs/user/openstack/install_upi.md VIRTUAL PRIVATE CLOUD PUBLIC CLOUDS 30
OpenShift 4.2 on OpenStack try.openshift.com 31
OpenStack Bare Metal Ironic Introduction 32
Bare Metal On-Trend Among users who run Kubernetes on OpenStack, adoption of Ironic is even stronger with 37% relying on it. OpenStack User Survey 2018 OpenStack User Survey 2017
Bare Metal On-Trend Popular Use Cases Kubernetes on Bare Metal High-Performance Computing Direct Access to Dedicated Hardware Devices Big Data and Scientific Applications
OpenStack Bare Metal Service - Ironic Hardware Lifecycle Management Hardware Inspection Servers and Network Switches (via LLDP) OS Image Provisioning Supporting qcow2 images Routed Spine/Leaf Networking Provisioning over routed networks Multi-Tenancy With network isolation between tenants Node Auto-discovery Broad Power Management Support Redfish, iDrac, iRMC, iLo, IPMI, oVirt, vBMC
OpenStack Bare Metal Features 36
Multi-Tenant Support with Isolation Between Tenants Available from OpenStack Queens Tenant-Dedicated Networks L2 Switch Instead of a shared flat network VLANs set by by ML2 plug-in Provisioning Over an Isolated NIC NIC Network BM BM Switch Ports Dynamically Configured At deployment time and on termination L2 Switch L2 Switch Link Aggregation Configured by ML2 plug-in LAG NIC NIC bond Configured by cloud-init using BM metadata
Multi-Tenant Support with Isolation Between Tenants Available from OpenStack Queens Neutron ML2 Networking-Ansible Driver Boot BM on ML2 Plug-in ML2 Plug-in BM is Tenant Configures Configures BM is ready Multiple Switch Platforms in a Provisioned Network Switch Switch Single ML2 Driver Leveraging the Networking Ansible modules Provisioning Network is Tenant Network is configured in the switch configured in the switch Available in OpenStack Queens | Red Hat OpenStack L2 Switch Platform 13 OSP 13 Long Life Support NIC NIC BM BM
L3 Routed Networks (Spine/Leaf Network Topologies) Available from OpenStack Queens IPv6 Support being added to OpenStack Train and above spine switch spine switch spine switch L3 routed L3 routed networks networks L3 Spine and Leaf Topologies Ironic provisioning bare metal nodes over ToR/leaf switch ToR/leaf switch ToR/leaf switch routed networks DHCP Relay DHCP Relay DHCP Relay Ironic Node Bare Metal Bare Metal DHCP Relay Allowing PXE booting over L3 routed Ironic Node Bare Metal Bare Metal networks Ironic Node Bare Metal Bare Metal Bare Metal Bare Metal Bare Metal Bare Metal Bare Metal Bare Metal
Recommend
More recommend
