May 2018 Magnum Project on Boarding, OpenStack Summit Vancouver Spyros Trigazis @strigazi, Ricardo Rocha @ahcorporto
What is Magnum? ● OpenStack API service for creation of container clusters ● Single-tenant clusters ● Credential management ● OpenStack integration, cloud provider ● Lifecycle operations ● Kubernetes, Docker Swarm, Mesos, DC/OS
Magnum Terminology - Cluster Template ● Set of parameters describing a cluster (base for cluster creation) +-----------------------+------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------+ | insecure_registry | - | | docker_volume_size | - | | labels | {u'kube_dashboard_enabled': u'false', | | server_type | vm | | | u'prometheus_monitoring': u'true', | | external_network_id | - | | | u'kube_tag': u'v1.8.0-1', | | cluster_distro | fedora-atomic | | | u'grafana_admin_passwd': u'changeme', | | image_id | 55e22657-74e5-46d9-ba28-47980986b42c | | | u'flannel_backend': u'vxlan'} | | volume_driver | - | | updated_at | - | | registry_enabled | False | | floating_ip_enabled | False | | docker_storage_driver | overlay | | fixed_subnet | - | | apiserver_port | - | | master_flavor_id | m2.medium | | name | kubernetes-alpha | | uuid | afee31b7-6f35-42d3-8a21-9328edd5acf3 | | created_at | 2017-09-25T10:47:17+00:00 | | no_proxy | - | | network_driver | flannel | | https_proxy | - | | fixed_network | - | | tls_disabled | False | | coe | kubernetes | | keypair_id | - | | flavor_id | m2.medium | | public | True | | master_lb_enabled | False | | http_proxy | - | | dns_nameserver | 8.8.8.8 | +-----------------------+------------------------------------------------+
Magnum Terminology - Cluster ● Configurable number of master nodes +---------------------+-------------------------------------------+ | Field | Value | ● Configurable number of worker nodes +---------------------+-------------------------------------------+ | status | CREATE_COMPLETE | ● Deployed as Heat Stacks | cluster_template_id | 27d0fef7-3a03-4a83-ae27-6c219a84e589 | | node_addresses | [u'yyy.yyy.yyy.yyy'] | | uuid | 89f79322-b574-4ea5-8169-606888d38b6f | ● A trustee user and a trust | stack_id | 7cbca34c-afe3-43f6-9443-d2cfc1232996 | | status_reason | Stack CREATE completed successfully | ● A Certificate Authority | created_at | 2018-04-30T14:08:26+00:00 | | updated_at | 2018-04-30T14:19:46+00:00 | | coe_version | v1.9.3 | ○ Stored in Barbican or Magnum DB | labels | {u'kube_tag': u'v1.10.1’} | | faults | | | keypair | strigazi-lxplus | ● 3 cluster orchestrator engines | api_address | https://xxx.xxx.xxx.xxx:6443 | | master_addresses | [u'xxx.xxx.xxx.xxx'] | ○ Kubernetes, Swarm, Mesos / DC/OS | create_timeout | 60 | | node_count | 1 | | discovery_url | https://discovery.etcd.io/bc41b65fe11669d | ● Multiple OS options | master_count | 1 | | container_version | 1.12.6 | ○ Fedora Atomic, CoreOS, Ubuntu, Centos | name | strigazi-kube | | master_flavor_id | m2.medium | | flavor_id | m2.medium | ● VM or Baremetal +---------------------+-------------------------------------------+ ● Cluster scaling up/down ● Possible Ceilometer integration
Usage ● https://docs.openstack.org/magnum/latest/user/ ● Operators: manage cluster templates ● End user: create clusters, custom templates $ openstack coe cluster create --cluster-template kubernetes --flavor m1.xlarge --node-count 32 ... kubernetes Request to create cluster ad418271-5232-466b-a4db-768a7ecae526 accepted $ ... $ $(openstack coe cluster config kubernetes) $ kubectl get componentstatuses NAME STATUS MESSAGE ERROR etcd-0 Healthy {"health": "true"} scheduler Healthy ok controller-manager Healthy ok $ kubectl proxy Starting to serve on 127.0.0.1:8001
Usage ● https://docs.openstack.org/magnum/latest/user/ ● Operators: manage cluster templates ● End user: create clusters, custom templates $ openstack coe cluster create --cluster-template kubernetes --flavor m1.xlarge --node-count 32 ... kubernetes Request to create cluster ad418271-5232-466b-a4db-768a7ecae526 accepted $ ... $ $(openstack coe cluster config kubernetes) $ kubectl get componentstatuses NAME STATUS MESSAGE ERROR etcd-0 Healthy {"health": "true"} scheduler Healthy ok controller-manager Healthy ok $ kubectl proxy Starting to serve on 127.0.0.1:8001
Magnum Queens Features ● Simplified cluster creation, new params available: ○ Master and node flavor ○ Docker volume size ○ Labels ● Cluster availability zone selection ● Kubernetes features ○ Calico as a network driver ○ Role Based Access Control - RBAC ○ Monitoring stack, heapster, influxDB and grafana ○ Traefik ingress ○ Support for versions v1.9.x and 1.10.x
Magnum Queens Features for Ops & Devs ● Admin operations ○ Can list, show, delete clusters on all projects ● Use custom certificate authorities ○ For self signed certificates or certificates signed by unknown issuers ● Extensible cluster drivers ○ Heat agent instead of Nova user data ○ Allows more complex and larger deployments
Rocky Goals Rocky blueprints ● Cluster Upgrades ● Cluster Node Replacement ● Cluster Health Monitoring / Healing ● Keystone Authentication for Kubernetes ● Cluster Log Collection ● Support for Federation (Kubernetes) ● Availability Zone Selection ● Run dockerd in system containers ● Different container runtimes (docker, cri-o, cri-containerd, gvisor, kata)
Cluster Upgrades ● Spec: https://review.openstack.org/#/c/433728/ ● Rolling upgrade per batch of nodes ● Two upgrade steps masters and workers ● Upgrade in place for containerized COEs ● Node rebuild or replace ● Upgrade parameters via cluster template ● Implementation (bp/cluster-upgrades)
Cluster Healing ● Specification: https://review.openstack.org/#/c/529897/ ● New fields for health status ○ HEALTHY | UNHEALTHY ○ Reason ● Periodic check of cluster components ○ Cluster API reachable? ○ Node status? ● Rebuild | Replace nodes based on policy
Running Magnum
Before you install Magnum Hard Requirements: ● Core OpenStack (Keystone V3, Glance, Nova, Neutron) and Heat ● Access to a container registry (Outside or Inside the cloud) ● Network Connectivity from Compute instances to the public OpenStack APIs ● Floating IPs and Tenant networks Optional Requirements: ● Cinder for extra container space and/or persistent data ● Octavia for cluster API LB or kubernetes LB service type ● Barbican, for secure secret storage ● Ironic, for running on Physical Servers
Common issues ● Cannot pull containers from the internet ○ Mirror all dependencies in your cloud and use container_infra_prefix label and insecure_registry field ● Self signed certificates for the OpenStack APIs or signed by unknown authorities ○ Use openstack_ca_file in magnum.conf ● Incompatible OS and COE versions, see compatibility matrix
Recommend
More recommend
