Naturalis Biodiversity Center At Naturalis Biodiversity Center we - PowerPoint PPT Presentation

No team No Problem! How to a single admin manages 70 Openstack nodes Atze de Vries | Naturalis Biodivercity Center

Naturalis Biodiversity Center At Naturalis Biodiversity Center we want to describe, understand and explore biodiversity for human wellbeing and the future of our planet. We are based in the Netherlands We also have a museum and a T-Rex

About me Atze de Vries - atze.devries@naturalis.nl Sysadmin / Devops

Our current Infrasturcture An deployment with HPC (35 nodes) Openstack An deployment for Backup to Disk (14 nodes) Openstack An older (to be migrated) (18 nodes) Openstack An older VMWare cluster All environments have as storage backend. Instances boot from localdisk. Openstack Ceph

Where we came from Windows only environment Virtualized using VMWare IT was not primary business at Naturalis

Scientists and computing Scientists wanted more computing power and more freedom Solution was to buy highend workstations Downsides: Hard to maintain Inefficient use

How was a solution Openstack With Openstack (or a other cloud) you have Easy sharing of resources Sandbox environment Almost full (maybe to much) freedom We also considered Cloudstack OpenNebula But at that time had the best feature set and community Openstack

Getting your science department into Openstack Workload is generally single-cpu / memory intensive Resource sharing, less costs Ultimate freedom Access anywhere Abillity to replay your calculations

Downsides of using Openstack Scientists need knowledge of computing

How to help your Scientists Teach them how to use SSH-keypairs / Security Groups / etc Create (great) documentation Organise workshops Recrute keyusers / early adopters

Whats missing from Openstack 3D More Cloud ready Science software

Maintaining your stack

Current status We are moving our infrastructure to . Openstack We are almost there

Get a default configuration Don't use exotic configurations. For hypervisor use KVM (or Xen) etc.

Get a sensible configuration Look at your needs Do you really need shared storage? Do you need HA controllers?

Go for the cloud way roast your pets, breed your cattle Get rid of big Windows installations which take hours to setup Try to split services

Go for config management puppet/ceph ansible etc Openstack is ideal for getting your whole infrastructure into code Config management tools make it easy to rebuild / update your servers fast When you do it, do it right and do it only once

Start using the API Openstack The API lets you automate operating Openstack You can build preconfigured networks, flavors or projects Let user accounts sync with Active Directory

Example configuration of flavors ### Specials # highmem server - cpu: 10 ram: 500 disk: 100 type: hpc_highmem - cpu: 20 ram: 500 disk: 200 type: hpc_highmem - cpu: 40 ram: 500 disk: 200 type: hpc_highmem

for f in flavors['flavors']: name = "%s.%sc.%sr.%sh" % (f['type'],f['cpu'],f['ram'],f['disk']) if name in inyaml: log.logger.debug("flavor %s is duplicate in flavor.yaml so it will be skipped" continue inyaml.append(name) if name in current: log.logger.debug("flavor %s exists! flavor will not be created" % name) continue try: flavor = nova.flavors.create(name,(f['ram']*1024),f['cpu'],f['disk'],is_public= flavor.set_keys({'type':f['type']}) log.logger.info("creating flavor %s" % name) except Exception as e: log.logger.warning("Failed to create flavor %s" % name) log.logger.debug(e)

Example of project (tenant) configuration project: # Wijziging W15110057 name: Ariantinae groups: - "adsync - Endless Forms" # quotas quotas: nova: cores: 4 instances: 2 ram: 16384 cinder: gigabytes: 100 volumes: 20 snapshots: 20

neutron: network: 4 subnet: 8 port: 50 router: 2 security_group: 10 security_group_rule: 10 floatingip: 10 flavors: - hpc.4c.16r.20h - hpc.4c.16r.40h - hpc.4c.16r.60h

Get 3rd party support Mirantis / Ubuntu / Whatever

Questions?