Self-service virtual environments at Deutsche Telekom based on - PowerPoint PPT Presentation

Self-service virtual environments at Deutsche Telekom based on OpenStack Alexander Stellwag Deutsche Telekom AG Products & Innovation

Agenda ● Introduction ● Short History Of Wolke-7 ● Planning and Setup ● Virtual Private Environments ● Version 2: havana and neutron ● Lessons Learned ● Q&A

Introduction ● Alex Stellwag ● 43 Years ● Started at Deutsche Telekom in 2001 ● Cloud Architect since 2011 ● OpenStack since „Diablo“

Short History of Wolke-7 ● Started back in late 2011 after major reorganization to gain knowledge of OpenStack ● Dec 2012: initiated two basic use cases Plain IaaS, configurable via EC2-API – Fully managed environments (PaaS) – ● Both available from one cloud environment ● Internal customers only, no public services ● Core principles Stay open source – Build over partnering –

Intranet Infrastructure ● Supermicro servers (2UTwin² and 2U 12 Disk) ● Ubuntu 12.04 LTS (Kernel 3.2 and 3.11) ● OpenStack packages from UCA ● Ceph packages from Inktank ● I/S automation via FAI 1 and puppet ● Monitoring via nagios / check_mk 1 http://fai-project.org

Some Figures ● 12 compute nodes with 2x Xeon X5650 (6 cores, 2.67 GHz, Intel HT enabled), 96 GB Ram, 2x 250 GB SSD each ● 3 Ceph nodes with 2x Xeon E5-2630, 64 GB Ram, 2x 240 GB SSD + 10x 2TB S-ATA (OSDs) each. ==> ~60 TB net ● 2x Cisco Nexus 5548 for tenant VLANs and floating IPs ● 2x Cisco 48 port 1 GE for administration

Planning and Setup ● Setting up OpenStack is quite painless thanks to the official docs ● We needed a complete DC – Location – Connectivity – Basic I/S services (DNS, Mail, Proxy) – Automation services (FAI, puppet) ● Complex Setup due to internal security requirements ● Adapting the puppet modules for our needs took a while

Virtual Private Environments I ● Simple applications (apache, mysql, tomcat) – Provided by user-data – Published on project website – User has to copy & paste them into dashboard – Designed to fulfill security requirements – Full self-service via OpenStack dashboard – Users have full control and responsibility

Virtual Private Environments II #cloud-config # this cloud config code installs apache2-worker webserver # and changes several configuration directives to already meet # some GIS requirements # after customization of the webserver the requirements have to be # checked agein # supported OS: Ubuntu 12.04 # addtional packages to be installed packages: - apache2 - libapache-mod-security # commands to modify the default installation runcmd: - a2dismod cgid # req 7 - sed -i 's/\(.*Alias \/\(cgi-bin\|doc\)\)/#\1/; //{s/^/#/}; //,/Options /{s/\(.*\n\t\t\tdeny from all\n\t\t \ <\/LimitExcept>/}' /etc/apache2/sites-available/default # req 9,11,12,13,16,18 - sed -i 's/^/#/' /etc/apache2/mods-available/alias.conf # req 16 - sed -i 's/^\s*ServerTokens .*/ServerTokens Prod/' /etc/apache2/conf.d/security # req 19 - service apache2 restart # restart webserver to apply changes

Virtual Private Environments III ● Complex environments (currently CI/CD for Java only) – Jenkins, git, tomcat – Set up via bash and python scripts – Not yet integrated in dashboard, ordered via e-mail – Fully managed by Test & Development team

Version 2: havana & neutron ● Started in 10/2013 ● New location, so again we needed to setup a complete DC ● Ready for production in May 2014 ● Currently looking for dedicated operations team ● Heat installed and enabled but Virtual Environments not ported over yet. ● Users may use their own stack templates

Infrastructure Changes ● No dedicated network node, all agents distributed amongst compute nodes ● Multiple L3 and DHCP agents per L3 network ● Custom python script to check availability of agents and reschedule if necessary ● Works, but we have no long-time experience ● Currently looking into Icehouse, but migration has issues (mainly OS-wise)

New Hardware ● 11 compute nodes with 2x Xeon E5-2660 v2 (10 cores, 2.2 GHz, Intel HT enabled), 128 GB Ram, 2x 400 GB SSD each ● 8 Ceph nodes with 2x Xeon E5-2620 v2 (6 cores, 2.1 GHz, Intel HT enabled), 32 GB Ram, 2x 100 GB SSD + 10x 4TB S-ATA (OSDs) each. ==> ~312 TB net ● 2x Alcatel Lucent OS 6900-T40 for VxLAN tunnels and floating IPs ● 2x Alcatel Lucent OS 6850 for administration

Lessons Learned ● I/S deployment is hard – even if you start on a green field ● Deploying OpenStack without the ability to fix the code is difficult (and sometimes frustrating) ● HA for neutron is a complex beast ● Deploying multi-tier applications without heat is no fun

Lessons Learned II ● Manage internal stakeholder for proper funding ● Many customers don't know the difference between virtualization and cloud computing ● Change management program needed because cloud impacts 95% of operations roles ● For large and inhomogeneous teams SCRUM is not the right project management method

Q & A Questions ?