USING OPENSTACK TO INTEGRATE NON-OPENSTACK SERVICE JUNHO YOON, ANDREW LIU, JACK NING
AGENDA • INTRODUCTION • MOTIVATIONS • INTEGRATE AUTHN/AUTHZ • INTEGRATE PLATFORM UI • INTEGRATE PLATFORM COMMUNICATION • CONTINOUS DEPLOYMENT WITH CUSTOMIZATION
Introduction JUNHO YOON ANDREW LIU JACK NING Senior developer of Senior developer of Senior developer of NAVER NAVER China NAVER China
Introduction • Established in 1999, South Korea • Handle more than half of internet search market in Korea • Have more than 8000 employees • Some apps have more than 100m users
Introduction • Have a own IDC and a public cloud service https://www.ncloud.com • However NOT OpenStack based
PASTA - IN-HOUSE PAAS 1000+ projects / 800+ daily user 10+ integrated platforms so far
PASTA – Architecture Today’s Topic platforms platforms platforms PASTA-web Shipdock (In-house docker cluster) keystone Company SSO Users cinder horizon ceph Nova Experimental
ADENDA • INTRODUCTION • MOTIVATIONS • INTEGRATE AUTHN/AUTHZ • INTEGRATE PLATFORM UI • INTEGRATE PLATFORM COMMUNICATION • CONTINOUS DEPLOYMENT WITH CUSTOMIZATION
Motivations – Too many platforms • About 40 platforms – It’s impossible even to remember URL • No single entrance/catalog • No resource utilization • No common user experience • Reinvent wheel
Motivations - authz / authn • Each platforms had its own authz/authn • Takes too much time for first access • Has different permission set • Requires even different user id/password sometimes Common problems in big company PlatformB PlatformA
AWS comes to our sight • What does AWS provide • Integrated UI/UX - consistency • Organized services catalog • Separated PaaS UI with the main UI • Centralized user management - AWS IAM • We decide to make our platforms as a PaaS like AWS
Component which enables PaaS PAAS DYNAMIC RESOURCE INTEGRATED CONSOLE PROVISIONING • Resource Provisioning on • Consistent UX demand • Integrated Authz/Authn • Docker Cluster? • Seamless integration b/w platforms https://www.slideshare.net/d eview/221-docker- orchestration
• Make new from scratch? • Start from opensouce or commercial system? • Or OpenStack …
Composable Infrastructure Decide to adopt openstack
ADENDA • INTRODUCTION • MOTIVATIONS • INTEGRATE AUTHN/AUTHZ • INTEGRATE PLATFORM UI • MAKE PLATFORMS INTEROPERATE • PACKAGE/DEPLOY WITH CUSTOMIZATION
Keystone • Authn/Authz in OpenStack • Feature • Configurable auth/identity backend • Easy to extend by Adding plugin for Authz/Authn • Abundant API interface OpenStack Services Keystone API Policy Token Catalog Identity Assignments Credentials Backend Backend Backend Backend Backend Backend
Keystone Problem we are facing: • Need to integrate into our existing SSO • Need to identify not logged-in user as well • Want to avoid to save user’s ID/PW in our DB HORIZON 3) connect to ENDPOINT with X-AUTH-TOKEN 1) ID/PW 2) issue X-AUTH-TOKEN PROJECT OPENSTACK KEYSTONE COMPONENT 4) ask the X-AUTH-TOKEN info (PROJECT ID + ROLE + USER)
extended Keystone v1 PASTA-WEB 4) ACCESS WITH X-AUTH-TOKEN (IN-HOUSE CONSOLE) 0) OAUTH Auth OAUTH2 PROVIDER 1) ID/OAUTH-TOKEN 3) ISSUE X-AUTH-TOKEN or ID/PASSWORD 5) verify X-AUTH-TOKEN KEYSTONE PLATFORMS 1.1) verify OAUTH-TOKEN 2) GET IDENTITY 1.2) verify PASSWORD AUTH PLUGIN LDAP IDENTITY PLUGIN IN-HOUSE LDAP USING COMPANY’S SSO USING COMPANY’S LDAP FOR IDENTITY
Auth Plugin PASSWORD AUTH default identity auth AUTH PLUGIN Success DEFAULT LOGIN AUTH Fail Success SSO LOGIN AUTH SSO HTTP API /api/Auth/tokenInfo to verify token Fail LOGIN FAILED
Auth Plugin • Keypoint is… ü Treat SSO token as password ü Try default auth method first. If failed, use auth using SSO next ü Extends auth handler Keystone.auth.plugins.password.Password • Keystone Configuration
Extended Keystone v1 PASTA-WEB (IN-HOUSE CONSOLE) 0) OAUTH Auth OAUTH2 PROVIDER 1) ID/OAUTH-TOKEN 3) ISSUE X-AUTH-TOKEN 4) ACCESS WITH X-AUTH-TOKEN or ID/PASSWORD 5) verify X-AUTH-TOKEN KEYSTONE PLATFORMS 1.1) verify OAUTH-TOKEN 2) GET IDENTITY 1.2) verify PASSWORD AUTH PLUGIN LDAP PLUGIN Problem • Do not have right to save OpenStack system IN-HOUSE LDAP users in LDAP • Deadly slow when retrieving all users.
Extended Keystone v2 PASTA-WEB (IN-HOUSE CONSOLE) 0) OAUTH Auth OAUTH2 PROVIDER 1) ID/OAUTH-TOKEN 3) ISSUE X-AUTH-TOKEN 4) ACCESS WITH X-AUTH-TOKEN or ID/PASSWORD 5) verify X-AUTH-TOKEN KEYSTONE PLATFORMS 1.1) verify OAUTH-TOKEN 2) GET IDENTITY 1.2) verify PASSWORD HYBRID IDENTITY PASTA PLUGIN AUTH HANDLER (AUTH + IDENTITY) IN-HOUSE LDAP SQL INTRODUCE HYBRID INDENTITY PLUGIN • Save new user in SQL • Read from only SQL when querying all users
Hybrid Backend Plugin • About the auth part • Based on keystone-hybrid-backend IDENTITY AUTH ü Implement LDAP Indentity ü extending SQL Indentity Success SQL LOGIN Auth Failed Success LDAP LOGIN Auth Failed LOGIN FAILED
Hybrid Backend Plugin • Identity ü For API like get/update user just like the auth flow • Why customzied for list large users LIST_USERS ü 2000+ LDAP USER ü List all user take 10~60s in horizon Yes Filter by SQL + LDAP ü No domain concept when adopting legacy platforms USERS Name No SQL USERS • Configuration
ADENDA • INTRODUCTION • MOTIVATIONS • INTEGRATE PLATFORM AUTHN/AUTHZ • INTEGRATE PLATFORM UI • MAKE PLATFORMS INTEROPERATE • PACKAGE/DEPLOY WITH CUSTOMIZATION
Previously our platforms…. • Have each own web based management console • No consistent user experience • Implemented using various tech set • Backend : Spring/Node.js/Golang (No python…) • Framework: Backbone.js/Angular/Vue.js/React/Jquery
Openstack - Horizon Horizon Nova Nova UI Neutron Neutron UI Cinder Cinder UI Manilla Plugin Manilla BlarBlar Plugin Blar Blar Keystone
Openstack - Horizon • Not fit for NAVER • Is not working very well with large user set • Seems “”little bit”” UGLY for us • Implemented with Python + Django • Need to restart and test whenever some platform’s UI upgraded • How to evenly distribute the UI development job to each platform’s developer guaranteeing consistency? • Make UI independently developed without forcing to use specific tech set
Micro Service Architecture UI / Horizon Monolithic App Microservice Microservice Microservice Microservice Microservice Microservice Logic + Database
Micro Service Architecture - modified UI PASTA Integrator Microservice+UI Microservice+UI </> </> Microservice+UI Microservice+UI Microservice+UI Microservice+UI </> </> </> </> Logic + Database
Micro Service Architecture - modified UI integrator handles this part Each platform handles this part
Spring Cloud – Netflix ZUUL HTTP Request “pre” filters “routing” filters “post” filters “custom” filters “error” filters BACKEND SERVER
Realized Runtime Flow OAUTH-PROVIDER 7. User permission check using X-AUTH-TOKEN KEYSTONE (OPENSTACK) 3. OAUTH PlatformA 4. Service Permission Check & Issue X-AUTH-TOKEN 6. https//{{platform-host}}/platform-id/* X-AUTH-TOKEN 8. Render platform page 1. Prepare routing table 5. Decide where to route based on context path Pasta WEB ZUUL 2. Access service-id .pasta.navercorp.com/ platform-id /a.txt PlatformB 8. Final HTML Rendering PlatformC
Platform Info Extension • Be able to keep each platform endpoint info in keystone’s Service catalog and endpoints • Service(Openstack Term) = Platform (PASTA Term) Use the platform name as a context path Pick internal interface URL for routing
Platform Info Extension • Need extra room to store extra routing info • Ex) Platform Icons / Display order … • Need separate DB to store these? • Use description section with JSON
Service Info Extension • Should store the project’s extra info into keystone • Project(Openstack Term) = Service (PASTA Term) • https://blueprints.launchpad.net/horizon/+spec/support-extra-prop- for-project-and-user • OpenStack4J
ADENDA • INTRODUCTION • MOTIVATIONS • INTEGRATE AUTHN/AUTHZ • INTEGRATE PLATFORM UI • MAKE PLATFORMS INTEROPERATE • PACKAGE/DEPLOY WITH CUSTOMIZATION
UI Level Interoperability • Already be able to call the other platform’s REST API • Because all platform UI share same authn/z in a user session http://sample.pasta-host/kaleido/ PLATFORM-A PLATFORM-B PLATFORM-C $.get(“/platform-a/api/functionA”) $.get(“/platform-b/api/functionB”) $.get(“/platform-c/api/functionC”)
Backend Interoperability • Need special way to communicate each other • ex) Batch / Event Handler which run outside of the user session • ex) Run user’s platform interoperation code when event is triggered • Introduce Serverless Framework (openwisk) OPENWISK PlatformA PlatformB • Not cover detail here
ADENDA • INTRODUCTION • MOTIVATIONS • INTEGRATE AUTHN/AUTHZ • INTEGRATE PLATFORM UI • MAKE PLATFORMS INTEROPERATE • PACKAGE/DEPLOY WITH CUSTOMIZATION
Recommend
More recommend
