Microservices Lessons Learned Susanne Kaiser Independent Tech Consultant @suksr CTO at Just Software @JustSocialApps #VoxxedMicroservices @suksr
Each Journey is Different “People try to copy Netflix, but they can only copy what they see. They copy the results, not the process.” Adrian Cockcroft, AWS VP Cloud Architect, former Netflix Chief Cloud Architect #VoxxedMicroservices @suksr
Each Journey is Different Affecting Circumstances Team Structure Skillset Journey Size #VoxxedMicroservices @suksr
Each Journey is Different Affecting Circumstances Legacy Maintenance Runtime Team effort environment Structure Skillset Journey Size #VoxxedMicroservices @suksr
Each Journey is Different Affecting Circumstances Legacy Maintenance Runtime Team effort environment Structure Skillset Journey Strategy Size New Features Timeline / Milestones #VoxxedMicroservices @suksr
Background JUST SOCIAL JUST DRIVE JUST CONNECT JUST LIST JUST WIKI JUST PEOPLE JUST NEWS #VoxxedMicroservices @suksr
At the Beginning A Monolith in Every Aspect One team One technology stack Single Unit One collaboration product #VoxxedMicroservices @suksr
Background After an Evolving Time #VoxxedMicroservices @suksr
Background After an Evolving Time Productivity suffered #VoxxedMicroservices @suksr
Background After an Evolving Time Productivity suffered Usability/UX suffered #VoxxedMicroservices @suksr
Background After an Evolving Time Productivity suffered New features released slowly Usability/UX suffered #VoxxedMicroservices @suksr
Background Separate Collaboration Apps JUST DRIVE JUST CONNECT JUST LIST JUST WIKI JUST SOCIAL JUST PEOPLE JUST NEWS #VoxxedMicroservices @suksr
Background Separate, Autonomous Teams Well-defined responsibilites #VoxxedMicroservices @suksr
Background In The Long Run Organisation Software Product #VoxxedMicroservices @suksr
Background Our Motivation for Microservices Autonomous Work at different parts Deploy independently Scale independently Develop independently teams independently At different speed #VoxxedMicroservices @suksr
Decomposition Strategy Identify Bounded Contexts Loose coupling between services High cohesion within a service #VoxxedMicroservices @suksr
Decomposition Strategy Identify Bounded Contexts Loose coupling between services High cohesion within a service Bounded Context Well-defined Related behaviour business function Semantic boundary around domain model #VoxxedMicroservices @suksr
Decomposition Strategy Identify Bounded Contexts Bounded Contexts JUST DRIVE JUST CONNECT JUST LIST JUST PEOPLE JUST NEWS JUST WIKI #VoxxedMicroservices @suksr
Decomposition Strategy Co-Existing Service From Scratch JUST DRIVE #VoxxedMicroservices @suksr
Decomposition Strategy Co-Existing Service From Scratch JUST PEOPLE JUST DRIVE #VoxxedMicroservices @suksr
Decomposition Strategy Co-Existing Service From Scratch REST API JUST DRIVE Application-Service Domain-Model Monolith DB Adapter owns document state #VoxxedMicroservices @suksr
Decomposition Strategy Co-Existing Service From Scratch REST API Application-Service Domain-Model Monolith DB Adapter owns profile document owns document created by state state author #VoxxedMicroservices @suksr
Decomposition Strategy Co-Existing Service From Scratch REST API Application-Service Domain-Model Message Broker Adapter Monolith subscribe publish DB Adapter Message Broker owns profile state Events owns document state local copy of author #VoxxedMicroservices @suksr
Decomposition Strategy Co-Existing Service From Scratch New UI REST API Good approach in general, New Business Logic Application-Service but we did too many steps at once Domain-Model Domain-Event Message Broker Adapter => Not optimal to start with DB Adapter New Data Structure vs. #VoxxedMicroservices @suksr
Decomposition Strategy Incremental Top Down REST API REST Client REST API Monolith Extracting Web App #VoxxedMicroservices @suksr
Decomposition Strategy Incremental Top Down REST API Monolith uses extracted business logic Application-Service REST Client Domain-Model Monolith DB Adapter Extracting Business Logic #VoxxedMicroservices @suksr
Decomposition Strategy Incremental Top Down REST API Application-Service Domain-Model Message Broker Adapter subscribe Monolith publish DB Adapter Message Broker Events Splitting Data Storage #VoxxedMicroservices @suksr
Decomposition Strategy Which One First? Changing Frequently Greatest benefit vs. after extraction Easy to Extract Different Resource Consumption Early experiences w/ Microservices #VoxxedMicroservices @suksr
Cross-Cutting Concerns Authorization JUST DRIVE JUST WIKI Inter-service dependency Fine-grained authorization #VoxxedMicroservices @suksr
Cross-Cutting Concerns Authorization I have a new service that needs authorization. Where is the authz service I could use? Not there, yet. Sorry! Ok, then I am putting my code Ok, then I am implementing authz to the place where authz handling in my local service. exists … to the monolith. Re-implementing authz w/ every Feeding the monolith new service #VoxxedMicroservices @suksr
Cross-Cutting Concerns Handle Them Early Feeding the monolith Re-implementing authz w/ every new service Handle Cross-Cutting Concerns Early #VoxxedMicroservices @suksr
Cross-Cutting Concerns Avoid A Distributed Monolith Authz Service #VoxxedMicroservices @suksr
Cross-Cutting Concerns Avoid A Distributed Monolith translate Authz Service translate One stable translate common contract #VoxxedMicroservices @suksr
Service Interaction Request-Driven / Event-Driven command query command Message Broker Message Broker Events query Events subscribe publish subscribe publish Request-Driven Hybrid Event-Driven #VoxxedMicroservices @suksr
How To Manage Shared Data? Hybrid Model Remote query directly to source REST API Message Broker Events for notification #VoxxedMicroservices @suksr
How To Manage Shared Data? Event Driven State Transfer ProfileUpdatedEvent Message Broker Local copy of profile data Events for data duplication #VoxxedMicroservices @suksr
How To Manage Shared Data? Source Of Truth Multiple sources of truth Single source of truth Internal source of truth Event Log Events as first-class citizens External source of truth “Traditional” Event-Driven System Event Store #VoxxedMicroservices @suksr
Message Broker Event Log Event Stream Messaging System Storage System Streaming Platform #VoxxedMicroservices @suksr
How To Manage Shared Data? Kafka Streams Unbounded, ordered sequence of data records Continuously Key-value pair updating #VoxxedMicroservices @suksr
How To Manage Shared Data? Kafka Streams Running in the same process Service of Microservice Loaded on startup of Microservice Kafka Topic Lightweight embedded state store (disk-backed) Streams make data available wherever it’s needed #VoxxedMicroservices @suksr
How To Manage Shared Data? Kafka Streams Kafka Stream-Table Duality Kafka Streams API join filter KStream KTable group by Changelog of state changes Snapshot of the latest value for each key aggregate (key1, value1), (key2, value2), (key 1, value 3) key1 value3 → key2 value2 → etc. #VoxxedMicroservices @suksr
How To Manage Data? Materialized Views w/ Kafka Streams Document Service REST API Stream Materialized View as State Store Stream-Table-Join Kafka Table for enrichment #VoxxedMicroservices @suksr
How To Manage Shared Data? Event Streams as a Shared Source of Truth Event streams as a Events for notification Events for data duplication shared source of truth Eliminating local copy => reduces Simple integration Simple integration Eliminating remote query => ● ● ● ● duplicating effort Remote query => increasing coupling Remote query => increasing coupling better decoupling ● ● Pushes data to where it’s needed Local copy => better autonomy ● ● Increases pluggability Duplicating effort to maintain ● ● Low barrier to entry for new local dataset ● service #VoxxedMicroservices @suksr
Infrastructure Complexities CI/CD Pipeline Backup Recovery Test Checkout Build Deploy SCM Config-Mngmt. Monitoring Log Aggreation Metrics Health Checks Distributed Tracing Timeout-Handling Retries Idempotency Bulkheads Circuit Breaker Message Broker Service Discovery Load-Balancer API-Gateway API Virtualization Hardware O/S Container Runtime Data Store µService #VoxxedMicroservices @suksr
Build the things that differentiate you Offload the things that don’t #VoxxedMicroservices @suksr
Managed Services µService Data Store Offload by getting common Runtime building blocks managed by cloud providers Container Orchestration O/S O/S Virtualization Hardware
Recommend
More recommend
