Programming the Cloud: Empowering Developers to Do Infrastructure - PowerPoint PPT Presentation

Programming the Cloud: Empowering Developers to Do Infrastructure Luke Hoban QCON San Francisco November 11th, 2019

Why do I care about this?

An Analogy What’s missing? ● Variables ● Loops ● Functions ● Abstraction ● Standard Library ● Types

Software Engineering Desiderata Application Frameworks Debugging High-Level Languages CI/CD Type Checking Source Control IDEs Package Management Unit Testing Componentization Rapid Iteration Integration Testing

Infrastructure as Code

Infrastructure as Code Text

Demo Infrastructure as Code for Developers

Process Models web server cloud Page Process Stack Transient Finite lifetime Lives “forever” Script tags ELF binaries Desired State Stateless Largely stateless Fundamentally Stateful 12

Application vs. Infrastructure

Application + Infrastructure Desired State: - There is a Fargate Task - It has 512MB memory reservation - It is running the Docker image built from ./docker-ffmpeg-thumb

Application + Infrastructure

Demo Unifying Application and Infrastructure

Transition to Cloud Native Pre-Cloud Cloud Native Lambda S3 API Gateway Docker MySQL PM2 DataDog CloudWatch DataDog EKS Aurora

Programming Architecture Diagrams

Demo Programming Architecture Diagrams

Architecture as Code

Demo Building a Cloud App in 90 Seconds

The Future? Frameworks on top of cloud provider building blocks Unifying application and infrastructure A process model native to the cloud Rapidly develop directly in the cloud “Every Developer Is a Cloud Developer”

Thanks! Program the Cloud lukehoban @lukehoban

An Analogy What’s missing? ● Variables ● Loops ● Functions ● Abstraction ● C Standard Library ● Types

Infrastructure as Code AWSTemplateFormatVersion: '2010-09-09' Description: Slack Bot Effect: Allow Parameters: BotName: Action: - 'lambda:ListFunctions' Type: String const lambda = new AWS.Lambda(); SlackToken: - 'lambda:InvokeFunction' Type: String Resource: '*' exports.handler = (event, context, callback) => { Properties: WitAIToken: Action: "lambda:InvokeFunction" Type: String botLambda: const account_id = context.invokedFunctionArn.split(":")[4]; FunctionName: !GetAtt [ botLambda, Arn ] - Type: "AWS::Lambda::Function" Outputs: Principal: "apigateway.amazonaws.com" StatusCode: 200 Url: Properties: return processEvent(event, account_id) SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${api}/*" Integration: Value: !Sub https://${api}.execute-api.${AWS::Region}.amazonaws.com/${stage} FunctionName: !Ref BotName .then(res => callback(null, formatResponse("200", Type: AWS_PROXY Handler: index.handler Resources: JSON.stringify({response_type: "in_channel", text: res })))) helpLambdaPermission: IntegrationHttpMethod: POST lambdaRole: Runtime: nodejs4.3 .catch(err => callback(null, formatResponse("400", err.message || err))) Type: "AWS::Lambda::Permission" PassthroughBehavior: WHEN_NO_TEMPLATES Role: !GetAtt [ lambdaRole, Arn ] Type: "AWS::IAM::Role" ; arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambdaArn}/invocations Uri: !Sub Properties: Timeout: 300 }; - - arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03- AssumeRolePolicyDocument: Environment: lambdaArn: !GetAtt helpLambda.Arn 31/functions/${lambdaArn}/invocations Variables: Version: "2012-10-17" function processEvent(event, account_id) { SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${api}/*" - Statement: SLACKTOKEN: !Ref SlackToken // Code for bot command goes here DependsOn: helpLambdaPermission lambdaArn: !GetAtt botLambda.Arn WITAITOKEN: !Ref WitAIToken - DependsOn: botLambdaPermission Code: Effect: Allow helpLambda: api: ZipFile: | Principal: Type: "AWS::Lambda::Function" Type: "AWS::ApiGateway::RestApi" deployment: 'use strict'; Service: lambda.amazonaws.com Properties: Properties: Type: "AWS::ApiGateway::Deployment" Action: "sts:AssumeRole" FunctionName: !Sub ${BotName}-help Name: !Ref BotName Properties: const AWS = require('aws-sdk'); Path: / Handler: index.handler RestApiId: !Ref api ManagedPolicyArns: const https = require('https'); Runtime: nodejs4.3 anyMethod: StageName: DummyStage - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" const url = require('url'); Role: !GetAtt [ lambdaRole, Arn ] Type: "AWS::ApiGateway::Method" DependsOn: anyMethod const qs = require('querystring'); Policies: Timeout: 300 Properties: - Code: AuthorizationType: NONE stage: const AWS_REGION = process.env.AWS_REGION; PolicyName: lambda ZipFile: | HttpMethod: ANY Type: "AWS::ApiGateway::Stage" PolicyDocument: const AWS_LAMBDA_FUNCTION_NAME = process.env.AWS_LAMBDA_FUNCTION_NAME; // Code for bot help goes here RestApiId: !Ref api Properties: Version: '2012-10-17' const slackToken = process.env.SLACKTOKEN; ResourceId: !GetAtt api.RootResourceId RestApiId: !Ref api const witaiToken = process.env.WITAITOKEN; Statement: botLambdaPermission: MethodResponses: StageName: bot Type: "AWS::Lambda::Permission" DeploymentId: !Ref deployment

Other Similar Approaches

Programming the Cloud Continue the march of JavaScript from Browser to Server to Cloud Apply Software Engineering to Cloud Infrastructure Work at the right level of abstraction - raw infra or “architecture diagram” Bridge the gap between App and Infra A different kind of application model - “stacks” instead of processes

Backup

Software Engineering Desiderata Application Frameworks Debugging High-Level Languages CI/CD Type Checking Source Control IDEs Package Management Unit Testing Componentization Rapid Iteration Integration Testing