Orchestrating Security Tooling With AWS Step Functions 1 - PowerPoint PPT Presentation

Orchestrating Security Tooling With AWS Step Functions 1

Background Jules Denardou Justin Massey @Pod_Sec @jmassey09 @JulesDT @th3r3p0 Security Engineers at Datadog ● Product Security team ● Improve security of product without detrimental impact to the developers ● workflows 2

3

Goals: Developers own the security of their app 4

Goals: Developers own the security of their app Find bugs early. Fix bugs early. 5

Security Tooling and CI/CD A Love/Hate Relationship 6

Security tooling and CI/CD ● Security tools often don’t integrate with CI/CD Long running jobs ○ False positives ○ Each tool has its own way of giving a report ○ 7

Security tooling and CI/CD ● Security tools often don’t integrate with CI/CD Long running jobs ○ False positives ○ Each tool has its own way of giving a report ○ 8

Security tooling and CI/CD ● Security tools often don’t integrate with Dev Workflows Blocking deployment pipelines ○ Reports in a third party website ○ 9

Security tooling and CI/CD ● Security tools often don’t integrate with Dev Workflows Blocking deployment pipelines ○ Reports in a third party website ○ 10

Plugin Based Workflow 11

Plugin Based Workflow Triggers 12

Plugin Based Workflow Triggers Processors 13

Plugin Based Workflow Triggers Processors Outputs 14

Our Solution 15

Our Solution Github PR 16

Our Solution Webhook Github PR API Gateway 17

Our Solution Webhook Github PR Lambda Function API Gateway 18

Our Solution API Security Scanning Webhook Github PR Lambda Function API Gateway 19

Our Solution API Security Scanning API Webhook Github PR Lambda Function API Gateway 20 Github PR Comment

Issues With This Solution Time Some scans take > 30 min ● Lambdas timeout out after 300 seconds ● 21

Issues with 1st Solution Time Some scans take > 30 min ● Lambdas timeout out after 300 seconds ● 22

Issues with 1st Solution Time Some scans take > 30 min ● Lambdas timeout out after 300 seconds ● Notifications Some scans have no way to notify you when complete ● 23

Design ● First implementation 24

Design ● First implementation AWS Lambdas 25

Design First implementation ● 26

Design ● First implementation AWS Lambdas ○ 27

Design ● Alternative: AWS Step Functions 28

Design ● Alternative: AWS Step Functions State Machine ○ Amazon States Language ◻ Individual States ○ Step Function console = State Machine GUI ○ 29

Design ● States Task ○ Choice ○ Pass ○ Delay ○ Parallel ○ Success or Failure ○ 30

Design Individual State State Machine 31

Design 32

Demo ! 33

34

Design 35

Design ● Integrating Github & AWS: Incompatible by default 36

Design ● Integrating Github & AWS: Incompatible by default Github uses HMAC signed events ○ 37

Design ● Integrating Github & AWS: Incompatible by default Github uses HMAC signed events ○ AWS API Gateway uses an “Authorizer” ○ 38

Design ● Integrating Github & AWS: Incompatible by default Implement a custom authorizer as a first step of the Step function ○ Yes No 39

Design ● Integrating Github & AWS: Incompatible by default Implement a custom authorizer as a first step of the Step function ○ 40

Design 41

Design 42

Design 43

Design 44

Design 45

Design 46

Design 47

Design 48

Design 49

Design 50

Design 51

Design 52

Design 53

Design 54

Design 55

Sifting through logs... 56

57

58

Searching for an ID unique to the state machine: print(event) print(context) 59

It 60

It doesn’t 61

It doesn’t exist 62

63

Solution 1. Generate a Unique ID in the first lambda (state_id) 64

Solution 1. Generate a Unique ID in the first lambda (state_id) 2. Override `logging.Filter.filter()` 65

Solution 1. Generate a Unique ID in the first lambda 2. Override `logging.Filter.filter()` 3. Use custom filter 66

Design and Implement 67

68

Developer feedback 69

Developer feedback ● Important part of the project: the developer Developer engagement is crucial ○ 70

Developer feedback ● Important part of the project: the developer Developer engagement is crucial ○ We need to give the results in the best way possible ○ 71

Developer feedback ● Important part of the project: the developer Developer engagement is crucial ○ We need to give the results in the best way possible ○ Low-noise, easy to read, where the developer is ○ 72

Developer feedback ● Important part of the project: the developer Developer engagement is crucial ○ We need to give the results in the best way possible ○ Low-noise, easy to read, where the developer is ○ Make issues found actionable ○ 73

Developer feedback ● Important part of the project: the developer Developer engagement is crucial ○ We need to give the results in the best way possible ○ Low-noise, easy to read, where the developer is ○ Make issues found actionable ○ ● The developer is our “customer” 74

Developer feedback ● We reach out to them Send organization wide emails asking for feedback ○ Application Security focus group ○ Direct discussion with developers ○ 75

Developer feedback ● We reach out to them Send organization wide emails asking for feedback ○ Application Security focus group ○ Direct discussion with developers ○ 76

Developer feedback ● Follow their workflows Use the same CI/CD tools ○ Try to use the same technologies ○ ● Helps understanding their constraints 77

Developer feedback ● Our current solution Comment on the PR ○ This is where the dev is looking 78

Developer feedback ● Our current solution Comment on the PR ○ This is where the dev is looking ● Our plan for the future All the scans in a single comment ○ “Auto-fix” PR (when possible) created and linked for review ○ 79

Deployment 80

Deployment GitHub Authorizer ● GitHub Trigger ● Plugin that uses Go Security Scanner ● GitHub PR Commenter ● Slack Output ● 81

Deployment ● Lambdas Serverless framework ○ 82

Deployment ● Lambdas Serverless framework ○ ● IAM, Step Function, API Gateway Terraform ○ 83

Thank you! Questions? Jules Denardou Justin Massey @Pod_Sec @jmassey09 @JulesDT @th3r3p0 We are hiring: Paris, New York, and remote! 84