gitlab ci and docker registry
play

GitLab-CI and Docker Registry Oleg Fiksel Security Consultant @ - PowerPoint PPT Presentation

Jul 17, 2023 •129 likes •1k views

A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND GitLab-CI and Docker Registry Oleg Fiksel Security Consultant @ CSPI GmbH oleg.fiksel@cspi.com | oleg@fiksel.info | Matrix: @oleg:fiksel.info FrOSCon 2017 A BOUT I

  1. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND GitLab-CI and Docker Registry Oleg Fiksel Security Consultant @ CSPI GmbH oleg.fiksel@cspi.com | oleg@fiksel.info | Matrix: @oleg:fiksel.info FrOSCon 2017

  2. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND A GENDA A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND Q & A

  3. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND A BOUT ME ◮ Security Consultant @ CSPI 1 (former MODCOMP 2 ) ◮ Main topics ◮ Architecture ◮ Development cycle ◮ Perl Coding 1 About CSPi 2 Wikipedia: MODCOMP

  4. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND G OALS OF THIS TALK

  5. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND G OALS OF THIS TALK ◮ This is not a comparision of CI tools

  6. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND G OALS OF THIS TALK ◮ This is not a comparision of CI tools ◮ Provide an overview of dependencies needed to deploy GitLab-CI Community Edition and Docker Registry on-premise

  7. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND G OALS OF THIS TALK ◮ This is not a comparision of CI tools ◮ Provide an overview of dependencies needed to deploy GitLab-CI Community Edition and Docker Registry on-premise ◮ Disclamer: The means and methods presented are my own expirience

  8. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND G IT L AB 101

  9. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W HAT IS G IT L AB ?

  10. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W HAT IS G IT L AB ? ◮ Web-based Git repository manager and more...

  11. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W HAT IS G IT L AB ? ◮ Web-based Git repository manager and more... ◮ Started as a pet-project in 2011 and now has more then 150 employees

  12. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W HAT IS G IT L AB ? ◮ Web-based Git repository manager and more... ◮ Started as a pet-project in 2011 and now has more then 150 employees ◮ Introduced Pipelines (CI) in version 8.8 (2016-05-28)

  13. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W HAT IS G IT L AB ? ◮ Web-based Git repository manager and more... ◮ Started as a pet-project in 2011 and now has more then 150 employees ◮ Introduced Pipelines (CI) in version 8.8 (2016-05-28) ◮ GitLab is used by many organisations such as: IBM, Sony, NASA, Alibaba, SpaceX and CSPi

  14. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W HAT IS D OCKER ?

  15. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W HAT IS D OCKER ? client docker host registry docker daemon docker build containers images docker pull docker run ...

  16. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W ORDING

  17. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W ORDING ◮ GitLab Server : git repository hosting service

  18. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W ORDING ◮ GitLab Server : git repository hosting service ◮ GitLab-CI Runner : user-space daemon that executes build/tests

  19. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W ORDING ◮ GitLab Server : git repository hosting service ◮ GitLab-CI Runner : user-space daemon that executes build/tests ◮ Artifacts : build results pushed into an internal GitLab storage

  20. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W ORDING ◮ GitLab Server : git repository hosting service ◮ GitLab-CI Runner : user-space daemon that executes build/tests ◮ Artifacts : build results pushed into an internal GitLab storage ◮ GitLab Container Registry : integrated docker registry frontend

  21. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND W ORDING ◮ GitLab Server : git repository hosting service ◮ GitLab-CI Runner : user-space daemon that executes build/tests ◮ Artifacts : build results pushed into an internal GitLab storage ◮ GitLab Container Registry : integrated docker registry frontend ◮ Docker Registry : mandatory container registry service

  22. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND D EPLOYING ON - PREMISE

  23. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND C HECKLIST

  24. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND C HECKLIST ◮ 2 VMs or Rancher/Kubernetes/Mesos cluster

  25. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND C HECKLIST ◮ 2 VMs or Rancher/Kubernetes/Mesos cluster ◮ Reverse proxy/loadabalancer for SSL offload (optional) supporting HTTP 1.1 to the backend (! Lighttpd)

  26. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND C HECKLIST ◮ 2 VMs or Rancher/Kubernetes/Mesos cluster ◮ Reverse proxy/loadabalancer for SSL offload (optional) supporting HTTP 1.1 to the backend (! Lighttpd) ◮ Direct internet connection (for pulling docker images)

  27. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND C HECKLIST ◮ 2 VMs or Rancher/Kubernetes/Mesos cluster ◮ Reverse proxy/loadabalancer for SSL offload (optional) supporting HTTP 1.1 to the backend (! Lighttpd) ◮ Direct internet connection (for pulling docker images) ◮ SSL Certificates (own CA or official)

  28. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND P ITFALLS

  29. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND P ITFALLS Internal CA ◮

  30. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND P ITFALLS Internal CA ◮ Forward proxy ◮

  31. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND P ITFALLS Internal CA ◮ Forward proxy ◮ DNS split horizon (not handled in this talk) ◮

  32. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND G IT L AB -CI RUNNER ARCHITECTURE

  33. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND G IT L AB -CI RUNNER ARCHITECTURE GitLab-CI-Runner Shell Container GitLab-CI-Runner Container GitLab-CI Docker Container GitLab-CI-Runner GitLab-CI-Runner GitLab-CI-Runner

  34. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND O N - PREMISE DEPLOYMENT ARCHITECTURE

  35. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND O N - PREMISE DEPLOYMENT ARCHITECTURE hub.docker.com Pull (HTTPS) GitLab-CI Runner run GitLab pull/push git clone Docker Container GitLab-CI (HTTPS) Test, Build, etc Artifacts push (HTTPS) Docker registry (frontend) Auth read/write access auth token auth (HTTPS) (HTTPS) [separate CA] local Docker client S3 Azure Docker registry GCS (container) store blob push/pull (HTTPS) Swift

  36. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND I NTERNAL CA

  37. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND I NTERNAL CA Every GitLab HTTPS client must trust internal CA including:

  38. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND I NTERNAL CA Every GitLab HTTPS client must trust internal CA including: ◮ gitlab-ci-runner

  39. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND I NTERNAL CA Every GitLab HTTPS client must trust internal CA including: ◮ gitlab-ci-runner ◮ docker container building docker images

  40. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND I NTERNAL CA ◮ Problem: docker images are pulled from docker hub and doesn’t trust intern CA.

  41. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND I NTERNAL CA ◮ Problem: docker images are pulled from docker hub and doesn’t trust intern CA. ◮ Solution: extend all base images with internal CA and use them for building.

  42. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND S WITCH D OCKER STORAGE BACKEND 1Source

  43. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND S WITCH D OCKER STORAGE BACKEND By default, when using docker:dind, Docker uses the vfs storage driver which copies the filesystem on every run. This is a very disk-intensive operation which can be avoided if a different driver is used, for example overlay. 1 1Source

  44. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND S WITCH D OCKER STORAGE BACKEND OS Setup:

  45. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND S WITCH D OCKER STORAGE BACKEND OS Setup: ◮ add overlay to / etc / modules (Ubuntu 16.04)

  46. A BOUT I NTRODUCTION GitLab 101 Deploying on-premise Known issues E ND S WITCH D OCKER STORAGE BACKEND OS Setup: ◮ add overlay to / etc / modules (Ubuntu 16.04) ◮ modprobe overlay or reboot the system

Recommend

- GitLab

- GitLab

- GitLab CI/CD Docker DevOps- What we have NGINX + Tomcat (Java) + MySQL + Redis + SOLR

1.55k views • 71 slides
Setup docker rm $(docker ps -aq) docker network rm my_net Demo - Install and activate yum -y

Setup docker rm $(docker ps -aq) docker network rm my_net Demo - Install and activate yum -y

Setup docker rm $(docker ps -aq) docker network rm my_net Demo - Install and activate yum -y install docker s ystemctl start docker systemctl enable docker docker images - Launch pre-canned container: hello-world (busybox is another good

480 views • 4 slides
docker service is the new docker run Getting Started with Docker Clustering Mike Goelzer /

docker service is the new docker run Getting Started with Docker Clustering Mike Goelzer /

docker service is the new docker run Getting Started with Docker Clustering Mike Goelzer / mgoelzer@docker.com / @mgoelzer Docker Inc. docker service is the new docker run docker run nginx 2013-14 docker run -p 3375:2375 swarm ; 2014-15

688 views • 34 slides
Resource Saturation Monitoring and Capacity Planning on GitLab.com Andrew Newdigate, GitLab 1

Resource Saturation Monitoring and Capacity Planning on GitLab.com Andrew Newdigate, GitLab 1

Resource Saturation Monitoring and Capacity Planning on GitLab.com Andrew Newdigate, GitLab 1 Introduction Andrew Newdigate Scalability Team, Infrastructure Group , GitLab @suprememoocow gitlab.com/andrewn 2 Resource Saturation Resource

614 views • 38 slides
INTRODUCTION TO DOCKER ADRIAN MOUAT SO WHAT IS DOCKER? SIMILAR TO A LIGHTWEIGHT VM Both

INTRODUCTION TO DOCKER ADRIAN MOUAT SO WHAT IS DOCKER? SIMILAR TO A LIGHTWEIGHT VM Both

INTRODUCTION TO DOCKER ADRIAN MOUAT SO WHAT IS DOCKER? SIMILAR TO A LIGHTWEIGHT VM Both provide isolated environments Docker is much more efficient 64-bit Linux only (currently) CONTAINERIZATION A Docker container is a portable store for a

674 views • 29 slides
Docker Provider The Docker provider is used to interact with Docker containers and images. It uses

Docker Provider The Docker provider is used to interact with Docker containers and images. It uses

Docker Provider The Docker provider is used to interact with Docker containers and images. It uses the Docker API to manage the lifecycle of Docker containers. Because the Docker provider uses the Docker API, it is immediately compatible not only

553 views • 34 slides
Roadmap for Section 12.2. Registry Fundamentals Registry Structure Registry Limits Monitoring

Roadmap for Section 12.2. Registry Fundamentals Registry Structure Registry Limits Monitoring

Unit OS12: Scripting 12.2. The Registry Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze Roadmap for Section 12.2. Registry Fundamentals Registry Structure Registry Limits Monitoring Registry

397 views • 19 slides
Docker: Testing the Waters LA-UR 15-25901 1 LA-UR 15-25901 Docker: Theres No Containing

Docker: Testing the Waters LA-UR 15-25901 1 LA-UR 15-25901 Docker: Theres No Containing

Eric Holm Skyler Manzanares Yuxin (Kelly) Wang http://gerardable.com/wp-content/uploads/2014/12/docker-whale-home-logo.png Docker: Testing the Waters LA-UR 15-25901 1 LA-UR 15-25901 Docker: Theres No Containing this Whale! Free,

997 views • 28 slides
Docker Orchestration: Beyond the Basics Aaron Lehmann Software Engineer, Docker About me

Docker Orchestration: Beyond the Basics Aaron Lehmann Software Engineer, Docker About me

Docker Orchestration: Beyond the Basics Aaron Lehmann Software Engineer, Docker About me Software engineer at Docker Maintainer on SwarmKit and Docker Engine open source projects Focusing on distributed state, task scheduling,

1.29k views • 62 slides
Agenda Caching Caching Gitlab Demo Caching Demos Mirroring Caching Limitations Manual

Agenda Caching Caching Gitlab Demo Caching Demos Mirroring Caching Limitations Manual

Agenda Caching Caching Gitlab Demo Caching Demos Mirroring Caching Limitations Manual Mirroring Caching Other Registries Summary 1 / 35 @sudo_bmitch How to Use Mirroring and Caching to Optimize Your Image Registry Brandon Mitchell

837 views • 35 slides
CSE 333 Section 1 C, Pointers, and Gitlab 1 Logistics Due Friday: Exercise 0 @ 10:00 am Due

CSE 333 Section 1 C, Pointers, and Gitlab 1 Logistics Due Friday: Exercise 0 @ 10:00 am Due

CSE 333 Section 1 C, Pointers, and Gitlab 1 Logistics Due Friday: Exercise 0 @ 10:00 am Due Monday: Exercise 1 @ 10:00 am HW0 (setup Gitlab ASAP) @11 pm 2 Icebreaker! 3 attu/CSE VM Setup and Gitlab Demo 4 Attu/CSE VM Updates and gcc 9

461 views • 32 slides
Docker meets Python A look on the Docker SDK for Python pip install docker Jan Wagner

Docker meets Python A look on the Docker SDK for Python pip install docker Jan Wagner

Docker meets Python A look on the Docker SDK for Python pip install docker Jan Wagner Data Science Consultant accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de |

664 views • 29 slides
Docker Review Basic Commands docker image ls # list images currently present locally docker

Docker Review Basic Commands docker image ls # list images currently present locally docker

Docker Review Basic Commands docker image ls # list images currently present locally docker container ls # list running containers docker run <image_name> # run an image as a container docker exec <container> <command> # run a

1.1k views • 6 slides
What is a Honeymoon Registry A Honeymoon Registry is a wedding gift registry experience for

What is a Honeymoon Registry A Honeymoon Registry is a wedding gift registry experience for

What is a Honeymoon Registry A Honeymoon Registry is a wedding gift registry experience for honeymoons and destination weddings. Our online Honeymoon Registry allows brides and grooms to list anything they want to do on their

547 views • 19 slides
Docker in the EGI Docker in the EGI Federated Cloud Federated Cloud Carlos Gimeno

Docker in the EGI Docker in the EGI Federated Cloud Federated Cloud Carlos Gimeno

Docker in the EGI Docker in the EGI Federated Cloud Federated Cloud Carlos Gimeno cgimeno@bifi.es Instituto de Biocomputacin y Fsica de Sistemas Complejos info@bifi.es http://bifi.es 0. Index Introduction What is Docker? A

401 views • 14 slides
Docker for fun and profit Solomon Hykes* about Docker: "It uses Linux containers and the

Docker for fun and profit Solomon Hykes* about Docker: "It uses Linux containers and the

Docker for fun and profit Solomon Hykes* about Docker: "It uses Linux containers and the Internet won't shut up about it." (LinuxCon 2014 keynote) *Founder of dotcloud and creator of the Docker project What are Linux containers or

206 views • 19 slides
Docker for Developers Michael Hrivnak Principal Software Engineer - Red Hat Inc.

Docker for Developers Michael Hrivnak Principal Software Engineer - Red Hat Inc.

Docker for Developers Michael Hrivnak Principal Software Engineer - Red Hat Inc. @michael_hrivnak Docker is Popular Deployment gets most of the attention. 1 Developer 1 Laptop 3 Skills Exit Codes $ sudo docker run -it --rm centos:centos7

347 views • 17 slides
Containers, Docker, and Security: State of the Union 1 / 38 Who am I? Jrme Petazzoni

Containers, Docker, and Security: State of the Union 1 / 38 Who am I? Jrme Petazzoni

Containers, Docker, and Security: State of the Union 1 / 38 Who am I? Jrme Petazzoni (@jpetazzo) French software engineer living in California Joined Docker (dotCloud) more than 4 years ago (I was at Docker before it was cool! ) I built

420 views • 38 slides
Registry Principles GMTA March 15, 2017 Key Elements of Registry Principles Definition

Registry Principles GMTA March 15, 2017 Key Elements of Registry Principles Definition

Registry Principles GMTA March 15, 2017 Key Elements of Registry Principles Definition Objectives for a registry Threshold questions Data Governance Committee Well-balanced registry design Registry data use

415 views • 9 slides
One year with GitLab Catalysts Samba Team Our journey today Bootstrap GitHub

One year with GitLab Catalysts Samba Team Our journey today Bootstrap GitHub

One year with GitLab Catalysts Samba Team Our journey today Bootstrap GitHub GitLab Background and statjstjcs Your custom image here Has it been it too hard to contribute to Samba? Where did our new contributors go?

655 views • 42 slides
CSCI 4152/6509 Natural Language Processing Lab 4: Git and GitLab Tutorial Lab Instructor:

CSCI 4152/6509 Natural Language Processing Lab 4: Git and GitLab Tutorial Lab Instructor:

CSCI 4152/6509 Natural Language Processing Lab 4: Git and GitLab Tutorial Lab Instructor: Dijana Kosmajac, Tukai Pain Faculty of Computer Science Dalhousie University 5-Feb-2020 (4) CSCI 4152/6509 1 Lab Overview GitLab Web interface

552 views • 17 slides
Give Me A REST! Amanda Folson Developer Advocate @ GitLab @AmbassadorAwsum Who Am I to Judge?

Give Me A REST! Amanda Folson Developer Advocate @ GitLab @AmbassadorAwsum Who Am I to Judge?

Give Me A REST! Amanda Folson Developer Advocate @ GitLab @AmbassadorAwsum Who Am I to Judge? Developer Advocate at GitLab Average consumer of APIs and IPAs Well RESTed (you can boo at my pun) Professional conference attendee

709 views • 45 slides
USING DOCKER SAFELY ADRIAN MOUAT NLUUG 28 MAY 2015 LOT OF NEGATIVE COMMENTS ON DOCKER SECURITY

USING DOCKER SAFELY ADRIAN MOUAT NLUUG 28 MAY 2015 LOT OF NEGATIVE COMMENTS ON DOCKER SECURITY

USING DOCKER SAFELY ADRIAN MOUAT NLUUG 28 MAY 2015 LOT OF NEGATIVE COMMENTS ON DOCKER SECURITY "Containers Don't Contain" Daniel Walsh, RedHat https://opensource.com/business/14/7/docker- security-selinux "... total systemic

502 views • 48 slides
Docker@OVH with Mesos/Marathon June 28th 2016 @devatoria @brouberol Devops / Python charmer

Docker@OVH with Mesos/Marathon June 28th 2016 @devatoria @brouberol Devops / Python charmer

Docker@OVH with Mesos/Marathon June 28th 2016 @devatoria @brouberol Devops / Python charmer Devops / Golang artist The Docker ecosystem Docker Engine : run containerized processes (aka: containers) Docker Compose : run multi-container

381 views • 13 slides

More recommend