Security and Privacy Implications of URL Shortening Services - PowerPoint PPT Presentation

Security and Privacy Implications of URL Shortening Services Alexander Neumann, Johannes Barnickel, Ulrike Meyer IT Security Research Group RWTH Aachen University Germany May 26th, 2011 Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 1

Introduction • Alexander Neumann ◮ Computer science student, RWTH Aachen University, Germany ◮ Working as Penetration Tester for RedTeam Pentesting GmbH • Johannes Barnickel ◮ Research Advisor ◮ Ph.D. Student, IT Security Group, RWTH Aachen University Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 2

Definitions • Short URL does not exceed 35 characters • Shortened URL belongs to a shortening service • Corner cases: ◮ Shortened URL but not short: http://urlshorteningservicefortwitter.com/4czyx ◮ Short URL but not shortened: http://www.google.de/search?q=IEEE Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 3

The Problem • Long URLs tend to wrap (especially in mail) • Twitter is restricted to 140 characters • URLs in books should be short • User tracking and statistics • “Solution”: URL shortening services (USS) Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 4

Technical Description 1. User posts long URL to service http://itsec.rwth-aachen.de/research 2. Service generates short URL http://nvg8.it/8b896c 3. Other user requests short URL 4. Server responds HTTP 301 or 302 5. Other user is redirected to long URL Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 5

Downsides • Link destination is not transparent • Reachability of the service is not guaranteed • USS can silently stop working ⇒ All shortened URLs defunct • Request is delayed when connection to USS has high latency • Service might be hacked and redirect to malware • Service might be hostile and serve malware to vulnerable browsers • Secret URLs submitted to USS cannot be deleted • . . . Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 6

Facts many users do not know • URL service accumulates click statistics and creates profiles of users • Short links might vanish after a few years • When the USS dies, all shortened URLs are dysfunctional • When the USS is hacked, links might point to other websites (malware, porn, violence, . . . ) • Short URLs can be enumerated ⇒ All submitted URLs are public(!) Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 7

Research Objectives • Analyze risks for clients, servers, and privacy of users • Empirical studies: 1. Determine popular USS on Twitter 2. Analyze the use of USS in spam 3. Test for malicious services 4. Analyze user tracking abilities 5. Enumerate shortened URLs 6. Submit honeypot URLs to USS 7. Test latency and availability of popular USS Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 8

Preliminaries: List of USS Sources: • Firefox add-on ShortenURL • List of USS on longurl.org (URL expansion service) • Lists of USS on several blogs • Hostnames of URLs in Spam E-Mails Leads to list of 610 USS (distinct host names), includes 527 general purpose USS. Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 9

0lv.ru clickmeter.com fwib.net kurl.nu nn.nf retwt.me snipurl.com twhub.com urlz.at 0rz.tw clickthru.ca get.sfu.ca kurzurl.net notifyurl.com rickroll.it snkr.me twip.us urlzen.com 1link.in cli.gs get- k.vu notlong.com r.im snurl.com twirl.at usat.ly 1url.com clk.my shorty.com l9k.net not.my ri.ms sokrati.ru twitclicks.com use.my 23o.net cl.lk get- lanjut.in n.pr riz.gd song.ly twitterpan.com u.to 2big.at cl.ly url.com lat.ms nsfw.in rmse.ru sp2.ro twitterurl.net vb.ly 2.gp clop.in gizmo.do liip.to nutshellurl.com rnk.me spedr.com twitterurl.org vdirect.com 2.ly coge.la gkurl.us liltext.com nvg8.it rnm.me sqrl.it twittu.ms v.gd 2su.de c-o.in gl.am lin.cr nxy.in rt.nu srnk.net twiturl.de vgn.am 2tu.us conta.cc go2cut.com lin.io nyti.ms rubyurl.com srs.li twtr.us vi.ly 2ya.com cort.as go2.me linkbee.com oboeyasui.com ru.ly starturl.com twurl.cc vl.am 2ze.us cot.ag go.9nl.com linkbun.ch oc1.us rurl.org sturly.com twurl.nl voizle.com voomr.com 301.to crks.me good.ly linkee.com odun.net rww.tw su.pr u28.de 301url.com crum.pl goo.gl linkl.ru omf.gd s4c.in surl.co.uk u6e.de vtc.es 307.to ctvr.us go.qb.by linkslice.com om.ly s7y.us surl.hu u76.org w3t.org 3.ly curio.us goshrink.com link.toolbot.com omoikane.net safe.mn ta.gd ub0.cc w55.de 4ms.me cut.im gourl.gr linxfix.com on.cnn.com sai.ly tbd.ly uforgot.me wapo.st 4sq.com cutt.us go.usa.gov liteurl.net on.mktw.net sdut.us t.co uik.in wapurl.co.uk 4url.cc cuturls.com gowat.ch liurl.cn ooqx.com services.digg.com tcrn.ch uiop.me webalias.com 5.gp daa.li g.ro.lt lnk.by orz.se sfu.ca tgr.me ulimit.com weturl.com 6url.com dai.ly gurl.es lnkd.in ow.ly shar.es tgr.ph ulu.lu w.hurl.ws 7.ly deadsmall.com gzurl.com lnk.gd o-x.fr sharetabs.com thesurl.com u.mavrev.com wipi.es 9mp.com decenturl.com hao.jp lnk.in parv.us shim.net thinfi.com unfake.it wp.me a2n.eu df9.net hex.io lnk.ly paulding.net shink.de thnlnk.com u.nu xaddr.com a.307.to dfl8.me hhvx.com lnk.ms pd.am shmyl.com tighturl.com updating.me xeeurl.com aa.cx digbig.com hiderefer.com lnk.nu pendek.in shorl.com timesurl.at ur1.ca xil.in abbrr.com digg.com hmm.ph lnk.sk pic.gd shortenurl.com tiniuri.com urizy.com xlurl.net abcurl.net digipills.com ho.io lnkurl.com piko.me shorterlink.com tini.us url360.me xr.com adf.ly disq.us hop.im ln-s.net ping.fm shorterlink.co.uk tinurl.mobi url4.eu xrl.in adjix.com dld.bz hosturl.com ln-s.ru pipes.yahoo.com short.ie tiny123.com url9.com xrl.us ad.vu dlvr.it hotredirect.com lookleap.com piurl.com shortio.com tinyarro.ws urlac.com x.se afx.cc dn.vc href.in low.cc pli.gs shortlinks.co.uk tiny.by url.ag xsm.us a.gd doiop.com hsblinks.com l.pr plumurl.com shortn.me tiny.cc urlao.com xs.to a.gg do.my htxt.it lru.jp plurl.me short.to tinylink.com url.az xurl.es aisr.us dopen.us hub.tm lt.tl p.ly shorturl.com tinylink.in urlbee.com xurl.jp all.fuseurl.com durl.me huff.to lurl.no pnt.me shorturl.de tiny.ly urlbit.us x.vu alturl.com durl.us hulu.com macte.ch politi.co shoturl.us tiny.pl urlborg.com xxsurl.de amzn.to dwarfurl.com hurl.it makeashorterlink.com poprl.com shout.to tinypl.us urlbrief.com y.ahoo.it a.nf dy.fi hurl.me makeitbrief.com post.ly show.my tinyuri.ca urlcorta.es yatuc.com

Results Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 11

Determine Popular USS on Twitter • Base: Two samples of Twitter messages (24 hours each, max 10 %) • Results: ◮ 7.5 million / 8.7 million messages ◮ 1.2 million / 1.1 million URLs ◮ 553,320 / 431,636 shortened URLs • Top ten services (cover 96% of all USS on Twitter): 1. bit.ly / j.mp 6. dlvr.it 2. t.co 7. is.gd 3. tinyurl.com 8. migre.me 4. goo.gl 9. dld.bz 5. ow.ly 10. lnk.ms Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 12

Top ten services 600,000 100% Rest Top 9 Proportion of shortened URLs bit.ly Number of shortened URLs 500,000 80% 400,000 60% 300,000 40% 200,000 20% 100,000 0 0% Sample 1 Sample 2 Sample 1 Sample 2 Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 13

USS in Spam • Spam e-mails from SCHNUCKI project • 7.9 million e-mails collected since 2003 • 12.8 million URLs, 0.3 % shortened URLs • Query shortened URLs and analyze response code • Calculate spam detection rate for relevant services Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 14

Spam detection rate of USS 100% 80% 60% 40% 20% 0% is.gd tinyurl.com snipurl.com bit.ly moourl.com su.pr migre.me tiny.cc redir.ec urlpass.com Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 15

Malicious USS Attack scenario: Setup a fast and attractive USS, serve all requests normally, but after a while start sending vulnerable browsers to malware sites. Analysis: • Query shortened URLs seen on Twitter for 187 different USS with 83 different User-Agent strings • Analyze HTTP response Location header • Result: No malicious behaviour found • But: One service handles browsers different Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 16

User-Tracking USS • Data from previous experiment • Analyze HTTP response Set-Cookie header • Results: ◮ 65 USS set cookies ◮ 38 USS set persistent cookies ◮ 28 USS set persistent cookies with validity period 6 months or more • Define Q as quotient: #all unique values received for the cookie #all values Alexander Neumann, Johannes Barnickel, Ulrike Meyer RWTH Aachen University 17