Research Overview Big Data Large Structured into multiple vertical - PDF document

To create the largest structured domain object database in Asia for rapid & rich mobile search Research Overview Big Data Large • Structured into multiple vertical Database of application domains (structured) Media Objects • For machine learning & testing Search Next ‐ • Fast & rich in content Generation • Real ‐ time & contextual (Real World Object Search consumer behaviour analysis Objects) • Scalable media search, Media Cloud processing, & delivery Platform • Testbed for experimentation 2

Object Categorisation • 2D (Planar) objects : Logos, book • Faces : Genders, age groups, covers, CD covers, labels, coins profiles, ethnicity, sentiment • 3D rigid objects : Cars, hardware, product packages • Landmark & Scenery • Deformable objects : Clothes, shoes, bags, toys 3 Solution Architecture Applications Smart Advertising Tourism Education Smart Surveillance Digital Archive Transportation/Logistics E ‐ Commerce SDK Tools API API API API API Rigid Object Deformable Consumer Media Media Search Object Retrieval Analysis Analysis Processing Algorithm Visual Object Infrastructure Cloud Database 4

ROSE Partner Ecosystem Commercial Partners Green Expo Technology Mobile Search with Contextual Mobility NTU/PKU Joint ‐ Lab Large ‐ Scale Object Dataset & Analytics Media Cloud Platform Research Technology Partners Partners Supported by 5 Collaboration with Partners Top University in #1 Internet Leading Solution China Company in Asia Provider in China • Structured • Mobile Visual • Media Cloud Object Database Search Platform • Object Search • Face Recognition • Scalable Media Search (Faces, • Media Cloud • Landmark Products, Logos) Platform Search & Streetview Maps 6

Collaboration with Partners Green Expo Green Expo Technology Technology Mobile A*Star – UIUC Virtual Expo Contextual Tradeshow Research Centre Platform Advertising Assistant • Joint • Logo Search • Mobile Object • Visual Object Development Search Search • Product • Shared Search • Sentiment • Face Resources Analysis Recognition • Face • Co ‐ Recognition • Video • GPU Supervision of Analytics Computing • Sentiment Students Analysis 7 ROSE Lab: Key Milestones RCA Signed MOU Signed Released RCA Signed RCA Signed Green Expo RCA Signed Technology MOU Signed WeChat MOU Signed Image Platform Dec Feb Mar Jul Aug Oct Nov Jan Feb Apr May Sep 2012 2013 2013 2013 2013 2013 2013 2014 2014 2014 2014 2014 Grant: Visual Launch: 2 SINGA Grant: Cloud Opinion WeMage PhD Students Computing Grant: Deep Computation Challenge Grant: Learning for 2014 IDM Futures Fashion Retrieval Visit by MPEG CDVS Media Mayor, Official Draft Int’l Exploits Jinan City, launch Std 2013 8 China

Big Data Search Cloud Computing P2: P3: P1: Object Media Cloud Structured Search Platform Object Database Engines RESEARCH OVERVIEW 9 P2: P3: P1: Structured Object Database P1: Object Media Structure Cloud Search d Object Platform Database Engines • Large ‐ scale domain object database and analytics – Two ‐ layer data solutions: collected and on ‐ demand – Large ‐ scale data structure, tagging, indexing, mining and storage – Collect and link multi ‐ modal data to facilitate fast mobile search on object database • Domains – eCommerce & Contextual Advertising – Tourism & Transport – Life Style & Hobbies 10

Large ‐ Scale Structured Object Database 50M structured objects (clean) Structured Object Database (Apr 2014) #Images 8,000,000 mCommerce & 7,000,000 Tourism & Lifestyle & Digital Transport Hobbies 6,000,000 Advertising 5,000,000 Handbag Landmark Fish 4,000,000 Shoes Place of interest Fruit 3,000,000 Clothes Road signs Cat 2,000,000 … 1,000,000 Book cover Car ‐ … Trademark & Logo eCommerce Tourism Lifestyle & & Digital Hobbies Advertising … P2: P3: P1: Object Media P2: Object Search Structured Cloud Search Object Platform Database Engines • Scalable Object Search – Efficient context ‐ aware, goal ‐ driven mobile object search – User ‐ contributed live data search through social networks – Develop real ‐ time location ‐ and situation ‐ based data analytics on user’s preferences and behaviour 12

Whole Image Retrieval Query Database Ranked Images = + …… Visual Object Search Ranked Detections from Query Database Cluttered Images + = ……

Branded Bag Recognition • For people, use bag image to find bag Street Scene Online shops M40249 TWD: $3800 Artsy MM m40249 LV Artsy mm USD: 100 Wholesale fashion USD:110 Face Recognition & Retrieval Retrieving images of a person Identifying people

P2: P3: P1: Object Media P3: Media Cloud Platform Structured Cloud Search Object Platform Database Engines • Testbed – Design an innovative multimedia cloud platform as a test ‐ bed for large ‐ scale applications • GPUs – For accelerating machine learning and object search • Media Processing Technologies – Develop new media processing technologies in transcoding, visual analytics and quality assessment 17 ROSE Lab Physical Infrastructure IT Cloud HPC Cloud • P3 Cloud cluster • GPU Cloud Cluster – 7x Dell R720 2U server without GPU – 1x Dell R720 2U server with 2x K20m GPU – 84x Intel SNB Processor @2.3GHz – 3x Dell R720 2U server with 1x K20m GPU – 434GB RAM @ 1600MHz – 1x Dell R720 2U server with 1x Intel Xeon Phil MIC • Network Infrastructure 1x CISCO Catalyst 3750 ‐ x Layer ‐ 3 Switch, 48 port – (with 1 Gbps link to NTU Campus Network) Big Data – 5x CISCO Catalyst 3560 ‐ x Layer ‐ 2 IP ‐ based Switch, 48 port • P1 Database cluster – 1x HP Proliant 2U server – 1x JBOD Storage Chasis Experimental Zone – 1x D ‐ Link NAS with 16TB storage • P2 Development cluster • Storage Cluster 1x HP Proliant 2U server – – 4x Novatte 12 ‐ Bay Storage Server • Experimental GPU Platform – Up to 160TB Storage Capacity – GPU Workstations: 2 x Titan Black, 2 x GTX770 18

ROSE Systems Infographic Servers & CPUs GPUs Storage • Includes Tesla K20 & • 23 Servers & • 41 CPUs K40, Titan Black, Workstations • 244 Physical GTX770, GTX645,… • 135TB Storage cores • 37 GPUs • 488 Hyper ‐ • 42,048 CUDA threaded Logical cores cores 19 ROSE Lab ‐ Summary Capacity & Capability Intellectual Property 18 RSEs Industry Engagement 1 Draft International 32 PhD Students Standard $7.1M Co ‐ Funding 83 Students Trained 1 Provisional Patent 7 Partners 4 Technology Disclosures 19 Projects 28 Publications 2 Products & Services 20

Thank You rose.ntu.edu.sg/index.html 21