Todd Stavish, In-Q-Tel CosmiQ Works SpaceNet Overview Inspiration - PowerPoint PPT Presentation

GTC 2017 Todd Bacastow, DigitalGlobe Radiant Todd Stavish, In-Q-Tel CosmiQ Works

SpaceNet Overview Inspiration Components Datasets Competitions • 1 st Release (8/16) • 1 st Competition Inspired by 1. Datasets 50cm 8-band over Rio de Completed 12/16 ImageNet Publicly available Janeiro • 2 nd Competition satellite imagery & • 2 nd Release (1/17) Launched on 3/20 labeled data Points of Interest (POI) 2. Competition over Rio Public challenges • 3 rd Release (2/17) against remote 30cm 8-band over Las sensing problems Vegas, Paris, Shanghai & Khartoum

Source: https://cdn.pixabay.com/photo/2016/04/10/19/20/colored-pencils-1320548_1280.jpg

Source: DigitalGlobe, Inc.

Source: https://commons.wikimedia.org/wiki/File:Openstreetmap_logo.svg

The data management challenge 10

How do we get 100 PB into the cloud? Home broadband: 300 years X 1,400 DirectConnect: 6-18 months ($$$) 11

… or a bigger “snowball” – a Snowmobile 12

13

SpaceNet Datasets SpaceNet on AWS is an open repository of 5,700+ km 2 of satellite imagery and 520,000+ vectors made available to developers to enable geospatial machine learning. Rio de Janeiro Rio de Janeiro Las Vegas, Paris, Khartoum, Buildings Points of Interest (POIs) and Shanghai Buildings Released August 2016 Released January 2017 Released February 2017 14 Imagery: 50 cm WV-2 mosaic Imagery: 50 cm WV-2 mosaic and Imagery: 30 cm WV-3 image strips 8-band MSI covering 1900 km 2 and 8-band MSI covering 3,880 km² POIs: 120,155 individual POIs from 460 feature classes Building Footprints: 220,594 Building Footprints: 221,376 covering 252 km 2 Released with NGA support

Rio Public Data Set Rio de Janeiro, Brazil Imagery: 50cm WV-2 mosaic + 8-band MSI covering 1900 km 2 Building Footprints: 220,594 covering a 252 km 2 AOI https://aws.amazon.com/public-data-sets/spacenet/ 13 14 15

Rio Points of Interest Dataset • 12 datasets with 35 unique layers containing more POI Datast Includes than 120,000 individual points of interest • Subset of 11,114 points across 139 features that have been identified as discernable in the provided satellite imagery Public Facilities • Released in GIS (geodatabase) and machine learning friendly formats (parsable JSON) • Provides quality estimation attributes (e.g. Utilities confirmation and resolution) • Introduces the concept of an object hierarchy akin to ImageNet’s use of WordNet (e.g. infrastructure- >buildings->apartments) Transportation

Rio POI Dataset

Newly Released Public Data Sets Imagery: 30cm WV-3 single strip images + 8-band MSI Total Building Footprints: 221,376 covering a 3,880 km 2 AOI across for 4 additional cities: Las Vegas, Paris, Shanghai, and Khartoum Shanghai Khartoum Las Vegas Paris 270 km 2 1,560 km 2 1,170 km 2 800 km 2 109,807 16,663 69,433 25,463 Footprints Footprints Footprints Footprints 302GB 373GB 69GB 402GB Raster Data Raster Data Raster Data Raster Data SpaceNet | March 2017 18

Lowering the Barrier of Entry for SpaceNet • SpaceNet contains a massive amount of labeled data in GeoJSON files, an unfamiliar format for most data scientists. • We released code to transform these labels into a multitude of other formats (NumPy arrays, image masks, etc.) more conducive to machine learning. * Naïve approach yields F1=0.57 Imagery Courtesy of DigitalGlobe Imagery Courtesy of DigitalGlobe Imagery Courtesy of DigitalGlobe 19

crowdsourcing.topcoder.com/spacenet 20

SpaceNet Challenges The SpaceNet Challenge is a series of coding competitions with cash prizes that make use of SpaceNet on AWS datasets to accelerate geospatial machine learning. Automated Mapping Automated Mapping High Revisit Activity Challenge - Round 1 Challenge - Round 2 Detection Challenge Nov. – Dec. 2016 March – May 2017 Mid-2017 Las Vegas, Paris, Khartoum, Shanghai Rio de Janeiro Imagery will show places with economic indicators and focus Building extraction w/ 2x Building extraction on activity-based analytics performance $35,000 in prizes $15,500 in prizes

SpaceNet Challenge Metric and Scoring • Metric was an IoU comparison with a threshold o IoU(A,B) = area(A intersection B) / area(A union B) • Top public leaderboard F1 score was 0.255 o precision = TP / (TP + FP) o recall = TP / (TP + FN) o F1= 2 * precision * recall / (precision + recall) Source: Walber (Own work) [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0), via Wikimedia Commons. https://commons.wikimedia.org/wiki/File%3APrecisionrecall.svg 22

SpaceNet Challenge - Round 1 Challenge Evaluation Cash Prizes Competition focused on Results were evaluated with $ 35,000 in prizes were paid automated feature scientifically grounded to the top performing teams metrics (F1 Score) extraction International Competitors Winning Result Submissions Top 5 submissions F1 Score of 0.255 42 competitors 242 submissions were international from Brazil worldwide Competition Timeline 10/24 10/31 11/7 11/14 12/8 12/22 Pre- Training Data Google OnAir Match Began Competition Winners Registration + Visualizer Hangout w/ (3-Week Ends Announced Released SpaceNet Competition) Experts The relatively low F1 scores of the winning submissions indicate that automated building footprint extraction remains a challenging problem that warrants further research 23

SpaceNet Challenge – Round 1: Winning Solution • The winning implementation was developed by a Brazilian Topcoder • Implementation was custom and used random forests with brute force polygon search • Results of the first challenge were promising given limited time and use of an early training dataset • More information Summary of approach: CosmiQ Works blog “ SpaceNet: 1. Classify pixels into 3 categories: Winning Implementations and New border, inside a building, and other. Imagery Release” 2. Based on individual pixel classification, generate candidate polygons that may contain buildings 3. Evaluate polygon candidates to select those with a confidence above a given threshold; discard remaining polygons 24

Round 1 Winning Solution

SpaceNet Challenge - Round 2 Challenge Evaluation Cash Prizes Competition on footprint Highest F1 per city and Up to $15,500 in prizes to extraction over four averaged across all cities be paid to the top diverse cities performing teams Competition Timeline (Estimated) 2/17 3/20 4/1 5/23 5/31 Training Data Match Began Competition Winners Early Released (9-Week Ends Announced Incentive Competition) Awarded 26

SpaceNet Challenge Round 2 Early Results • F-score: ~0.6, average of all four cities • Improvements in imagery resolution and vector labels • Higher F-scores in Round 2 initially seems to be directly related to better training data - imagery and labels

How to Get Involved 1. Utilize SpaceNet on AWS data for research • Use the data to train models for research or commercial uses • Publish open source code, blog posts, and research papers 2. Participate in current/future SpaceNet Challenges • SpaceNet Challenge Round 2 is live • Tell your friends 3. Contribute/sponsor future open data releases • Looking for new participants to contribute to the release of additional data sets • The data must have an ‘open’ license and come ‘prepared’ 28

Thank You