TRUEBRANCH METRIC LEARNING-BASED VERIFICATION OF FOREST - PowerPoint PPT Presentation

TRUEBRANCH METRIC LEARNING-BASED VERIFICATION OF FOREST CONSERVATION PROJECTS Simona Santamaria* ssimona@ethz.ch David Dao* david.dao@inf.ethz.ch Björn Lütjens* lutjens@mit.edu Prof. Dr. Ce Zhang ce.zhang@inf.ethz.ch

Motivation • Deforestation and forest degradation accounts for ~15% of all global greenhouse gas emissions [1] • International stakeholders are paying landowners for forest conservation, if they can verify it [2] www.reforestationworld.org [1] IPCC. 2019: Summary for policymakers, WWF 2 [2] UN-REDD Programme, www.goldstandard. org

Monitoring, reporting and verification process • On-ground inspection is expensive (300 USD/ha [3] ), biased, hard to scale, and corruptible • Carbon estimates by satellites can have high uncertainties and long lead times [3] Interviews with Ministry of Agriculture Peru 3

Verification with Drones • Low-cost monitoring via drones Landowner International Reported High-Resolution Stakeholder Imagery Payment for Ecosystem Services (PES) 4

Challenge • Opens up possibility of untruthfully reported imagery • Attack vectors Reported Land-Use true time wrong time true time modified image true location true location wrong location Detected high medium high high Forest Cover 5

Approach - True Branch Verification System Public Low-Resolution Imagery TrueBranch Automated Valuation International Landowner Reported High-Resolution Verification of Forest Ecosystem Stakeholder Imagery Services Payment for Ecosystem Services (PES) 6

Forest validation algorithm How to distinguish truthful imagery from untruthful imagery? • Image Registration: Matching Drone images with Satellite images Satellite image Same location, same time? Model and Classifier Yes/No Drone image 7

Nominal Metrics not able to detect attacks • Nominal distance metrics of MSE in pixels space • MSE in pixel space Satellite image ground truth wrong time wrong location MSE Drone image 8

Nominal Metrics • Nominal distance metrics of MSE in feature space • MSE in feature space Satellite image ground truth wrong time g e1 wrong location Pretrained MSE Embeddings Drone image Model g e2 9

Learned Metrics • MSE in pixel space and RESISC-45 feature space not sufficient rt-wl Satellite anchor rt-rl wt-rl MSE MSE 10

Learned Metrics • MSE in pixel space and RESISC-45 feature space not sufficient • Metric learning with triplet loss function wt-rl rt-rl Satellite anchor adv. pert. Learned distance threshold Learned 11 distance

Metric Learning with Triplet loss Training Triplet loss e1 anchor Embeddings Model positive negative e2 12

Metric Learning with Triplet loss Inference e1 same location, same time? Classifier Yes/No Embeddings Model e2 13

First Results on Dataset in Indonesia • Dataset: 10 different locations, 3 different years 14

First Results on Indonesian Dataset • Difference between satellite image and drone images rt-rl wt/wl Satellite anchor Location Distance 15

Conclusion and Further Work • Model with metric learning is able to distinguish truthfully reported imagery from untruthfully reported imagery • Model evaluation on more training and testing data to ensure high reliability • Protecting model from Adversarial perturbation • Metric learning with images from different sources with different resolutions 16

Thank you very much for your attention Simona Santamaria* ssimona@ethz.ch David Dao* david.dao@inf.ethz.ch Björn Lütjens* lutjens@mit.edu Prof. Dr. Ce Zhang ce.zhang@inf.ethz.ch