Evaluating the Spread of Climate Model Ensembles Based on Computing - PowerPoint PPT Presentation

Evaluating the Spread of Climate Model Ensembles Based on Computing Environment Selection Tom Robinson Multicore Workshop 2019

Outline • Motivation • Ensemble method • Ensemble description • Ensemble spreads and comparison • Conclusions

Motivation • Reproducibility is important • Floating point and rounding differences between runs prevents bit-for-bit reproducibility • “Climate answers” are dependent on the selection of platform/compiler (options) • What is the “model spread” due to rounding error? • Is the model spread platform dependent?

Ensemble Method • GFDL AM4 (github.com/NOAA-GFDL/AM4) • Simulate rounding error – Single random point – Initial mid-level T 10 -13 K – Different point for each ensemble member • Model run for one year

Ensembles Ensemble Name Compiler Platform Processor # of ensembles Base Production intel 16 Gaea B/H 300 AVX intel 16 Gaea B/H 100 Intel 18 intel 18 Gaea B/H 100 Cray cray Gaea B/H 95 Theta intel 16 theta KNL 118 Hera intel 19 Hera Skylake 47

Average standard deviation • Find the point-by-point standard deviation – Take a global average • Plot and compare – Point by point mean • Are the means similar? – Point by point standard deviation – Compare across ensembles • Is spread platform dependent?

Global Spread Surface Pressure 7 Base 7 Base Average spread 30 members Average spread 50 members BH/Cray BH/Cray 6 Skylake/Intel19 6 Skylake/Intel19 KNL/Intel16 KNL/Intel16 5 5 BH/Intel18 BH/Intel18 BH/intel16avx 4 4 3 3 2 2 1 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 7 Base Average spread 100 members BH/Cray 6 BH/Intel18 KNL/Intel16 5 BH/Intel18 4 3 2 1 0 1 2 3 4 5 6 7 8 9 10 11 12

Global Spread U wind 4.5 4.5 Average spread 30 members Average spread 50 members 4.25 4 4 3.75 3.5 3.5 3.25 Base 3 3 BH/Cray Base Skylake/Intel19 2.75 BH/Cray KNL/Intel16 2.5 2.5 Skylake/Intel19 BH/Intel18 BH/Intel16avx2 KNL/Intel16 2.25 BH/Intel18 2 2 JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 4.5 1.75 u Average spread 100 members u Standard Deviation of Standard Deviation 50 members 1.65 4 1.55 1.45 3.5 1.35 1.25 Base 3 1.15 BH/Cray BH/Cray Skylake/Intel19 47Skylake/Intel19 1.05 KNL/Intel16 KNL/Intel16 2.5 0.95 BH/Intel18 BH/Intel18 BH/Intel16avx2 0.85 Base 2 0.75 JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 1 2 3 4 5 6 7 8 9 10 11 12

Mean ps (base)

ps Standard Deviation

KNL-Base Mean Difference *All values within 1 standard deviation

Standard Deviation Diff (theta-base)

Standard Deviation Diff (cray-base)

Standard Deviation %Diff (KNL-base)

Standard Deviation %Diff (cray-base)

Standard Deviation %Diff (KNL-base)

Standard Deviation %Diff (skylake-base)

Base30-Base %diff

Conclusions • Ensemble means are not platform dependent • Ensemble spreads over a local region are platform/compiler dependent • You should use a large ensemble to report the error due to rounding on your computing platform. – Global Average for summary – Map of values for patterns/weaker areas