What can we expect from grid point AROME ? Thomas Burgot (PhD - PowerPoint PPT Presentation

What can we expect from grid point AROME ? Thomas Burgot (PhD Student, CNRM/GMAP/ALGO) Supervisors : Ludovic Auger, Pierre Bénard Dynamic Day 28/05/2019

Introduction AROME 2D AROME 3D Steep slopes Conclusion Problem Two issues : • Scalability • Steep slopes A common solution ? • Grid Point approach 1/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion AROME 2D - presentation • ICI constant coefficient, SL, A-grid, mass-based coordinate, etc • No physics, idealised test cases • Spectral or grid point versions Grid point version : • Explicit diffusion • Krylov solver Stop criterion : � ( Ax − b ) T ( Ax − b ) ε = √ bb T 2/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Density current test case Spectral vs Grid Point 4 th order, N iter ≈ 10 3/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Hypothesis Parallelisation : • No MPI, no Open-MP Geometry : • 500 x 500 pts (vs 1536 x 1440 pts in operational AROME) Spectral computations : • From D t +∆ t to U t +∆ t and V t +∆ t • Implicit diffusion • RHS Grid point computations : • Derivatives in the linear operator • Krylov solver 4/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Spectral vs Grid Point Spectral AROME Grid Point AROME 297.0 297.0 47°N 47°N 293.9 293.9 290.8 290.8 46°N 46°N 287.7 287.7 284.6 284.6 45°N 45°N 281.5 281.5 278.4 278.4 44°N 44°N 275.3 275.3 272.2 272.2 43°N 43°N 269.1 269.1 266.0 266.0 3°E 4°E 5°E 6°E 7°E 8°E 9°E 10°E 3°E 4°E 5°E 6°E 7°E 8°E 9°E 10°E T80, δ t = 50 s , T = 2 h , ∆ x = 1 . 3 km , N iter ≈ 13, 8 th order 5/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Spectral vs Grid Point Difference 1.5 47°N 1.2 0.9 46°N 0.6 0.3 45°N 0.0 0.3 44°N 0.6 0.9 43°N 1.2 1.5 3°E 4°E 5°E 6°E 7°E 8°E 9°E 10°E δ ( T 80 ) , δ t = 50 s , T = 2 h , ∆ x = 1 . 3 km , N iter ≈ 13, 8 th order 6/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Sensitivity test case Difference 0.20 0.16 47°N 0.12 46°N 0.08 0.04 45°N 0.00 0.04 44°N 0.08 0.12 43°N 0.16 0.20 3°E 4°E 5°E 6°E 7°E 8°E 9°E 10°E δ ( T 80 ) , δ t = 50 s , T = 2 h , ∆ x = 1 . 3 km One more iteration in the ICI 7/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Sensitivity test case Difference 0.5 47°N 0.4 0.3 46°N 0.2 0.1 45°N 0.0 0.1 44°N 0.2 0.3 43°N 0.4 0.5 3°E 4°E 5°E 6°E 7°E 8°E 9°E 10°E δ ( T 80 ) , δ t = 50 s , T = 2 h , ∆ x = 1 . 3 km Random noise at the bottom of the atmosphere ( σ = 0 . 01 K ) at t = 0 8/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Comparisons • Comparisons between AROME and some observations : nearly identical scores after 4 hours (not shown) Perspectives : • To extend study to 8 ∗ 24 hours forecast 9/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Solver for constant coefficient SI D t + δ t = D •• � � 1 − δ t 2 B ∆ B non-symmetric matrix (boundary conditions) : GMRES method 10/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Solver for constant coefficient SI By projecting in the eigenspace of B : QD t + δ t = QD •• � � 1 − δ t 2 b m ∆ where b m ∈ [ 10 − 2 , 10 5 ] m 2 s − 2 � b m ∈ [ 0 . 1 , 320 ] ms − 1 ) ( 10/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Solver for constant coefficient SI By projecting in the eigenspace of B : � � QD t + δ t = QD •• 1 − δ t 2 b m ∆ where b m ∈ [ 10 − 2 , 10 5 ] m 2 s − 2 � b m ∈ [ 0 . 1 , 320 ] ms − 1 ) ( 1 +∆ t 2 b m π 2 / ∆ x 2 π 2 1 +∆ t 2 b m 4 π 2 / L 2 ≈ 1 + δ t 2 b m ∆ x 2 = 1 + C 2 cond ≈ ∗ where C ∗ is the CFL number 10/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Convergence behaviour ε = 10 − 8 160 140 Number of iterations required 120 100 80 60 40 20 0 0 10 20 30 40 50 60 70 80 90 Vertical mode index 11/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Numerical cost AROME operational configuration (2019) • 170 nodes on Bull SX supercomputer • output bandwidth from a node : 7 Go/s • network latency : 0.864 ms 12/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Numerical cost AROME operational configuration (2019) • 170 nodes on Bull SX supercomputer • output bandwidth from a node : 7 Go/s • network latency : 0.864 ms Without projection on vertical modes (150 iterations required) : "Total" cost : 27 . 5 + 0 . 1 ≈ 27 . 6 s With projection on vertical modes (13 iterations required) : "Total" cost : 2 . 4 + 1 ≈ 3 . 4 s 12/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Comparison with an HEVI model Cost of 1 iteration in the solver ≈ Cost of 1 acoustic time step 13/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Comparison with an HEVI model Cost of 1 iteration in the solver ≈ Cost of 1 acoustic time step δ t = 50 s , ∆ x = 1300 m , c = 350 m/s HEVI model (if we suppose CFL < 1) : ∆ t ≈ 4 s 13/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Comparison with an HEVI model Cost of 1 iteration in the solver ≈ Cost of 1 acoustic time step δ t = 50 s , ∆ x = 1300 m , c = 350 m/s HEVI model (if we suppose CFL < 1) : ∆ t ≈ 4 s SI grid point model : δ t 50 ∆ τ = = 2 ∗ 13 ≈ 2 s 2 N iter 13/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Conclusion Results • Non exact derivatives –> Order ≥ 6 • Convergence –> N iter ≈ 13 • Technical viability • Simulated computational cost seems low 14/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Linear equations without orographic terms ∂π ′ � 1 � 1 ∂ U ′ ∂ t = − RT ∗ ∂ q ′ ∂ x − RT ∗ m ∗ ∂ T ′ m ∗ ∂ q ′ ∂ x d η ′ + RT ∗ S ∂ x d η ′ ∂ x − R π ∗ π ∗ π ∗ η η S ∂ d ′ ∂ t = − g rH ∗ ∂ ∗ ( ∂ ∗ + 1 ) q ′ � η C p ∂ q ′ � ∂ U ′ 0 m ∗ ∂ U ′ � + 1 ∂ x + d ′ ∂ x d η ′ ∂ t = − π ∗ C v ∂ T ′ ∂ t = − RT ∗ � ∂ U ′ � ∂ x + d ′ C v � 1 ∂π ′ 0 m ∗ ∂ U ′ S ∂ t = − ∂ x d η 15/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Linear equations σ -coor with orographic terms ∂π ∗ ∂ U ′ ∂ t = ... + RT ∗ ∂φ ∗ ∂ x T ′ − ∂φ ∗ ∂ x q ′ − π ∗ ∂φ ∗ ∂ q ′ S + 1 ∂ x π ′ S T ∗ m ∗ π ∗ 2 ∂ x ∂η S ∂ d ′ ∂ t = ... C p ∂ q ′ π ∗ ∂φ ∗ ∂ U ′ ∂π ∗ � 1 � − 1 ∂ x U ′ + ... ∂ t = − ... + RT ∗ m ∗ π ∗ C v ∂ x ∂η ∂ T ′ ∂ t = − RT ∗ π ∗ ∂φ ∗ ∂ U ′ � 1 � ... + RT ∗ m ∗ ∂ x ∂η C v � 1 ∂π ′ 0 U ′ ∂ m ∗ S ∂ t = ... − ∂ x d η 16/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Linear equations η -coor with orographic terms � 1 � 1 ∂ U ′ � m ∗ � m ∗ ∂ � ∂ � qd η ′ − R d ∂ t = ... + R d T ∗ T ′ d η ′ π ∗ π ∗ ∂ x ∂ x η η ∂ d ′ ∂ t = ... ∂ q ′ ∂ t = ... ∂ T ′ ∂ t = ... ∂π ′ S ∂ t = ... 17/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Linear equations η -coor with orographic terms In general : ∂ X ∂ t = L ( X ) With a 2-TL discretisation : � � I − δ t X + = X • 2 L 17/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion SI scheme A I U + U • B C d + d • q + q • = T + T • π + π • S S 18/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion SI scheme A I U + U • B C Φ + Φ • = 18/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion SI scheme U + + A Φ + = U • BU + + C Φ + = Φ • We can reduce the problem to only one equation : U + = U • − AC − 1 Φ • � � I − AC − 1 B 19/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion SI scheme U + + A Φ + = U • BU + + C Φ + = Φ • In constant coefficient approach : U + = U • − AC − 1 Φ • � � I − A ∗ C − 1 B ∗ ∆ 19/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion SI scheme U + + A Φ + = U • BU + + C Φ + = Φ • With orography in σ -coordinate : U + = U • − AC − 1 Φ • � � I − AC − 1 B • Orographic idealised test cases • Identify the instability contribution of each orographic term 19/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion SI scheme U + + A Φ + = U • BU + + C Φ + = Φ • With orography in η -coordinate : U + = U • − AC − 1 Φ • � � I − AC − 1 B 19/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion Conclusion & perspectives Scalability : • Grid point approach seems viable in AROME • Grid point approach seems competitive Perspectives • To test some preconditioners ? • To remove completely spectral computations in AROME ? Steep slopes : Perspectives • To test it in AROME 2D • To measure the interest/potential 20/ 21

Introduction AROME 2D AROME 3D Steep slopes Conclusion End Thank you for your attention ! Do you have some questions ? 21/ 21