ci l o p
play

Ci L O P t SCI entific Q t application for L earning from O - PowerPoint PPT Presentation

Feb 14, 2023 •96 likes •557 views

Ci L O P t SCI entific Q t application for L earning from O bservations of P lasmas from S pace D ata Center for Data Sience Paris Saclay Groupe de Travail SPU - Donnes spatiales - meeting 1er fvrier 2016. IN THIS PRESENTATION 1. 2.

  1. Ci L O P t SCI entific Q t application for L earning from O bservations of P lasmas from S pace D ata Center for Data Sience Paris Saclay Groupe de Travail SPU - Données spatiales - meeting 1er février 2016.

  2. IN THIS PRESENTATION 1. 2. 3.

  3. S UN -E ARTH SYSTEM

  4. C ORONAL MASS EJECTIONS Soleil You are 1-2 days Here

  5. S OLAR WIND

  6. L OTS OF MISSIONS

  7. G RAPHICAL M ULTI & MISSIONS F LEXIBLE C OMMUNITY S MART L earning from S pace Automatic event D ata detection with Machine Learning

  8. TEAM Nico A. « typical user » interface with observers @ LPP H UGO W INTER - Alexis Jeandet Erwan Le Pennec CDD - 12 MOIS Main code designer Expert/consulting expert C++/GUI Main developer Machine Learning GUI Qt Signal Rodrigue Piberne Space Data products / scientific visualization

  9. G RAPHICAL M ULTI & MISSIONS F LEXIBLE C OMMUNITY S MART

  10. W HY M ULTI - MISSIONS ? S AME DATA P LASMA D ENSITY , TEMPERATURE FLOWS P RESSURES D ISTRIBUTIONS E LECTROMAG . E, B, POTENTIAL …

  11. W HY M ULTI - MISSIONS ? SAME DATA FORMAT : CDF e.g. Mission ESA/Cluster, 130TB since 2001 mission NASA/MMS, launched 2015 > 10TB/year

  12. • Multi-mission, intuitive GUI • E ASILY BROWSE DATA PRODUCTS • I NTEROPERABILITY WITH CDPP , NASA, ETC . • LOAD ASCII /CDF FILES … • S IMPLY DRAG PRODUCTS TO PLOT AREA Search data any text will be searched in product meta-data All known products Dynamically filtered data products local, distant…

  13. • R EAD CDF, ASCII… • MISSIONS P LUGINS • AMDA/NASA INTEROPERABILITY G RAPHICAL M ULTI & MISSIONS F LEXIBLE Just get data C OMMUNITY S MART

  14. G RAPHICAL M ULTI & MISSIONS F LEXIBLE C OMMUNITY S MART

  15. E XISTING TOOLS ? S CRIPTING GUI S NO: NO: VERY BAD FOR JUST DATA BROWSING NOT VERY FLEXIBLE R EINVENTING ALL WHEELS TENDENCY FOR « USINES À GAZ » L OTS OF CRAPPY CODE IN NATURE … YES: YES: E ASY DATA BROWSING B RINGS STRONG FLEXIBILITY E ASY ROUTINE TREATMENTS REQUIRED BY RESEARCH E ASY FOR STUDENTS SHARING CODE BATCH ANALYSIS

  16. E XISTING TOOLS ? S CRIPTING GUI S NO: NO: VERY BAD FOR JUST DATA BROWSING NOT VERY FLEXIBLE R EINVENTING ALL WHEELS TENDENCY FOR « USINES À GAZ » L OTS OF CRAPPY CODE IN NATURE … YES: YES: E ASY DATA BROWSING B RINGS STRONG FLEXIBILITY E ASY ROUTINE TREATMENTS REQUIRED BY RESEARCH E ASY FOR STUDENTS SHARING CODE BATCH ANALYSIS

  17. • Technology choices C++ Q T S IMPLE CODE , PORTABLE , PERFORMANCE , GOOD HUGE COMMUNITY COMMUNITY O PEN S OURCE

  18. • Multi-mission, intuitive GUI Scroll and transparently Interactive high download data perf panels easy browsing of data products real time based on update keywords specific toolboxes

  19. • Embedded iPython : power of custom toolkits (homemade or not) • Easy access to user libraries • terminal <—> plots • enable very specific data manipulation (not GUI)

  20. • ELEGANT AND ERGONOMIC • PERFORMANCE AND REAL TIME PLOTTING • INTERACT WITH DATA AND PLOTS • POWER AND FLEXIBILITY OF PYTHON SCRIPTS G RAPHICAL M ULTI & MISSIONS F LEXIBLE C OMMUNITY S MART Visualize complex data interact with data

  21. G RAPHICAL M ULTI & MISSIONS F LEXIBLE C OMMUNITY S MART

  22. « EVENT » Time interval where measures show signatures associated with a physical phenomenon of interest. Usually group them to do statistical studies

  23. • Catalogs of data Gather data for statistics • Catalog = group of data intervals • Data can belong to multiple catalogs • « add to catalog » directly from plot panels • clone/extend features

  24. • Visualizing catalogs Extract and visualize metadata ex : where are all my • Rich automatic metadata intervals located? (user, spacecraft, data products etc.) not just start/stop date and optional description • Easily extract and visualize information from your catalog

  25. • Online community based catalog (> SciQLOP v.1) Improve reproducibility - ANTI-reinventing-the-wheel-tool • Public and group catalogs • Online sharing between all SciQLOP instances • Build catalogs with colleagues

  26. • Catalogs and published studies (> SciQLOP v.1) Improve reproducibility - ANTI-reinventing-the-wheel-tool • Export to publishable additional material catalogs with custom fields • Catalog type = « published event » • Register an event as « published » and add DOI/ paper meta data • SciQLOP will let you know visually that the event you’re looking at has been published and let you easily grab the paper

  27. Share science G RAPHICAL M ULTI & MISSIONS F LEXIBLE • O RGANIZE DATA INTO CATALOGS C OMMUNITY S MART • C OLLABORATIVE CATALOGS • PUSH AND PULL PUBLISHED DATA

  28. G RAPHICAL M ULTI & MISSIONS F LEXIBLE C OMMUNITY S MART L earning from S pace Automatic event D ata detection with Machine Learning

  30. S PACE TIME AMBIGUITY d n i w r a l o s TIME VARYING BOUNDARY CONDITIONS bow shock d n i w r a l o s magnetosheath magnetopause S URFACE WAVES AND PROCESSES

  31. S PACE TIME AMBIGUITY

  32. M ACHINE LEARNING Auto select M’sphere regions M’pause M’sheath Shock Solar wind

  33. Reconnection Visual signatures detection

  34. Trenchi et al. 2008

  35. K ELVIN H ELMHOLTZ

  36. S HOCK CROSSING

  37. C OLLECTING ( AUTOMATICALLY ) DATA IS HARD • DATA IS COMPLEX , NOT REPRODUCIBLE • N AÏVE DETECTION ALGO . BASED ON FIXED RULES GIVE > 70% FALSE DETECTIONS EASIEST THING IS STILL THE EYE • EVERYONE KNOWS THE « TEXTBOOK » EXAMPLE OF OUR FAVORITE PHENOMENA REPRESENTS LESS THAN 1% OF EVENTS • P REVENTS STATISTICAL STUDIES OF PHENOMENA • H OW DO WE USE YEARS OF ARCHIVED DATA ?? • L ISTS ARE COMPILED HERE AND THERE … BAD REPRODUCIBILITY • WHAT DO WE DO WHEN WE RUN OUT OF INTERNS TO SELECT INTERVALS ?

  38. • ML from and for catalogs • Learn from catalogs • suggest new events • scan databases • Extend catalogs • Test performance

  39. • Using catalogs to do science. E.g. shock model as a function of IMF and Sw Mach nber. • What is the 3D shape/position of the shock as a function of solar wind control parameters ? bow shock magnetosheath magnetopause • build and share models based on catalogued data • Export model to 3DView (collaboration with CDPP)

  40. • Using catalogs to do science. e.g. reconnection at the magnetopause • What is the position of the X line on the magnetopause as a function of solar wind control parameters ? magnetopause • build and share models based on catalogued data • Export model to 3DView (collaboration with CDPP)

  41. CDS RAMP I CME A UTOMATIC DETECTION OF ICME S Magnetic clouds: Very geoefficient structure Huge structure lasting typically 1 day cloud Start with a discontinuity : jumps in B, V, n, T than in preceding solar wind sheath Then 2 parts: (1) sheath: large fluctuations (2) Magnetic cloud itself: - smooth variations - Smooth B rotation

  42. Learn data from/for users G RAPHICAL M ULTI & MISSIONS F LEXIBLE • L EARN FROM CATALOGS C OMMUNITY S MART • SUGGEST DATA AND EXTEND CATALOGS • BUILD COMPLEX MODELS FROM DATA

  43. Visualize complex data Just get data interact with data G RAPHICAL M ULTI & MISSIONS F LEXIBLE C OMMUNITY S MART Share science Learn data from/for users

  44. L earning from S pace • D EFINE STRATEGIES TO DETECT : D ata • REGIONS / BOUNDARIES • TAIL / M’ PAUSE / SHOCK / ETC . • SOLAR WIND EVENTS • • SOLAR WIND SHOCKS • M’ PAUSE RECONNECTION • M’ PAUSE KH • I NTEGRATION IN S CI QLOP • LEARN FROM CATALOGS • SCAN DATABASES • SUGGEST EVENTS

  45. FUTURE : Built-in SciQLOP engine?

Recommend

More recommend