VOParis Data Centre Pierre Le Sidaner Observatoire de Paris COSADI - PowerPoint PPT Presentation

VOParis Data Centre Pierre Le Sidaner Observatoire de Paris COSADI – Heidelberg, June 2013 1

VOParis Organisation Started 10 years ago to develop Virtual Observatory knowledge for data distribution at Observatoire de Paris  Now a thematic organisation split in project groups mixing scientists and IT engineers to develop VO projects: – Atomic and Molecular Physics – Theory – Solar system and Planetology – Heliophysics – Reference Systems – Stars & Far Universe – Interoperability, workflow and Big data – Learning and public outreach 2

VOParis Data dissemination Use of VO Protocol CS, SIA, SSA, TAP (PDAP) Use of web portal for VOParis data discovery http://voparis-srv.obspm.fr/portal/ 3

Softs & Protocols SIA – SSA – CS – PDAP have been developed in perl first, then PHP. Databases are MySql or PostgreSQL UWS is developed in PHP that talks to Torque/Maui Scheduler For TAP a first simple version has been done in PHP, then DaCHs was used : http://voparis-tap.obspm.fr/ The Registry framework is written in Python using CouchDB and ElasticSearch 4

Infrastructure 5

PUE 1.35 Power Usage Effectiveness PUE 2? 6 PUE 1.1

Container Total free cooling PUE 1.1

Data preservation – time scale Preservation : for what time scale & what future uses Preservation : for what time scale & what future uses Creation of the Paris Observatory (1667), engraving by Thibault, from a painting by Charles Lebrun. Colbert presents the members of the Science Academy to the King. The structure is the oldest active Observatory (since 1667) with a short interruption during French Revolution in 1789

Context  OAIS standard for data archive (ISO) Archive information package Data + relative information for preservation 9

Data preservation  10

Data preservation Reference information The information that identifies, and if necessary describes one or more mechanisms used to provide assigned identifiers for the Content Information. It also provides identifiers that allow outside systems to refer, unambiguously, to a particular Content Information. An example of Reference Information is an ISBN. Do ivo identifiers correspond to this ? Ex: ivo://data_provider/service#IDnumber Provenance Information. The information that documents the history of the Content Information. This information tells the origin or source of the Content Information, any changes that may have taken place since it was originated, and who has had custody of it since it was originated. Examples of Provenance Information are the principal investigator who recorded the data, and the information concerning its storage, handling, and migration 11

Data preservation Context Information The information that documents the relationships of the Content Information to its environment. This includes why the Content Information was created and how it relates to other Content Information objects. Within the VO, it was mainly presented as “provenance data model” Fixity Information. The information which documents the authentication mechanisms and provides authentication keys to ensure that the Content Information object has not been altered in an undocumented manner. An example is a Cyclical Redundancy Check (CRC) code for a file. Use classical md5 sum? 12

Data preservation Structure information + Semantic information => This outscopes the VO because the VO deals with exchange formats, not archive native formats. Package description : partly used by DAL 13

Data preservation How to standardize information for an Image Atlas Define an XML schema with all related metadata for ESO-R, SRC-J POSS-E All digitized at MAMA (Gepi) First draft model at http://voplus.obspm.fr/xml/ OAIS Standard 14

Data preservation  15

Data centre Conclusion  Performing backups is necessary and discussions are still active on open backup systems : Tape / Disk But digital preservation is one layer over systems technology and it's evolution. We must have in mind that future users should have access to full information for future uses of data.  The VO handles the problem of data distribution standards concerning both access and format. More and more data types are now handled by the VO(s). Communities are active (Solar, Planetology, Atomic & Molecular physics, Plasma physics). 16

Registry consistency  There have been some cleaning in registry content  Next time stat will be done using voparis registry when interface will be final one 17