Development of New Standards for OAIS Interoperability Open - PowerPoint PPT Presentation

Development of New Standards for OAIS Interoperability Open Repositories 2019 June 2019 Mike Kearney KearneySolutions@gmail.com From the CCSDS DAI WG …and the Google Digital Vellum Project …and the Space Infrastructure Foundation Supporting Co-authors: David Giaretta, PTAB Ltd.; DAI WG co-chair John Garrett, Garrett Software; DAI WG co-chair Steve Hughes, NASA JPL CalTech; DAI WG Architecture Document Book Editor

Intro to CCSDS Consultative Committee for Space Data Systems  Started in 1982 developing data interoperability standards for space programs.  CCSDS is organized by space agencies, but inclusive of other (non-space) agencies, industry and academia  About 22 working groups, one of which is the DAI WG  Data Archive Interoperability Working Group  Focused on long-term Digital Preservation Archives  Developed the Open Archival Info System (OAIS) Reference Model  When OAIS became broadly adopted, CCSDS and DAI became even more inclusive of non-space preservation orgs (libraries, archives)  CCSDS and DAI standards are procedurally adopted by and published by ISO (CCSDS = ISO TC20/SC13) 2

OAIS and the next steps  Initially the DAI WG worked on processes for digital preservation.  That resulted in the OAIS Reference Model first published in 2002  Subsequently set up ISO certification process for trustworthy repositories, Continued to optimize OAIS Processes in 5-year reviews.  Result - OAIS is widely accepted and practiced among national archives and libraries around the world.  Now: DAI is starting work on OAIS-based interoperability standards for archives  Google’s Vint Cerf approached us and advocated stronger focus on technical interoperability for preservation archives.  Purpose of this presentation:  Peer review/feedback on the concept.  Invitation to participate in this next phase of work. 3

New Architecture Concept  OAIS Reference Model and Certification Processes will not change  Enables trustworthiness of important digital preservation archives regardless of implementations or the interoperability component.  Referred to here as the OAIS Process Framework (OAIS-PF) or OAIS Reference Model (OAIS-RM)  Proposed new specifications for technical implementable standards for interoperability are referred to here as the OAIS Interoperability Framework (OAIS-IF)  OAIS-IF supports OAIS-PF, but OAIS-IF is not required for OAIS Compliance.  OAIS-RM archives are trustworthy, just not necessarily OAIS-IF interoperable  “Core” portions of the architecture will support all disciplines.  “Protocols/plug - ins/APIs” will be adapted to archives for specific disciplines (“Designated Communities” in OAIS terminology)  OAIS-IF interfaces can work in parallel with existing archive interfaces. 4

Basic Concept – Layered Architecture = Part of OAIS-IF  User Interfaces Discipline application and interfaces  Producer, Consumer  Abstraction Layer* C P  Modularizes  Interoperability  Archive Interfaces Abstraction Layer  “Plug - ins” for specific Space Science Housekeeping applications Enterprise Data Spacecraft Web / HTML Documents Astronomy Geology  For specific designated communities  For specific data formats Archive 5

Basic Concept – Applicability and Development  Core capabilities for all archives Discipline application and interfaces developed in CCSDS/ISO  Unique plug-in bindings for C P spaceflight developed in CCSDS  Common data types  Developed elsewhere Abstraction Layer  Discipline-unique capabilities Space Science Housekeeping Enterprise Data Spacecraft Web / HTML Documents Astronomy  Developed by and within each Geology discipline Archive 6

Basic Concept – Cross Discipline Capabilities 1. Researcher in their own discipline C P (e.g. Heliophysics) Plug-in Libraries 2. Researcher crossing • CCSDS SANA? • IIPC Online? disciplines to study • RDA Online? Abstraction Layer how sun affects climate change (1) Heliophysics plug-in (2) Earth Obs plug-in 3. Researcher needs another data type Election Results (HTML) Archive 4. Researcher crossing disciplines to study Web / HTML how climate change Archive Earth Observation Heliophysics effects elections Archive Archive 7

Overall OAIS-IF System Architecture Producer Archive Consumer Archive Application Application Producer Archive I/F Consumer Archive I/F Archive Abstraction Layer Science Data Engineering Web/HTML Data Enterprise Data Web/HTML Doc/PDF Binding Binding Binding Binding Data Binding Binding Web Doc/PDF Science Data Engineering Data Archive I/F Archive I/F Archive I/F Archive I/F Web Doc/PDF Science Data Engineering Archive Archive I/F Data Archive Archive

System Executing Typical OAIS Functions SIP DIP Producer Archive Consumer Archive SIP = Submission Info Pkg Application Application DIP = Dissemination Info Pkg Producer Archive I/F Consumer Archive I/F Archive Abstraction Layer Science Data Engineering Web/HTML Data Enterprise Data Web/HTML Doc/PDF Binding Binding Binding Binding Data Binding Binding Web Doc/PDF Science Data Engineering Data Archive I/F Archive I/F Archive I/F Archive I/F Web Doc/PDF Science Data Engineering Archive Archive I/F Data Archive Archive AIP = Archive Info Pkg AIP

Specific Standards To Be Developed CAIS Producer Archive Consumer Archive Application Application CAIP PAIS PAIP PAIS – Producer Archive Producer Archive I/F Consumer Archive I/F Interface Specification AAL Archive Abstraction Layer PAIP – Producer Archive Binds to lower-layer Interface Protocol Plugins for Archive Science Data application layer functions Engineering Web/HTML Data Enterprise Data Web/HTML Doc/PDF Binding Binding Binding Binding Data Binding CAIS – Consumer Archive Binding Interface Spec Multiple Future Bindings/plugins For many various CAIP – Consumer Archive Archive types. Including vendor- Interface Protocol unique archives or datastores. Web Doc/PDF Science Data Engineering Data AAL – Archive Abstraction Archive I/F Archive I/F Archive I/F Archive I/F Layer Web Doc/PDF Science Data Engineering ADD Archive Archive I/F Data Archive Archive ADD – Architecture Overall System Description Description Document

More Detailed Draft Concept Standardized Archive System Architecture Archive Consumer Application Archive Consumer Archive Producer Application Application Archive Producer I/F Archive Consumer I/F User Systems Archive Abstraction Layer Archive Abstraction Layer Archive Doc/PDF Web/HTML Enterprise Data Science Data Other TBD Archive Doc/PDF Web/HTML Enterprise Data Science Data Other TBD Common Calls Binding Binding Binding Binding Bindings Common Calls Binding Binding Binding Binding Bindings Communications Layers Communications Layers Archive Common Calls Doc/PDF Web/HTML Enterprise Science Other Archive Common Calls Doc/PDF Web/HTML Enterprise Science Other Archive Common Calls Doc/PDF Web/HTML Enterprise Science Other Archive Common Calls Doc/PDF Web/HTML Enterprise Science Other Data Data Representation Representation Information Information Preservation Preservation Object layer Description Info Object layer Description Info Archive Systems Access Rights Provenance Info Reference Info Access Rights Provenance Info Reference Info Context Info Context Info Fixity Info Semantic layer Fixity Info Semantic layer Info Info Structure layer Structure layer Bit layer Bit layer Archival Information Package Interface Layer Archive Preservation activities/processes

Relationship of CCSDS DAI WG standards Defines how to check that the information is being preserved Guides the collection/creation of the metadata required for long AUDIT & term use and preservation CERTIFICATION IPELTU Defines a process to transfer Defines a process to transfer Data /metadata from the archive Data /metadata to the archive OAIS RM PAIMAS CAIMAS DATA CONSUMERS DATA PRODUCERS Defines how the information should be preserved OAIS Process Framework (OAIS- PF ) OAIS Interoperability Framework (OAIS- IF ) Abstracts the details of the Underlying archive transactions Defines a spec/protocol to transfer Defines a spec/protocol to transfer Data /metadata to the archive Data /metadata from the archive AAL PAIS PAIP CAIS CAIP ARCHIVE ABSTRACTION LAYER Describes Data Architecture – how Software components fit together DA-ADD Data Archive Architecture EAST Bindings/APIs/ Description Document ENGINEERING Protocols for DEDSL ENTERPRISE TELEMETRY WEB/HTML Legend: OTHER… AAL interfaces to SCIENCE DATA Blue Book Normative, SW implementable for interoperability Archives PVL CCSDS/ISO for specific Magenta Book Normative, but not SW implementable Standards for Disciplines XFDU Representation Green Book Informative, not normative or data types Information Completed (or in systematic review) CCSDS/ISO In work DATA PRESERVERS (ARCHIVES) Standard for Future work Packaging

Overall DAI WG schedule 13

UML rendering of CCSDS DA Architecture  Architecture Description Document starting as a UML model.  “Eye chart” for offline review, as an indicator of DAI architecture methodology  This standard-development methodology should make implementation of OAIS easier for developers (hence more widespread adoption of OAIS) 14