Documenting cross-national comparability of survey data: the Online - PDF document

Documenting cross-national comparability of survey data: the Online Codebook & Analysis of the Generations & Gender Programme Arianna Caporali arianna.caporali@ined.fr Institut national d’études démographiques, INED Abstract In international survey projects, metadata are paramount to evaluate cross-national comparability of data. They help identify issues of conceptual or methodological equivalence between country datasets. This paper presents the ways in which metadata are provided in the framework of the Generations and Gender Programme (GGP). This is a longitudinal demographic survey of 18-79 year olds in 19 countries in Europe and beyond. It is managed by a consortium of research institutions on the basis of a decentralized model that is relatively dependent upon post hoc harmonization of data. The challenge of documenting GGP data is the need to combine information from existing surveys with information on the harmonization process. To face this challenge, metadata are provided in compliance with the Data Documentation Initiative (DDI), an international standard for documenting social science data, and are made available in an online codebook and analysis tool through the software package Nesstar. The paper illustrates the DDI elements chosen to document cross-national comparability of GGP surveys and discusses the experience of the GGP team in providing the relevant information. It ends with an outlook on possible future development. Content 1. Introduction: metadata in international survey programs ........................................... 2 2. The Data documentation Initiative and the software Nesstar ...................................... 3 3. The case of the Generations and Gender Surveys ......................................................... 3 3.1. Main characteristics and methodology ....................................................................... 3 3.2. Implications for data documentation ........................................................................... 5 4. The Generations and Gender Programme Online Codebook and Analysis ............... 5 4.1. Description of the workflow steps ............................................................................... 5 4.2. Online GGP surveys data files and their DDI metadata items ................................... 6 4.2.1. The pooled datasets .............................................................................................. 7 4.2.2. The country-specific data files ............................................................................. 8 4.2.3. The variable availability data file ....................................................................... 10 5. Conclusions ...................................................................................................................... 11 6. References ........................................................................................................................ 12 1

1. Introduction: metadata in international survey programs Survey data can be usable only if accompanied by comprehensive metadata. These are ‘data about the data’ necessary to transform the numbers into meaningful knowledge. Without metadata, data sharing and secondary analysis would not be possible, and researchers could not test and replicate each other’s studies (King 1995). In international survey programs, metadata are paramount to document data quality across countries and evaluate issues of cross-national equivalence between country methodologies and datasets (Mohler et al . 2010). This is especially crucial for survey programs based on a decentralized management model. Indeed, in these programs, survey instruments and guidelines can be either adapted to the specific country contexts or ‘incorporated’ into existing surveys. Such surveys require full documentation of country specificities in fieldwork methodologies, as well as of the harmonization procedures that are implemented to ensure data comparability. Metadata are even more important in the framework of longitudinal surveys, where subsequent waves need to be described. Metadata preparation concerns both survey producers and survey data archives (Caporali, Morisset, and Legleye 2015). Data producers create coherent data files and prepare survey documentation. Survey data archives must ensure their reusability even behind national boundaries. To this end, they review quality of data files, ensure that they are sufficiently anonymised to avoid risk of re-identification of respondents, and create exhaustive and structured metadata files following international standards. Today the standard called Data Documentation Initiative (DDI) is widely used in social sciences, both for national and for cross-national comparative surveys (Mohler et al. 2010). It is often used in connection with Nesstar (Networked Social Science Tools and Resources), a user-friendly software (Vardigan, Heus, and Thomas 2008), which can be used to prepare survey metadata according to the DDI standard and to disseminate surveys online. DDI and Nesstar are recommended by the Consortium of European Social Science Data Archives (CESSDA). What is it to document cross-national comparability? What are the documentation challenges posed by longitudinal surveys? How can they be overcome? This paper presents the ways in which metadata are prepared and disseminated in the framework of the cross- national longitudinal survey project called Generations and Gender Programme (GGP; see section 3). GGP has a decentralized management model. It therefore relies on considerable post hoc harmonization of data. The specific challenge that data documentation poses in this project is the need to combine information from already collected surveys with information on post hoc harmonization processes over subsequent survey waves. Then metadata have to follow international recommendations, be compliant to DDI standard and disseminated online through the Nesstar software. After an explanation of what DDI and Nesstar are (section 2), the paper focuses on the GGP characteristics and methodology and on the issues that are to be overcome to document its surveys in a way that ensures reusability by international researchers (section 3). This is a collaborative work between GGP national data collectors and GGP central coordination team and follows a standardised procedure (section 4.1). The paper also explains the DDI elements chosen to document the surveys, as well as the ways in which Nesstar functionalities are used to present the surveys online. The content of the GGP Online Codebook and Analysis is detailed and illustrated with some examples (section 4.2). The paper ends with some reflexions about the GGP experience in data documentation and the plans for improving the data documentation process in future rounds of GGP data collection. 2