The Office for National Statistics Big Data Project Nigel Swier - PowerPoint PPT Presentation

Sep 05, 2022 •476 likes •612 views

The Office for National Statistics Big Data Project Nigel Swier NTTS Conference, 10-12 March 2015, Brussels Background Set up as a12 month project Jan-Dec 2014 Aims: Investigate the potential of big data for official statistics and

The Office for National Statistics Big Data Project Nigel Swier NTTS Conference, 10-12 March 2015, Brussels
Background • Set up as a12 month project Jan-Dec 2014 • Aims: • Investigate the potential of big data for official statistics and to understand the challenges • Establish an ONS policy and longer term strategy • Recommend practical next steps • 3 month extension to prepare business case for next phase
Work packages • Management and Strategy • Stakeholder Engagement • Communication • Analysis and infrastructure: #Twitter Smart meters Technology Pilots Mobile phones Prices
Stakeholder Engagement Commercial Sector International Privacy Groups Academia Government
Pilot 1: Smart meters Potential of data from electricity smart-type meters to identify unoccupied households • More efficient response chasing • Data from smart meter trials in Great Britain and Republic of Ireland • A range of potential methods identified • Privacy and ethics
Smart-type Meter Energy Use Profiles Occupied profile Unoccupied profile Anomaly
Pilot 2: Mobile Phones Mobile phone data to model population flows, e.g. Commuting statistics • Building relationships with mobile network operators and other parts of UK Government • No data yet. Seeking better coordinated data access for Government • Privacy and ethics (again)
Pilot 3: Prices Use of web scraped price data for use in price statistics • ONS prices collection is manual • Web scraping promises more detailed, more frequent and cheaper data • Prototype web scrapers: • 35 CPI/RPI item categories • 3 supermarkets • Daily collection (around 6500 a day) • Data ‘wrangling’ is a big challenge
Daily Price Index (Whiskey)
Pilot 4: Twitter Potential of geo-located Twitter to gain new insights mobility and migration • 7 months of geo-located tweets within Great Britain (about 100 million data points) • Methodology to infer place of usual residence: - Identify user ‘anchor points’ by clustering tweets using a DBSCAN algorithm - Identify residential anchor points using AddressBase and nearest neighbour analysis
Use case: Student mobility
Conclusion • A range of potential benefits (not just about replacing existing outputs) • Challenges can be overcome: • Technical/Skills => Innovation labs • Legal/ethical => Policy and guidance • Statistical (bias) => Benchmarking survey • Affordable access => ??? • Long term investment is required Project recommends a further 3 year project to deliver both tangible benefits and a broader capability to support big data projects

Recommend

Exit Checks data so far? Nicky Rogers, Office for National Statistics 19 October 2018 1

How have we used Home Office Exit Checks data so far? Nicky Rogers, Office for National Statistics 19 October 2018 1 Presentation outline Non-EU international students what do they do at the end of their studies? Non-EU migration

178 views • 14 slides

Data Collection International Labour Office Department of Statistics Data Collection data

Data Collection International Labour Office Department of Statistics Data Collection data collection activities should be established, adapted to countries national circumstances main sources of information establishment surveys

129 views • 10 slides

Statistics Liberia Office of National Statistics The compilation of External Trade

UNITED NATIONS DEPARTMENT OF ECONOMIC AND SOCIAL AFFAIRS STATISTICS DIVISION Workshop on compilation of international merchandise trade statistics, Abuja, Nigeria, 30 aot - 2 septembre 2005 Country Presentation Statistics Liberia Office

68 views • 4 slides

The Effects of Relocation on Blaise Support in ONS Lucy Fletcher, Office for National Statistics,

The Effects of Relocation on Blaise Support in ONS Lucy Fletcher, Office for National Statistics, UK Blaise support team (BDSS) ONS and BDSS Office for National Statistics Employs around 4000 people Collects and publishes a wide

401 views • 15 slides

PAATA SHAVISHVILI Deputy Executive Director National Statistics Office of Georgia The image part

OVERVIEW OF EXTERNAL MIGRATION STATISTICS IN GEORGIA THE USE OF ADMINISTRATIVE DATA FOR MEASURING MIGRATION IN GEORGIA PAATA SHAVISHVILI Deputy Executive Director National Statistics Office of Georgia The image part with relationship ID rId2 was

239 views • 12 slides

National Statistics Conference 2012 (MyStats 2012) Enhancing National Statistics to Meet Public

National Statistics Conference 2012 (MyStats 2012) Enhancing National Statistics to Meet Public and Private Sector Needs During a Period of Transformation Session 2: Promoting Data Sharing and Communication of Statistics Maximising the

473 views • 13 slides

Mobile phone data for Mobility statistics Emanuele Baldacci Italian National Institute of

International Conference on Big Data for Official Statistics Organised by UNSD and NBS China Beijing, China, 28-30 October 2014 Mobile phone data for Mobility statistics Emanuele Baldacci Italian National Institute of Statistics (Istat) Head,

303 views • 16 slides

Session 2: Promoting Data Sharing and Communication of Statistics Enhancing the communication of

NATIONAL STATISTICS CONFERENCE 2012 Enhancing National Statistics to Meet Public and Private Sectors Needs during a Period of Transformation Organized jointly by Department of Statistics Malaysia and Bank Negara Malaysia Session 2:

394 views • 21 slides

National Health Care Spending In 2016 Office of the Actuary National Health Statistics Group 1

National Health Care Spending In 2016 Office of the Actuary National Health Statistics Group 1 Overview National Health Expenditures reached $3.3 trillion, or $10,348 per person in 2016. National health spending increased 4.3 percent in

408 views • 21 slides

USDA National Agricultural Statistics Service New England Field Office 53 Pleasant St, Rm 3450

USDA National Agricultural Statistics Service, New England Field Office USDA National Agricultural Statistics Service New England Field Office 53 Pleasant St, Rm 3450 Concord, NH 03301 Telephone: 603-224-9639 800-642-9571 Fax

794 views • 56 slides

Green National Accounting in India: A Framework-Bio-diversity related findings Dr.V.

Green National Accounting in India: A Framework-Bio-diversity related findings Dr.V. Parameswaran Deputy Director General 04-12-2013 Social Statistics Division Central Statistics Office National Statistical Organization Ministry of Statistics

720 views • 34 slides

Population and Migration Statistics Committee Projections Sub-Group 17 August 2016 Meeting

Population and Migration Statistics Committee Projections Sub-Group 17 August 2016 Meeting Sub-National Population Projections (SNPP) Review: Update Overview of Project Why are we doing this? The Office for National Statistics (ONS)

443 views • 20 slides

Dementia Update 2015 The Challenge Office for National Statistics 2012 Implications for GWH

Dementia Update 2015 The Challenge Office for National Statistics 2012 Implications for GWH 70% of inpatients are older people (>75) 50% of these patients have cognitive impairment COMPLICATIONS Hospital acquired infections

371 views • 22 slides

Th The Integrated Household I t t d H h ld Survey: The next instalment Alessio Fiacco Office

Th The Integrated Household I t t d H h ld Survey: The next instalment Alessio Fiacco Office for National Statistics (ONS) UK Office for National Statistics (ONS), UK Outline Background Implementation plan Survey instrument

347 views • 13 slides

2015 National Conference on Health Statistics Community Health Data, Data Stewardship and Data

2015 National Conference on Health Statistics Community Health Data, Data Stewardship and Data Access and Use: Tools and Resources North Bethesda, MD August 25, 2015 Speakers James Scanlon NCVHS Role and Contributions in HHS NCVHS

654 views • 46 slides

2 This work is funded under National Data Storage 2 project (2011-2013), Project number

2 National Data Storage 2 Secure sharing, publishing and exchanging data Maciej Brze niak, Norbert Meyer, Micha Jankowski, Gracjan Jankowski Supercomputing Department, PSNC 2 This work is funded under National Data Storage 2 project

477 views • 30 slides

development plan Nicola White, Office for National Statistics 19 October 2018 Context: Changes

International migration development plan Nicola White, Office for National Statistics 19 October 2018 Context: Changes in demand for evidence Brexit negotiations Implications for European migration Impact on sectors Labour market and

598 views • 7 slides

Existing Legislations on Data Privacy: A Change to Data Sharing? National Statistics Conference 2

Existing Legislations on Data Privacy: A Change to Data Sharing? National Statistics Conference 2 0 1 2 Professor Abu Bakar Munir Faculty of Law , University of Malaya 7 Novem ber 2 0 1 2 1 Some of my books on ICT Law In Print Privacy and

721 views • 24 slides

and challenges for the 2021 Census and beyond Andy Teague Jane Naylor Office for National

Administrative and big data opportunities and challenges for the 2021 Census and beyond Andy Teague Jane Naylor Office for National Statistics 1 Overview Context and background Aspirations, expectations and challenges in using

653 views • 27 slides

Modelling sample data from smart-type meter electricity usage Susan Williams NTTS Conference,

Modelling sample data from smart-type meter electricity usage Susan Williams NTTS Conference, 10-12 March 2015, Brussels Introduction Big Data Project in the UKs Office For National Statistics Practical pilot on the potential of

312 views • 12 slides

Exercise & Fitness Did you know? According to the National Center for Health Statistics, and

Exercise & Fitness Did you know? According to the National Center for Health Statistics, and the Office of the Surgeon General of The United States: Adults aged 18 and older need an average of 30 minutes 5 or more days a week of physical

289 views • 13 slides

1. Consider the wholesale data in the sheet Wholesale. (a) For the grocery sales in region

Statistics and Data Analysis Descriptive Statistics (2) Summarization Instructor: Ling-Chieh Kung Department of Information Management National Taiwan University 1. Consider the wholesale data in the sheet Wholesale. (a) For the

265 views • 9 slides

Non-traditional data sources in Social Statistics of Statistics Finland Pasi Piela,

Non-traditional data sources in Social Statistics of Statistics Finland Pasi Piela, pasi.piela@stat.fi Non-traditional data sources in the National Statistical Systems, 17 th Meeting of ECLAC, Santiago de Chile, 1-2 October 2018 Contents

488 views • 48 slides

Statistics and Data Analysis Descriptive Statistics (2): Summarization Ling-Chieh Kung

Central tendency Variability Correlation Statistics and Data Analysis Descriptive Statistics (2): Summarization Ling-Chieh Kung Department of Information Management National Taiwan University Descriptive Statistics 1 / 33 Ling-Chieh Kung

338 views • 33 slides