Data Quality Challenges ACM JDIQ EiC Open Knowledge Networks - PowerPoint PPT Presentation

Apr 03, 2023 •272 likes •482 views

Data Quality Challenges ACM JDIQ EiC Open Knowledge Networks (Biomedicine) Data Science for Finance (DSfin) Louiqa Raschid Smith School of Business Computer Science and UMIACS Technical Review of Data Quality Provenance

Data Quality Challenges ● ACM JDIQ EiC ● Open Knowledge Networks (Biomedicine) ● Data Science for Finance (DSfin) Louiqa Raschid Smith School of Business Computer Science and UMIACS
Technical Review of Data Quality ● Provenance Cleaning Annotation ● Data cleaning infrastructure and tools: ○ Robust first generation. ○ Big data and scalability. ○ Human-in-the-loop (HumInt). ● Process ○ Fitness to task. ○ Understanding workflows.
First Gen methodologies and products
Technical Review of Data Quality ● Provenance Cleaning Annotation ● Data cleaning infrastructure and tools: ○ Robust first generation. ○ Big data and scalability. ○ Human-in-the-loop (HumInt). ● Process ○ Fitness to task. ○ Understanding workflows.
Scenarios ● Lung cancer data (primary) generated by clinicians: ○ Patient entity identification in clinical notes, e.g., JM, J.M., etc. (cleaning) ○ Scale: barthel 4 ○ Stages: P0T0, Stage4, etc. (annotation) ● Analytics over (secondary) sources: ○ Drug induced liver injury (DILI): phenotype includes elevated levels of liver enzymes, etc. ○ (fitness to task; HumInt): There are many causes for elevated liver enzymes including transplants, some infants, etc.
Scenarios ● Privacy preserving data mining: ○ Entity linkage in the de-identified space. ○ Different entries contribute hashed identifiers but they may be missing a variety of fields. (provenance; fitness to task) ● iASiS SEMANTIC Data Cleaning / Annotation Pipeline ● Finding patterns in OKN: DILI Case Study ○ (Provenance; fitness to task; HumInt; Annotation.)
iASiS
DILI Case Study o Given a knowledge graph and a DILI phenotype (keywords) ... o Create profiles, e.g., [Phenotype | Drug | Gene | Pathway] o Rank the DRUG at most risk for DILI.
DILI Case Study
DILI Case Study
Tamr: Understanding Workflows
Tamr: Understanding Workflows
Tamr: Understanding Workflows
Lessons learned ● First generation tools work well. ● Next generation needs to focus on processes and workflows and HumInt. ● Scientists still spend huge amounts of time on cleaning. How can we fix this problem? ● Is Open Knowledge Networks a solution? ● An unexpected case study ...

Recommend

Challenges in Improving Information Quality NISS Data Quality Conference November 30

Challenges in Improving Information Quality NISS Data Quality Conference November 30 December 1 Ann Thornton National Director, Data Quality and Integrity Deloitte & Touche Perspective on Information Quality Inclusion within

705 views • 21 slides

The challenges of capturing clinically accurate data for electronic Clinical Quality Measures

The challenges of capturing clinically accurate data for electronic Clinical Quality Measures (eCQM) Joe Kunisch PhD, RN-BC, CPHQ Enterprise Director of Clinical Quality Informatics Memorial Hermann Hospital System Quality Patient Safety &

294 views • 14 slides

The Data Quality Management Challenges of Solvency II Massimiliano Neri, Associate Director,

The Data Quality Management Challenges of Solvency II Massimiliano Neri, Associate Director, Moodys analytics 2 Agenda 1. Introduction 2. Criteria to Assess Data Quality 3. Data Quality Systems and Procedures 4. Moodys Analytics Best

292 views • 26 slides

OPPORTUNITIES AND CHALLENGES FOR HEALTH DATA ORGANIZATIONS (USING AHRQ QUALITY INDICATORS)

ICD-10 IMPLEMENTATION: OPPORTUNITIES AND CHALLENGES FOR HEALTH DATA ORGANIZATIONS (USING AHRQ QUALITY INDICATORS) Patrick S. Romano, MD MPH Professor of Medicine and Pediatrics UC Davis School of Medicine Co-Editor in Chief, HSR AHRQ Quality

625 views • 25 slides

Data Collection and Aggregation Data Collection and Aggregation 1 Challenges: data Challenges:

Data Collection and Aggregation Data Collection and Aggregation 1 Challenges: data Challenges: data Data type: numerical sensor readings. Rich and massive data, spatially distributed and correlated. Data dynamics: data

714 views • 48 slides

DATA QUALITY AND DATA DATA QUALITY AND DATA PROGRAMMING PROGRAMMING "Data cleaning and

DATA QUALITY AND DATA DATA QUALITY AND DATA PROGRAMMING PROGRAMMING "Data cleaning and repairing account for about 60% of the work of data scientists." Christian Kaestner Required reading: Schelter, S., Lange, D., Schmidt, P.,

1.1k views • 86 slides

Census Data Quality Assurance 17 May 2010 Types of Quality Assurance (QA) Quality assurance of

Census Data Quality Assurance 17 May 2010 Types of Quality Assurance (QA) Quality assurance of captured and coded data Quality assurance of downstream processes (including data security and integrity) Quality assurance of population counts,

129 views • 12 slides

ESOs role as data provider: Strategies and Challenges ESOs mandate address the challenge:

ESOs role as data provider: Strategies and Challenges ESOs mandate address the challenge: Data Flow System provide quality content: Science Data Products future opportunities: ESO archive ASTERICS European Data Provider Forum, Heidelberg,

453 views • 16 slides

The challenges of f the data and how to obtain good quality Karsten Elmsted/Bent Reiter

The challenges of f the data and how to obtain good quality Karsten Elmsted/Bent Reiter EasyX Member of XBRL DK and XBRL SE About EasyX Worked with XBRL more than 15 years DK Gaap (2012), IFRS (2014), EBA (2014), EIOPA (2016)

241 views • 13 slides

After this session you should be able to: 1. Describe what is data quality and why it is

After this session you should be able to: 1. Describe what is data quality and why it is important. 2. Define methods of data quality check. 3. List reasons of poor data quality and explain how to address them. 4. Address process to improve

632 views • 42 slides

Data Quality Blueprint for Pentaho: Better Data Leads to Better Results Charles Gaddy Director

Data Quality Blueprint for Pentaho: Better Data Leads to Better Results Charles Gaddy Director Global Sales & Alliances, Melissa Data Agenda What is Data Quality, and What Role Does it Play? 6 Concepts of Data Quality Full Data

621 views • 26 slides

Quality data is needed to inform efforts to reduce disparities Have come a long way in the past

5/6/2014 Quality data is needed to inform efforts to reduce disparities Have come a long way in the past 10 years in identifying and exploring MCH issues experienced by the AI population Identified key challenges and barriers to better data,

367 views • 9 slides

R in Grenoble DATA CHALLENGES Magali Richard & Florent Chuffart Introduct ction Data

R in Grenoble DATA CHALLENGES Magali Richard & Florent Chuffart Introduct ction Data challenges in class Data challenges for scientists Tutorial CIT-SAB MEETING 2 What is a data challenge? Ellrott et al. Genome Biology (2019) 20:195

373 views • 21 slides

METROLOGY AND METROLOGY AND QUALITY ASSURANCE QUALITY ASSURANCE Challenges for the analytical

METROLOGY AND METROLOGY AND QUALITY ASSURANCE QUALITY ASSURANCE Challenges for the analytical chemist Challenges for the analytical chemist Ale Fajgelj Fajgelj Ale IAEA International Atomic Energy Agency Agencys Laboratories

1.3k views • 16 slides

Service Data Quality and Improvement Quality: When Session T31 99.7% Isnt Good May 17,

ASQ World Conference on Service Data Quality and Improvement Quality: When Session T31 99.7% Isnt Good May 17, 2016 Enough Jennifer Wilson Dodd Starbird Learning Objectives 1. Discuss the value of using quality data for process and

449 views • 22 slides

Childcare Challenges Finding and Providing Quality Childcare Overview Why does Quality

Childcare Challenges Finding and Providing Quality Childcare Overview Why does Quality Childcare Matter What does it look like? Challenges in providing quality childcare Why does Quality Child Care Matter? City

545 views • 25 slides

HL3 Data Collection (its finally published!) Data Quality Overview of published data

HL3 Data Collection (its finally published!) Data Quality Overview of published data Group Discussion on future data analysis Data Quality 2017/18 - 262 (2%) difference between HL3 and HL2 - largely known reasons for the

68 views • 6 slides

Data Collection & Challenges with Cost Transparency Executive Summary Key Issues Level of

Data Collection & Challenges with Cost Transparency Executive Summary Key Issues Level of effort Theres a lot of work to do . Lack of ownership If no one else will, Ill have to own it . Quality data

582 views • 22 slides

Data Quality: finding data and evaluating its quality for your research David Dorr, MD, MS

Data Quality: finding data and evaluating its quality for your research David Dorr, MD, MS Professor and Vice Chair, Medical Informatics and Clinical Epidemiology Professor, Medicine Oregon Health & Science University AcademyHealth 2017!

268 views • 22 slides

Air Quality Challenges In Hells Kitchen South mul9ple large bus problems DO YOU SEE IT ?

Air Quality Challenges In Hells Kitchen South mul9ple large bus problems DO YOU SEE IT ? Environmental Protec9on Agency (EPA) New York County ManhaKan Non AKainment of Na9onal Ambient Air Quality Standards (NAAQS)

355 views • 13 slides

Big Da Big Data ta Challenges Challenges in Delivering Health Coaching Interventions to the

Big Da Big Data ta Challenges Challenges in Delivering Health Coaching Interventions to the Home Holly Jimison, PhD, FACMI Consortium on Technology for Proactive Care College of Computer & Information Science & School of Nursing

551 views • 37 slides

Project Management Professionals Hot Topics & Challenges Quality Management Topic: Seven

Project Management Professionals Hot Topics & Challenges Quality Management Topic: Seven Basic Quality Management Tools Presenter: Sohel Akhter, PMP, CCNA,ISMS PMP Instructor, Netcom Learning Adjunct Professor , MBA program, CUNY PMP

181 views • 17 slides

NAHDO Data Quality Forum Cross State Metrics to Assess Data Quality September 2019 Identify

NAHDO Data Quality Forum Cross State Metrics to Assess Data Quality September 2019 Identify cross-state metrics to compare data quality benchmarks Use NAHDOs Data Quality Forum to gather input and consensus on an agreed upon set of

275 views • 6 slides

Data Collection and Aggregation 1 Challenges: data Data type: numerical sensor readings.

Data Collection and Aggregation 1 Challenges: data Data type: numerical sensor readings. Rich and massive data, spatially distributed and correlated. Data dynamics: data streaming and aging. Uncertainty, noise, erroneous

1.45k views • 99 slides