01110101 01101101 01110011 01100001 01101110 00100000 01101110 01100101 01110001 01110101 01100101 00100000 01101100 01101111 01110010 01100101 01101101 00100000 01110000 01110101 01101100 01110110 01101001 01101110 01100001 01110010 00100000 01110100 01110101 01110010 01110000 01101001 01110011 00101110 00100000 01010000 01110010 01101111 01101001 01101110 00100000 01100001 00100000 01100011 01101111 01101110 01110110 01100001 01101100 01101100 01101001 01110011 00100000 01101100 01100101 01100011 01110100 01110101 01110011 00101110 00100000 01010110 01101001 01110110 01100001 01101101 01110101 01110011 00100000 01100001 01110101 01100011 01110100 01101111 01110010 00100000 01101100 01100101 01100011 01110100 01110101 01110011 00100000 01110101 01110100 00100000 01100101 01110011 01110100 00100000 01110000 01110010 01100101 01110100 01101001 01110101 01101101 00101100 00100000 01110110 01101001 01110100 01100001 01100101 00100000 01100011 01101111 01101110 01110011 01100101 01100011 01110100 01100101 01110100 01110101 01110010 00100000 01100101 01111000 00100000 01100001 01101100 01101001 01110001 01110101 01100001 01101101 00101110 00100000 01000001 01101100 01101001 01110001 01110101 01100001 01101101 00100000 01110100 01100101 01101101 01110000 01110101 01110011 00100000 01101110 01101111 01101110 00100000 01100101 01110010 01100001 01110100 00100000 01110001 01110101 01101001 01110011 00100000 01100110 01110010 01101001 01101110 01100111 01101001 01101100 01101100 01100001 00101110 A simple tool to 00100000 01001001 01101110 01110100 01100101 01100111 01100101 01110010 00100000 01100001 01100011 00100000 01100101 01100111 01100101 01110011 01110100 01100001 01110011 00100000 01100101 01111000 00101100 00100000 increase confjdence 01101001 01101110 00100000 01110110 01100101 01101000 01101001 01100011 01110101 01101100 01100001 00100000 01101101 01101001 00101110 00100000 01010001 01110101 01101001 01110011 01110001 01110101 01100101 00100000 in results 01101101 01100001 01110100 01110100 01101001 01110011 00100000 01110011 01100011 01100101 01101100 01100101 01110010 01101001 01110011 01110001 01110101 01100101 00100000 01101101 01101001 00100000 01101110 01100101 01100011 00100000 01100011 01101111 01101110 01100100 01101001 01101101 01100101 01101110 01110100 01110101 SALLY DUCKWORTH 01101101 00101110 00100000 01001101 01100001 01110101 01110010 01101001 01110011 00100000 01100110 01100101 Research Evaluation 01110101 01100111 01101001 01100001 01110100 00100000 & Design Specialist 01100001 01110010 01100011 01110101 00100000 01101110 01101111 01101110 00100000 01100100 01101001 01100001 01101101 00100000 01110010 01110101 01110100 01110010 01110101 01101101 00100000 01101100 01101111 01100010 01101111 01110010 01110100 01101001 01110011 00100000
Data quality is a concept that academics, scientists and IT specialists have been taking seriously for years.
Ethical Valid Data quality is Depth the fjtness of data Coherent for an intended Accurate purpose. Precise Complete Reliable Culturally Intuitive responsive
Having quality data leads to confjdence in results.
And poor quality data RECOMMENDATIONS is a house of cards An expression dating CONCLUSION back to 1645 meaning a structure or argument built on a shaky foundation FINDINGS that would collapse if a necessary but overlooked element is removed. DATA
The main way of making data quality explicit in formal evaluation products is by stating limitations.
Limitations are occurrences that the evaluator did not or could not control.
Acknowledging limitations should not be seen as a weakness of the evaluator, but rather a signifjer of credibility. They are also an opportunity for refmective learning.
Best practice for acknowledging limitations points to 3 things: 1. Identifying limitations and their importance 2. Discussing constraints 3. Recommending further research and data collection
We did a stocktake of evaluation reports and found that 28% did not refer to limitations Evaluation reports 25 published by 6 state service organisations in 2015 18 17 7 7 72% discussed 68% talked about 28% recommended 28% did not make data limitations how limitations further research comments constrain fjndings
Then we asked senior evaluators* 3 questions: 1. How important is the quality of data used in evaluations? 2. How does your organisation determine the quality of data used in evaluations? 3. Does your organisation have standards for reporting the quality of data in evaluation products? *6 in 4 state sector organisations
We heard that data $ quality is important when conducting evaluation. And people have different expectations of data quality depending on the scope, size, duration and location of an evaluation.
Highly qualifjed We also hear d people that data quality is Co-coding Teamwork determined by having trust and Peer review confjdence that Triangulation evaluators are Sense checking following ‘good professional Completeness practice.’ Response rates Co-interviewing
We learnt that organisations do not have standards for stating data quality but are more explicit (risk averse) when publishing externally.
We found that limitations slide off accessible and real time products such as A3’s, correspondence and progress reports.
‘ Decision makers and other stakeholders are wanting things instantly and digestible. We can provide a one or two page thing, but ’ you lose the nuances and limitations.
Finally we learnt that limitations are seen as weakness of the evaluator or evaluation, rather than enhancing credibility or confjdence. We need to fmip this around and start talking about and celebrating good data quality to enhance the confjdence C R of results. E D I B I L I T Y
A simple tool to increase the confjdence in results is to: 1 State the quality DATA SOURCE QUALITY RATING COMMENT of your data ON QUALITY LITERATURE HIGH REVIEW QUALITATIVE HIGH INTERVIEW STOCKTAKE MEDIUM Y T I L A U Q
A simple tool to increase the confjdence in results is to: 1 2 3 State the quality ...Adjust it so it ...And apply it of your data fjts the context to all evaluation products Y T I L A U Q DATA SOURCE QUALITY RATING COMMENT ON QUALITY LITERATURE HIGH REVIEW QUALITATIVE HIGH INTERVIEW Y T I L A U Q Y T STOCKTAKE MEDIUM I L A U Q
Data quality... ...let’s start a conversation @LitmusNz #ANZEA2016 #dataqual
Recommend
More recommend
