Expectancy bias and Bias and forensic evidence Bias and speech - PDF document

Overview • Bias effects Expectancy bias and • Bias and forensic evidence • Bias and speech research forensic speech research • Blind forensic speech research Maartje Schreuder TMFI / Maastricht University I AFPA 2011 Bundeskrim inalam t Vienna, Austria July 27, 2011 Bias effects Bias effects, expectancy effects • Bias • Madrid bombings 11 maart 2004 • Expectancy • Finger prints on bag with detonating devices • Confirmation bias (“tunnel vision”) • Several FBI- and other finger print experts: ‘The most obvious danger in forensic science is – 100% match Brandon Mayfield, sollicitor in USA, that an examiner's observations and conclusions converted to Muslim will be influenced by extraneous, potentially – Mayfield arrested biasing information’ (Risinger et al., 2002) Merckelbach, Crombach & Van Koppen, 2003 Bias-effects, expectancy effects Bias effects, expectancy effects • Spanish police: match with Algerian Ouhnane Daoud • Mayfield released

Bias-effects, expectancy effects Bias-effects, expectancy effects Results: Experiment Dror and colleagues (2006a): • 5 finger print experts (USA, UK, Israel, Netherlands, Australia) • 2 finger prints from case they had done before: match • Information: “these are the finger prints that were falsely matched by the FBI as belonging to the Madrid bomber” – � non-match – Instruction: ignore all context information � Experts are susceptible to irrelevant and misleading information! DNA and interpretation DNA and interpretation • Problematic DNA cases: partial or mixed DNA profiles • Thompson, as part of lecture for DNA experts – Profile of evidentiary material – Profile of suspect � Tom – Not all peaks of profile evidentiary material in material of suspect, nor victim • Wrong suspect? – No 12 on D3, no OL on FGA • Experts: “Obvious that these peaks are artefacts , can be ignored” Electropherograms in an easy-to-interpret case. Thompson, 2009 Thompson, 2009 DNA and interpretation DNA and interpretation – 2 nd lecture, same case: profile of Dick labeled as suspect. “I’m not sure”. – Is 12 peak on locus D3 true allele or a drop-in artefact? – No 20 peak at locus FGA. • Experts: “How can you doubt that? Morphology, peak height disparity, stochastic effects , …” • Thompson: “suspect was actually Tom” • Experts: “Oops” Electropherogram of a saliva sample and four suspect profiles. Thompson, 2009

DNA and interpretation DNA and interpretation – 3 rd lecture: profile of Harry labeled as suspect. “Problematic: D3 14, 17”. • Experts: “no problem at all. Allelic dropout, masking by an artefact, …” Electropherogram of a saliva sample and four suspect profiles. Thompson, 2009 DNA and interpretation DNA and interpretation – 4 th lecture: profile Sally labeled as suspect. “Do you agree that this defendant should be excluded?” • Experts: “no, evidentiary profile could be a mixed profile ” • (mix-theory was not mentioned before, when ‘suspect’ had the other profiles.) Electropherogram of a saliva sample and four suspect profiles. Thompson, 2009 DNA and interpretation DNA and interpretation • Standards may shift to encompass the profile of the suspects! Electropherogram of a saliva sample and four suspect profiles.

Bias-effects, expectancy effects Bias-effects, expectancy effects • Unintentional, subconscious process • Psychological phenomenon: fooled by our brains! • Almost impossible to disregard background • And we’re not even talking about speech information research yet! • Science: (double) blind to control for observer – Inherently variable! effects and expectancy/confirmation bias • Forensic labs: regulations – against contamination of samples – for reporting and conclusions, etc. – hardly any precautions against bias! Expectancy Bias and Speech intelligibility Expectancy Bias and Speech intelligibility Cue sentence : “ En toen zei ze: ik heb er geen zin meer in, ik wil naar Experiment: huis ” • Noisy speech fragment (music) [And then she said: “I don’t feel like • Target sentence follows given cue sentence it anymore, I want to go home”] • Participant writes orthographic transcription • 2 conditions: – Introduction from crime reporter (television) Write the sentence down that – No introduction follows the cue sentence. Participants Expectancy Bias and Speech intelligibility Target sentence : “ En eeh toen heb ik • 87 participants haar gebracht ” – 20 excluded: understood ‘other’ word [And eh then I brought her] (wrong + not crime-related) • Of which 14 in Control condition � not ‘aided’ by context Some phonological similarity: • 67 participants remained gebracht [brought] � verkracht [raped] � gepakt [grabbed] – 38 Context condition – 29 Control condition

Results Results Transcriptions (totals): • Understood by the 67 (remaining) participants: • Correct transcription – gebracht [brought] 49 Context condition • Crime-related transcriptions 17 (45%) – Verkracht [raped] 11 – Gepakt [grabbed] 7 21 (55%) – Vastgepakt [grabbed] 1 Correct word – Geraakt [hit] 1 Crime-related word • Other transcriptions – Gepakt* [grasped, hugged] 6 Non-context condition – Hard [hard] 2 � 2 (2) = 9.291, (p = 0.002) – Ontmoet [met] 2 26 (90%) – Bekijk het maar [whatever] 1 – … … 3 (10%) Thanks to my students of research practicum 2011 Context, transcripts, degradation Lange, Tomas, Dana, & Dawes, 2011: Experiment with ‘wire-tapping • 2 experiments with degraded recordings interpreters’ – Contextual information: criminal justice system – Dubious transcripts along with ‘evidentiary’ recordings • Results: – systematic misinterpretations – Confidence in expectation-induced misinterpretations – Information leads to miscalibration to quality of recording: poor quality goes undetected Lange, Tomas, Dana, & Dawes, 2011 Martin Robert 5 6 7 1 2 3 4 1 2 3 4 5 6 7 06- 06- 06- 06- 56991078 56991078 06- 06- 56991078 56991078 56991078 56991078 Mart in Mart in Mart in Mart in Mart in Mart in 06- 06- 06- 06- 06-3635406 06-3635406 56991078 56991078 06-13635406 06-3635406 56991078 56991078 Mart in Mart in Mart in Mart in Robe rt Robe rt Robe rt Robe rt False positives with Martin Police interviews Police interview False & negatives with Martin False negative & with Robert Martin Robert False positive with Robert

Speaker identification (auditory) Blind analysis 1. Blind analysis – “ Evidence line-up ”: • Stadia of analysis • anonimized material 1. ‘Blind’ analysis • addition of control group: “foils” 2. Analysis of the questioned material • presented in random order 3. Comparison of questioned material with – Expert has no knowledge of the source of the reference material materials, and searches unprejudiced for salient similarities/differences. Vergelijkend spraakonderzoek Speaker identification 1. Blind analysis – No a priori expectations – Unprejudiced – Objective – “foils”: test of the expert + ‘ground truth’ 33/ 21 Speaker identification Speaker identification 2. Non-blind analysis: – Full materials Limitations of (blind) analysis: – Speaker’s variation range • Speaker’s variation range – Context • Comparison complicated when language – Communicative circumstances use situation and/or emotional/physical – Line quality condition of speaker differ – Educated Native Speaker – But: bias effects – Limited basis for comparison • Only influences strength of conclusion, conclusion should stay the same after blind analysis