Accrediting a small Forensic Speaker Comparison Lab Text Jonas Lindh Forensic Phonetic Analyst Voxalys AB University of Gothenburg Department of Clinical Neuroscience Sahlgrenska Academy University of Gothenburg 1
Voxalys AB • Performed casework for 11 years • Sweden, Norway and US • approximately 400 cases • 3 employees part time (also employed at the university) • Subcontractor of NFC (National Forensic Centre, Sweden) 2
Outline • Applied methods • Evaluations for accreditation • Forensic Conclusions in Sweden • Inference? 3
3 part casework analysis • NFC screening, in-house screening, punting samples during analysis… • Part 1 - Linguistic phonetic perceptual analyses (FSASR) • blind testing • Part 2 - Acoustic measurement AR, F0 & LTF (FSASR) • Part 3 - AVC, Automatic Voice Comparison (FASR) (2 systems active, 1 evaluating, 1 for research) 4
Evaluations for accreditation • Guidelines • Drygajlo, A., Jessen, M., Gfroerer, S., Wagner, I., Vermeulen, J., Niemi, T., 2015 . Methodological Guidelines for best practice in forensic semiautomatic and automatic speaker recognition, including guidance on the conduct of proficiency testing and collaborative exercises. Wiesbaden, Germany. European Network of Forensic Science Institutes. • Meuwly, D., Ramos, D., Haraksim, R., A guideline for the validation of likelihood ratio methods used for forensic evidence evaluation , Forensic Sci. Int. (2016), http://dx.doi.org/10.1016/j.forsciint.2016.03.048 5
Evaluations for accreditation logLRs LR values 0.6 0.18 Likelihood Ratios After PAV DET plot 2 discrimination loss LR=1 always 0.16 calibration loss log LR = 0 DET curve 1.8 E min 0.5 40 0.14 true log LR 1.6 30 false alarms 0.12 0.4 normalized Bayes error-rate pError 30 misses 0.1 1.4 20 C llr [bits] 0.08 0.3 1.2 Miss probability (in %) 10 0.06 1 0.2 0.04 5 0.8 0.02 0.1 0.6 0 2 -6 -4 -2 0 2 4 6 logit prior 0.4 1 0 0.5 0.2 logLRs 0.2 0 LR values -10 -8 -6 -4 -2 0 2 4 6 8 10 Tippett plot 1 0.1 After PAV logit : 100 100 LR=1 always H p true H p true 0.5 0.1 0.2 0.5 1 2 5 10 20 40 H p Bound 90 H d true H d true False Alarm probability (in %) 0.1 H d Bound 80 80 logLRs 0.4 Proportion of cases (%) 70 Proportion of cases (%) 0.08 Normalized count 0.3 60 60 0.06 50 0.2 40 40 0.04 30 0.1 20 20 0.02 10 0 0 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 -3 -2 -1 0 1 2 3 Prior log 10 (odds) 0 0 -6 -5 -4 -3 -2 -1 0 1 2 3 -6 -5 -4 -3 -2 -1 0 1 2 3 Log 10 (LR) Greater Than log 10 LRs log 10 LR Tippett plot 2 100 H p true 90 H d true 80 Proportion of cases (%) 70 60 50 40 30 20 6 10 0 -6 -5 -4 -3 -2 -1 0 1 2 3 log 10 LR
Evaluations for accreditation • GSM-GSM performance durations 1x100 10x10 1x110 11x10 1x120 12x10 1x10 1x20 2x10 1x30 3x10 1x40 2x20 4x10 1x50 5x10 1x60 2x30 3x20 6x10 1x70 7x10 1x80 2x40 4x20 8x10 1x90 3x30 9x10 2x50 5x20 2x60 3x40 4x30 6x20 CLLRmins 10s ,573 ,557 ,495 ,532 ,428 ,481 ,430 ,425 ,460 ,413 ,435 ,393 ,342 ,324 ,395 ,329 ,401 ,341 ,314 ,366 ,394 ,327 ,300 ,403 ,327 ,284 ,293 ,368 ,257 ,212 ,319 ,326 ,325 ,281 ,287 20s ,495 ,457 ,400 ,444 ,318 ,404 ,312 ,288 ,345 ,284 ,334 ,285 ,245 ,219 ,301 ,212 ,299 ,243 ,220 ,232 ,290 ,234 ,210 ,299 ,215 ,193 ,201 ,268 ,151 ,273 ,217 ,226 ,224 ,195 ,163 30s ,544 ,394 ,351 ,397 ,263 ,373 ,256 ,234 ,300 ,236 ,279 ,224 ,199 ,175 ,268 ,152 ,258 ,196 ,173 ,195 ,246 ,172 ,159 ,270 ,190 ,157 ,160 ,236 ,117 ,235 ,175 ,197 ,176 ,156 ,142 40s ,438 ,382 ,338 ,362 ,234 ,350 ,258 ,215 ,261 ,209 ,235 ,207 ,194 ,132 ,214 ,113 ,214 ,173 ,162 ,160 ,204 ,152 ,142 ,231 ,145 ,162 ,147 ,214 ,107 ,201 ,150 ,172 ,151 ,148 ,122 50s ,380 ,320 ,300 ,304 ,180 ,327 ,192 ,168 ,212 ,168 ,190 ,156 ,130 ,083 ,190 ,104 ,200 ,143 ,101 ,096 ,175 ,101 ,088 ,215 ,121 ,105 ,084 ,193 ,068 ,187 ,125 ,137 ,089 ,079 ,075 60s ,370 ,339 ,293 ,290 ,198 ,327 ,200 ,159 ,204 ,167 ,186 ,149 ,137 ,082 ,153 ,105 ,184 ,123 ,119 ,069 ,168 ,074 ,097 ,206 ,095 ,108 ,091 ,176 ,083 ,161 ,106 ,129 ,068 ,086 ,094 70s ,420 ,311 ,295 ,281 ,210 ,305 ,194 ,161 ,213 ,153 ,178 ,145 ,123 ,068 ,161 ,124 ,156 ,131 ,120 ,077 ,135 ,091 ,090 ,169 ,109 ,115 ,087 ,164 ,081 ,154 ,104 ,121 ,070 ,087 ,090 80s ,434 ,300 ,341 ,244 ,206 ,295 ,185 ,154 ,184 ,149 ,152 ,122 ,104 ,062 ,145 ,096 ,142 ,105 ,071 ,074 ,122 ,068 ,054 ,145 ,075 ,078 ,028 ,126 ,024 ,132 ,064 ,094 ,056 ,053 ,031 90s ,370 ,278 ,337 ,243 ,155 ,300 ,177 ,140 ,181 ,132 ,141 ,126 ,100 ,048 ,118 ,085 ,121 ,106 ,070 ,076 ,117 ,069 ,049 ,144 ,074 ,083 ,025 ,104 ,024 ,128 ,065 ,093 ,056 ,049 ,022 100s ,339 ,278 ,270 ,254 ,118 ,242 ,130 ,081 ,178 ,099 ,143 ,128 ,077 ,053 ,125 ,068 ,123 ,113 ,058 ,020 ,094 ,068 ,018 ,127 ,067 ,077 ,032 ,109 ,014 ,113 ,067 ,094 ,060 ,032 ,023 110s ,283 ,261 ,230 ,241 ,100 ,203 ,077 ,062 ,138 ,067 ,110 ,101 ,073 ,046 ,107 ,052 ,127 ,101 ,053 ,003 ,087 ,061 ,013 ,106 ,055 ,094 ,020 ,080 ,010 ,124 ,051 ,090 ,060 ,023 ,024 120s ,280 ,236 ,248 ,233 ,087 ,218 ,077 ,061 ,127 ,075 ,096 ,092 ,074 ,026 ,079 ,051 ,115 ,095 - ,003 ,074 ,061 ,007 ,075 ,046 ,094 ,014 ,068 ,006 ,099 ,037 ,078 ,051 ,020 ,020 • from “FRITS” - David van der Vloed at NFI 7
Evaluations for accreditation • Microphone (Olympus dictaphone recordings and Zoom H4) vs GSM • Microphone vs microphone (short or long distance with room acoustics) • Mobile video recordings vs GSM and/or Mic • With and without face cover • In and outside car • Indoor and outdoor • Different languages • Compressions (DSS, MP3, WMV, AMR, 3GP, Speex) 128kbps ->8kbps 8
Evaluations for accreditation • Perceptual phonetic analysis • Training, testing -> Evaluation blind • small scale - very time consuming 9
Conclusions in Sweden • National Forensic Centre (NFC) uses 2 hypotheses and a 9 point ordinal scale with verbal expressions • Level +4 …the results are extremely much more probable if the main hypothesis is true compared to if the alternative hypothesis is true. • Level -2 …the results are more probable if the alternative hypothesis is true compared to if the main hypothesis is true. Behind each level is a span of likelihood ratios… 10
Inference • Every case is unique… • How much can you infer from evaluations to an actual case? 11
General questions • What does it mean to have a transparent report? • Who has to be able to understand it? 12
Thank you for the attention! http://www.voxalys.se http://www.ling.gu.se/~jonas
Recommend
More recommend
