Score Distribution Based Term Specific Thresholding for Spoken Term - PowerPoint PPT Presentation

Score Distribution Based Term Specific Thresholding for Spoken Term Detection D. Can M. Sarac ¸lar Bo˘ gazic ¸i University Department of Electrical & Electronics Engineering BUSIM Lab

Introduction Thresholding for Spoken Term Detection Experiments Summary Outline Introduction 1 Thresholding for Spoken Term Detection 2 Global Thresholding Term Weighted Value Based Term Specific Thresholding Score Distribution Based Term Specific Thresholding Experiments 3 Setup Results Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Thresholding for Spoken Term Detection Experiments Summary Application: Sign Dictionary Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Thresholding for Spoken Term Detection Experiments Summary Anatomy of a Spoken Term Detection (STD) System Speech User Retrieve Database yes larger Query ASR no than τ ? Dispose Search Preprocess Index Engine Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Thresholding for Spoken Term Detection Experiments Summary Anatomy of a Spoken Term Detection (STD) System Speech User Retrieve Database yes larger Query ASR no than τ ? Dispose INDEXING Search Preprocess Index Engine Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Thresholding for Spoken Term Detection Experiments Summary Anatomy of a Spoken Term Detection (STD) System Speech User Retrieve Database yes SEARCH larger Query no than τ ? Dispose Search Preprocess Index Engine Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Thresholding for Spoken Term Detection Experiments Summary Anatomy of a Spoken Term Detection (STD) System Speech User Retrieve Database yes RETRIEVAL larger Query no than τ ? Dispose Search Preprocess Index Engine Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Outline Introduction 1 Thresholding for Spoken Term Detection 2 Global Thresholding Term Weighted Value Based Term Specific Thresholding Score Distribution Based Term Specific Thresholding Experiments 3 Setup Results Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Global Thresholding 15 Incorrect Class Distribution Correct Class Distribution Incorrect Class EM Estimate Correct Class EM Estimate 10 Normalized histogram of posterior n scores for an example query 5 0 0 0.2 0.4 0.6 0.8 1 Posterior Score Pick a global threshold θ for all query terms Apply binary thresholding Vary θ for different operating points No term specific behavior, no joint processing of candidates, hence poor performance! Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Global Thresholding 15 Incorrect Class Distribution Correct Class Distribution Incorrect Class EM Estimate Correct Class EM Estimate 10 Normalized histogram of posterior n scores for an example query 5 0 0 0.2 0.4 0.6 0.8 1 Posterior Score Pick a global threshold θ for all query terms Apply binary thresholding Vary θ for different operating points No term specific behavior, no joint processing of candidates, hence poor performance! Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Global Thresholding 15 Incorrect Class Distribution Correct Class Distribution Incorrect Class EM Estimate Correct Class EM Estimate 10 Normalized histogram of posterior Reject Accept n scores for an example query 5 0 0 0.2 0.4 0.6 0.8 1 Posterior Score Pick a global threshold θ for all query terms Apply binary thresholding Vary θ for different operating points No term specific behavior, no joint processing of candidates, hence poor performance! Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Global Thresholding 15 Incorrect Class Distribution Correct Class Distribution Incorrect Class EM Estimate Correct Class EM Estimate 10 Normalized histogram of posterior Reject Accept n scores for an example query 5 0 0 0.2 0.4 0.6 0.8 1 Posterior Score Pick a global threshold θ for all query terms Apply binary thresholding Vary θ for different operating points No term specific behavior, no joint processing of candidates, hence poor performance! Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Global Thresholding 15 Incorrect Class Distribution Correct Class Distribution Incorrect Class EM Estimate Correct Class EM Estimate 10 Normalized histogram of posterior Reject Accept n scores for an example query 5 0 0 0.2 0.4 0.6 0.8 1 Posterior Score Pick a global threshold θ for all query terms Apply binary thresholding Vary θ for different operating points No term specific behavior, no joint processing of candidates, hence poor performance! Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Outline Introduction 1 Thresholding for Spoken Term Detection 2 Global Thresholding Term Weighted Value Based Term Specific Thresholding Score Distribution Based Term Specific Thresholding Experiments 3 Setup Results Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Term Weighted Value (TWV) [NIST, 2006] Q TWV = 1 − 1 � { P miss ( q k ) + β P FA ( q k ) } Q k =1 P miss ( q k ) = 1 − C ( q k ) P FA (q k ) = A ( q k ) − C ( q k ) R ( q k ) , T − C ( q k ) Q Number of queries R ( q k ) Number of occurrences of query q k A ( q k ) Total number of retrieved documents for q k C ( q k ) Number of correctly retrieved documents for q k T Total duration of the speech archive β Cost of false alarms relative to hits Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary TWV Based Term Specific Thresholding [Miller et al., 2007] 1 β ˆ ˆ V hit ( q k ) = , C FA ( q k ) = ˆ T − ˆ N true ( q k ) N true ( q k ) ˆ C FA ( q k ) ˆ θ ( q k ) = C FA ( q k ) + ˆ ˆ V hit ( q k ) ˆ N true ( q k ) Expected count of occurrences of q k ˆ θ ( q k ) Optimal threshold for q k maximizing TWV in the expected sense Term specific expected counts → Term specific thresholds Vary β for different operating points Only the sum of individual scores affects the threshold! Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary TWV Based Term Specific Thresholding [Miller et al., 2007] 1 β ˆ ˆ V hit ( q k ) = , C FA ( q k ) = ˆ T − ˆ N true ( q k ) N true ( q k ) ˆ C FA ( q k ) ˆ θ ( q k ) = C FA ( q k ) + ˆ ˆ V hit ( q k ) ˆ N true ( q k ) Expected count of occurrences of q k ˆ θ ( q k ) Optimal threshold for q k maximizing TWV in the expected sense Term specific expected counts → Term specific thresholds Vary β for different operating points Only the sum of individual scores affects the threshold! Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD

Introduction Global Thresholding Thresholding for Spoken Term Detection Term Weighted Value Based Term Specific Thresholding Experiments Score Distribution Based Term Specific Thresholding Summary Outline Introduction 1 Thresholding for Spoken Term Detection 2 Global Thresholding Term Weighted Value Based Term Specific Thresholding Score Distribution Based Term Specific Thresholding Experiments 3 Setup Results Can, Sarac ¸lar Score Distribution Based Term Specific Thresholding for STD