Cortana / Speech Platform 2 Cortana truly uly person sonal - PowerPoint PPT Presentation

Cortana / Speech Platform 2

Cortana truly uly person sonal acros oss s all your r device ces s get t things ngs done. e.

Cortana Across your Devices

Cortana Usage is Growing In 9 months since Windows 10 launched Over 270 milli lion on active Windows 10 devices • Over 5 b billion ion questions asked of Cortana • One millio ion n voice questions are directed towards • Cortana on a daily basis On any given day, 25% of Cortana PC users use speech & Of these, 15% use the keyword to initiate Hey Cortana tana

Cortana Drives Customer Satisfaction inten ent t to reco commend mmend improve ove customer tomer satis isfac faction tion increase “re - purchase” intent

With Windows 10 Anniversary Update

Nearly 1,000 Apps with Voice Commands

Geographies Cortana functionality for Windows 10 is available in the following languages and countries Japanese-Japan English-US English-India English-UK English-Canada Chinese-China In plan for Windows 10 English-Australia German-Germany Anniversary Update French-France Portuguese-Brazil Italian-Italy French-Canada Spanish-Spain Spanish-Mexico

Cortana with Voice (CwV) Marketing Assets Icons only Icons + Titles Cortana with Voice Cortana with Voice Titles and Your truly personal digital assistant who helps you get things done, even descriptions hands-free - just say “Hey Cortana” to get started. T echnical *The “Cortana with Voice” title, icon, description, and associated assets Requirements such as messaging can only be used if all following criteria are met: to market 1. The device is tested using Microsoft’s “Speech platform tools” Cortana with 2. The device’s test results meet or exceed the Standard spec Voice (CwV)

Standard vs. Premium Spec Windows 10 Anniversary Update & Beyond Standard • Normal ambient noise level • Normal ambient noise level • 0.5m • 0.5m • Standard experience • Standard experience • Cortana with voice (CwV) • Cortana with Voice (CwV) • Additional tests for Voice Activation (optional) Premium • More challenging conditions • More challenging conditions • <1m • 4m • Talk level: 89 dBA • Talk level: 99 dBA • Premium experience • Far-field experience • Additional tests for Voice Activation *Early draft, subject to change

Cortana with Voice Testing Scenarios

Cortana with Voice Validation Process Run the OEM Verifier tool and ensure the driver settings aligns with the design Utility - OEMVerificationx86.exe • Driver reports these key elements correctly • Number of microphones Audio effects supported   Microphone geometry Exposes Audio Pipeline being used   Default microphone gain Driver mode supported   Follow the test guidance to validate Cortana with Voice performance Validate in 3 scenarios: Quite, Ambient Noise (café and pub), Echo • Parse data and ensure the test result are at or above the provided benchmark Utility - OEMScoreUtilityFarAndNearx64.exe • Calculate the speech accuracy score & troubleshoot the issues •

Examples of Test Result Microsoft Pipeline Speech Accuracy Score OEM Pipeline

Future Planning for CwV Requirements *Early draft, subject to change

Platform Experience Release People Scenario Near-field Far-field • Platform providers test voice 0 50 0 50 activation false accepts (FA) degrees degrees degrees degrees Quiet (Private Office, Living Room) and correct accepts (CA) for Male Medium (Coffee Shop, Kitchen) 90% 90% Music listening in Quiet every supported locale 2016 Quiet (Private Office, Living Room) Female Medium (Coffee Shop, Kitchen) 90% 90% These two tests are not needed • Music listening in Quiet to be run by OEMs Quiet (Private Office, Living Room) Music listening in Quiet Male 92% 92% 90% 90% Medium (Coffee Shop, Kitchen) Music listening in Medium Ambient Quiet (Private Office, Living Room) Music listening in Quiet Future Female 92% 92% 90% 90% Medium (Coffee Shop, Kitchen) Music listening in Medium Ambient Quiet (Private Office, Living Room) Music listening in Quiet Children 92% 92% 90% 90% Medium (Coffee Shop, Kitchen) (5-12) Music listening in Medium Ambient

OEM Experience OEMs test specific devices for good OEMs test specific devices for good voice speech recognition on EN-US only. activation accuracy on EN-US and JA-JP . These tests will become quality gates beyond 2016.

Recommended Array for Far-Field When the Microsoft far-field processing is used the recommendation is to use the microphone array geometry shown above For 360 degree operation an 8-mic circular array is recommended For 3 rd party solutions use their spec recommendations for microphone array and geometry

Technical Requirement References Requirement Detail Reference / Spec Speech Platform Application and Microsoft Speech Platform Test Tools http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=59735 test tools content guidance for testing Speech Platform Audio Calibration files http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=57500 v1.1 “Standard” Quiet, Ambient, Echo Speech Platform Input Device Recommendations v1.2 (§2 – Device.SpeechRecognition recommendations) benchmark scores http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=58099 https://msdn.microsoft.com/en-us/library/windows/hardware/dn915051(v=vs.85).aspx v1.1 Test setup Measurement setup Speech Platform Input Device Test Setup v1.1 http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=58099 https://msdn.microsoft.com/en-us/library/windows/hardware/dn915051(v=vs.85).aspx Test setup for Measurement setup Speech Platform Input Device Recommendations – Accessories Support v1.0 http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=59861 accessories v2.0 pre3 Specs Benchmarks and Speech Platform Input Device Recommendations v2.0pre3 Speech Platform Input Device Test Setup v2.0pre3 measurement setup https://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=60086

Call to Action • Ensure your drivers have the correct settings (mic geometry, mic gain, number of mics, expose audio pipeline being used) • Download and test the latest specs and tools • Provide feedback on Speech Platform v2.0pre3 specs

Join WinHEC LINE Online Survey Form Community @winhec http://aka.ms/winhecfeedback Download WinHEC presentations here: * G Gifts s are limit ited. ed. They y will be offered ed at https://channel9.msdn.com/Blogs/WinHEC “a first come, first serve” basis.

Please provide feedback on this session: http://aka.ms/winhecfeedback