2020 LAS Collaborators’ Week Dr. Alyson Wilson Dr. Matt Schmidt Jamie Roseborough LAS Principal Investigator LAS Director of Programs LAS Director of Outreach and Engagement Dr. Christine Brugh Dr. Jascha Swisher Lori Wachter LAS Technical Program Manager LAS Technical Program Manager LAS Technical Program Manager June 15-18, 2020 1
Contact Info General Inquiries: lasoutreach@ncsu.edu ● Specific Inquiries: ● Alyson Wilson, agwilso2@ncsu.edu ○ Matt Schmidt, mcschmid@ncsu.edu ○ Jamie Roseborough, jvrosebo@ncsu.edu ○ ● LAS Collaborators Week Website: https://ncsu-las.org/2020-las-collaborators-day/ 2
LAS Collaborators Week Schedule Monday, June 15: Plenary Session ● Tuesday, June 16: “How to Work with LAS” Sessions ● Wednesday, June 17: Technical “Office Hour” Sessions ● Analytic Rigor and Performance ○ Data Triage ○ Influence Campaigns ○ Thursday, June 18: Technical “Office Hour” Sessions ● Machine Learning Integrity ○ Human Machine Collaboration ○ Selected Cybersecurity Challenges ○ Additional Use Cases ○ 3
Plenary Session Monday, June 15 Overview of LAS and how we work ● Overview of 2021 LAS interest areas ● Overview of the white paper submission process ● 4
“How to Work with LAS” Sessions Tuesday, June 16 Purpose ● Answer questions about logistics of working with LAS ○ Provide general suggestions about how your research interests might align ○ with the different LAS interest areas. Individual sessions conducted via Zoom ● Sign up for 10-minute time slot at: https://ncsu-las.org/2020-las-collaborators-day/ ○ Two available blocks of time slots ○ 09:00a – 11:00a (EDT) ■ 01:00p – 03:00p (EDT) ■ 5
Technical “Office Hour” Sessions Wednesday, June 17 & Thursday, June 18 Purpose ● Provide an opportunity to speak with LAS staff who have related interests ○ about potential project ideas and collaborations. Individual sessions conducted via Zoom ● Sign up for 10-minute time slot at: https://ncsu-las.org/2020-las-collaborators-day/ ○ Wednesday, June 17 ○ 09:00a – 11:30a (EDT) : Analytic Rigor and Performance (CFWP Section 3.1) ■ 12:00p – 02:30p (EDT) : Influence Campaigns (CFWP Section 4.1) ■ 02:30p – 05:00p (EDT) : Data Triage (CFWP Section 3.4) ■ Thursday, June 18 ○ 09:00a – 11:30a (EDT) : Machine Learning Integrity (CFWP Section 3.2) ■ 12:00p – 02:30p (EDT) : Selected Cyber Security Challenges (CFWP Section 4.2) ■ 12:00p – 02:30p (EDT) : Additional Use Cases (CFWP Section 4.3) ■ 02:30p – 05:00p (EDT) : Human Machine Collaboration (CFWP Section 3.3) ■ 6
Questions If you would like to ask a question please use the Q&A feature in ● Zoom We have multiple places in the talk where we will pause to answer ● questions from the Q&A If you are unable to ask your question through the Q&A feature ● today, please email lasoutreach@ncsu.edu with your question, and we will get back to you. 7
What is the Laboratory for Analytic Sciences? LAS is a mission-oriented academic-industry-government research collaboration that works at the intersection of technology and tradecraft. https://ncsu-las.org/ 8
Advance the tradecraft of intelligence analysis while leveraging novel and recent advances in research and technology ● Investigate technical approaches with the potential to address analysis challenges ● Develop analytic tradecraft that leverages research and technology to address mission needs ● Transition technology and tradecraft to partners who can operationalize and scale solutions 9
How do we work at LAS? Mission-relevant projects ● ~90% of our work is unclassified ● Integrated, team-based approach ● Guidance is intentionally open-ended, as we are expecting you to ● help shape the direction of the projects 10
Who is participating with LAS in 2020? 24 faculty (and ≈ 35 students) at 9 unique universities ● 7 industry partners and 1 national lab ● ≈ 50 government staff/IC partners ● 14 NCSU staff ● 11
What are we looking for in 2021 Collaborators? Immersive ● Iterative approaches to solutions ○ Opportunistic approaches to solutions ○ Interdisciplinary ● Researchers, developers, and practitioners ○ STEM, humanities, and social sciences ○ Relevant Expertise ● Relevant to their own activities ○ Potentially relevant to other activities ○ 12
What are we looking for in 2021 Projects? Impact ● Will a successful outcome have a positive impact for intelligence analysts? Innovation ● Is a new approach proposed, or does it utilize new capabilities? Engagement ● Are LAS stakeholders interested in collaborating on the project? 13
What are we looking for in 2021 Outcomes? New Understanding ● Experimental data ○ Research Papers ○ New Tradecraft ● Storyboards ○ Documented Workflows ○ New Capabilities ● Proofs-of-concept (e.g. Jupyter Notebooks) ○ Prototypes ○ 14
Questions If you would like to ask a question please use the Q&A feature in ● Zoom 15
What are our areas of interest for 2021? Human-Machine Machine Learning Analytic Rigor Triage Collaboration Integrity and Performance Influence, Cybersecurity, and Other Use Cases 16
Analytic Rigor and Performance Defining and Evaluating rigor and its components Identifying the “Fundamental Five” of analyst performance Analytic Evaluating rigor Augmenting Applying rigor to production and in analytic analytic language journalism workflows performance analysis 17
Machine Learning Integrity ML in Production from Amershi et al (2019) Define and support best practices for machine learning operations Label, build, deploy, monitor R&D not finished products People and ML Improve interactions between humans and algorithms End Users Data Scientists Encourage appropriate trust in automated predictions Accelerate development of reliable models ML human factors ML explainability End Users as Data Scientists Empower individual end users to address their own challenges through ML User-centric document classification 18
Human Machine Collaboration Recognizing Intent Understand what an analyst is trying to do Modeling intent in open-world environments Useful interventions Effectively support analysts in achieving their goals from Crouser et al (2020) Comparative studies Microsoft Office Assistant, used with permission from Microsoft. From Wikipedia User:Norm from Guo et 19 from Farrell and Ware (2020) from Hong and Watson (to appear) al (2020)
Data Triage Data Triage concerns the Classic Information Retrieval Challenges of Big Data Data Retention Data Tagging Data Exploration & Survey Data Prioritization 20
Influence Campaigns “The collection of tactical information Indicators, about an adversary as well as the origins, & dissemination of propaganda in provenance pursuit of a competitive advantage over an opponent” (RAND) Message Influence can be: content • Online or offline Impact & • Authentic or effectiveness inauthentic activity Countering • Targeted or broad malign influence • “Innocuous” or malign 21
Selected Cybersecurity Challenges Vulnerability Detection ● Symbolic Execution without Source Code ○ Malware Evolution and Triage ● Polymorphic vs Metamorphic Obfuscation ○ & Detection Techniques Endpoint Detection and Response ● Machine Learning Research for EDR ○ Cybersecurity Policy ● Effectiveness Assessment ○ 22
Additional Use Cases Prioritization of User-Centric Processing Handwriting Voice Data Document Uniquely Recognition in Categorization Structured Forms Scanned Docs 23
Questions If you would like to ask a question please use the Q&A feature in ● Zoom 24
LAS White Paper and Proposal Timeline May 28, 2020 Call for White Papers ● ● June 15-18, 2020 LAS Collaborators Week July 17, 2020 White Papers Due ● Sept 15, 2020 Preliminary Notifications ● Nov 1, 2020 Final Notifications ● Jan 1, 2021 Begin Period of Performance ● Dec 31, 2021 End Period of Performance ● 25
White Paper Submissions In order to propose work, you must submit a white paper ● More than one submission is fine. You should submit one white ● paper for each project idea you have. You may submit team white papers with more than one performer. ● White Papers Due July 17, 2020 26
White Paper Submissions All white papers must be submitted through web-based tool ● Link: https://whitepapers.ncsu-las.net ○ Each white paper submission must include: ● Title ○ All Funded PIs and Main POC ○ NOTE: These will be entered separately in Abstract ○ the submission tool and do not have to be Budget Request ○ repeated in the Technical Description Technical Description ○ We ask that your abstract and white paper NOT contain classified, proprietary, or sensitive information of any kind. 27
Recommend
More recommend
