Analytic Component System (ACS) Matt Schmidt and Andrew Crerar Presentation to LAS Weekly Research Meeting 13 April 2015
Collaborators • Tonya Adelsperger (LAS-G) • John Slankas (LAS-I2I) • Joe Aguayo (LAS-G) • Colleen Stacy (LAS-G) • Edward Browne (LAS-G) • Pankaj Telang (LAS-I2I) • Samuel Christie (NCSU) • Eli Typhina (NCSU) • Bill Elm (RCS) • Kathleen Vogel (NCSU) • Rob Johnston (Johnston Analytics) • Lori Wachter (LAS-G) • Judy Johnston (Johnston Analytics) • Alyson Wilson (NCSU) • James Kaiser (LAS-G) • Mark Wilson (NCSU) • Munindar Singh (NCSU) • Jorge Zuniga (RCS)
Outline Motivation and Overview Tradecraft Storyboard Analytic Component Interface Background and Motivation Approach Initial Results Benefits Analytic Computing Environment Background Demo #1: Execution of Composite Workflow Demo #2: Inspection of Workflow Demo #3: Modification of Workflow Future Work and Summary
Goal of ACS Problem Problem Desired Outcome Desired Outcome • Analysts have access to a wide variety of An organized set of analytic tools and methods analytic tools and methods (components) that an analyst could use to • Applying these tools and methods to a problem construct their desired analytic workflow can require a great deal of effort to: 1. Learn about the tool/method and; 2. Adapt the analyst’s workflow to incorporate it Provide analysts a library of analytic components that can be Provide analysts a library of analytic components that can be combined to create analytic workflows combined to create analytic workflows
Technical Challenge: Component Integration Integration Costs Integration Costs • Analytic components consume and produce certain types of information • Integration occurs when one component consumes information produced by another • Integration costs occur when there is a need to VS translate the information before consumption • Time • Cognitive Load • Mis-Communication • Integration costs can arise at many different places • Component to Component • Component to Human • Human to Human The costs of integrating different components into a single The costs of integrating different components into a single workflow can overwhelm any benefits workflow can overwhelm any benefits
Technical Challenge: Workflows are not Applications • The analytic components used are not always known before execution • Components can be computational, manual, or interactive • Analysts may want to revisit and modify previously executed workflows Analytic workflows do not look like procedural applications and Analytic workflows do not look like procedural applications and therefore require a different computation environment therefore require a different computation environment
Technical Approach Component Interface Component Interface Computing Environment Computing Environment Component Library Component Library Goal Goal Enable compatible component Enable execution of workflows Develop library of novel and interfaces composed of analytic components existing analytic components Approach Approach Define standard analytic Develop service-oriented Decompose and wrap components constructs computing platform using ACI to execute in the ACE The ACS effort consists of three related development efforts The ACS effort consists of three related development efforts
Outline Motivation and Overview Tradecraft Storyboard Analytic Component Interface Background and Motivation Approach Initial Results Benefits Analytic Computing Environment Background Demo #1: Execution of Composite Workflow Demo #2: Inspection of Workflow Demo #3: Modification of Workflow Future Work and Summary
ACS Technology-Enabled Tradecraft Storyboard Backstory Backstory Question Question Resources Resources • Single Analyst • Components in ACS plus front-end Who is the most likely next target of the Anonymous hacktivist group? • Multiple open data sources • Collected reporting Task of the analyst is to report on likely targets of the Task of the analyst is to report on likely targets of the Anonymous hacktivist group Anonymous hacktivist group
Storyboard Scene 1: Question to Data Source Scene Description Scene Description Action Action Analyst selects a background report on Anonymous Entities Involved Entities Involved • Analyst • Analytic Component from ACL • Produces Data Source given a Question • ACE • Sends question to component and sends data source to next component • Background Report Different components in ACS use different combinations of Different components in ACS use different combinations of computation and manual processes to generate constructs computation and manual processes to generate constructs
Storyboard Scene 2: Data to Indicators Scene Description Scene Description Actions Actions • Analyst reads line in background report saying Anonymous members often have strong libertarian beliefs • Analyst creates an indicator question: Who are the most visible anti-libertarian actors right now? Entities Involved Entities Involved • Analyst • ACL Components • ACI Constructs • Background report The ACI enables the data inspection component to generate The ACI enables the data inspection component to generate constructs understood by question resolution components constructs understood by question resolution components
Storyboard Scene 3: Indicators to Ranking Scene Description Scene Description Action Action Analyst inspects automated workflow to see how list of anti-libertarian actors was produced Entities Involved Entities Involved • Analyst • ACL enables automated construction of workflow to produce ranked list • ACE logs all constructs produced in workflow • Front-end allows analyst to inspect The ACL and ACE enables a question resolution workflow to be The ACL and ACE enables a question resolution workflow to be automatically composed and executed automatically composed and executed
Storyboard Scene 4: Manually introduce new data source Scene Description Scene Description Action Action • Analyst adds a missing data source to the workflow • Workflow automatically runs Entities Involved Entities Involved • Analyst • ACE enables workflow modification • Front-end enables inspection of workflow and addition of constructs The ACE enables analyst modification and control of an The ACE enables analyst modification and control of an automated piece of the analytic workflow automated piece of the analytic workflow
Storyboard Scene 5: New Hypothesis Alert Scene Description Scene Description Action Action Analyst is alerted to new anti-libertarian actor because of a new data being added to the data source Entities Involved Entities Involved • Front-end determines how to alert user • ACE enables streaming ingest and analysis of data using workflows originally for historical data • ACL consists of components that can be run in a streaming fashion • ACI enables the use of either batch or streaming methods of producing construct The ACS along with User Experience efforts supports transition of The ACS along with User Experience efforts supports transition of analysis effort to monitoring effort analysis effort to monitoring effort
Storyboard Scene 6: Workflow Capture and Analysis Scene Description Scene Description Action Action • Draft report generated automatically • Workflow becomes part of report • Analysis of workflow can provide estimate of analytic rigor Entities Involved Entities Involved • Elastic Search logs all analytic constructs • Auto-report generation uses constructs to produce draft report • Analytic rigor maps production of constructs to metrics of analytic rigor The constructs logged by the ACS provide a rich source of The constructs logged by the ACS provide a rich source of instrumentation data for reporting and analysis instrumentation data for reporting and analysis
Storyboard Slide 7: Develop new components Scene Description Scene Description Action Action Researchers use logged pairings of constructs to develop automated method of generating the constructs Entities Involved Entities Involved • Researchers • Logged ACI constructs in Elastic Search • Analyst production of constructs Previously captured workflows provide training data for the Previously captured workflows provide training data for the development of new components and workflow recommenders development of new components and workflow recommenders
Related Efforts Instrumentation Instrumentation Analytic Rigor Analytic Rigor ACI constructs define the resolution of ACI constructs provide data from which to workflow logs derive metrics on analytic rigor Analytic Component Development Analytic Component Development User Experience User Experience ACS makes explicit the needs for new User Experience will be required enable components for existing components that analyst to observe and control workflows are inefficient executed in the ACS Auto-Report Generation Auto-Report Generation Recommender Systems Recommender Systems Logged ACI construct should provide Logged ACI constructs provide rich training enough information to create a draft report data source for recommender system efforts automatically Multiple LAS efforts can benefit from or contribute to the ACS Multiple LAS efforts can benefit from or contribute to the ACS
Recommend
More recommend
