Solving Time-to-Market and Data Flexibility problems with IMDG Appar Singh IT Architect Agilent Technologies Inc
About Agilent Agilent is a leader in life sciences, diagnostics and applied chemical markets. The company provides laboratories worldwide with instruments, services, consumables, applications and expertise, enabling customers to gain the insights they seek. Agilent’s expertise and trusted collaboration give them the highest confidence in our solutions. Agilent focuses its expertise on six key markets, where we help our customers achieve their goals: Environmental Chemical and Diagnostics Pharmaceutical Research Food and Forensics Energy • Agilent gives • From disease • We provide • Instruments • Help customers • Helps to ensure doctors a head research and fast, accurate and S/W help maximize their our food supply start in the fight drug discovery, and sensitive Research with production of remain free of against cancer to drug methods for Scientists fuels and contaminants. and other development, monitoring across Globe. predict failures. diseases. manufacturing contaminants. and quality control. 2
Outline • Agility and Data Flexibility • Time to Market for Products • Data Flexibility and its complexity • Digital Data Touchpoints • Normalized Data Catalog • Product Catalog • Regular Data Access Patterns • Leveraging Skinny Integrations and IMDG • Skinny Integrations with Web/e-Store • Data Flexibility using Data Grid • De-Normalized Access • In-Memory Data Grid Design • Access from Downstream Apps • SLA’s post IMDG • Architecture post IMDG 3 • Recommendations Engine
Agility & Data Flexibility Overview
Agility Measured through Time to Market Industry Demand 100% • Increasingly, Agility of a Digital Team 90% is being measured through time to 80% markets for products and product 70% related changes. 60% 50% • Whether we talk about new features for customers or the addition of new 40% product portfolios for a company. 30% 20% • Digital Edge on competition can 10% really be achieved if we can manage and reduce time to market SLA’s. 0% FY16 FY17 FY18 FY19 IT Budgets IT Ability to Deliver Business Needs Expectations 5
Data Flexibility and its Complexity Deign and Coding 15% Test and Build • Really means Relational Data 25% Schema Flexibility. • Schema changes to product and Brainstroming related content attribution from 10% Upstream systems. • Integrate product data models to propagate attribution structure changes to downstream targets. • Agile adaptability of attribution for downstream systems. Administrative Tasks • Complete Normalization helps to 20% Data Model focus on core customer needs and changes product portfolios. 30% 6
Digital Data Touchpoints Application Stack Standard Devices Experience & Delivery Layer HTTPS MDM + Non-MDM E-Store CMS / CXM Search Catalog Publish Catalog Publish Preview & Indexes Delivery Content & Catalog Content & Import Data Delivery Data Delivery Full Data Model Full Data Model Copy Copy Processing & Analysis Layer ETL ETL Dataflow Dataflow Digital Asset Product Data ERP / CRM Management Management / MDM Asset Data Sources 7 Associations/Tagging
MDM-to-Web SLA’s TIME TO MARKET SYSTEM SLA'S Time to Market SLA's (Hours) 10 9 8 7 6 5 4 3 2 1 0 Marketing Updates Product Enrichments New Product Intro Support Material Additions 8
Normalized Data Selling Points and its Problems
Product Catalog Normalized Product Attribute Classification/ Catalog Parts Groups Marketing Support Digital Assets Specifications Attributes Assets Content 10
Data Fetch for Product Involves… Product Attribute Classification/ Catalog Parts Groups Marketing Support Digital Assets Specifications Attributes Assets Content 11
What if we could… Accelerate Avoid Dynamically Avoid Focus on Time to chasing Upgrade Spending Customer Market for MDM data Experience time in Innovation Products model for Newer integrations and data changes for Products with MDM driven Downstream repeatedly. experience. 12
Leveraging Skinny Integrations and IMDG TTM and Data Flexibility Solutions
Skinny Integrations with Web/e-Store • Import attributes in downstream systems only if needed. - Core Attributes • Attributes absolutely needed inside of the platform E-Store CMS / CXM Content & Content & • Contain channel specific attributes. Data Delivery Data Delivery • Contain attributes used for channel specific business logic. I just need the I just need the • Core attributes live in delivery apps like CXM and e- reference! reference! Store. - Improved Performance of Imports by factor of 5. - Improved Internal Publishing performance of downstream apps. • Don’t disturb primary downstream apps for any updates. - Creates Only MDM Product • New Product Introductions. Information • Complete new Product Portfolios. Management • Merger and Acquisitions. • Product Line Splits and Mergers. 14
Data Flexibility using Data Grid • Import all your normalized data in GridGain’s Data Grid. - Neighbor Attributes • Attributes which are needed by the channel but Data Grid CMS / CXM E-Store Cached Content Content & Content & can be referred in runtime. Immutable Data Data Delivery Data Delivery • Contain UI only attributes. • Contain omni-channel attributes which doesn’t Step Aside. Give I just need the I just need the need to be channel specific. me everything you reference! reference! • Neighbors live in Data Grid got. - Heavy attribution imports through Streaming imports. • Do disturb Data Grid every time there is an update upstream. - Creates and Updates • Marketing Updates on products. • Digital Asset Associations. MDM • Support Asset Additions. Product Information • Product Enrichments. Management 15
De-Normalized Access Amalgamate data for Downstream
In-Memory Data Grid Design Service Grid Higher Level Caches Content Products Cache Store e-Catalog Cache Content Classification More… Cache Compute Grid Low Level Caches MDM Products MDM Marketing MDM Parts MDM Classification MDM Digital Objects 50 More… 17
Access from Downstream Apps • Access Patterns bypasses any CMS / CXM Service Grid Content & Data Delivery understanding of MDM Data Model Structure . Higher Level Caches • Downstream apps can focus on Content Store Left customer experience. E-Store Products Nav Cache Content & Cache Data Delivery • As soon as Data changes, Content Catalog More… experience changes. attribution • Improved performance by Compute Grid accessing de-normalized data. Low Level Caches 18
MDM-to-Web SLA’s post IMDG TIME TO MARKET SYSTEM SLA'S Time to Market SLA's (Minutes) 30 25 20 15 10 5 0 Marketing Updates Product Enrichments New Product Intro Support Material Additions 19
Architecture and Integrations Application Stack Standard Devices Experience & Delivery Layer Non-MDM Catalog Search HTTPS Pricing Pull Publish Indexes E-Store CMS / CXM Content & Delivery Preview & Data Delivery Content & Catalog Import Data Delivery * Data Grid Skinny Model Skinny Model Cached Content Immutable Data Flexible Data Model * Processing & Analysis Layer Message Broker Queue’s & Listeners * JMS Queue’s & Listeners Digital Asset Product Data ERP / CRM Management Management / MDM Asset Data Sources 20 Associations/Tagging
Recommendations Engine Machine Learning and Data Delivery
Concept • A scalable front end that records user interactions to collect data. • Permanent storage that can be accessed by a machine learning platform. Loading the data into this storage can include several steps, such as import- export and transformation of the data. • A machine learning platform that can analyze the existing content to create relevant recommendations. • Storage that can be used by the front end, in real time or later, based on the timeliness requirements for recommendations. 22
Methodology • Used Collaborative Filtering model, which generates recommendations based on the relationship between the visitors and products. • No explicit information regarding the visitors or the products required in the approach. • Implicit: Not as obvious in terms of preference, such as views, clicks, purchase. • Solving the problem requires a matrix of user-item interactions. • We utilized Matrix Factorization method to figure out the latent (hidden) features that relate them to each other in a much smaller matrix of user features and item features. 23
Recommendation Engine Design Ingest Data Storage Data Processing Analytics Storage/Analysis ML Jobs Compute Standard Storage Processing Bucket Dataproc Cloud Storage Processing Pipelines Data Grid Data Grid Data Grid Fast Storage Fast Storage Fast Storage Data ready Data ready Data ready Serverless bucket with Nested Serverless Functions objects functions OnPrem Services Localization Entitlements Pricing 24
Recommend
More recommend
