Data Warehouse of German Federal Police From Raw Data to Flexible - PowerPoint PPT Presentation

Data Warehouse of German Federal Police From Raw Data to Flexible Analytics

Data Warehouse German Federal Police Table of Contents • Organizational overview of German Federal Police • Pain points (professional and technical requirements) • Technical overview • Benefits • Q&A

Data Warehouse German Federal Police Requirements • Replace the previous Excel-based reporting system • Provide standardized quality-assured data for all cases (parliamentary questions, FRONTEX, Federal Ministry of the Interior) • Provide more flexibility in delivering analyses requested by governmental and EU institutions in short notice

Data Warehouse German Federal Police Our Philosophy Don’t try to boil the ocean! • "The artist who aims at perfection achieves in nothing” • The user doesn´t know the professional requirements -> agile development Keep it short and simple but be prepared for the future • More catalogs, little text needed • more automation, little manual work needed Stay independent • Use open source whenever possible

Data Warehouse German Federal Police Technical Requirements • Flexibility • Data quality assurance (“washing machine”) • Ability to rearchitect the DWH at any time • Reduced maintenance and development costs • Automation and job control / monitoring

Data Warehouse German Federal Police Overview Data Mining Cubes Presentation layer (User) Dashboards Reports Data Mart Data Mart Central Database Data Warehouse E xtraction T ransformation L oading Meta database PES Backup PES Web (QS) Application / Backup E xtraction T ransformation L oading Washing machine External sources Data sources

Data Warehouse German Federal Police Data Management Data sources PES (washing machine) DWH dimension tables PES (washing machine) backup DWH validation DWH fact tables

Data Warehouse German Federal Police Data Management LB virtualized server Washing Galera cluster PES 8 GB RAM ☺ machine DB DB Siko Backup DB PES LB dedicated server 256 GB RAM App App App DWH DWH DB DB

Data Warehouse German Federal Police ETL – Manhunt (job) Slow Fast

Data Warehouse German Federal Police ETL – Manhunt (job)

Data Warehouse German Federal Police PES Web (washing machine)

Data Warehouse German Federal Police ETL – PES -> DWH (validation)

Data Warehouse German Federal Police ETL – fact tables

Data Warehouse German Federal Police Automation / Job control

Data Warehouse German Federal Police Presentation layer

Data Warehouse German Federal Police Presentation layer / BI server

Thank you!

Data Warehouse of German Federal Police From Raw Data to Flexible - PowerPoint PPT Presentation

Data Warehouse of German Federal Police From Raw Data to Flexible Analytics Data Warehouse German Federal Police Table of Contents Organizational overview of German Federal Police Pain points (professional and technical requirements)

An Overview of Data Warehousing and OLAP T echnology What is a data warehouse? A

Data Warehouse Update March 19, 2019 Agenda Why a data warehouse? Why THIS data

Financial Data Financial Data Financial Data Financial Data Warehouse Warehouse Warehouse

BI and WIC Data Warehouse Project Overview Reason for the Data Warehouse project EBT

DATA WAREHOUSE How Business Intel and Data Warehouse works Information DEG uses Teacher

Data Warehouse and Business Intelligence Webinar October 23, 2014 Objectives What is the

Data Warehouse Chronic Conditions Data Warehouse 1 Your source for national CMS Medicare and

Data Warehouse Chronic Conditions Data Warehouse 1 Your source for national CMS Medicare and

Financial Data Financial Data Warehouse Warehouse Some day, on the corporate balance sheet,

Multi-dimensional index structures Part I: motivation 144 Motivation: Data Warehouse A de fi

Data Warehouse Presentation Toto.Horvli@Teradata-NCR.com November 10th 2004 A LARGE Data

ED-FI DATA DASHBOARD TRAINING SPRING 2016 What is Ed-Fi? Ed-Fi is a data warehouse that:

A Data Warehouse-based A Data Warehouse-based Gene Expression Analysis Gene Expression Analysis

DATA WAREHOUSE BUILT FOR THE CLOUD QCON San Francisco, November 2019 Thierry Cruanes, Co-Founder

Building a Big Data DWH Data Warehousing on Hadoop Friso van Vollenhoven @fzk CTO

Data Warehouse Ali Kamandi Sharif University of Technology Spring 2007 kamandi@ce.sharif.edu

Data Warehouse and OLAP Data Warehouse and OLAP Week 5 1 Midterm I Midterm I Friday, March

Tool (CaRT) Student Records Data Key Topics Data Warehouse Modeling Key Subject Areas

The Snowflake Elastic Data Warehouse SIGMOD 2016 and beyond Ashish Motivala, Jiaqi Yan 1 Our

Summary of Last Chapter Principles of Knowledge Discovery in Data What is a data warehouse

An Overview of Data An Overview of Data and facts. Data is usually in different databases and

STRATEGIC LOCATION Add: Warehouse No.5, 2 nd Floor, Tan Cang-Cat Lai warehouse, Cat Lai Ward,

and Web Applications 03 Data Warehousing Alexandros Labrinidis University of Pittsburgh

Hive* A Petabyte Scale Data Warehouse Using Hadoop Authors Facebook Data Infrastructure